This fix tries to address the issue raised in 25304 to support
`--group-add` and `--group-rm` in `docker service create`.
This fix adds `--group-add` to `docker service create` and `docker service update`,
adds `--group-rm` to `docker service update`.
This fix updates docs for `docker service create` and `docker service update`:
1. Add `--group-add` to `docker service create` and `docker service update`
2. Add `--group-rm` to `docker service update`
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
There are cases such as migrating from classic overlay network to the
swarm-mode networking (without kv-store), such a mechanism to allow
disconnecting a container even when a network isnt available will be
useful.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
There can be a race between getting the container ids for matches and
getting the actual container. This makes sure that we check that the
container returned by `Get` is non-nil before adding it to the list of
matches.
Fixes#25991
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
This fix tries to fix the issue raised in 25863 where `--ip` value
is not validated for `docker create`. As a result, the IP address
passed by `--ip` is not used for `docker create` (ignored silently).
This fix adds validation in the daemon so that `--ip` and `--ip6`
are properly validated for `docker create`.
An integration test has been added to cover the changes.
This fix fixes 25863.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Support args to RunCommand
Fix docker help text test.
Fix for ipv6 tests.
Fix TLSverify option.
Fix TestDaemonDiscoveryBackendConfigReload
Use tempfile for another test.
Restore missing flag.
Fix tests for removal of shlex.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Also consolidate the leftover packages under cli.
Remove pkg/mflag.
Make manpage generation work with new cobra layout.
Remove remaining mflag and fix tests after rebase with master.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Cleanup cobra integration
Update windows files for cobra and pflags
Cleanup SetupRootcmd, and remove unnecessary SetFlagErrorFunc.
Use cobra command traversal
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Warn the user and fail daemon start if the graphdir path has any
elements which will deny access to the remapped root uid/gid.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
No user of GetNetworkDriverList needs to access the map by key.
The only user of GetNetworkDriverList is in docker info and with a map
the network list is always flipping because loop is not deterministic.
Fix this by returning a string slice which instead is.
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
This fix tries to address the issue in 25000 where `docker stats`
will not show network stats with `NetworkDisabled=true`.
The `NetworkDisabled=true` could be either invoked through
remote API, or through `docker daemon -b none`.
The issue was that when `NetworkDisabled=true` either by API or
by daemon config, there is no SandboxKey for container so an error
will be returned.
This fix fixes this issue by skipping obtaining SandboxKey if
`NetworkDisabled=true`.
Additional test has bee added to cover the changes.
This fix fixes 25000.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
In the common case where the user is using /var/lib/docker and
an image with less than 60 layers, forking is not needed. Calculate
whether absolute paths can be used and avoid forking to mount in
those cases.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
It makes little sense to have swarm related code into the daemon
package. This refactor the `daemon` and `cluster` package to remove
`ListContainersForNode` from the daemon.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Right now docker puts swarm's control socket into the docker root dir
(e.g. /var/lib/docker).
This can cause some nasty issues with path length being > 108
characters, especially in our CI environment.
Since we already have some other state going in the daemon's exec root
(libcontainerd and libnetwork), I think it makes sense to move the
control socket to this location, especially since there are other unix
sockets being created here by docker so it must always be at a path that
works.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This fix tries to address the issue in raised #23367 where an out-of-band
volume driver deletion leaves some data in docker. This prevent the
reuse of deleted volume names (by out-of-band volume driver like flocker).
This fix adds a `--force` field in `docker volume rm` to forcefully purge
the data of the volume that has already been deleted.
Related documentations have been updated.
This fix is tested manually with flocker, as is specified in #23367.
An integration test has also been added for the scenario described.
This fix fixes#23367.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
`--log-opt splunk-format=inline|json|raw` allows to change how logging
driver sends data to Splunk, where
`inline` - default value, format used before, message is injected as a
line in JSON payload
`json` - driver will try to parse each line as a JSON object and embed it
inside of the JSON payload
`raw` - driver will send Raw payload instead of JSON, tag and attributes
will be prefixed before the message
`--log-opt splunk-verify-connection=true|false` - allows to skip
verification for Splunk Url
Signed-off-by: Denis Gladkikh <denis@gladkikh.email>
fixes#25766
If a container's AutoRemove is enabled, client will wait until it's
removed after container exits, this is implemented based on "destroy"
event.
Currently an "AutoRemove" container will report "destroy" event to
notify a hanging client to exit before all volumes are removed, this is
wrong, we should wait container until everything is cleaned up.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
Fixes a race where the log reader would get events for both an actual
rotation as we from fsnotify (`fsnotify.Rename`).
This issue becomes extremely apparent when rotations are fast, for
example:
```
$ docker run -d --name test --log-opt max-size=1 --log-opt max-file=2
busybox sh -c 'while true; do echo hello; usleep 100000; done'
```
With this change the log reader for jsonlogs can handle rotations that
happen as above.
Instead of listening for both fs events AND rotation events
simultaneously, potentially meaning we see 2 rotations for only a single
rotation due to channel buffering, only listen for fs events (like
`Rename`) and then wait to be notified about rotation by the logger.
This makes sure that we don't see 2 rotations for 1, and that we don't
start trying to read until the logger is actually ready for us to.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
If we are running in a user namespace, don't try to mknod as
it won't be allowed. libcontainer will bind-mount the host's
devices over files in the container anyway, so it's not needed.
The chrootarchive package does a chroot (without mounting /proc) before
its work, so we cannot check /proc/self/uid_map when we need to. So
compute it in advance and pass it along with the tar options.
Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
During image pulls on docker service create, logs will only show status
updates and updates every 1 second on long-running actions like
downloading and extracting. Adds golang.org/x/time/rate as dependency.
Ports docker/swarmkit#1352 to docker/docker.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Since we added labels for volume, it is desired to have
filter support label for volume
Closes: #21416
Signed-off-by: Kai Qiang Wu(Kennan) <wkqwu@cn.ibm.com>
Remove checks that prevent overlay and SELinux from working together.
Fixes are arriving in the 4.9 kernel.
Signed-off-by: Dan Walsh <dwalsh@redhat.com>
In cases there are failures in task start, swarmkit might be trying to
restart the task again in the same node which might keep failing. This
creates a race where when a failed task is getting removed it might
remove the associated network while another task for the same service
or a different service but connected to the same network is proceeding
with starting the container knowing that the network is still
present. Fix this by reacting to `ErrNoSuchNetwork` error during
container start by trying to recreate the managed networks. If they
have been removed it will be recreated. If they are already present
nothing bad will happen.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Unlike `docker run -v..`, `docker service create --mount`
does not allow bind-mounting non-existing host paths.
This adds validation for the specified `source`, and
produces an error if the path is not found on the
host.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before this, container's auto-removal after exit is done in a goroutine,
this commit will get ContainerRm out of the goroutine.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
If AutoRemove is set, wait until client get `destroy` events, or get
`detach` events that implies container is detached but not stopped.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
`--rm` is a client side flag which caused lots of problems:
1. if client lost connection to daemon, including client crash or be
killed, there's no way to clean garbage container.
2. if docker stop a `--rm` container, this container won't be
autoremoved.
3. if docker daemon restart, container is also left over.
4. bug: `docker run --rm busybox fakecmd` will exit without cleanup.
In a word, client side `--rm` flag isn't sufficient for garbage
collection. Move the `--rm` flag to daemon will be more reasonable.
What this commit do is:
1. implement a `--rm` on daemon side, adding one flag `AutoRemove` into
HostConfig.
2. Allow `run --rm -d`, no conflicting `--rm` and `-d` any more,
auto-remove can work on detach mode.
3. `docker restart` a `--rm` container will succeed, the container won't
be autoremoved.
This commit will help a lot for daemon to do garbage collection for
temporary containers.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
This fix tries to address the issue raised in #23498 to allow unset
`--entrypoint` in `docker run` or `docker create`.
This fix checks the flag `--entrypoint` and, in case `--entrypoint=` (`""`)
is passed, unset the Entrypoint during the container run.
Additional integration tests have been created to cover changes in this fix.
This fix fixes#23498.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the issue raised in #23528 where
docker labels caused journald log error because journald
has special requirements on field names.
This fix addresses this issue by sanitize the labels per
requirements of journald.
Additional unit tests have been added to cover the changes.
This fix fixes#23528.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the issue raised in 25374 where the
output of `docker ps --filter` is in random order and
not deterministic.
This fix sorts the list of containers by creation time so that the
output is deterministic.
An integration test has been added.
This fix fixes 25374.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Problem Description:
An example scenario that involves deferred removal
1. A new base image gets created (e.g. 'docker load -i'). The base device is activated and
mounted at some point in time during image creation.
2. While image creation is in progress, a privileged container is started
from another image and the host's mount name space is shared with this
container ('docker run --privileged -v /:/host').
3. Image creation completes and the base device gets unmounted. However,
as the privileged container still holds a reference on the base image
mount point, the base device cannot be removed right away. So it gets
flagged for deferred removal.
4. Next, the privileged container terminates and thus its reference to the
base image mount point gets released. The base device (which is flagged
for deferred removal) may now be cleaned up by the device-mapper. This
opens up an opportunity for a race between a 'kworker' thread (executing
the do_deferred_remove() function) and the Docker daemon (executing the
CreateSnapDevice() function).
This PR cancel the deferred removal, if the device is marked for it. And reschedule the
deferred removal later after the device is resumed successfully.
Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>
Fixes: #25073
Update kernel memory on running containers without initialized
is forbidden only on kernel version older than 4.6.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
This is intended as a minor fix for 1.12.1 so that task creation doesn't
do unexpected things when the user supplies erroneous paths.
In particular, because we're currently using hostConfig.Binds to setup
mounts, if a user uses an absolute path for a volume mount source, or a
non-absolute path for a bind mount source, the engine will do the
opposite of what the user requested since all absolute paths are
treated as binds and all non-absolute paths are treated as named
volumes.
Fixes#25253
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This fix tries to improve error messages when IP address
autodetection fails, as is specified in 25141.
Previously, error messages only indicate that multiple IPs
exist when autodetection fails. In this fix, if one
interface consists of multiple addresses or multiple
interfaces consist of addresses, the error messages output
the address names and interface names so that end user could
take notice.
This fix is verified manually.
When multiple addresses exist on multiple interfaces:
```
$ sudo docker swarm init
Error response from daemon: could not choose an IP address
to advertise since this system has multiple addresses on different
interfaces (192.168.186.128 on ens33 and 192.168.100.199 on eth10)
- specify one with --advertise-addr
```
When multiple addresses exist on single interface:
```
$ sudo docker swarm init
Error response from daemon: could not choose an IP address
to advertise since this system has multiple addresses
on interface ens33 (192.168.186.128 and 192.168.55.199)
- specify one with --advertise-addr
```
This fix fixes 25141.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Windows base layers are no longer the special "layers+base" type, so we can remove all the special handling for that.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
A kernel may support any of these, but an admin may have unmounted
certain cgroups, so let's include that possibility in the error so
as to avoid users thinking they have a kernel issue.
Signed-off-by: Christy Perez <christy@linux.vnet.ibm.com>
join-tokens are not needed for this endpoint, and should not be as part
of /info
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
(cherry picked from commit d8f3b3f006)
With digests being added by default, all images have multiple references.
The check for whether force is required to remove the reference should use the new check for single reference which accounts for digest references.
This change restores pre-1.12 behavior and ensures images are not accidentally left dangling while a container is running.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Context cancellations were previously causing `Prepare` to fail
completely on re-entrant calls. To prevent this, we filtered out cancels
and deadline errors. While this allowed the service to proceed without
errors, it had the possibility of interrupting long pulls, causing the
pull to happen twice.
This PR forks the context of the pull to match the lifetime of
`Controller`, ensuring that for each task, the pull is only performed
once. It also ensures that multiple calls to `Prepare` are re-entrant,
ensuring that the pull resumes from its original position.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Remove the swarm inspect command and use docker info instead to display
swarm information if the current node is a manager.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Ensure that cancellation of a pull propagates rather than continuing to
container creation. This ensures that the `Prepare` method is properly
re-entrant.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Instead reserve exit code 2 to be future proof, document that it should
not be used. Implementation-wise, it is considered as unhealthy, but
users should not rely on this as it may change in the future.
Signed-off-by: Tibor Vass <tibor@docker.com>
This changes the default behavior so that rolling updates will not
proceed once an updated task fails to start, or stops running during the
update. Users can use docker service inspect --pretty servicename to see
the update status, and if it pauses due to a failure, it will explain
that the update is paused, and show the task ID that caused it to pause.
It also shows the time since the update started.
A new --update-on-failure=(pause|continue) flag selects the
behavior. Pause means the update stops once a task fails, continue means
the old behavior of continuing the update anyway.
In the future this will be extended with additional behaviors like
automatic rollback, and flags controlling parameters like how many tasks
need to fail for the update to stop proceeding. This is a minimal
solution for 1.12.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Truncated dir name can't give any useful information, print whole dir
name will.
Bad debug log is like this:
```
DEBU[2449] aufs error unmounting /var/lib/doc: no such file or directory
```
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
