Also consolidate the leftover packages under cli.
Remove pkg/mflag.
Make manpage generation work with new cobra layout.
Remove remaining mflag and fix tests after rebase with master.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Warn the user and fail daemon start if the graphdir path has any
elements which will deny access to the remapped root uid/gid.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
This fix tries to address the issue raised in 25927 where
the HTTP headers have been chaged when AUthZ plugin is in
place.
This issue is that in `FlushAll` (`pkg/authorization/response.go`),
the headers have been written (with `WriteHeader`) before all the
headers have bee copied.
This fix fixes the issue by placing `WriteHeader` after.
A test has been added to cover the changes.`
This fix fixes 25927
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Fix delete containers and make sure it prints errors correctly.
Rename Result.Fails to Result.Assert()
Create a constant for the default expected.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Remove some run functions and replace them with the unified run command.
Remove DockerCmdWithStdoutStderr
Remove many duplicate runCommand functions.
Also add dockerCmdWithResult()
Allow Result.Assert() to ignore the error message if an exit status is expected.
Fix race in DockerSuite.TestDockerInspectMultipleNetwork
Fix flaky test DockerSuite.TestRunInteractiveWithRestartPolicy
Signed-off-by: Daniel Nephin <dnephin@docker.com>
if there is no cookie set in dm task, or flag DM_UDEV_DISABLE_LIBRARY_FALLBACK
is cleared for a DM_DEV_REMOVE task, libdevmapper will fallback to clean up the
symlink under /dev/mapper by itself, no matter the device removal is executed
immediately or deferred by the kernel.In some cases, the removal is deferred by the
kernel, while the symlink is deleted directly by libdevmapper, when docker tries to
activate the device again, the deferred removal will be canceld, but the symlink will
not show up again, so docker's attempt to mount the device by the symlink will fail,
and it will eventually leads to a `docker start/diff` error.
Fixes#24671
Signed-off-by: Ji.Zhilong <zhilongji@gmail.com>
Plugin discovery on Windows is not possible using named pipes. However,
it is possible using spec file (tcp based). This adds Windows specific
paths for discovery.
Fixes#23605
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
The namespace unshare+pivot root is not possible when running inside a
user namespace, so fallback to the original "real" chroot code.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
If we are running in a user namespace, don't try to mknod as
it won't be allowed. libcontainer will bind-mount the host's
devices over files in the container anyway, so it's not needed.
The chrootarchive package does a chroot (without mounting /proc) before
its work, so we cannot check /proc/self/uid_map when we need to. So
compute it in advance and pass it along with the tar options.
Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
This patch implements an Ellipsis utility to
append an ellipsis (...) when truncating
strings in output.
It also fixes the existing Truncate() utility
to be compatible with unicode/multibyte characters.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Fixes case where shutdown occurs before content is synced to disked
on layer creation. This case can leave the layer store in an bad
state and require manual recovery. This change ensures all files
are synced to disk before a layer is committed. Any shutdown that
occurs will only cause the layer to not show up but will allow it to
be repulled or recreated without error.
Added generic io logic to ioutils package to abstract it out of
the layer store package.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Problem Description:
An example scenario that involves deferred removal
1. A new base image gets created (e.g. 'docker load -i'). The base device is activated and
mounted at some point in time during image creation.
2. While image creation is in progress, a privileged container is started
from another image and the host's mount name space is shared with this
container ('docker run --privileged -v /:/host').
3. Image creation completes and the base device gets unmounted. However,
as the privileged container still holds a reference on the base image
mount point, the base device cannot be removed right away. So it gets
flagged for deferred removal.
4. Next, the privileged container terminates and thus its reference to the
base image mount point gets released. The base device (which is flagged
for deferred removal) may now be cleaned up by the device-mapper. This
opens up an opportunity for a race between a 'kworker' thread (executing
the do_deferred_remove() function) and the Docker daemon (executing the
CreateSnapDevice() function).
This PR cancel the deferred removal, if the device is marked for it. And reschedule the
deferred removal later after the device is resumed successfully.
Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>
Fixes: #25073
Update kernel memory on running containers without initialized
is forbidden only on kernel version older than 4.6.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
Following #22729, enable to dynamically reload/remove the daemon
authorization plugins (via standard reloading mechanism).
https://docs.docker.com/engine/reference/commandline/daemon/#daemon-
configuration-file
Daemon must store a reference to the authorization middleware to refresh
the plugin on configuration changes.
Signed-off-by: Liron Levin <liron@twistlock.com>
SetConsoleMode() on input handles appears to remember invalid bits that
were set, causing problems for other programs (such as xcopy.exe) trying
to set the console mode after docker.exe has exited. Always restore
the input console mode on set failure.
Signed-off-by: John Starks <jostarks@microsoft.com>
This slightly simplifies TruncateID() by only
trimming the string if needed.
Also improved unit-tests for this package;
- Add a test for GenerateNonCryptoID()
- Add a test for shortening a sha-256 ID
- Make TestShortenId() more "unit", by using a fixed string, instead of calling GenerateRandomID()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Legacy plugin model maintained a map of plugins. This is
not used by the new model. Using this map in the new model
causes incorrect lookup of plugins. This change uses adds
a plugin to the map only if its legacy.
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
Use module name logrus instead of log.
Use logrus.[Error|Warn|Debug|Fatal|Panic|Info]f instead of w/o f
Signed-off-by: Daehyeok Mun <daehyeok@gmail.com>
This reduces memory usage with a lot of docker proxy processes.
On Docker for Mac we are currently carrying a patch to replace
the binary as we modify it to forward ports to the Mac rather
than the Linux VM, this allows us to simply replace this binary
in our packaging with one that has a compatible interface. This
patch does not provide an easy way to substitute a binary as
the interface is complex and there are few use cases, but where
needed this can be done.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
In the API:
`Writable` changed to `ReadOnly`
`Populate` changed to `NoCopy`
Corresponding CLI options updated to:
`volume-writable` changed to `volume-readonly`
`volume-populate` changed to `volume-nocopy`
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
British mathematician Andrew Wiles is notable for solving Fermat's LAst
Theorem. Florence Nightingale was the first female inducted into the
Royal Statistical Society. Thomas Alva Edison was a prolific inventor,
noted for inventing the incandescent light bulb, the phonograph and the
motion picture camera (among many other).
Signed-off-by: Shourya Sarcar <shourya.sarcar@gmail.com>
This fix tries to fix the issue raised in #24168 where
golang.org/x/sys causes s390x build failure.
This fix removed the import of "golang.org/x/sys/unix".
This fix fixes#24168.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Perform chmod before rename with the atomic file writer.
Ensure writeErr is set on short write and file is removed on write error.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
This fix tries to fix wrong CPU count after CPU hot-plugging.
On windows, GetProcessAffinityMask has been used to probe the
number of CPUs in real time.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address issues raised in #23768 where the CPU count
is not updated after cpu ho-plugging.
This fix follows the suggestion from #23768 and replace go's `runtime.NumCPU()`
with `sysconf(_SC_NPROCESSORS_ONLN)` so that correct CPU count could
be obtained even after CPU hot-plugging.
This fix is tested manually, as is suggested in #23768.
This fix fixes#23768.
The NumCPU() in Linux is based on @wmark 's implementation.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Currently when overlay creates a whiteout file then the overlay2 layer is archived,
the correct tar header will be created for the whiteout file, but the tar logic will then attempt to open the file causing a failure.
When tar encounters such failures the file is skipped and excluded for the archive, causing the whiteout to be ignored.
By skipping the copy of empty files, no open attempt will be made on whiteout files.
Fixes#23863
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Always enable VT output emulation when starting the process so that
non-attaching commands can still output VT codes.
Also remove the version block for using the native console and just rely
on supported flags being present.
Signed-off-by: John Starks <jostarks@microsoft.com>
This patch introduces a new experimental engine-level plugin management
with a new API and command line. Plugins can be distributed via a Docker
registry, and their lifecycle is managed by the engine.
This makes plugins a first-class construct.
For more background, have a look at issue #20363.
Documentation is in a separate commit. If you want to understand how the
new plugin system works, you can start by reading the documentation.
Note: backwards compatibility with existing plugins is maintained,
albeit they won't benefit from the advantages of the new system.
Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
This fix tries to fix logrus formatting by removing `f` from
`logrus.[Error|Warn|Debug|Fatal|Panic|Info]f` when formatting string
is not present.
This fix fixes#23459.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
When pivot_root fails we need to unmount the bind mounted path we
previously mounted in preparation for pivot_root.
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
There might be other (valid) reasons for setxattr(2) to fail, so only
ignore it when it's a not supported error (ENOTSUP). Otherwise, bail.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
This is similar to network scopes where a volume can either be `local`
or `global`. A `global` volume is one that exists across the entire
cluster where as a `local` volume exists on a single engine.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This fix tries to address the issue raised in #22420. When
`--tmpfs` is specified with `/tmp`, the default value is
`rw,nosuid,nodev,noexec,relatime,size=65536k`. When `--tmpfs`
is specified with `/tmp:rw`, then the value changed to
`rw,nosuid,nodev,noexec,relatime`.
The reason for such an inconsistency is because docker tries
to add `size=65536k` option only when user provides no option.
This fix tries to address this issue by always pre-progating
`size=65536k` along with `rw,nosuid,nodev,noexec,relatime`.
If user provides a different value (e.g., `size=8192k`), it
will override the `size=65536k` anyway since the combined
options will be parsed and merged to remove any duplicates.
Additional test cases have been added to cover the changes
in this fix.
This fix fixes#22420.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
For me when I run the test I see:
```
Downloading from http://nourl/bad
Importing 283 B
Untar re-exec error: exit status 1: output: unexpected EOF
```
and nothing about "dial tcp" so it appears that the output is
system dependent and therefore we can't really check it. I think
checking for non-zero exit code is sufficient so I'm removing this
string check.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Using golang 1.6, is it now possible to ignore SIGPIPE events on
stdout/stderr. Previous versions of the golang library cached 10
events and then killed the process receiving the events.
systemd-journald sends SIGPIPE events when jounald is restarted and
the target of the unit file writes to stdout/stderr. Docker logs to stdout/stderr.
This patch silently ignores all SIGPIPE events.
Signed-off-by: Jhon Honce <jhonce@redhat.com>
The path we're trying to remove doesn't exist after a successful
chroot+chdir because a / is only appended after pivot_root is
successful and so we can't cleanup anymore with the old path.
Also fix leaking .pivot_root dirs under /var/lib/docker/tmp/docker-builder*
on error.
Fix https://github.com/docker/docker/issues/22587
Introduced by https://github.com/docker/docker/pull/22506
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
