Commit graph

145 commits

Author SHA1 Message Date
Daniel Nephin
93fbdb69ac Remove error return from RootPair
There is no case which would resolve in this error. The root user always exists, and if the id maps are empty, the default value of 0 is correct.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-07 11:45:33 -04:00
Daniel Nephin
df248d31d9 Remove ToHost and replace it with IDMappings.ToHost
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-07 11:44:34 -04:00
Daniel Nephin
acdbc285e2 Fix vfs unit test and port VFS to the new IDMappings
The test was failing because TarOptions was using a non-pointer for
ChownOpts, which meant the check for nil was never true, and
createTarFile was never using the hdr.UID/GID

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-07 11:44:34 -04:00
Daniel Nephin
5672eeb5e0 Convert tarAppender to the newIDMappings.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-07 11:44:34 -04:00
Daniel Nephin
967ef7e6d2 Remove unused functions from archive.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-07 11:44:33 -04:00
Daniel Nephin
8cd6c30a48 Upadte archive.ReplaceFileTarWrapper() to not expect a sorted archive
Improve test coverage of ReplaceFileTarWrapper()

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-06 13:36:34 -04:00
Daniel Nephin
56bf6de9f7 Factor out adding dockerfile from stdin.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-05 19:42:31 -04:00
David Sheets
3f6dc81e10 build: accept -f - to read Dockerfile from stdin
Heavily based on implementation by David Sheets

Signed-off-by: David Sheets <sheets@alum.mit.edu>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-05 19:42:31 -04:00
unclejack
087f7307a6 pkg/{chroot,}archive: clean up a few small issues
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2017-03-30 16:37:54 +03:00
Tonis Tiigi
f95f58283b Add support for COPY from previous rootfs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-03-23 15:12:15 -07:00
Tonis Tiigi
adeb3e3684 Fix inefficient file paths filter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-03-08 18:59:25 -08:00
yuexiao-wang
11454e1c97 Fix a bit typos
Signed-off-by: yuexiao-wang <wang.yuexiao@zte.com.cn>
2016-12-09 03:05:11 +08:00
John Howard
7c29f5beed Windows: Use sequential file access
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-11-14 17:41:52 -08:00
Vincent Demeester
bcdce2a6b0 Merge pull request #27705 from dmcgowan/fix-27298
Ensure opaque directory permissions respected
2016-11-02 10:13:30 +01:00
Qiang Huang
e6866492c4 Fix bunch of typos
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2016-10-29 15:03:26 +08:00
Derek McGowan
daa7019517
Ensure opaque directory permissions respected
When converting an opaque directory always keep the original
directory tar entry to ensure directory is created with correct
permissions on restore.

Closes #27298

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-10-28 13:13:10 -07:00
Stephen J Day
aa2cc18745
pkg/archive: remove unnecessary Archive and Reader type
The `archive` package defines aliases for `io.ReadCloser` and
`io.Reader`. These don't seem to provide an benefit other than type
decoration. Per this change, several unnecessary type cases were
removed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-10-20 19:31:24 -07:00
Tonis Tiigi
57e12037ac Fix error reporting in CopyFileWithTar
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-09-30 11:46:37 -07:00
Serge Hallyn
617c352e92 Don't create devices if in a user namespace
If we are running in a user namespace, don't try to mknod as
it won't be allowed.  libcontainer will bind-mount the host's
devices over files in the container anyway, so it's not needed.

The chrootarchive package does a chroot (without mounting /proc) before
its work, so we cannot check /proc/self/uid_map when we need to.  So
compute it in advance and pass it along with the tar options.

Signed-off-by: Serge Hallyn <serge.hallyn@ubuntu.com>
2016-08-12 16:26:58 -04:00
Vincent Demeester
37fe4bdb59 Merge pull request #23912 from dmcgowan/fix-overlay2-whiteout-exclusion
Fix overlay2 ignoring whiteout files
2016-06-24 01:06:38 +02:00
Derek McGowan
bd13c53f8d Fix overlay2 ignoring whiteout files
Currently when overlay creates a whiteout file then the overlay2 layer is archived,
the correct tar header will be created for the whiteout file, but the tar logic will then attempt to open the file causing a failure.
When tar encounters such failures the file is skipped and excluded for the archive, causing the whiteout to be ignored.
By skipping the copy of empty files, no open attempt will be made on whiteout files.

Fixes #23863

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-23 13:34:38 -07:00
bin liu
950073aabb fix some typos
Signed-off-by: bin liu <liubin0329@gmail.com>
2016-06-21 15:29:25 +08:00
Michael Crosby
8a2f9a249c Merge pull request #22126 from dmcgowan/overlay-native-diff
Overlay multiple lower directory support
2016-06-13 13:15:39 -07:00
Yong Tang
a72b45dbec Fix logrus formatting
This fix tries to fix logrus formatting by removing `f` from
`logrus.[Error|Warn|Debug|Fatal|Panic|Info]f` when formatting string
is not present.

This fix fixes #23459.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-06-11 13:16:55 -07:00
Aleksa Sarai
e6d856df43 pkg: archive: only ignore ENOTSUP when xattr fails
There might be other (valid) reasons for setxattr(2) to fail, so only
ignore it when it's a not supported error (ENOTSUP). Otherwise, bail.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2016-06-10 00:56:40 +10:00
Derek McGowan
8222c86360 Update archive package to support overlay whiteouts
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-08 00:10:16 -07:00
Viktor Stanchev
b03d3232d1 aufs compatibility
Signed-off-by: Viktor Stanchev <me@viktorstanchev.com>
2016-06-08 00:10:16 -07:00
allencloud
3f77baf2ce fix typos
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-05-06 18:23:11 +08:00
Alexander Morozov
9b2c3aa894 pkg/archive: use more narrow interface for CompressStream
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2016-04-07 13:21:05 -07:00
Tõnis Tiigi
553edd1d19 Merge pull request #21251 from cyphar/refactor-copyonbuild
pkg: archive: don't fail Untar if xattrs are not supported
2016-03-28 16:15:18 -07:00
Dmitri Logvinenko
6dede879fd Fix misspellings
Signed-off-by: Dmitri Logvinenko <dmitri.logvinenko@gmail.com>
2016-03-28 16:21:02 +03:00
Aleksa Sarai
75a271603c pkg: archive: don't fail Untar if xattrs are not supported
Since certain filesystems don't support extended attributes, ignore
errors produced (emitting a warning) when attempting to apply extended
attributes to file.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2016-03-24 02:05:36 +11:00
Phil Estes
06d95003b3 Fix CopyWithTar creation of new destination dir as remapped root
If the destination does not exist, it needs to be created with ownership
mapping to the remapped uid/gid ranges if user namespaces are enabled.
This fixes ADD operations, similar to the prior fixes for COPY and WORKDIR.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2016-03-12 23:05:45 -05:00
Doug Davis
842b8d8784 Optimize .dockerignore when there are exclusions
Closes #20470

Before this PR we used to scan the entire build context when there were
exclusions in the .dockerignore file (paths that started with !). Now we
only traverse into subdirs when one of the exclusions starts with that dir
path.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2016-03-03 05:46:24 -08:00
Sebastiaan van Stijn
67b16c7381 Merge pull request #20782 from estesp/new-parent-dir-ownership
Fix ownership of non-existing parent dir
2016-03-01 01:45:38 +01:00
Phil Estes
7a61b9ae42 Fix ownership of non-existing parent dir
During "COPY" or other tar unpack operations, a target/destination
parent dir might not exist and should be created with ownership of the
root in the right context (including remapped root when user namespaces
are enabled)

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2016-02-29 09:14:23 -05:00
Zhang Wei
439433099e Clean redundant error message for export
When execute `docker export -o path xxx` and path is a directory docker
has no privilege to write to, daemon will print lots of error logs that
most of them are duplicated and redundant.

This will remove unnecessary error logs and print only once.

Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2016-02-19 18:50:11 +08:00
Justas Brazauskas
927b334ebf Fix typos found across repository
Signed-off-by: Justas Brazauskas <brazauskasjustas@gmail.com>
2015-12-13 18:04:12 +02:00
Phil Estes
1626c9dae0 Skip aufs whiteout files on userns translation to container
aufs kernel module creates whiteout files on upper layer delete (and
other situations) and those files already are 'translated' regarding
ownership in host terms (e.g. they are already "0:0" owned), so when
these layers are copied around with pkg/archive we don't want to try and
translate these files regarding ownership.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2015-12-09 10:11:32 -05:00
Tibor Vass
1f8efc687c Merge pull request #18123 from aidanhs/aphs-fail-on-broken-tar
Ensure adding a broken tar doesn't silently fail
2015-12-07 14:38:21 +01:00
Doug Davis
7bb9fc415a Fix for zero-sized layers
Moved a defer up to a better spot.

Fixed TestUntarPathWithInvalidDest to actually fail for the right reason

Closes #18170

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-11-25 18:02:03 -08:00
Aidan Hobson Sayers
3243e504d0 Ensure adding a broken tar doesn't silently fail
Signed-off-by: Aidan Hobson Sayers <aidanhs@cantab.net>
2015-11-23 14:18:58 +00:00
Stefan J. Wernli
37ba67bf63 Fixing last access time on Windows to unblock python.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
2015-11-05 12:15:18 -08:00
Jess Frazelle
98c01c2c90 Merge pull request #17088 from Microsoft/sjw/archive_hang_fix
Fixing hang in archive.CopyFileWithTar with invalid dst
2015-10-19 17:18:25 -07:00
Stefan J. Wernli
a150eee308 Fixing hang in archive.CopyWithTar with invalid dst
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
2015-10-19 11:56:05 -07:00
Dmitry Vorobev
4102537cfd Fixes #9283. Consider hardlinks in image size.
Based on #8984. This patch fixes behavior when image size calculation
didn't consider hardlinks.

Signed-off-by: Dmitry Vorobev <dimahabr@gmail.com>
2015-10-17 12:24:05 +02:00
Derek Ch
b48f4bf523 fix a race crash when building with "ADD some-broken.tar.xz ..."
The race is between pools.Put which calls buf.Reset and exec.Cmd
doing io.Copy from the buffer; it caused a runtime crash, as
described in #16924:

``` docker-daemon cat the-tarball.xz | xz -d -c -q | docker-untar /path/to/... (aufs ) ```

When docker-untar side fails (like try to set xattr on aufs, or a broken
tar), invokeUnpack will be responsible to exhaust all input, otherwise
`xz` will be write pending for ever.

this change add a receive only channel to cmdStream, and will close it
to notify it's now safe to close the input stream;

in CmdStream the change to use Stdin / Stdout / Stderr keeps the
code simple, os/exec.Cmd will spawn goroutines and call io.Copy automatically.

the CmdStream is actually called in the same file only, change it
lowercase to mark as private.

[...]
INFO[0000] Docker daemon                                 commit=0a8c2e3 execdriver=native-0.2 graphdriver=aufs version=1.8.2

DEBU[0006] Calling POST /build
INFO[0006] POST /v1.20/build?cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=gentoo-x32&ulimits=null
DEBU[0008] [BUILDER] Cache miss
DEBU[0009] Couldn't untar /home/lib-docker-v1.8.2-tmp/tmp/docker-build316710953/stage3-x32-20151004.tar.xz to /home/lib-docker-v1.8.2-tmp/aufs/mnt/d909abb87150463939c13e8a349b889a72d9b14f0cfcab42a8711979be285537: Untar re-exec error: exit status 1: output: operation not supported
DEBU[0009] CopyFileWithTar(/home/lib-docker-v1.8.2-tmp/tmp/docker-build316710953/stage3-x32-20151004.tar.xz, /home/lib-docker-v1.8.2-tmp/aufs/mnt/d909abb87150463939c13e8a349b889a72d9b14f0cfcab42a8711979be285537/)
panic: runtime error: slice bounds out of range

goroutine 42 [running]:
bufio.(*Reader).fill(0xc208187800)
    /usr/local/go/src/bufio/bufio.go:86 +0x2db
bufio.(*Reader).WriteTo(0xc208187800, 0x7ff39602d150, 0xc2083f11a0, 0x508000, 0x0, 0x0)
    /usr/local/go/src/bufio/bufio.go:449 +0x27e
io.Copy(0x7ff39602d150, 0xc2083f11a0, 0x7ff3960261f8, 0xc208187800, 0x0, 0x0, 0x0)
    /usr/local/go/src/io/io.go:354 +0xb2
github.com/docker/docker/pkg/archive.func·006()
    /go/src/github.com/docker/docker/pkg/archive/archive.go:817 +0x71
created by github.com/docker/docker/pkg/archive.CmdStream
    /go/src/github.com/docker/docker/pkg/archive/archive.go:819 +0x1ec

goroutine 1 [chan receive]:
main.(*DaemonCli).CmdDaemon(0xc20809da30, 0xc20800a020, 0xd, 0xd, 0x0, 0x0)
    /go/src/github.com/docker/docker/docker/daemon.go:289 +0x1781
reflect.callMethod(0xc208140090, 0xc20828fce0)
    /usr/local/go/src/reflect/value.go:605 +0x179
reflect.methodValueCall(0xc20800a020, 0xd, 0xd, 0x1, 0xc208140090, 0x0, 0x0, 0xc208140090, 0x0, 0x45343f, ...)
    /usr/local/go/src/reflect/asm_amd64.s:29 +0x36
github.com/docker/docker/cli.(*Cli).Run(0xc208129fb0, 0xc20800a010, 0xe, 0xe, 0x0, 0x0)
    /go/src/github.com/docker/docker/cli/cli.go:89 +0x38e
main.main()
    /go/src/github.com/docker/docker/docker/docker.go:69 +0x428

goroutine 5 [syscall]:
os/signal.loop()
    /usr/local/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
    /usr/local/go/src/os/signal/signal_unix.go:27 +0x35

Signed-off-by: Derek Ch <denc716@gmail.com>
2015-10-13 15:58:06 -07:00
Phil Estes
442b45628e Add user namespace (mapping) support to the Docker engine
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.

Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2015-10-09 17:47:37 -04:00
Darren Stahl
40b77af234 Fixed file modified time not changing on Windows
Signed-off-by: Darren Stahl <darst@microsoft.com>
2015-10-01 10:45:32 -07:00
Stefan J. Wernli
230cfc6ed2 Fixing Tar functions to support Windows long paths.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
2015-08-11 16:07:36 -07:00