Commit graph

2867 commits

Author SHA1 Message Date
Fabian Kramm
eea4977d02 Fix unmount redeclaration on darwin in github.com/docker/docker/pkg/mount
Signed-off-by: Fabian Kramm <kramm@covexo.com>
2018-12-14 11:10:52 +01:00
Kir Kolyshkin
6533136961 pkg/mount: wrap mount/umount errors
The errors returned from Mount and Unmount functions are raw
syscall.Errno errors (like EPERM or EINVAL), which provides
no context about what has happened and why.

Similar to os.PathError type, introduce mount.Error type
with some context. The error messages will now look like this:

> mount /tmp/mount-tests/source:/tmp/mount-tests/target, flags: 0x1001: operation not permitted

or

> mount tmpfs:/tmp/mount-test-source-516297835: operation not permitted

Before this patch, it was just

> operation not permitted

[v2: add Cause()]
[v3: rename MountError to Error, document Cause()]
[v4: fixes; audited all users]
[v5: make Error type private; changes after @cpuguy83 reviews]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-12-10 20:07:02 -08:00
Kir Kolyshkin
90be078fe5 pkg/mount: refactor Unmount()
It has been pointed out that we're ignoring EINVAL from umount(2)
everywhere, so let's move it to a lower-level function. Also, its
implementation should be the same for any UNIX incarnation, so
let's consolidate it.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-12-10 20:06:10 -08:00
Kir Kolyshkin
8072e62d83 pkg/system/stat_unix: wrap errors in PathError
syscall.Stat (and Lstat), unlike functions from os pkg,
return "raw" errors (like EPERM or EINVAL), and those are
propagated up the function call stack unchanged, and gets
logged and/or returned to the user as is.

Wrap those into os.PathError{} so the error message will
at least have function name and file name.

Note we use Capitalized function names to distinguish
between functions in os and ours.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-12-10 20:06:10 -08:00
Akihiro Suda
ec153ccfc8 pkg/archive: add TestReexecUserNSOverlayWhiteoutConverter
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-12-11 07:18:37 +09:00
Akihiro Suda
037e370de1 pkg/archive: annotate tests that requires initial userns
`rootlesskit go test ./pkg/archive` now succeeds

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-12-11 07:18:37 +09:00
Akihiro Suda
f8ed19c8b4 pkg/archive: support overlayfs in userns (Ubuntu kernel only)
Ubuntu kernel supports overlayfs in user namespaces.

However, Docker had previously crafting overlay opaques directly
using mknod(2) and setxattr(2), which are not supported in userns.

Tested with LXD, Ubuntu 18.04, kernel 4.15.0-36-generic #39-Ubuntu.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-12-11 07:18:37 +09:00
Tibor Vass
6e3113f700
Merge pull request #38327 from andrewhsu/ctrd
update containerd to v1.2.1
2018-12-10 17:28:50 +01:00
Andrew Hsu
78045a5419
use empty string as cgroup path to grab first find
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-07 18:44:00 +01:00
Akihiro Suda
9ddd6e47a9 pkg/archive: fix TestTarUntarWithXattr failure on recent kernel
Recent kernel has strict check for security.capability value.
Fix #38289

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-11-29 16:14:35 +09:00
Yong Tang
f023816608 Add memory.kernelTCP support for linux
This fix tries to address the issue raised in 37038 where
there were no memory.kernelTCP support for linux.

This fix add MemoryKernelTCP to HostConfig, and pass
the config to runtime-spec.

Additional test case has been added.

This fix fixes 37038.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-11-26 21:03:08 +00:00
Yong Tang
ea3ac621e3
Merge pull request #37982 from Microsoft/jjh/archive
pkg/archive fixes, and port most unit tests to Windows
2018-11-27 05:49:39 +09:00
John Howard
56b732058e pkg/archive fixes, and port most unit tests to Windows
Signed-off-by: John Howard <jhoward@microsoft.com>

If fixes an error in sameFsTime which was using `==` to compare two times. The correct way is to use go's built-in timea.Equals(timeb).

In changes_windows, it uses sameFsTime to compare mTim of a `system.StatT` to allow TestChangesDirsMutated to operate correctly now.

Note there is slight different between the Linux and Windows implementations of detecting changes. Due to https://github.com/moby/moby/issues/9874,
and the fix at https://github.com/moby/moby/pull/11422, Linux does not consider a change to the directory time as a change. Windows on NTFS
does. See https://github.com/moby/moby/pull/37982 for more information. The result in `TestChangesDirsMutated`, `dir3` is NOT considered a change
in Linux, but IS considered a change on Windows. The test mutates dir3 to have a mtime of +1 second.

With a handful of tests still outstanding, this change ports most of the unit tests under pkg/archive to Windows.

It provides an implementation of `copyDir` in tests for Windows. To make a copy similar to Linux's `cp -a` while preserving timestamps
and links to both valid and invalid targets, xcopy isn't sufficient. So I used robocopy, but had to circumvent certain exit codes that
robocopy exits with which are warnings. Link to article describing this is in the code.
2018-11-26 10:20:40 -08:00
Kir Kolyshkin
8abadb36fa pkg/mount: add MakeMount()
This function ensures the argument is the mount point
(i.e. if it's not, it bind mounts it to itself).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-10-11 23:30:52 -07:00
Kir Kolyshkin
f01297d1ae pkg/mount: simplify ensureMountedAs
1. There is no need to specify rw argument -- bind mounts are
   read-write by default.

2. There is no point in parsing /proc/self/mountinfo after performing
   a mount, especially if we don't check whether the fs is mounted or
   not -- the only outcome from it could be an error from our mountinfo
   parser, which makes no sense in this context.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-10-11 23:16:10 -07:00
Yong Tang
ffaa0b7b3f
Merge pull request #37936 from benny-gold/master
Added two new scientists to the namesgenerator
2018-10-08 23:40:51 +00:00
Yong Tang
2cc338c100
Merge pull request #37967 from thaJeztah/upstream_dos_fix
Fix denial of service with large numbers in cpuset-cpus and cpuset-mems
2018-10-08 13:23:03 +00:00
Vincent Demeester
a3f54d4570
Merge pull request #37977 from mooncak/remove_duplicated_word
Remove duplicated words in pkg files
2018-10-08 11:40:10 +02:00
Vincent Demeester
5f6aca784f
Merge pull request #37491 from poizan42/fix-mips-signals
SIGSKTFLT does not exist on MIPS, instead SIGEMT does. SIGRTMAX is also 127 on MIPS
2018-10-08 09:28:20 +02:00
mooncake
35739fd290 Remove duplicated words in pkg files
Signed-off-by: mooncake <xcoder@tenxcloud.com>
2018-10-05 22:32:14 +08:00
Justin Cormack
f8e876d761
Fix denial of service with large numbers in cpuset-cpus and cpuset-mems
Using a value such as `--cpuset-mems=1-9223372036854775807` would cause
`dockerd` to run out of memory allocating a map of the values in the
validation code. Set limits to the normal limit of the number of CPUs,
and improve the error handling.

Reported by Huawei PSIRT.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-05 15:09:02 +02:00
David P Hilton
272759f268 make xattr EPERM non-fatal in createTarFile
Signed-off-by: David P Hilton <david.hilton.p@gmail.com>
2018-10-04 00:00:23 -06:00
Ben Gould
733d777bc0 added an adjective to kick off another build!
Signed-off-by: Ben Gould <ben@bengould.co.uk>
2018-09-30 21:00:09 +01:00
Ben Gould
13c36d00fd Added two new scientists to the namesgenerator
Signed-off-by: Ben Gould <ben@bengould.co.uk>
2018-09-29 20:26:24 +01:00
Lifubang
00eb3480dc fixes display text in Multiple IDs found with provided prefix
Signed-off-by: Lifubang <lifubang@acmcoder.com>
2018-09-24 12:17:31 +08:00
Tibor Vass
11841b70d0
Merge pull request #37780 from tiborvass/fix-progress-panic
pkg/progress: work around closing closed channel panic
2018-09-13 14:45:15 -07:00
Oli
68906e6dcd Update LICENSE
Signed-off-by: Oliver Reason <oli@overrateddev.co>

Update LICENSE.APACHE

Signed-off-by: Oliver Reason <oli@overrateddev.co>

Update LICENSE.BSD

Signed-off-by: Oliver Reason <oli@overrateddev.co>
2018-09-12 14:27:53 +01:00
Kir Kolyshkin
9b0097a699 Format code with gofmt -s from go-1.11beta1
This should eliminate a bunch of new (go-1.11 related) validation
errors telling that the code is not formatted with `gofmt -s`.

No functional change, just whitespace (i.e.
`git show --ignore-space-change` shows nothing).

Patch generated with:

> git ls-files | grep -v ^vendor/ | grep .go$ | xargs gofmt -s -w

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-09-06 15:24:16 -07:00
Tibor Vass
7dac70324d pkg/progress: work around closing closed channel panic
I could not reproduce the panic in #37735, so here's a bandaid.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-09-06 20:20:12 +00:00
John Howard
b1b9937bc7 Windows: Go1.11: Use long path names in build context (TestBuildSymlinkBreakout)
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-05 17:01:05 -07:00
Anusha Ragunathan
c129828151
Merge pull request #37666 from Yadnyawalkya/docker_names
Added two new adjectives and four new names to names-generator.go
2018-09-04 12:15:24 -07:00
Kir Kolyshkin
fffa8958d0 pkg/filenotify/poller: fix Close()
The code in Close() that removes the watches was not working,
because it first sets `w.closed = true` and then calls w.close(),
which starts with
```
        if w.closed {
                return errPollerClosed
	}
```

Fix by setting w.closed only after calling w.remove() for all the
files being watched.

While at it, remove the duplicated `delete(w.watches, name)` code.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-08-29 22:16:04 -07:00
Kir Kolyshkin
dfbb64ea7d pkg/filenotify/poller: close file asap
There is no need to wait for up to 200ms in order to close
the file descriptor once the chClose is received.

This commit might reduce the chances for occasional "The process
cannot access the file because it is being used by another process"
error on Windows, where an opened file can't be removed.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-08-29 13:51:09 -07:00
Kir Kolyshkin
88bcf1573c pkg/filenotify: poller.Add: fix fd leaks on err
In case of errors, the file descriptor is never closed. Fix it.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-08-29 13:46:46 -07:00
Yadnyawalkya Tale
3cd0bfc68e
Add adjectives, male and female populer personalities
* Add cool, crazy, charming, magical and sweet as a adjectives (Aug 18)
* Add four male scientists to the list - faraday, maxwell, sutherland, and moore (Aug 21)
* Add four female scientists to the list - cannon, moser and rhodes (Aug 28)

Signed-off-by: Yadnyawalkya Tale <yadnyawalkyatale@gmail.com>
2018-08-28 23:58:10 +05:30
Sebastiaan van Stijn
e0ad6d045c
Merge pull request #37092 from cpuguy83/local_logger
Add "local" log driver
2018-08-20 07:01:41 +01:00
John Stephens
b3e9f7b13b
Merge pull request #35521 from salah-khan/35507
Add --chown flag support for ADD/COPY commands for Windows
2018-08-17 11:31:16 -07:00
Yong Tang
74c43af1a3
Merge pull request #37600 from edrevo/conemu-fix
Fix #28814: use emulation for ConEmu and ConsoleZ
2018-08-15 16:50:05 -07:00
Salahuddin Khan
763d839261 Add ADD/COPY --chown flag support to Windows
This implements chown support on Windows. Built-in accounts as well
as accounts included in the SAM database of the container are supported.

NOTE: IDPair is now named Identity and IDMappings is now named
IdentityMapping.

The following are valid examples:
ADD --chown=Guest . <some directory>
COPY --chown=Administrator . <some directory>
COPY --chown=Guests . <some directory>
COPY --chown=ContainerUser . <some directory>

On Windows an owner is only granted the permission to read the security
descriptor and read/write the discretionary access control list. This
fix also grants read/write and execute permissions to the owner.

Signed-off-by: Salahuddin Khan <salah@docker.com>
2018-08-13 21:59:11 -07:00
Yong Tang
1fd7e4c28d
Merge pull request #37443 from Projjol/new_names
Add in two new names for names-generator
2018-08-12 19:04:56 -07:00
Brian Goff
874867d4e5 Add func to get an io.Reader for tail operations
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-08-10 21:01:15 -07:00
Ximo Guanter Gonzálbez
ea65a0b134 Fix #28814
Signed-off-by: Ximo Guanter Gonzálbez <joaquin.guantergonzalbez@telefonica.com>
2018-08-07 10:20:13 +02:00
Sebastiaan van Stijn
09f5e9dae3
Merge pull request #37465 from Russtopia/master
namesgenerator: new adjective, two inspiring Computing Science authors
2018-08-01 23:50:05 +02:00
Russ Magee
72a5e621e5 Added additional scientists for balance per names-generator.go policy
Signed-off-by: Russ Magee <rmagee@gmail.com>
2018-07-31 11:34:56 -07:00
Iskander Sharipov
cda9d5f7f0 pkg/system: return nil explicitly
Makes code less confusing.
Otherwise it looks like an error (typo of "==" instead "!=").

Signed-off-by: Iskander Sharipov <quasilyte@gmail.com>
2018-07-29 01:31:08 +03:00
Kasper Fabæch Brandt
12804540f3 SIGSKTFLT does not exist on MIPS, instead SIGEMT does. SIGRTMAX is also 127 on MIPS.
Signed-off-by: Kasper Fabæch Brandt <poizan@poizan.dk>
2018-07-19 12:26:52 +02:00
Kir Kolyshkin
05cbe23db9 pkg/tarsum: fix unit test for Go 1.11+
Since go-1.11beta1 archive/tar, tar headers with Typeflag == TypeRegA
(numeric 0) (which is the default unless explicitly initialized) are
modified to have Typeflag set to either tar.TypeReg (character value
'0', not numeric 0) or tar.TypeDir (character value '5') [1].
This results in different Typeflag value in the resulting header,
leading to a different Checksum, and causing the following test
case errors:

> 12:09:14 --- FAIL: TestTarSums (0.05s)
> 12:09:14 tarsum_test.go:393: expecting
> [tarsum+sha256:8bf12d7e67c51ee2e8306cba569398b1b9f419969521a12ffb9d8875e8836738],
> but got
> [tarsum+sha256:75258b2c5dcd9adfe24ce71eeca5fc5019c7e669912f15703ede92b1a60cb11f]
> ... (etc.)

All the other code explicitly sets the Typeflag field, but this test
case is not, causing the incompatibility with Go 1.11. Therefore,
the fix is to set TypeReg explicitly, and change the expected checksums
in test cases).

Alternatively, we can vendor archive/tar again (for the 100th time),
but given that the issue is limited to the particular test case it
does not make sense.

This fixes the test for all Go versions.

[1] https://go-review.googlesource.com/c/go/+/85656

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-07-19 10:38:45 +03:00
Russ Magee
c1ab5c98a2 namesgenerator: new adjective and two inspiring Computing Science authors
Signed-off-by: Russ Magee <rmagee@gmail.com>
2018-07-15 00:59:27 -07:00
Projjol Banerji
ad5d34ad0e Add in two new names for names-generator
Added in : Blase Pascal and Kathleen Antonelli

Signed-off-by: Projjol Banerji <probaner23@gmail.com>
2018-07-13 00:41:17 +05:30
Sebastiaan van Stijn
656036ce2c
Merge pull request #37157 from debayande/even-more-names
Adds a few more names to the name generator.
2018-07-05 21:06:12 +02:00
Tibor Vass
ca8022ec63 builder: return image ID in API when using buildkit
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-07-03 19:11:02 +00:00
Debayan De
9d61eee75a Adds a few more names to the name generator.
Signed-off-by: Debayan De <debayande@users.noreply.github.com>
2018-07-03 02:12:46 +01:00
Sebastiaan van Stijn
1d7132e3d2
Merge pull request #37356 from Microsoft/jjh/new37316
LCOW: lazycontext: Use correct lstat, fix archive check
2018-06-28 12:32:31 +02:00
Tonis Tiigi
f099771665 system: add back lcow validation function
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-27 15:24:26 -07:00
John Howard
a5aed699cf LCOW: lazycontext: Use correct lstat, fix archive check
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-06-27 13:29:21 -07:00
Tonis Tiigi
48b93419dc builder: buildkit rebase update
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-26 22:59:17 -07:00
Dennis Chen
7f334d3acf Initial support for OCI multi-platform image
Add the OCI spec compatible image support in client side.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-06-26 11:39:29 -07:00
John Howard
35193c0e7d LCOW: Auto-select OS
Signed-off-by: John Howard <jhoward@microsoft.com>

Addresses https://github.com/moby/moby/pull/35089#issuecomment-367802698.
This change enables the daemon to automatically select an image under LCOW
that can be used if the API doesn't specify an explicit platform.

For example:

FROM supertest2014/nyan
ADD Dockerfile /

And docker build . will download the linux image (not a multi-manifest image)

And similarly docker pull ubuntu will match linux/amd64
2018-06-26 11:37:43 -07:00
Sebastiaan van Stijn
cb37987ee1
Merge pull request #37301 from Mischi/fix-define-self
fix build on OpenBSD by defining Self()
2018-06-25 20:25:38 +02:00
Yong Tang
0ed55cd220 Fix a small spacing issue
As a follow up to https://github.com/moby/moby/pull/37331#pullrequestreview-131411625

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-06-24 17:27:50 +00:00
Martin Muzatko
7015fc03b7 Just satisfying my OCD - fixed comment spacing and removed a hidden character
Signed-off-by: Martin Muzatko <martin@happy-css.com>
2018-06-22 17:17:08 +02:00
Derek McGowan
ac5d363e67
Update overlay2 to use naive diff for changes
The archive changes function is not implemented correctly
to handle opaque directories.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-06-20 11:07:36 -07:00
Fabian Raetz
21537b818d fix build on OpenBSD by defining Self()
Signed-off-by: Fabian Raetz <fabian.raetz@gmail.com>
2018-06-16 19:09:04 +02:00
Vincent Demeester
3845728524
Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-13 09:04:30 +02:00
Joe Ferguson
76e99e1a8b Close readclosers returned by DecompressStream
Signed-off-by: Joe Ferguson <joe@infosiftr.com>
2018-06-07 11:16:13 -07:00
Sebastiaan van Stijn
5037c5a8ce
Merge pull request #36688 from cpuguy83/volumes_service
Extract volume interaction to a volumes service
2018-06-05 02:16:20 +02:00
Anusha Ragunathan
71cd53e4a1
Merge pull request #37177 from cpuguy83/minor_plugin_addr_change
Move plugin client to separate interface
2018-05-31 08:22:04 -07:00
Vincent Demeester
2ea4ece674
Merge pull request #37174 from thaJeztah/no_additional_magic_matching
Be explicit about github.com prefix being a legacy feature
2018-05-31 09:45:59 +02:00
Brian Goff
7c77df8acc Move plugin client to separate interface
This makes it a bit simpler to remove this interface for v2 plugins
and not break external projects (libnetwork and swarmkit).

Note that before we remove the `Client()` interface from `CompatPlugin`
libnetwork and swarmkit must be updated to explicitly check for the v1
client interface as is done int his PR.

This is just a minor tweak that I realized is needed after trying to
implement the needed changes on libnetwork.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-30 15:22:10 -04:00
Sebastiaan van Stijn
babb0c14fd
Be explicit about github.com prefix being a legacy feature
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-30 11:09:14 +02:00
Brian Goff
f51a96c016 Move plugin client creation to the extension point
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 15:18:53 -04:00
Brian Goff
e4b6adc88e Extract volume interaction to a volumes service
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 14:21:07 -04:00
Sebastiaan van Stijn
f23c00d870
Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-23 17:50:54 +02:00
Kir Kolyshkin
d78e885326 pkg/mount/TestMount: fix wrt selinux
Sometimes docker-master CI fails on rhel4+selinux configuration,
like this:

--- FAIL: TestMount (0.12s)
    --- FAIL: TestMount/none-remount,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "rw,size=128k"
    --- FAIL: TestMount/none-remount,ro,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "ro,size=128k"

Earlier, commit 8bebd42df2 (PR #34965) fixed this failure,
but not entirely (i.e. the test is now flaky). It looks like
either selinux detection code is not always working (it won't
work in d-in-d), or the kernel might or might not add 'seclabel'
option).

As the subject of this test case is definitely not selinux,
it can just ignore the option added by it.

While at it, fix error messages:
 - add missing commas;
 - fix a typo;
 - allow for clear distinction between mount
   and vfs (per-superblock) options.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-22 23:30:47 -07:00
Sebastiaan van Stijn
3646562a5c
Merge pull request #37088 from ohbarye/fix-typos-duplicated-the
Fix typos: remove duplicated "the"
2018-05-19 20:57:47 +02:00
Sebastiaan van Stijn
4f8c870d62
Fix some linting issues
These showed locally when running `make validate`. CI doesn't seem to have the
same (possibly it's disabled in the configuration)

    builder/fscache/fscache.go:618::error: github.com/docker/docker/vendor/github.com/tonistiigi/fsutil.StatInfo composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:44::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:20::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:113::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:110::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:171::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:413::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:203::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:584::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:109::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:388::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/volumes_windows.go:27::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    integration/service/network_test.go:31::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    api/server/server.go:129:10⚠️ should not use basic type string as key in context.WithValue (golint)
    integration/service/network_test.go:54::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:61::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:74::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    pkg/archive/archive_windows.go:76::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    plugin/manager_linux.go:56::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-17 19:28:27 +02:00
Vincent Demeester
6a9bcc51df
Merge pull request #37074 from skyc024/fix-typos
Fix typos
2018-05-17 16:00:26 +02:00
ohbarye
0f95b23d98 Fix typos: remove duplicated "the"
Signed-off-by: Masato Ohba <over.rye@gmail.com>
2018-05-17 21:49:51 +09:00
chenyuzhu
5ac0d2f0b8 Fix typos
Signed-off-by: chenyuzhu <chenyuzhi@oschina.cn>
2018-05-16 15:31:12 +08:00
Tonis Tiigi
f784907eae jsonmessage: pass message to aux callback
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 15:48:25 -07:00
Sebastiaan van Stijn
d4e48af48d
Merge pull request #36918 from Sh4d1/master
add verification to check if escapeKeys have elements (pkg/term/proxy.go)
2018-04-30 11:57:21 +02:00
Patrik Cyvoct
d339130f30
Add verification and test to check if escapeKeys is not empty
In pkg/term/proxy.go and pkg/term/proxy_test.go, check if escapeKeys is empty and if it is, return the one key read

Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
2018-04-30 00:32:43 +02:00
Kir Kolyshkin
7d62e40f7e Switch from x/net/context -> context
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".

Many vendored packages still use x/net/context, so vendor entry remains
for now.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-23 13:52:44 -07:00
Yong Tang
5c233cf431
Merge pull request #36913 from vdemeester/test-skip-non-root
Skip some tests requires root uid when run as user…
2018-04-23 11:42:42 -07:00
Sebastiaan van Stijn
20b524bf2e
Merge pull request #36920 from kolyshkin/cancel-func
context.WithTimeout: do call the cancel func
2018-04-23 20:14:25 +02:00
Vincent Demeester
a7999aaa53
Skip some tests requires root uid when run as user
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-04-23 10:14:39 +02:00
Vincent Demeester
5d8b88b114
Fix typo in idtools tests
It should check `os.Geteuid` with `uid` instead of `os.Getegid`.
On the container (where the tests run), the uid and gid seems to be
the same, thus this doesn't fail.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-04-23 10:14:07 +02:00
Vincent Demeester
53982e3fc1
Merge pull request #36091 from kolyshkin/mount
pkg/mount improvements
2018-04-21 11:03:54 +02:00
Kir Kolyshkin
05e2f7e2fa context.WithTimeout: do call the cancel func
govet complains (when using standard "context" package):

> the cancel function returned by context.WithTimeout should be called,
> not discarded, to avoid a context leak (vet)

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-20 12:27:13 -07:00
Kir Kolyshkin
a1d095199d mount.Unmount(): don't look into /proc/self/mountinfo
Now, every Unmount() call takes a burden to parse the whole nine yards
of /proc/self/mountinfo to figure out whether the given mount point is
mounted or not (and returns an error in case parsing fails somehow).

Instead, let's just call umount() and ignore EINVAL, which results
in the same behavior, but much better performance.

Note that EINVAL is returned from umount(2) not only in the case when
`target` is not mounted, but also for invalid flags. As the flags are
hardcoded here, it can't be the case.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:50 -07:00
Kir Kolyshkin
c611f18a7f pkg/mount/mountinfo_linux: parser speed up
The mountinfo parser implemented via `fmt.Sscanf()` is slower than the one
using `strings.Split()` and `strconv.Atoi()`. This rewrite helps to speed it
up to a factor of 8x, here is a result from go bench:

> BenchmarkParsingScanf-4      	     300	  22294112 ns/op
> BenchmarkParsingSplit-4      	    3000	   2780703 ns/op

I tried other approaches, such as using `fmt.Sscanf()` for the first
three (integer) fields and `strings.Split()` for the rest, but it slows
things down considerably:

> BenchmarkParsingMixed-4      	    1000	   8827058 ns/op

Note the old code uses `fmt.Sscanf`, when a linear search for '-' field,
when a split for the last 3 fields. The new code relies on a single
split.

I have also added more comments to aid in future development.

Finally, the test data is fixed to now have white space before the first field.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:42 -07:00
Kir Kolyshkin
871c957242 getSourceMount(): simplify
The flow of getSourceMount was:
 1 get all entries from /proc/self/mountinfo
 2 do a linear search for the `source` directory
 3 if found, return its data
 4 get the parent directory of `source`, goto 2

The repeated linear search through the whole mountinfo (which can have
thousands of records) is inefficient. Instead, let's just

 1 collect all the relevant records (only those mount points
   that can be a parent of `source`)
 2 find the record with the longest mountpath, return its data

This was tested manually with something like

```go
func TestGetSourceMount(t *testing.T) {
	mnt, flags, err := getSourceMount("/sys/devices/msr/")
	assert.NoError(t, err)
	t.Logf("mnt: %v, flags: %v", mnt, flags)
}
```

...but it relies on having a specific mount points on the system
being used for testing.

[v2: add unit tests for ParentsFilter]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:17 -07:00
Kir Kolyshkin
bb934c6aca pkg/mount: implement/use filter for mountinfo parsing
Functions `GetMounts()` and `parseMountTable()` return all the entries
as read and parsed from /proc/self/mountinfo. In many cases the caller
is only interested only one or a few entries, not all of them.

One good example is `Mounted()` function, which looks for a specific
entry only. Another example is `RecursiveUnmount()` which is only
interested in mount under a specific path.

This commit adds `filter` argument to `GetMounts()` to implement
two things:
 1. filter out entries a caller is not interested in
 2. stop processing if a caller is found what it wanted

`nil` can be passed to get a backward-compatible behavior, i.e. return
all the entries.

A few filters are implemented:
 - `PrefixFilter`: filters out all entries not under `prefix`
 - `SingleEntryFilter`: looks for a specific entry

Finally, `Mounted()` is modified to use `SingleEntryFilter()`, and
`RecursiveUnmount()` is using `PrefixFilter()`.

Unit tests are added to check filters are working.

[v2: ditch NoFilter, use nil]
[v3: ditch GetMountsFiltered()]
[v4: add unit test for filters]
[v5: switch to gotestyourself]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:48:09 -07:00
Alexandre Jomin
0d82bc142c pkg/term add build tag for netbsd
Signed-off-by: Alexandre Jomin <alexandrejomin@gmail.com>
2018-04-18 15:16:06 +02:00
Tibor Vass
61138fb5fc
Merge pull request #35739 from thaJeztah/bump-go-1.10
Bump Golang to 1.10.1
2018-04-13 13:56:38 -10:00
Anusha Ragunathan
ff6a103067
Merge pull request #36778 from odg0318/master
Using authorization plugin, I changed Content-Type check routine.
2018-04-12 17:39:38 -07:00
Derek McGowan
fb170206ba
Set format in archiver
Prevent changing the tar output by setting the format to
PAX and keeping the times truncated.
Without this change the archiver will produce different tar
archives with different hashes with go 1.10.
The addition of the access and changetime timestamps would
also cause diff comparisons to fail.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-04-12 13:34:33 -07:00
Derek McGowan
a422774e59
Fix tarsum for go 1.10
Remove invalid flush commands, flush should only occur when file
has been completely written. This is already handle, remove these calls.
Ensure data gets written after EOF in correct order and before close.
Remove gname and uname from sum for hash compatibility.
Update tarsum tests for gname/uname removal.
Return valid length after eof.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-04-12 13:34:14 -07:00
Brian Goff
74f8e47352 Limit authz response buffer
When the authz response buffer limit is hit, perform a flush.
This prevents excessive buffer sizes, especially on large responses
(e.g. `/containers/<id>/archive` or `/containers/<id>/export`).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-04-11 15:36:36 -04:00
Vincent Demeester
5219725890
Merge pull request #35518 from cyphar/libdm-dlsym-deferred_remove
pkg: devmapper: dynamically load dm_task_deferred_remove
2018-04-11 14:11:16 +02:00
odg0318
6ac73d34e4 If Content-Type is application/json;charset=UTF-8, RequestBody is empty.
Signed-off-by: odg0318 <odg0318@gmail.com>
2018-04-05 04:03:45 -04:00
Manuel Rüger
108bbd96cb Fix tests for pkg/archive
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2018-04-03 01:17:52 +02:00
Brian Goff
9d46c4c138 Support cancellation in directory.Size()
Makes sure that if the user cancels a request that the daemon stops
trying to traverse a directory.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-03-29 15:49:15 -04:00
Vincent Demeester
7270b01e5d
Merge pull request #36617 from Microsoft/jjh/kernel-registry
Windows: Move kernel_windows to use golang registry functions
2018-03-19 11:47:17 +01:00
John Howard
29f93c4bc7 Windows: Move kernel_windows to use golang registry functions
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-16 09:47:45 -07:00
Daniel Nephin
7d8815ea70 Cleanup pkg/jsonmessage progress tests
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:47 -04:00
Daniel Nephin
c9e52bd0da Post migration assertion fixes
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:46 -04:00
Daniel Nephin
6be0f70983 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:43 -04:00
Daniel Nephin
ef01dea893 Cleanup some assertions
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 10:54:54 -04:00
Vincent Demeester
a21d5bf669
Merge pull request #36506 from kolyshkin/pkg-mount-slice
pkg/mount: use sort.Slice
2018-03-09 09:46:53 +01:00
Akihiro Suda
4a1d35c546
Merge pull request #36505 from kolyshkin/pkg-mount-tests
pkg/mount unit tests: skip some test under non-root
2018-03-08 17:27:31 +09:00
John Stephens
8cf8fe9cf8
Merge pull request #36451 from Microsoft/jjh/ubr
Windows: Report Version and UBR
2018-03-07 06:27:01 -08:00
Kir Kolyshkin
a00310b54c pkg/mount: use sort.Slice
Sorting by mount point length can be implemented in a more
straightforward fashion since Go 1.8 introduced sort.Slice()
with an ability to provide a less() function in place.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-06 12:46:58 -08:00
Kir Kolyshkin
4aae77602a pkg/mount unit tests: skip some test under non-root
This makes `go test .` to pass if run as non-root user, skipping
those tests that require superuser privileges (for `mount`).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-06 12:37:27 -08:00
Sebastiaan van Stijn
390b74cb63
Merge pull request #36493 from cpuguy83/add_plugin_timeout_test
Adds a unit test for plugin request timeout
2018-03-06 12:45:17 +01:00
John Stephens
3e1505e3e6
Merge pull request #36327 from Microsoft/jjh/block-pulling-uplevel
Windows: Block pulling uplevel images
2018-03-05 15:12:52 -08:00
Brian Goff
7ca971fb49 Adds a unit test for plugin request timeout
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-03-05 15:38:56 -05:00
John Howard
6de9f90417 Windows: Report Version and UBR
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-05 08:20:45 -08:00
John Howard
83908836d3 Windows: Block pulling uplevel images
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-02-26 12:33:54 -08:00
Sebastiaan van Stijn
20028325da
Merge pull request #35829 from cpuguy83/no_private_mount_for_plugins
Perform plugin mounts in the runtime
2018-02-21 12:28:13 +01:00
Tonis Tiigi
f6c8266afd builder: fix wrong cache hits building from tars
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-15 23:42:42 -08:00
Aleksa Sarai
98fe4bd8f1
pkg: devmapper: dynamically load dm_task_deferred_remove
dm_task_deferred_remove is not supported by all distributions, due to
out-dated versions of devicemapper. However, in the case where the
devicemapper library was updated without rebuilding Docker (which can
happen in some distributions) then we should attempt to dynamically load
the relevant object rather than try to link to it.

This can only be done if Docker was built dynamically, for obvious
reasons.

In order to avoid having issues arise when dlsym(3) was unnecessary,
gate the whole dlsym(3) logic behind a buildflag that we disable by
default (libdm_dlsym_deferred_remove).

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2018-02-16 17:23:23 +11:00
bin liu
7a7a8a33a4 Fix typos in pkg
Signed-off-by: bin liu <liubin0329@gmail.com>
2018-02-10 19:43:13 +08:00
Brian Goff
0e5eaf8ee3 Ensure plugin returns correctly scoped paths
Before this change, volume management was relying on the fact that
everything the plugin mounts is visible on the host within the plugin's
rootfs. In practice this caused some issues with mount leaks, so we
changed the behavior such that mounts are not visible on the plugin's
rootfs, but available outside of it, which breaks volume management.

To fix the issue, allow the plugin to scope the path correctly rather
than assuming that everything is visible in `p.Rootfs`.
In practice this is just scoping the `PropagatedMount` paths to the
correct host path.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-07 15:48:27 -05:00
Daniel Nephin
4f0d95fa6e Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
Anusha Ragunathan
6e5c2d639f
Merge pull request #36119 from cpuguy83/fix_plugin_scanner
Fix issue with plugin scanner going to deep
2018-02-02 20:19:01 -08:00
Brian Goff
b27f70d45a Fix issue with plugin scanner going to deep
The plugin spec says that plugins can live in one of:

- /var/run/docker/plugins/<name>.sock
- /var/run/docker/plugins/<name>/<name>.sock
- /etc/docker/plugins/<name>.[json,spec]
- /etc/docker/plugins/<name>/<name>.<json,spec>
- /usr/lib/docker/plugins/<name>.<json,spec>
- /usr/lib/docker/plugins/<name>/<name>.<json,spec>

However, the plugin scanner which is used by the volume list API was
doing `filepath.Walk`, which will walk the entire tree for each of the
supported paths.
This means that even v2 plugins in
`/var/run/docker/plugins/<id>/<name>.sock` were being detected as a v1
plugin.
When the v1 plugin loader tried to load such a plugin it would log an
error that it couldn't find it because it doesn't match one of the
supported patterns... e.g. when in a subdir, the subdir name must match
the plugin name for the socket.

There is no behavior change as the error is only on the `Scan()` call,
which is passing names to the plugin registry when someone calls the
volume list API.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-02 16:49:14 -05:00
Sebastiaan van Stijn
c41c80026b
Merge pull request #35911 from ASMfreaK/Russian-Scientsists-addition-to-names-generator
Add more russian scientists in names-generator.go
2018-01-26 15:48:58 -08:00
Vincent Demeester
59e86068e6
Merge pull request #34379 from cpuguy83/mount_optimizations
Optimizations for recursive unmount
2018-01-24 14:00:58 -08:00
Sebastiaan van Stijn
db5c006bc8
Merge pull request #35919 from yongtang/35333-carry
Carry #35333: Devicemapper: ignore Nodata errors when delete thin device
2018-01-20 18:47:16 +01:00
John Howard
5b24976ad4 Bump RS3 final build, and remove LCOW_SUPPORTED
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-19 12:22:56 -08:00
John Howard
0cba7740d4 Address feedback from Tonis
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-18 12:30:39 -08:00
John Howard
afd305c4b5 LCOW: Refactor to multiple layer-stores based on feedback
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-18 08:31:05 -08:00
John Howard
ce8e529e18 LCOW: Re-coalesce stores
Signed-off-by: John Howard <jhoward@microsoft.com>

The re-coalesces the daemon stores which were split as part of the
original LCOW implementation.

This is part of the work discussed in https://github.com/moby/moby/issues/34617,
in particular see the document linked to in that issue.
2018-01-18 08:29:19 -08:00
unclejack
871afbb304
Merge pull request #35697 from sargun/use-pgzip
Make image (layer) downloads faster by using pigz
2018-01-17 11:18:20 -08:00
Sebastiaan van Stijn
f0b0f2038d
Merge pull request #35441 from cpuguy83/plugin_timeout
Add timeouts for volume plugin ops
2018-01-17 14:49:41 +01:00
Brian Goff
b15f8d2d4f Add timeouts for volume plugin ops
This protects the daemon from volume plugins that are slow or
deadlocked.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-16 20:30:49 -05:00
Pavel Pletenev
e0c3fe7836
Add more russian scientists in names-generator.go
Signed-off-by: ASM <cpp.create@gmail.com>
2018-01-16 21:09:17 +02:00
Sargun Dhillon
fd35494a25 Make image (layer) downloads faster by using pigz
The Golang built-in gzip library is serialized, and fairly slow
at decompressing. It also only decompresses on demand, versus
pipelining decompression.

This change switches to using the pigz external command
for gzip decompression, as opposed to using the built-in
golang one. This code is not vendored, but will be used
if it autodetected as part of the OS.

This also switches to using context, versus a manually
managed channel to manage cancellations, and synchronization.
There is a little bit of weirdness around manually having
to cancel in the error cases.

Signed-off-by: Sargun Dhillon <sargun@sargun.me>
2018-01-16 10:49:18 -08:00
Sebastiaan van Stijn
b4a6313969
Golint: remove redundant ifs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 00:42:25 +01:00
Brian Goff
dd21087660 Optimizations for recurrsive unmount
When a recursive unmount fails, don't bother parsing the mount table to check
if what we expected to be a mountpoint is still mounted. `EINVAL` is
returned when you try to unmount something that is not a mountpoint, the
other cases of `EINVAL` would not apply here unless everything is just
wrong. Parsing the mount table over and over is relatively expensive,
especially in the code path that it's in.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-11 16:13:16 -05:00
Yong Tang
99c22b0034
Merge pull request #35822 from keyolk/35709-fix-docker-build-overlayfs-whiteout
skip container ID remapping, if the file is overlayfs whiteout.
2018-01-03 22:21:00 +08:00
Chanhun Jeong
b013c1541d skip container ID remapping, if the file is overlayfs whiteout.
Signed-off-by: Chanhun Jeong <chanhun.jeong@navercorp.com>
2018-01-03 17:06:59 +09:00
Liu Hua
8451d03d8e Devicemapper: ignore Nodata errors when delete thin device
if thin device is deteled and the metadata exists, you can not
delete related containers. This patch ignore Nodata errors for
thin device deletion

Signed-off-by: Liu Hua <sdu.liu@huawei.com>
2018-01-02 18:04:25 +00:00
Sebastiaan van Stijn
6ed1163c98
Remove redundant build-tags
Files that are suffixed with `_linux.go` or `_windows.go` are
already only built on Linux / Windows, so these build-tags
were redundant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-12-18 17:41:53 +01:00
John Howard
7caa30e893 Windows: Case-insensitive filename matching against builder cache
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-12-14 13:49:40 -08:00
Tõnis Tiigi
cefb33700c
Merge pull request #35589 from keloyang/close-fd
Close pipe in chrootarchive.invokeUnpack when cmd.Start()/json.NewEncoder failed
2017-12-06 17:48:02 -08:00
Michael Crosby
72e45fd54e
Merge pull request #35618 from kolyshkin/mkdir-all
Fix MkdirAll* and its usage
2017-11-30 11:19:29 -05:00
Kir Kolyshkin
516010e92d Simplify/fix MkdirAll usage
This subtle bug keeps lurking in because error checking for `Mkdir()`
and `MkdirAll()` is slightly different wrt to `EEXIST`/`IsExist`:

 - for `Mkdir()`, `IsExist` error should (usually) be ignored
   (unless you want to make sure directory was not there before)
   as it means "the destination directory was already there"

 - for `MkdirAll()`, `IsExist` error should NEVER be ignored.

Mostly, this commit just removes ignoring the IsExist error, as it
should not be ignored.

Also, there are a couple of cases then IsExist is handled as
"directory already exist" which is wrong. As a result, some code
that never worked as intended is now removed.

NOTE that `idtools.MkdirAndChown()` behaves like `os.MkdirAll()`
rather than `os.Mkdir()` -- so its description is amended accordingly,
and its usage is handled as such (i.e. IsExist error is not ignored).

For more details, a quote from my runc commit 6f82d4b (July 2015):

    TL;DR: check for IsExist(err) after a failed MkdirAll() is both
    redundant and wrong -- so two reasons to remove it.

    Quoting MkdirAll documentation:

    > MkdirAll creates a directory named path, along with any necessary
    > parents, and returns nil, or else returns an error. If path
    > is already a directory, MkdirAll does nothing and returns nil.

    This means two things:

    1. If a directory to be created already exists, no error is
    returned.

    2. If the error returned is IsExist (EEXIST), it means there exists
    a non-directory with the same name as MkdirAll need to use for
    directory. Example: we want to MkdirAll("a/b"), but file "a"
    (or "a/b") already exists, so MkdirAll fails.

    The above is a theory, based on quoted documentation and my UNIX
    knowledge.

    3. In practice, though, current MkdirAll implementation [1] returns
    ENOTDIR in most of cases described in #2, with the exception when
    there is a race between MkdirAll and someone else creating the
    last component of MkdirAll argument as a file. In this very case
    MkdirAll() will indeed return EEXIST.

    Because of #1, IsExist check after MkdirAll is not needed.

    Because of #2 and #3, ignoring IsExist error is just plain wrong,
    as directory we require is not created. It's cleaner to report
    the error now.

    Note this error is all over the tree, I guess due to copy-paste,
    or trying to follow the same usage pattern as for Mkdir(),
    or some not quite correct examples on the Internet.

    [1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-11-27 17:32:12 -08:00
Kir Kolyshkin
2aa13f86f0 idtools.MkdirAs*: error out if dir exists as file
Standard golang's `os.MkdirAll()` function returns "not a directory" error
in case a directory to be created already exists but is not a directory
(e.g. a file). Our own `idtools.MkdirAs*()` functions do not replicate
the behavior.

This is a bug since all `Mkdir()`-like functions are expected to ensure
the required directory exists and is indeed a directory, and return an
error otherwise.

As the code is using our in-house `system.Stat()` call returning a type
which is incompatible with that of golang's `os.Stat()`, I had to amend
the `system` package with `IsDir()`.

A test case is also provided.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-11-27 13:25:44 -08:00
Chetan Birajdar
7c50d39480 added more awesome and notable women to names-generator.go
Signed-off-by: Chetan Birajdar <birajdar.chetan@gmail.com>
2017-11-26 21:40:02 -08:00
y00316549
f5f8f00860 Close pipe in chrootarchive.invokeUnpack when cmd.Start()/json.NewEncoder failed.
Signed-off-by: y00316549 <yangshukui@huawei.com>
2017-11-24 13:41:08 +08:00
Akihiro Suda
84b1f813a4
Merge pull request #35541 from thaJeztah/remove-deprecated-mkdirallas
Remove uses of deprecated MkdirAllAs(), MkdirAs()
2017-11-22 17:49:37 +09:00
Sebastiaan van Stijn
47af9f625d
Remove unused "testutil" package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 15:09:23 +01:00
Sebastiaan van Stijn
38b3af567f
Remove deprecated MkdirAllAs(), MkdirAs()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 13:53:54 +01:00
Sebastiaan van Stijn
ce66470f54
Minor refactor in idtools
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 13:49:58 +01:00
Sebastiaan van Stijn
9aba019b72
Update idtools tests to test non-deprecated functions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 13:47:17 +01:00
Chao Wang
97e406678c Split and remove pkg/stringutils
Signed-off-by: Chao Wang <wangchao.fnst@cn.fujitsu.com>
2017-11-13 10:01:11 +08:00
Chao Wang
5c154cfac8 Copy Inslice() to those parts that use it
Signed-off-by: Chao Wang <wangchao.fnst@cn.fujitsu.com>
2017-11-10 13:42:38 +08:00
Yong Tang
a4bdb304e2
Merge pull request #35250 from joppich/patch-1
Update names-generator.go
2017-11-09 06:21:02 -08:00
Victor Vieux
5745a8531e
Merge pull request #35265 from cpuguy83/32609_defreference_voldriver_on_error
Fixup some issues with plugin refcounting
2017-11-07 09:47:07 -08:00
Yong Tang
4785f1a7ab Remove solaris build tag and `contrib/mkimage/solaris
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2017-11-02 00:01:46 +00:00
Yong Tang
0bcca5ea86
Merge pull request #35341 from tklauser/utsname-x-sys
Simplify Utsname string conversion
2017-11-01 11:36:38 -07:00
joppich
f69f8f3679 Update names-generator.go to include Cédric Villani and Valentina
Tershkova

Signed-off-by: Jorit Kleine-Möllhoff <joppich@bricknet.de>
2017-10-31 15:38:57 +01:00
Tobias Klauser
6d068bc25b Simplify Utsname string conversion
Update golang.org/x/sys to 95c6576299259db960f6c5b9b69ea52422860fce in
order to get the unix.Utsname with byte array instead of int8/uint8
members.

This allows to use simple byte slice to string conversions instead of
using charsToString or its open-coded version.

Also see golang/go#20753 for details.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-10-31 10:59:32 +01:00
Sebastiaan van Stijn
ba3bf8191e
Merge pull request #35313 from charrywanganthony/RandomAlpha
Separate the GenerateRandomAlphaOnlyString function from stringutils
2017-10-29 22:33:57 +01:00
Yong Tang
e8730d052e
Merge pull request #35289 from zuiurs/namesgeneratorCmd
Fix a names-generator binary
2017-10-28 13:16:33 -07:00
chaowang
7c35a24182 Separate the GenerateRandomAlphaOnlyString function from stringutils
Signed-off-by: chaowang <chaowang@localhost.localdomain>
2017-10-28 09:03:02 +08:00
Sebastiaan van Stijn
17bb1d3663 Merge pull request #35285 from crosbymichael/solaris
Remove solaris files
2017-10-25 15:14:04 +02:00
Vincent Demeester
d040d637a9
Rename a few docker to moby
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-10-25 13:56:12 +02:00
Mizuki Urushida
eaab2f7150 Fix a names-generator binary
To ensure that namesgenerator binary outputs random name
by initializing Seed.

Signed-off-by: Mizuki Urushida <z11111001011@gmail.com>

not use init function.

Signed-off-by: Mizuki Urushida <z11111001011@gmail.com>
2017-10-25 18:21:17 +09:00
Sebastiaan van Stijn
d891f2e3ca Merge pull request #35217 from cpuguy83/fix_chroot_mount_race
Use rslave instead of rprivate in chrootarchive
2017-10-24 22:13:28 +02:00
Michael Crosby
5a9b5f10cf Remove solaris files
For obvious reasons that it is not really supported now.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-24 15:39:34 -04:00
Brian Goff
402540708c Merge pull request #34895 from mlaventure/containerd-1.0-client
Containerd 1.0 client
2017-10-23 10:38:03 -04:00
Brian Goff
3816b51438 Fixup some issues with plugin refcounting
In some circumstances we were not properly releasing plugin references,
leading to failures in removing a plugin with no way to recover other
than restarting the daemon.

1. If volume create fails (in the driver)
2. If a driver validation fails (should be rare)
3. If trying to get a plugin that does not match the passed in capability

Ideally the test for 1 and 2 would just be a unit test, however the
plugin interfaces are too complicated as `plugingetter` relies on
github.com/pkg/plugin/Client (a concrete type), which will require
spinning up services from within the unit test... it just wouldn't be a
unit test at this point.
I attempted to refactor this a bit, but since both libnetwork and
swarmkit are reliant on `plugingetter` as well, this would not work.
This really requires a re-write of the lower-level plugin management to
decouple these pieces.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-10-21 15:17:57 -04:00
Yong Tang
113bebe0ee Merge pull request #35059 from cpuguy83/35002_handle_bind_create_errors
idtools don't chown if not needed
2017-10-20 08:39:23 -07:00
Kenfe-Mickael Laventure
ddae20c032
Update libcontainerd to use containerd 1.0
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-20 07:11:37 -07:00
Brian Goff
fa9709a3fc idtools don't chown if not needed
In some cases (e.g. NFS), a chown may technically be a no-op but still
return `EPERM`, so only call `chown` when neccessary.

This is particularly problematic for docker users bind-mounting an NFS
share into a container.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-10-19 16:06:25 -04:00
Tobias Klauser
86f080cff0 Use Mkdev, Major and Minor functions from golang.org/x/sys/unix
Update golang.org/x/sys to 8dbc5d05d6edcc104950cc299a1ce6641235bc86 in
order to get the Major, Minor and Mkdev functions for every unix-like
OS. Use them instead of the locally defined versions which currently use
the Linux specific device major/minor encoding.

This means that the device number should now be properly encoded on e.g.
Darwin, FreeBSD or Solaris.

Also, the SIGUNUSED constant was removed from golang.org/x/sys/unix in
https://go-review.googlesource.com/61771 as it is also removed from the
respective glibc headers.

Remove it from signal.SignalMap as well after the golang.org/x/sys
re-vendoring.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-10-19 08:28:38 +02:00
Kate Heddleston
736fafb2d0 Removing unused code with a TODO that no longer needs to be done
Signed-off-by: Kate Heddleston <kate.heddleston@gmail.com>
2017-10-16 14:57:48 -07:00
Brian Goff
5ede64d63f Use rslave instead of rprivate in chrootarchive
With `rprivate` there exists a race where a reference to a mount has
propagated to the new namespace, when `rprivate` is set the parent
namespace is not able to remove the mount due to that reference.
With `rslave` unmounts will propagate correctly into the namespace and
prevent the sort of transient errors that are possible with `rprivate`.

This is a similar fix to 117c92745b

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-10-16 13:49:31 -04:00
Sebastiaan van Stijn
be4586fffd Merge pull request #33488 from raja-sami-10p/pkg/idtools
Increase Coverage of pkg/idtools
2017-10-10 18:37:49 +02:00
John Howard
d98ecf2d6c LCOW: API change JSON header to string POST parameter
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-10-06 15:26:48 -07:00
John Howard
0380fbff37 LCOW: API: Add platform to /images/create and /build
Signed-off-by: John Howard <jhoward@microsoft.com>

This PR has the API changes described in https://github.com/moby/moby/issues/34617.
Specifically, it adds an HTTP header "X-Requested-Platform" which is a JSON-encoded
OCI Image-spec `Platform` structure.

In addition, it renames (almost all) uses of a string variable platform (and associated)
methods/functions to os. This makes it much clearer to disambiguate with the swarm
"platform" which is really os/arch. This is a stepping stone to getting the daemon towards
fully multi-platform/arch-aware, and makes it clear when "operating system" is being
referred to rather than "platform" which is misleadingly used - sometimes in the swarm
meaning, but more often as just the operating system.
2017-10-06 11:44:18 -07:00
Sebastiaan van Stijn
1c4fad8135 Merge pull request #35056 from tklauser/win-console-mode-consts
Use windows console mode constants from Azure/go-ansiterm
2017-10-05 00:41:22 +02:00
Alexander Morozov
889cfd1b44 pkg/locker: add benchmarks
Signed-off-by: Alexander Morozov <lk4d4math@gmail.com>
2017-10-02 11:20:21 -07:00
Tobias Klauser
9335683fa5 Use all console mode constants from go-ansiterm
The missing console mode constants were added to go-ansiterm in
Azure/go-ansiterm#23. Use these constants instead of defining them
locally.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-10-02 09:49:20 +02:00
Danyal Khaliq
00c0ee885c Increase Coverage of pkg/idtools
Signed-off-by: Danyal Khaliq <danyal.khaliq@tenpearls.com>
2017-09-27 12:55:04 +05:00
Vincent Demeester
7d47823c22 Merge pull request #34946 from dnephin/fix-jsonlog
Move pkg/jsonlog to be a subpackage of the single consumer
2017-09-26 10:49:57 +02:00
Daniel Nephin
035604cca6 Move jsonlog to a subpackage of jsonfilelog
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
a06ad2792a Fix benchmarks and remove more unnecessary code.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
231c5cbd50 Remove unused JSONLog marshaling
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
638d4cc7e4 Remove unused Format
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
7de92de636 Unexport FastTimeMarshalJSON
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
27cfa68af1 Move RFC3339NanoFixed to a more appropriate package.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:24 -04:00
Vincent Demeester
8bebd42df2
Fix TestMount under a selinux system
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-09-25 11:54:30 +02:00
Stephen J Day
0cd4ab3f9a
pkg/package: remove promise package
The promise package represents a simple enough concurrency pattern that
replicating it in place is sufficient. To end the propagation of this
package, it has been removed and the uses have been inlined.

While this code could likely be refactored to be simpler without the
package, the changes have been minimized to reduce the possibility of
defects. Someone else may want to do further refactoring to remove
closures and reduce the number of goroutines in use.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-09-21 17:56:45 -07:00
Yong Tang
777d4a1bf4 Merge pull request #34861 from tklauser/fix-cstring-leaks
Fix CString memory leaks
2017-09-21 09:14:07 -07:00
Yong Tang
149f3aceb2 Merge pull request #31993 from dccurtis/issue-31816
Fixing ‘docker cp’ to allow new target file name in a host symlinked directory
2017-09-19 10:54:53 -07:00
Kir Kolyshkin
46833ee1c3 devmapper: show dmesg if mount fails
If mount fails, the reason might be right there in the kernel log ring buffer.
Let's include it in the error message, it might be of great help.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-09-17 22:04:31 -07:00