Commit graph

2750 commits

Author SHA1 Message Date
John Howard
6de9f90417 Windows: Report Version and UBR
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-05 08:20:45 -08:00
John Howard
83908836d3 Windows: Block pulling uplevel images
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-02-26 12:33:54 -08:00
Sebastiaan van Stijn
20028325da
Merge pull request #35829 from cpuguy83/no_private_mount_for_plugins
Perform plugin mounts in the runtime
2018-02-21 12:28:13 +01:00
Tonis Tiigi
f6c8266afd builder: fix wrong cache hits building from tars
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-15 23:42:42 -08:00
Aleksa Sarai
98fe4bd8f1
pkg: devmapper: dynamically load dm_task_deferred_remove
dm_task_deferred_remove is not supported by all distributions, due to
out-dated versions of devicemapper. However, in the case where the
devicemapper library was updated without rebuilding Docker (which can
happen in some distributions) then we should attempt to dynamically load
the relevant object rather than try to link to it.

This can only be done if Docker was built dynamically, for obvious
reasons.

In order to avoid having issues arise when dlsym(3) was unnecessary,
gate the whole dlsym(3) logic behind a buildflag that we disable by
default (libdm_dlsym_deferred_remove).

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2018-02-16 17:23:23 +11:00
bin liu
7a7a8a33a4 Fix typos in pkg
Signed-off-by: bin liu <liubin0329@gmail.com>
2018-02-10 19:43:13 +08:00
Brian Goff
0e5eaf8ee3 Ensure plugin returns correctly scoped paths
Before this change, volume management was relying on the fact that
everything the plugin mounts is visible on the host within the plugin's
rootfs. In practice this caused some issues with mount leaks, so we
changed the behavior such that mounts are not visible on the plugin's
rootfs, but available outside of it, which breaks volume management.

To fix the issue, allow the plugin to scope the path correctly rather
than assuming that everything is visible in `p.Rootfs`.
In practice this is just scoping the `PropagatedMount` paths to the
correct host path.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-07 15:48:27 -05:00
Daniel Nephin
4f0d95fa6e Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
Anusha Ragunathan
6e5c2d639f
Merge pull request #36119 from cpuguy83/fix_plugin_scanner
Fix issue with plugin scanner going to deep
2018-02-02 20:19:01 -08:00
Brian Goff
b27f70d45a Fix issue with plugin scanner going to deep
The plugin spec says that plugins can live in one of:

- /var/run/docker/plugins/<name>.sock
- /var/run/docker/plugins/<name>/<name>.sock
- /etc/docker/plugins/<name>.[json,spec]
- /etc/docker/plugins/<name>/<name>.<json,spec>
- /usr/lib/docker/plugins/<name>.<json,spec>
- /usr/lib/docker/plugins/<name>/<name>.<json,spec>

However, the plugin scanner which is used by the volume list API was
doing `filepath.Walk`, which will walk the entire tree for each of the
supported paths.
This means that even v2 plugins in
`/var/run/docker/plugins/<id>/<name>.sock` were being detected as a v1
plugin.
When the v1 plugin loader tried to load such a plugin it would log an
error that it couldn't find it because it doesn't match one of the
supported patterns... e.g. when in a subdir, the subdir name must match
the plugin name for the socket.

There is no behavior change as the error is only on the `Scan()` call,
which is passing names to the plugin registry when someone calls the
volume list API.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-02 16:49:14 -05:00
Sebastiaan van Stijn
c41c80026b
Merge pull request #35911 from ASMfreaK/Russian-Scientsists-addition-to-names-generator
Add more russian scientists in names-generator.go
2018-01-26 15:48:58 -08:00
Vincent Demeester
59e86068e6
Merge pull request #34379 from cpuguy83/mount_optimizations
Optimizations for recursive unmount
2018-01-24 14:00:58 -08:00
Sebastiaan van Stijn
db5c006bc8
Merge pull request #35919 from yongtang/35333-carry
Carry #35333: Devicemapper: ignore Nodata errors when delete thin device
2018-01-20 18:47:16 +01:00
John Howard
5b24976ad4 Bump RS3 final build, and remove LCOW_SUPPORTED
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-19 12:22:56 -08:00
John Howard
0cba7740d4 Address feedback from Tonis
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-18 12:30:39 -08:00
John Howard
afd305c4b5 LCOW: Refactor to multiple layer-stores based on feedback
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-18 08:31:05 -08:00
John Howard
ce8e529e18 LCOW: Re-coalesce stores
Signed-off-by: John Howard <jhoward@microsoft.com>

The re-coalesces the daemon stores which were split as part of the
original LCOW implementation.

This is part of the work discussed in https://github.com/moby/moby/issues/34617,
in particular see the document linked to in that issue.
2018-01-18 08:29:19 -08:00
unclejack
871afbb304
Merge pull request #35697 from sargun/use-pgzip
Make image (layer) downloads faster by using pigz
2018-01-17 11:18:20 -08:00
Sebastiaan van Stijn
f0b0f2038d
Merge pull request #35441 from cpuguy83/plugin_timeout
Add timeouts for volume plugin ops
2018-01-17 14:49:41 +01:00
Brian Goff
b15f8d2d4f Add timeouts for volume plugin ops
This protects the daemon from volume plugins that are slow or
deadlocked.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-16 20:30:49 -05:00
Pavel Pletenev
e0c3fe7836
Add more russian scientists in names-generator.go
Signed-off-by: ASM <cpp.create@gmail.com>
2018-01-16 21:09:17 +02:00
Sargun Dhillon
fd35494a25 Make image (layer) downloads faster by using pigz
The Golang built-in gzip library is serialized, and fairly slow
at decompressing. It also only decompresses on demand, versus
pipelining decompression.

This change switches to using the pigz external command
for gzip decompression, as opposed to using the built-in
golang one. This code is not vendored, but will be used
if it autodetected as part of the OS.

This also switches to using context, versus a manually
managed channel to manage cancellations, and synchronization.
There is a little bit of weirdness around manually having
to cancel in the error cases.

Signed-off-by: Sargun Dhillon <sargun@sargun.me>
2018-01-16 10:49:18 -08:00
Sebastiaan van Stijn
b4a6313969
Golint: remove redundant ifs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 00:42:25 +01:00
Brian Goff
dd21087660 Optimizations for recurrsive unmount
When a recursive unmount fails, don't bother parsing the mount table to check
if what we expected to be a mountpoint is still mounted. `EINVAL` is
returned when you try to unmount something that is not a mountpoint, the
other cases of `EINVAL` would not apply here unless everything is just
wrong. Parsing the mount table over and over is relatively expensive,
especially in the code path that it's in.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-11 16:13:16 -05:00
Yong Tang
99c22b0034
Merge pull request #35822 from keyolk/35709-fix-docker-build-overlayfs-whiteout
skip container ID remapping, if the file is overlayfs whiteout.
2018-01-03 22:21:00 +08:00
Chanhun Jeong
b013c1541d skip container ID remapping, if the file is overlayfs whiteout.
Signed-off-by: Chanhun Jeong <chanhun.jeong@navercorp.com>
2018-01-03 17:06:59 +09:00
Liu Hua
8451d03d8e Devicemapper: ignore Nodata errors when delete thin device
if thin device is deteled and the metadata exists, you can not
delete related containers. This patch ignore Nodata errors for
thin device deletion

Signed-off-by: Liu Hua <sdu.liu@huawei.com>
2018-01-02 18:04:25 +00:00
Sebastiaan van Stijn
6ed1163c98
Remove redundant build-tags
Files that are suffixed with `_linux.go` or `_windows.go` are
already only built on Linux / Windows, so these build-tags
were redundant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-12-18 17:41:53 +01:00
John Howard
7caa30e893 Windows: Case-insensitive filename matching against builder cache
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-12-14 13:49:40 -08:00
Tõnis Tiigi
cefb33700c
Merge pull request #35589 from keloyang/close-fd
Close pipe in chrootarchive.invokeUnpack when cmd.Start()/json.NewEncoder failed
2017-12-06 17:48:02 -08:00
Michael Crosby
72e45fd54e
Merge pull request #35618 from kolyshkin/mkdir-all
Fix MkdirAll* and its usage
2017-11-30 11:19:29 -05:00
Kir Kolyshkin
516010e92d Simplify/fix MkdirAll usage
This subtle bug keeps lurking in because error checking for `Mkdir()`
and `MkdirAll()` is slightly different wrt to `EEXIST`/`IsExist`:

 - for `Mkdir()`, `IsExist` error should (usually) be ignored
   (unless you want to make sure directory was not there before)
   as it means "the destination directory was already there"

 - for `MkdirAll()`, `IsExist` error should NEVER be ignored.

Mostly, this commit just removes ignoring the IsExist error, as it
should not be ignored.

Also, there are a couple of cases then IsExist is handled as
"directory already exist" which is wrong. As a result, some code
that never worked as intended is now removed.

NOTE that `idtools.MkdirAndChown()` behaves like `os.MkdirAll()`
rather than `os.Mkdir()` -- so its description is amended accordingly,
and its usage is handled as such (i.e. IsExist error is not ignored).

For more details, a quote from my runc commit 6f82d4b (July 2015):

    TL;DR: check for IsExist(err) after a failed MkdirAll() is both
    redundant and wrong -- so two reasons to remove it.

    Quoting MkdirAll documentation:

    > MkdirAll creates a directory named path, along with any necessary
    > parents, and returns nil, or else returns an error. If path
    > is already a directory, MkdirAll does nothing and returns nil.

    This means two things:

    1. If a directory to be created already exists, no error is
    returned.

    2. If the error returned is IsExist (EEXIST), it means there exists
    a non-directory with the same name as MkdirAll need to use for
    directory. Example: we want to MkdirAll("a/b"), but file "a"
    (or "a/b") already exists, so MkdirAll fails.

    The above is a theory, based on quoted documentation and my UNIX
    knowledge.

    3. In practice, though, current MkdirAll implementation [1] returns
    ENOTDIR in most of cases described in #2, with the exception when
    there is a race between MkdirAll and someone else creating the
    last component of MkdirAll argument as a file. In this very case
    MkdirAll() will indeed return EEXIST.

    Because of #1, IsExist check after MkdirAll is not needed.

    Because of #2 and #3, ignoring IsExist error is just plain wrong,
    as directory we require is not created. It's cleaner to report
    the error now.

    Note this error is all over the tree, I guess due to copy-paste,
    or trying to follow the same usage pattern as for Mkdir(),
    or some not quite correct examples on the Internet.

    [1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-11-27 17:32:12 -08:00
Kir Kolyshkin
2aa13f86f0 idtools.MkdirAs*: error out if dir exists as file
Standard golang's `os.MkdirAll()` function returns "not a directory" error
in case a directory to be created already exists but is not a directory
(e.g. a file). Our own `idtools.MkdirAs*()` functions do not replicate
the behavior.

This is a bug since all `Mkdir()`-like functions are expected to ensure
the required directory exists and is indeed a directory, and return an
error otherwise.

As the code is using our in-house `system.Stat()` call returning a type
which is incompatible with that of golang's `os.Stat()`, I had to amend
the `system` package with `IsDir()`.

A test case is also provided.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-11-27 13:25:44 -08:00
Chetan Birajdar
7c50d39480 added more awesome and notable women to names-generator.go
Signed-off-by: Chetan Birajdar <birajdar.chetan@gmail.com>
2017-11-26 21:40:02 -08:00
y00316549
f5f8f00860 Close pipe in chrootarchive.invokeUnpack when cmd.Start()/json.NewEncoder failed.
Signed-off-by: y00316549 <yangshukui@huawei.com>
2017-11-24 13:41:08 +08:00
Akihiro Suda
84b1f813a4
Merge pull request #35541 from thaJeztah/remove-deprecated-mkdirallas
Remove uses of deprecated MkdirAllAs(), MkdirAs()
2017-11-22 17:49:37 +09:00
Sebastiaan van Stijn
47af9f625d
Remove unused "testutil" package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 15:09:23 +01:00
Sebastiaan van Stijn
38b3af567f
Remove deprecated MkdirAllAs(), MkdirAs()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 13:53:54 +01:00
Sebastiaan van Stijn
ce66470f54
Minor refactor in idtools
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 13:49:58 +01:00
Sebastiaan van Stijn
9aba019b72
Update idtools tests to test non-deprecated functions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 13:47:17 +01:00
Chao Wang
97e406678c Split and remove pkg/stringutils
Signed-off-by: Chao Wang <wangchao.fnst@cn.fujitsu.com>
2017-11-13 10:01:11 +08:00
Chao Wang
5c154cfac8 Copy Inslice() to those parts that use it
Signed-off-by: Chao Wang <wangchao.fnst@cn.fujitsu.com>
2017-11-10 13:42:38 +08:00
Yong Tang
a4bdb304e2
Merge pull request #35250 from joppich/patch-1
Update names-generator.go
2017-11-09 06:21:02 -08:00
Victor Vieux
5745a8531e
Merge pull request #35265 from cpuguy83/32609_defreference_voldriver_on_error
Fixup some issues with plugin refcounting
2017-11-07 09:47:07 -08:00
Yong Tang
4785f1a7ab Remove solaris build tag and `contrib/mkimage/solaris
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2017-11-02 00:01:46 +00:00
Yong Tang
0bcca5ea86
Merge pull request #35341 from tklauser/utsname-x-sys
Simplify Utsname string conversion
2017-11-01 11:36:38 -07:00
joppich
f69f8f3679 Update names-generator.go to include Cédric Villani and Valentina
Tershkova

Signed-off-by: Jorit Kleine-Möllhoff <joppich@bricknet.de>
2017-10-31 15:38:57 +01:00
Tobias Klauser
6d068bc25b Simplify Utsname string conversion
Update golang.org/x/sys to 95c6576299259db960f6c5b9b69ea52422860fce in
order to get the unix.Utsname with byte array instead of int8/uint8
members.

This allows to use simple byte slice to string conversions instead of
using charsToString or its open-coded version.

Also see golang/go#20753 for details.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-10-31 10:59:32 +01:00
Sebastiaan van Stijn
ba3bf8191e
Merge pull request #35313 from charrywanganthony/RandomAlpha
Separate the GenerateRandomAlphaOnlyString function from stringutils
2017-10-29 22:33:57 +01:00
Yong Tang
e8730d052e
Merge pull request #35289 from zuiurs/namesgeneratorCmd
Fix a names-generator binary
2017-10-28 13:16:33 -07:00
chaowang
7c35a24182 Separate the GenerateRandomAlphaOnlyString function from stringutils
Signed-off-by: chaowang <chaowang@localhost.localdomain>
2017-10-28 09:03:02 +08:00
Sebastiaan van Stijn
17bb1d3663 Merge pull request #35285 from crosbymichael/solaris
Remove solaris files
2017-10-25 15:14:04 +02:00
Vincent Demeester
d040d637a9
Rename a few docker to moby
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-10-25 13:56:12 +02:00
Mizuki Urushida
eaab2f7150 Fix a names-generator binary
To ensure that namesgenerator binary outputs random name
by initializing Seed.

Signed-off-by: Mizuki Urushida <z11111001011@gmail.com>

not use init function.

Signed-off-by: Mizuki Urushida <z11111001011@gmail.com>
2017-10-25 18:21:17 +09:00
Sebastiaan van Stijn
d891f2e3ca Merge pull request #35217 from cpuguy83/fix_chroot_mount_race
Use rslave instead of rprivate in chrootarchive
2017-10-24 22:13:28 +02:00
Michael Crosby
5a9b5f10cf Remove solaris files
For obvious reasons that it is not really supported now.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-24 15:39:34 -04:00
Brian Goff
402540708c Merge pull request #34895 from mlaventure/containerd-1.0-client
Containerd 1.0 client
2017-10-23 10:38:03 -04:00
Brian Goff
3816b51438 Fixup some issues with plugin refcounting
In some circumstances we were not properly releasing plugin references,
leading to failures in removing a plugin with no way to recover other
than restarting the daemon.

1. If volume create fails (in the driver)
2. If a driver validation fails (should be rare)
3. If trying to get a plugin that does not match the passed in capability

Ideally the test for 1 and 2 would just be a unit test, however the
plugin interfaces are too complicated as `plugingetter` relies on
github.com/pkg/plugin/Client (a concrete type), which will require
spinning up services from within the unit test... it just wouldn't be a
unit test at this point.
I attempted to refactor this a bit, but since both libnetwork and
swarmkit are reliant on `plugingetter` as well, this would not work.
This really requires a re-write of the lower-level plugin management to
decouple these pieces.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-10-21 15:17:57 -04:00
Yong Tang
113bebe0ee Merge pull request #35059 from cpuguy83/35002_handle_bind_create_errors
idtools don't chown if not needed
2017-10-20 08:39:23 -07:00
Kenfe-Mickael Laventure
ddae20c032
Update libcontainerd to use containerd 1.0
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-20 07:11:37 -07:00
Brian Goff
fa9709a3fc idtools don't chown if not needed
In some cases (e.g. NFS), a chown may technically be a no-op but still
return `EPERM`, so only call `chown` when neccessary.

This is particularly problematic for docker users bind-mounting an NFS
share into a container.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-10-19 16:06:25 -04:00
Tobias Klauser
86f080cff0 Use Mkdev, Major and Minor functions from golang.org/x/sys/unix
Update golang.org/x/sys to 8dbc5d05d6edcc104950cc299a1ce6641235bc86 in
order to get the Major, Minor and Mkdev functions for every unix-like
OS. Use them instead of the locally defined versions which currently use
the Linux specific device major/minor encoding.

This means that the device number should now be properly encoded on e.g.
Darwin, FreeBSD or Solaris.

Also, the SIGUNUSED constant was removed from golang.org/x/sys/unix in
https://go-review.googlesource.com/61771 as it is also removed from the
respective glibc headers.

Remove it from signal.SignalMap as well after the golang.org/x/sys
re-vendoring.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-10-19 08:28:38 +02:00
Kate Heddleston
736fafb2d0 Removing unused code with a TODO that no longer needs to be done
Signed-off-by: Kate Heddleston <kate.heddleston@gmail.com>
2017-10-16 14:57:48 -07:00
Brian Goff
5ede64d63f Use rslave instead of rprivate in chrootarchive
With `rprivate` there exists a race where a reference to a mount has
propagated to the new namespace, when `rprivate` is set the parent
namespace is not able to remove the mount due to that reference.
With `rslave` unmounts will propagate correctly into the namespace and
prevent the sort of transient errors that are possible with `rprivate`.

This is a similar fix to 117c92745b

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-10-16 13:49:31 -04:00
Sebastiaan van Stijn
be4586fffd Merge pull request #33488 from raja-sami-10p/pkg/idtools
Increase Coverage of pkg/idtools
2017-10-10 18:37:49 +02:00
John Howard
d98ecf2d6c LCOW: API change JSON header to string POST parameter
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-10-06 15:26:48 -07:00
John Howard
0380fbff37 LCOW: API: Add platform to /images/create and /build
Signed-off-by: John Howard <jhoward@microsoft.com>

This PR has the API changes described in https://github.com/moby/moby/issues/34617.
Specifically, it adds an HTTP header "X-Requested-Platform" which is a JSON-encoded
OCI Image-spec `Platform` structure.

In addition, it renames (almost all) uses of a string variable platform (and associated)
methods/functions to os. This makes it much clearer to disambiguate with the swarm
"platform" which is really os/arch. This is a stepping stone to getting the daemon towards
fully multi-platform/arch-aware, and makes it clear when "operating system" is being
referred to rather than "platform" which is misleadingly used - sometimes in the swarm
meaning, but more often as just the operating system.
2017-10-06 11:44:18 -07:00
Sebastiaan van Stijn
1c4fad8135 Merge pull request #35056 from tklauser/win-console-mode-consts
Use windows console mode constants from Azure/go-ansiterm
2017-10-05 00:41:22 +02:00
Alexander Morozov
889cfd1b44 pkg/locker: add benchmarks
Signed-off-by: Alexander Morozov <lk4d4math@gmail.com>
2017-10-02 11:20:21 -07:00
Tobias Klauser
9335683fa5 Use all console mode constants from go-ansiterm
The missing console mode constants were added to go-ansiterm in
Azure/go-ansiterm#23. Use these constants instead of defining them
locally.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-10-02 09:49:20 +02:00
Danyal Khaliq
00c0ee885c Increase Coverage of pkg/idtools
Signed-off-by: Danyal Khaliq <danyal.khaliq@tenpearls.com>
2017-09-27 12:55:04 +05:00
Vincent Demeester
7d47823c22 Merge pull request #34946 from dnephin/fix-jsonlog
Move pkg/jsonlog to be a subpackage of the single consumer
2017-09-26 10:49:57 +02:00
Daniel Nephin
035604cca6 Move jsonlog to a subpackage of jsonfilelog
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
a06ad2792a Fix benchmarks and remove more unnecessary code.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
231c5cbd50 Remove unused JSONLog marshaling
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
638d4cc7e4 Remove unused Format
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
7de92de636 Unexport FastTimeMarshalJSON
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
27cfa68af1 Move RFC3339NanoFixed to a more appropriate package.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:24 -04:00
Vincent Demeester
8bebd42df2
Fix TestMount under a selinux system
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-09-25 11:54:30 +02:00
Stephen J Day
0cd4ab3f9a
pkg/package: remove promise package
The promise package represents a simple enough concurrency pattern that
replicating it in place is sufficient. To end the propagation of this
package, it has been removed and the uses have been inlined.

While this code could likely be refactored to be simpler without the
package, the changes have been minimized to reduce the possibility of
defects. Someone else may want to do further refactoring to remove
closures and reduce the number of goroutines in use.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-09-21 17:56:45 -07:00
Yong Tang
777d4a1bf4 Merge pull request #34861 from tklauser/fix-cstring-leaks
Fix CString memory leaks
2017-09-21 09:14:07 -07:00
Yong Tang
149f3aceb2 Merge pull request #31993 from dccurtis/issue-31816
Fixing ‘docker cp’ to allow new target file name in a host symlinked directory
2017-09-19 10:54:53 -07:00
Kir Kolyshkin
46833ee1c3 devmapper: show dmesg if mount fails
If mount fails, the reason might be right there in the kernel log ring buffer.
Let's include it in the error message, it might be of great help.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-09-17 22:04:31 -07:00
Kir Kolyshkin
84f1c054e6 devmapper gd: disable for static build
Static build with devmapper is impossible now since libudev is required
and no static version of libudev is available (as static libraries are
not supported by systemd which udev is part of).

This should not hurt anyone as "[t]he primary user of static builds
is the Editions, and docker in docker via the containers, and none
of those use device mapper".

Also, since the need for static libdevmapper is gone, there is no need
to self-compile libdevmapper -- let's use the one from Debian Stretch.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-09-17 22:04:31 -07:00
Tobias Klauser
593dbfd144 Fix CString memory leaks
Make sure to call C.free on C string allocated using C.CString in every
exit path.

C.CString allocates memory in the C heap using malloc. It is the callers
responsibility to free them. See
https://golang.org/cmd/cgo/#hdr-Go_references_to_C for details.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-09-15 09:57:26 +02:00
Victor Vieux
ff686743c5 Add LCOW behind experimental,
might not be the cleanest way, but it's definitly the way with the
minimum code change.

Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2017-09-14 13:51:16 -07:00
Akash Gupta
7a7357dae1 LCOW: Implemented support for docker cp + build
This enables docker cp and ADD/COPY docker build support for LCOW.
Originally, the graphdriver.Get() interface returned a local path
to the container root filesystem. This does not work for LCOW, so
the Get() method now returns an interface that LCOW implements to
support copying to and from the container.

Signed-off-by: Akash Gupta <akagup@microsoft.com>
2017-09-14 12:07:52 -07:00
Daniel Nephin
f7f101d57e Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-12 12:09:59 -04:00
Marcel Edmund Franke
6d3bc28289 Fix typos in Locker example
Signed-off-by: Marcel Edmund Franke <marcel.edmund.franke@gmail.com>
2017-09-10 11:34:19 +02:00
Daniel Nephin
09652bf878 Add ineffassign linter
Also enable GC in linting to reduce memory usage.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-08 18:23:21 -04:00
Yong Tang
2dcb77b24c Merge pull request #34738 from wgliang/optimization1
Optimize some wrong usage and spelling
2017-09-07 09:45:14 -07:00
wangguoliang
94cefa2145 Optimize some wrong usage and spelling
Signed-off-by: wgliang <liangcszzu@163.com>
2017-09-07 09:44:08 +08:00
Tibor Vass
be97c66708 Merge pull request #34748 from dnephin/cleanup-client-deps
Cleanup client/ package dependencies
2017-09-06 16:14:29 -07:00
Daniel Nephin
6916c215b0 Move tlsconfig to client package.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-06 16:39:55 -04:00
Simon Ferquel
7c9e64a2e1 re-enable some conversion for darwin-amd64 platform
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2017-09-06 12:22:41 +02:00
Brian Goff
a127269803 Merge pull request #34700 from tklauser/pkg-term-winsize
pkg/term: use IoctlGetWinsize/IoctlSetWinsize from golang.org/x/sys/unix
2017-09-01 15:58:25 -04:00
Yong Tang
cb952bf006 Merge pull request #34625 from dnephin/more-linters
Add interfacer and unconvert linters
2017-09-01 08:46:08 -07:00
Tobias Klauser
32dfc0cb64 pkg/term: use IoctlGetWinsize/IoctlSetWinsize from golang.org/x/sys/unix
Use unix.IoctlGetWinsize and unix.IoctlSetWinsize instead of manually
reimplementing them.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-09-01 14:43:09 +02:00
Yong Tang
6ed5db6243 Merge pull request #34615 from dnephin/remove-pkg-testutil-assert
Move ErrorContains to an internal package
2017-08-26 19:09:27 -07:00
Daniel Nephin
a5631c75e1 Merge pull request #34613 from dnephin/remove-pkg-testutil-cmd
Remove pkg testutil cmd
2017-08-25 15:09:52 -04:00
Vincent Demeester
475c9926db Merge pull request #34590 from shouze/add-support-of-ChownOpts-to-TarWithOptions
archive: add ChownOpts support to TarWithOptions
2017-08-25 18:21:31 +02:00
Daniel Nephin
c2c127fa5a Remove pkg/testutil/cmd
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-25 12:07:31 -04:00
Daniel Nephin
e62b2d410c Move ErrorContains to an internal package.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-25 12:04:58 -04:00
Yong Tang
2cea2f5469 Merge pull request #34583 from tklauser/win-event-functions
Use windows event functions from golang.org/x/sys/windows
2017-08-25 07:43:03 -07:00
Daniel Nephin
2f5f0af3fd Add unconvert linter
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-24 15:08:31 -04:00
Daniel Nephin
468154a841 Remove pkg/testutil tempfile and golden
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-23 17:26:27 -04:00
Sébastien HOUZÉ
8a34c67a7e
archive: add ChownOpts support to TarWithOptions
Signed-off-by: Sébastien HOUZÉ <cto@verylastroom.com>
2017-08-23 20:52:29 +02:00
Daniel Nephin
e885af2a6b Remove RunCommandPipelineWithOutput
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
9db68f4dea Remove RandomTmpDirPath
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
e8bd671815 Remove testutil.ParseCgroupPaths
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
1455086c4b Remove ConsumeWithSpeed
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
4f304e72a2 Remove testutil.ReadBody
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
6a0105b452 Remove ChannelBuffer
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
6ffbe3f6a8 Remove ListTar
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
6151d55ee7 Remove testutil.CompareDirectoryEntries and IsKilled
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
d30e51495a Remove testutil.ConvertSliceOfStringsToMap
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
a0c9089971 Remove unused helpers.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
62c1f0ef41 Add deadcode linter
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-21 18:18:50 -04:00
Daniel Nephin
372670b507 Add goimports to linters.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-21 18:15:08 -04:00
Tobias Klauser
e942513ac4 Use event functions from golang.org/x/sys/windows
Use CreateEvent, OpenEvent (which both map to the respective *EventW
function) and PulseEvent from golang.org/x/sys instead of local copies.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-08-21 12:58:09 +02:00
Daniel Nephin
9b47b7b151 Fix golint errors.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-18 14:23:44 -04:00
Douglas Curtis
cd7489f2b7 Replacing os.Lstat with os.Stat to determine directory status in CopyInfoDestinationPath
Signed-off-by: Douglas Curtis <dougcurtis1@gmail.com>

Commenting out tests for now

Signed-off-by: Doug Curtis <dougcurtis1@gmail.com>

Added unit test for CopyInfoDestionationPath.

Signed-off-by: Doug Curtis <dougcurtis1@gmail.com>

Removing integration-cli test case additions

Signed-off-by: Doug Curtis <dougcurtis1@gmail.com>

Removing extra spaces between archive_unix_test.go test cases

Signed-off-by: Doug Curtis <dougcurtis1@gmail.com>

Fixed gofmt issues in archive_unix_test.go

Signed-off-by: Doug Curtis <dougcurtis1@gmail.com>
2017-08-15 23:19:42 +00:00
Brian Goff
ebcb7d6b40 Remove string checking in API error handling
Use strongly typed errors to set HTTP status codes.
Error interfaces are defined in the api/errors package and errors
returned from controllers are checked against these interfaces.

Errors can be wraeped in a pkg/errors.Causer, as long as somewhere in the
line of causes one of the interfaces is implemented. The special error
interfaces take precedence over Causer, meaning if both Causer and one
of the new error interfaces are implemented, the Causer is not
traversed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-08-15 16:01:11 -04:00
Yong Tang
479cc38bb9 Merge pull request #33450 from naveed-jamil-tenpearls/pkg/signal/trap
Add test coverage to signal/trap.go
2017-08-15 00:53:37 -07:00
Brian Goff
31d8d2253e Merge pull request #34362 from kolyshkin/update-libdevmapper
Dockerfile*: bump devmapper library version
2017-08-14 09:56:43 -04:00
Yong Tang
187cd25517 Merge pull request #34276 from keloyang/testcase-docker-cp-hardlink
Add unit  test  for RebaseArchiveEntries
2017-08-09 07:29:09 -07:00
Naveed Jamil
bc756061a3 Add test coverage to signal/trap.go
Signed-off-by: Naveed Jamil <naveed.jamil@tenpearls.com>
2017-08-09 15:13:38 +05:00
yangshukui
1f1e8e9c0e use testify to check error for test unit test of archive package
Signed-off-by: yangshukui <yangshukui@huawei.com>
2017-08-09 10:27:01 +08:00
Tibor Vass
c7911389e6 Merge pull request #34441 from vdemeester/move-pkg-templates-away
Move pkg/templates away
2017-08-08 17:54:17 -07:00
Vincent Demeester
9ef3b53597
Move pkg/templates away
- Remove unused function and variables from the package
- Remove usage of it from `profiles/apparmor` where it wasn't required
- Move the package to `daemon/logger/templates` where it's only used

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-08-08 18:16:41 +02:00
Vincent Demeester
c204fce2ee
Move pkg/listeners away
It is only used in `daemon` and should really live there.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-08-08 18:14:04 +02:00
yangshukui
82eb9002e9 Add unit test for RebaseArchiveEntries
Signed-off-by: yangshukui <yangshukui@huawei.com>
2017-08-07 16:40:54 +08:00
Tobias Klauser
8a3730d251 pkg/term: set termios VMIN and VTIME in MakeRaw on Linux
The BSD and Solaris versions of term.MakeRaw already set VMIN and VTIME
explicitly such that a read returns when one character is available.
cfmakeraw (which was previously used) in glibc also sets these values
explicitly, so it should be done in the Linux version of MakeRaw as well
to be consistent.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-08-03 13:21:42 +02:00
yuexiao-wang
cc4da81128 Remove the logrus from pkg/signal
Signed-off-by: yuexiao-wang <wang.yuexiao@zte.com.cn>
2017-08-02 18:15:25 +08:00
Kir Kolyshkin
c4fde49a5c Dockerfile*: bump devmapper library version
Let's use latest lvm2 sources to compile the libdevmapper library.

Initial reason for compiling devmapper lib from sources was a need to
have the static version of the library at hand, in order to build
the static dockerd, but note that the same headers/solib are used
for dynamic build (dynbinary) as well.

The reason for this patch is to enable the deferral removal feature.
The supplied devmapper library (and headers) are too old, lacking the
needed functions, so the daemon is built with 'libdm_no_deferred_remove'
build tag (see the check in hack/make.sh). Because of this, even if the
kernel dm driver is perfectly able to support the feature, it can not
be used. For more details and background story, see [1].

Surely, one can't just change the version number. While at it:
 - improve the comments;
 - remove obsoleted URLs;
 - remove s390 and ppc configure updates that are no longer needed;
 - use pkg-config instead of hardcoding the flags (newer lib added
   some more dependencies);

 [1] https://github.com/moby/moby/issues/34298

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-08-01 22:08:24 -07:00
John Howard
8af4db6f00 Merge pull request #34272 from dmcgowan/update-logrus
Update logrus to v1.0.1 (Sirupsen -> sirupsen)
2017-08-01 18:53:33 -07:00
Kir Kolyshkin
adce3ca48e pkg/devicemapper: comment nitpicks
1. devmapper_wrapper_{,no_}deferred_remove.go:
Comments about LibraryDeferredRemovalSupport were very totally
misleading to me. This thing has nothing to do with either static
or dynamic linking (but with build tags). Fix the comment accordingly.

2. devmapper.go:
Reveal the source of those magic device* constants.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-07-31 20:05:26 -07:00
Derek McGowan
1009e6a40b
Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-31 13:16:46 -07:00
Tobias Klauser
01f70b028e Switch Stat syscalls to x/sys/unix
Switch some more usage of the Stat function and the Stat_t type from the
syscall package to golang.org/x/sys. Those were missing in PR #33399.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-27 10:09:02 +02:00
Sebastiaan van Stijn
f8c434366e Merge pull request #33987 from dnephin/cleanup-more-hack
Move test-unit out of hack/make
2017-07-27 00:29:13 +02:00
Victor Vieux
fa7db52e26 Merge pull request #34135 from tklauser/more-unix-fns
Replace manually written wrappers by functions from x/sys/unix
2017-07-18 20:43:13 +02:00
Kir Kolyshkin
7da12bcfa9 devmapper_wrapper.go: fix gcc warning
I am getting the following warning from gcc when compiling the daemon:

> # github.com/docker/docker/pkg/devicemapper
> pkg/devicemapper/devmapper_wrapper.go: In function ‘log_cb’:
> pkg/devicemapper/devmapper_wrapper.go:20:2: warning: ignoring return
> value of ‘vasprintf’, declared with attribute warn_unused_result
> [-Wunused-result]
>  vasprintf(&buffer, f, ap);
>  ^

vasprintf(3) man page says if the function returns -1, the buffer is
undefined, so we should not use it. In practice, I assume, this never
happens so we just return.

Introduced by https://github.com/moby/moby/pull/33845 that resulted in
commit 63328c6 ("devicemapper: remove 256 character limit of libdm logs")

Cc: Aleksa Sarai <asarai@suse.de>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-07-17 17:46:44 -07:00
Evan Hazlett
458f6712d4 Merge pull request #33886 from aaronlehmann/names-in-memdb
Store container names in memdb
2017-07-17 14:16:41 -04:00
Daniel Nephin
1fb615599a Remove test-unit from hack/make
Also remove the test flag from pkg/term and jsut checkuid directly.
Fixed a problem with a pkg/term test that was leaving the terminal in a bad
state.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-07-17 11:39:33 -04:00
Tobias Klauser
bedf09363c loopback: use IoctlGetInt/IoctlSetInt from x/sys/unix
Use IoctlGetInt/IoctlSetInt from golang.org/x/sys/unix (where
applicable) instead of manually reimplementing them.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-17 10:37:42 +02:00
Tobias Klauser
6476504695 [pkg/term] use IoctlGetTermios/IoctlSetTermios from x/sys/unix
Use IoctlGetTermios/IoctlSetTermios from golang.org/x/sys/unix instead
of manually reimplementing them.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-17 10:37:42 +02:00
Tobias Klauser
6c9d715a8c sysinfo: use Prctl() from x/sys/unix
Use unix.Prctl() instead of manually reimplementing it using
unix.RawSyscall. Also use unix.SECCOMP_MODE_FILTER instead of locally
defining it.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-17 10:37:42 +02:00
Vincent Demeester
97e730ae57 Merge pull request #34106 from tonistiigi/fix-tar-prefix-header
Fix tar prefix header
2017-07-17 09:40:39 +02:00
Tonis Tiigi
4a3cfda45e archive: add test for prefix header
With docker-17.06.0 some images pulled do not extract properly. Some files don't appear in correct directories. This may or may not cause the pull to fail. These images can't be pushed or saved. 17.06 is the first version of Docker built with go1.8.

Cause

There are multiple updates to the tar package in go1.8.

https://go-review.googlesource.com/c/32234/ disables using "prefix" field when new tar archives are being written. Prefix field was previously set when a record in the archive used a path longer than 100 bytes.

Another change https://go-review.googlesource.com/c/31444/ makes the reader ignore the "prefix" field value if the record is in GNU format. GNU format defines that same area should be used for access and modified times. If the "prefix" field is not read, a file will only be extracted by the basename.

The problem is that with a previous version of the golang archive package headers could be written, that use the prefix field while at the same time setting the header format to GNU. This happens when numeric fields are big enough that they can not be written as octal strings and need to be written in binary. Usually, this shouldn't happen: uid, gid, devmajor, devminor can use up to 7 bytes, size and timestamp can use 11. If one of the records does overflow it switches the whole writer to GNU mode and all next files will be saved in GNU format.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-14 10:20:48 -07:00
Brian Goff
f7ce35f47b Merge pull request #34077 from justincormack/mount_remount_opts
In the case of remounting with changed data, need to call mount
2017-07-14 09:58:30 -04:00
Aaron Lehmann
1128fc1add Store container names in memdb
Currently, names are maintained by a separate system called "registrar".
This means there is no way to atomically snapshot the state of
containers and the names associated with them.

We can add this atomicity and simplify the code by storing name
associations in the memdb. This removes the need for pkg/registrar, and
makes snapshots a lot less expensive because they no longer need to copy
all the names. This change also avoids some problematic behavior from
pkg/registrar where it returns slices which may be modified later on.

Note that while this change makes the *snapshotting* atomic, it doesn't
yet do anything to make sure containers are named at the same time that
they are added to the database. We can do that by adding a transactional
interface, either as a followup, or as part of this PR.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-13 12:35:00 -07:00
Justin Cormack
3a1ab5b479 In the case of remounting with changed data, need to call mount
The case where we are trying to do a remount with changed filesystem specific options was missing,
we need to call `mount` as well here to change those options.

See #33844 for where we need this, as we change `tmpfs` options.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-13 15:48:58 +01:00
Sebastiaan van Stijn
96bf279c7d Merge pull request #33539 from tklauser/x-sys-unix-symlink-xattrs
Use symlink xattr functions from x/sys/unix
2017-07-12 18:04:33 -07:00
Sebastiaan van Stijn
00b218216d Merge pull request #33845 from cyphar/devicemapper-show-me-your-logs
devicemapper: rework logging and add --storage-opt dm.libdm_log_level
2017-07-12 17:46:14 -07:00
Sebastiaan van Stijn
e04dbe5ac2 Merge pull request #33877 from rhvgoyal/sync-removal
devicemapper: Wait for device removal if deferredRemoval=true and deferredDeletion=…
2017-07-12 17:35:45 -07:00
Tobias Klauser
4d966409bc system: Use symlink xattr functions from x/sys/unix
Use the symlink xattr syscall wrappers Lgetxattr and Lsetxattr from
x/sys/unix (introduced in golang/sys@b90f89a) instead of providing own
wrappers. Leave the functionality of system.Lgetxattr intact with
respect to the retry with a larger buffer, but switch it to use
unix.Lgetxattr. Also leave system.Lsetxattr intact (even though it's
just a wrapper around the corresponding function from unix) in order to
keep moby building for !linux.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-12 14:52:42 +02:00
Tobias Klauser
ef5252fc5d [pkg/term] temporarily use STD_*_HANDLE from syscall again
Due to the CL https://go-review.googlesource.com/c/39608/ in
x/sys/windows which changed the definitions of STD_INPUT_HANDLE,
STD_OUTPUT_HANDLE and STD_ERROR_HANDLE, we get the following failure
after re-vendoring x/sys/windows:

  07:47:01 # github.com/docker/docker/pkg/term
  07:47:01 pkg/term/term_windows.go:82: constant 4294967286 overflows int
  07:47:01 pkg/term/term_windows.go:88: constant 4294967285 overflows int
  07:47:01 pkg/term/term_windows.go:94: constant 4294967284 overflows int
  07:47:12 Build step 'Execute shell' marked build as failure

Temporarily switch back pkg/term to use these constants from the syscall
package and add a comment about it.

To really fix this, go-ansiterm should probably be switched to use
x/sys/windows.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-12 14:52:42 +02:00
Christopher Jones
069fdc8a08
[project] change syscall to /x/sys/unix|windows
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>

[s390x] switch utsname from unsigned to signed

per 33267e036f
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2017-07-11 08:00:32 -04:00
Sebastiaan van Stijn
9789530b46 Merge pull request #33956 from keloyang/fix-cp-with-hardlink
Fix docker cp dir with hard link
2017-07-10 14:11:26 -07:00
Justin Cormack
b7bd959294 Split homedir files by operating system
libcontainer/user does not build at all on Windows any more, and
this was breaking the client on Windows with upstream `runc`. As
these functions are not used anyway, just split out and stop
checking `runtime`.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-10 14:10:43 +01:00
Yong Tang
05c7c31139 Merge pull request #33575 from cpuguy83/plugins_on_swarm
Plugins on swarm
2017-07-07 02:23:07 -07:00
Tibor Vass
0fcd082d88 Merge pull request #33935 from AkihiroSuda/pkg-archive-go1.9
pkg/archive.FileInfoHeader: fill file type bits
2017-07-06 14:22:45 -07:00
Brian Goff
72c3bcf2a5 Make plugin emit strongly typed, consumable events
Enables other subsystems to watch actions for a plugin(s).

This will be used specifically for implementing plugins on swarm where a
swarm controller needs to watch the state of a plugin.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-07-06 14:26:06 -04:00
yangshukui
d58ffa0364 Fix docker cp dir with hard link
docker run --name=test ubuntu /bin/sh -c "cd /tmp && echo hi > a && ln a b" && docker cp test:/tmp tmp_
test
link /root/tmp/a /root/tmp_/b: no such file or directory

Signed-off-by: yangshukui <yangshukui@huawei.com>
2017-07-06 17:03:38 +08:00
Akihiro Suda
1a451d9a7b pkg/archive.FileInfoHeader: fill file type bits
Go 1.9 (golang/go@66b5a2f) removed file type bits from
archive/tar.FileInfoHeader().

This commit ensures file type bits are filled even on Go 1.9 for
compatibility.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-07-05 05:24:32 +00:00
Josh Soref
39bcaee47b
Spelling fixes
* additional
* ambiguous
* anonymous
* anything
* application
* because
* before
* building
* capabilities
* circumstances
* commit
* committer
* compresses
* concatenated
* config
* container
* container's
* current
* definition
* delimiter
* disassociates
* discovery
* distributed
* doesnotexist
* downloads
* duplicates
* either
* enhancing
* enumerate
* escapable
* exactly
* expect
* expectations
* expected
* explicitly
* false
* filesystem
* following
* forbidden
* git with
* healthcheck
* ignore
* independent
* inheritance
* investigating
* irrelevant
* it
* logging
* looking
* membership
* mimic
* minimum
* modify
* mountpoint
* multiline
* notifier
* outputting
* outside
* overridden
* override
* parsable
* plugins
* precedence
* propagation
* provided
* provides
* registries
* repositories
* returning
* settings
* should
* signals
* someone
* something
* specifically
* successfully
* synchronize
* they've
* thinking
* uninitialized
* unintentionally
* unmarshaling
* unnamed
* unreferenced
* verify

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-03 13:13:09 -07:00
Josh Soref
6284cf5e9a
gofmt: 5 files
gofmt from go1.8.3

hg locate '*.go' |xargs ~/go/bin/gofmt -s -w

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-03 10:32:47 -07:00
Aleksa Sarai
cfd39e8d6d
devicemapper: change LogInit and move all callbacks to pkg
LogInit used to act as a manual way of registering the *necessary*
pkg/devicemapper logging callbacks. In addition, it was used to split up
the logic of pkg/devicemapper into daemon/graphdriver/devmapper (such
that some things were logged from libdm).

The manual aspect of this API was completely non-sensical and was just
begging for incorrect usage of pkg/devicemapper, so remove that semantic
and always register our own libdm callbacks.

In addition, recombine the split out logging callbacks into
pkg/devicemapper so that the default logger is local to the library and
also shown to be the recommended logger. This makes the code
substantially easier to read. Also the new DefaultLogger now has
configurable upper-bound for the log level, which allows for dynamically
changing the logging level.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2017-07-04 02:04:26 +10:00
Aleksa Sarai
c654147f06
devicemapper: actually remove DmLogInitVerbose
e07d3cd9a ("devmapper: Fix libdm logging") removed all of the callers of
DmLogInitVerbose, but we still kept around the wrapper. However, the
libdm dm_log_init_verbose API changes the verbosity of the *default*
libdm logger. Because pkg/devicemapper internally *relies* on using
logging callbacks to understand what errors were encountered by libdm,
this wrapper is useless (it only makes sense for the default logger
which we do not user).

Any user not inside Docker of this function almost certainly was not
using this API correctly, because pkg/devicemapper will misbehave if our
logging callbacks were not registered.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2017-07-04 02:04:26 +10:00
Aleksa Sarai
63328c6882
devicemapper: remove 256 character limit of libdm logs
This limit is unecessary and can lead to the truncation of long libdm
logs (which is quite annoying).

Fixes: b440ec013 ("device-mapper: Move all devicemapper spew to log through utils.Debugf().")
Signed-off-by: Aleksa Sarai <asarai@suse.de>
2017-07-04 02:04:26 +10:00
Vivek Goyal
36cb6efebc Wait for device removal if deferredRemoval=true and deferredDeletion=false
There have been some cases where umount, a device can be busy for a very
short duration. Maybe its udev rules, or maybe it is runc related races
or probably it is something else. We don't know yet.

If deferred removal is enabled but deferred deletion is not, then for the
case of "docker run -ti --rm fedora bash", a container will exit, device
will be deferred removed and then immediately a call will come to delete
the device. It is possible that deletion will fail if device was busy
at that time.

A device can't be deleted if it can't be removed/deactivated first. There
is only one exception and that is when deferred deletion is on. In that
case graph driver will keep track of deleted device and try to delete it
later and return success to caller.

Always make sure that device deactivation is synchronous when device is
being deleted (except the case when deferred deletion is enabled).

This should also take care of small races when device is busy for a short
duration and it is being deleted.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2017-06-30 14:27:26 -04:00
Victor Vieux
48e625ccca Merge pull request #33690 from raja-sami-10p/pkg/authorization
Increase test Coverage of pkg/authorization
2017-06-27 10:09:13 -07:00
Sebastiaan van Stijn
d3d1aabcc6
Move IsGitTransport() to gitutils
This function was only used inside gitutils,
and is written specifically for the requirements
there.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-06-26 10:07:04 -07:00
Sebastiaan van Stijn
4964b09238 Merge pull request #33674 from raja-sami-10p/pkg/templates
Increase test Coverage of pkg/templates
2017-06-24 01:04:24 -07:00
Raja Sami
27e771b714 Increase test Coverage of pkg/templates
Signed-off-by: Raja Sami <raja.sami@tenpearls.com>
2017-06-23 12:38:33 +05:00
Raja Sami
f1eb0c0ebb Increase test Coverage of pkg/authorization
Signed-off-by: Raja Sami <raja.sami@tenpearls.com>
2017-06-23 12:35:46 +05:00
Brian Goff
db57b85fcb Merge pull request #33653 from raja-sami-10p/pkg/streamformatter
Increase the Coverage of pkg/streamformatter
2017-06-22 19:52:02 -07:00
Tonis Tiigi
5c3d2d552b Implement incremental file sync using client session
Also exposes shared cache and garbage collection/prune
for the source data.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-22 11:52:35 -07:00
Raja Sami
956c5b61ce Increase the Coverage of pkg/streamformatter
Signed-off-by: Raja Sami <raja.sami@tenpearls.com>
2017-06-22 12:29:10 +05:00
John Stephens
930e689668 Merge pull request #33241 from Microsoft/jjh/multi-layerstore
LCOW: Support most operations excluding remote filesystem
2017-06-21 15:45:23 -07:00
John Howard
ed10ac6ee9 LCOW: Create layer folders with correct ACL
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:50:12 -07:00
John Howard
f154588226 LCOW: OCI Spec and Environment for container start
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:50:11 -07:00
John Howard
ed4d2aa981 LCOW: Graphdriver
Signed-off-by: John Howard <jhoward@microsoft.com>

Adds the graphdriver for Linux containers on Windows
2017-06-20 19:49:24 -07:00
Brian Goff
b5f68d7ed3 Merge pull request #33097 from stevvooe/remove-random-pkg
pkg: remove random package
2017-06-20 13:43:38 -07:00
John Howard
fe5b34ba88 LCOW: Add environment variable to enable
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 08:55:46 -07:00
Vincent Demeester
99c72eb268 Merge pull request #33454 from dnephin/refactor-builder-remove-copy-on-build
[Builder] Move file coping from the daemon to the builder
2017-06-20 10:12:57 +02:00
Akihiro Suda
93c06b8013 Merge pull request #33725 from ripcurld0/deprecate_go16
Remove Go 1.6 code from moby
2017-06-20 08:54:01 +03:00
Sebastiaan van Stijn
16380b3ee6 Merge pull request #33522 from naveed-jamil-tenpearls/pkg/promise
Added Test Case Coverage for PKG/PROMISE
2017-06-19 13:53:42 -07:00
Neil Horman
23dcfec140 devmapper: ensure that UdevWait is called after calls to setCookie
Recent changes to devmapper broke the implicit requirement that UdevWait be
called after every call to task.setCookie.  Failure to do so results in leaks of
semaphores in the LVM code, eventually leading to semaphore exhaustion.
Previously this was handled by calling UdevWait in a ubiquitous defer function.
While there was initially some concern with deferring the UdevWait function
would cause some amount of race possibiliy, the fact that we never return the
cookie value or any value used to find it, makes that possibility seem unlikely,
so lets go back to that method

Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
2017-06-19 13:50:57 -04:00
Boaz Shuster
6bfd0f6b5d Remove Go 1.6 code from moby
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
2017-06-19 15:59:00 +03:00
Sebastiaan van Stijn
acf23c7367 Merge pull request #33517 from naveed-jamil-tenpearls/pkg/reexec
Added Test Case Coverage for PKG/REEXEC
2017-06-16 15:24:51 +02:00
Vincent Demeester
11293d91f9 Merge pull request #33655 from dsheets/authz-disable-race
Eliminate authz plugin disable race
2017-06-14 11:07:23 +02:00
Victor Vieux
9c446a4d00 Merge pull request #33518 from naveed-jamil-tenpearls/pkg/sysinfo
Increased Unit Test Coverage for PKG/SYSINFO
2017-06-13 10:57:49 -07:00
David Sheets
24264697c5 authz: remove and hide unused and local-only methods respectively
Signed-off-by: David Sheets <dsheets@docker.com>
2017-06-13 13:51:11 +01:00
David Sheets
7da3986297 authz: eliminate race during plugin removal from middleware
Also, this removes the use of a questionable golang range feature which
corrects for mutation of a slice during iteration over that slice. This
makes the filter operation easier to read and reason about.

Signed-off-by: David Sheets <dsheets@docker.com>
2017-06-13 13:51:07 +01:00
Naveed Jamil
cffa6d52a2 Added Test Case Coverage for PKG/PROMISE
Signed-off-by: Naveed Jamil <naveed.jamil@tenpearls.com>
2017-06-13 15:17:40 +05:00
Sebastiaan van Stijn
8183b9d12c Merge pull request #33374 from raja-sami-10p/pkg/plugins
Increase the Coverage of pkg/plugins
2017-06-13 02:50:54 +02:00
Brian Goff
2ae085f309 Merge pull request #33414 from darrenstahlmsft/IoTServerContainers
Check for Windows 10 IoT Core to use process isolation on IoT
2017-06-12 18:02:15 -05:00
Raja Sami
8dd100a229 Increase the Coverage of pkg/plugins
Increases the test coverage of pkg/plugins.
Changed signature of function NewClientWithTimeout in pkg/plugin/client, to
take time.Duration instead of integers.

Signed-off-by: Raja Sami <raja.sami@tenpearl.com>
2017-06-12 12:23:10 +05:00
unclejack
ba40f4593f pkg/pools: add buffer32KPool & use it for copy
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2017-06-10 21:12:48 +03:00