Commit graph

2462 commits

Author SHA1 Message Date
Tobias Klauser
01f70b028e Switch Stat syscalls to x/sys/unix
Switch some more usage of the Stat function and the Stat_t type from the
syscall package to golang.org/x/sys. Those were missing in PR #33399.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-27 10:09:02 +02:00
Sebastiaan van Stijn
f8c434366e Merge pull request #33987 from dnephin/cleanup-more-hack
Move test-unit out of hack/make
2017-07-27 00:29:13 +02:00
Victor Vieux
fa7db52e26 Merge pull request #34135 from tklauser/more-unix-fns
Replace manually written wrappers by functions from x/sys/unix
2017-07-18 20:43:13 +02:00
Kir Kolyshkin
7da12bcfa9 devmapper_wrapper.go: fix gcc warning
I am getting the following warning from gcc when compiling the daemon:

> # github.com/docker/docker/pkg/devicemapper
> pkg/devicemapper/devmapper_wrapper.go: In function ‘log_cb’:
> pkg/devicemapper/devmapper_wrapper.go:20:2: warning: ignoring return
> value of ‘vasprintf’, declared with attribute warn_unused_result
> [-Wunused-result]
>  vasprintf(&buffer, f, ap);
>  ^

vasprintf(3) man page says if the function returns -1, the buffer is
undefined, so we should not use it. In practice, I assume, this never
happens so we just return.

Introduced by https://github.com/moby/moby/pull/33845 that resulted in
commit 63328c6 ("devicemapper: remove 256 character limit of libdm logs")

Cc: Aleksa Sarai <asarai@suse.de>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-07-17 17:46:44 -07:00
Evan Hazlett
458f6712d4 Merge pull request #33886 from aaronlehmann/names-in-memdb
Store container names in memdb
2017-07-17 14:16:41 -04:00
Daniel Nephin
1fb615599a Remove test-unit from hack/make
Also remove the test flag from pkg/term and jsut checkuid directly.
Fixed a problem with a pkg/term test that was leaving the terminal in a bad
state.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-07-17 11:39:33 -04:00
Tobias Klauser
bedf09363c loopback: use IoctlGetInt/IoctlSetInt from x/sys/unix
Use IoctlGetInt/IoctlSetInt from golang.org/x/sys/unix (where
applicable) instead of manually reimplementing them.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-17 10:37:42 +02:00
Tobias Klauser
6476504695 [pkg/term] use IoctlGetTermios/IoctlSetTermios from x/sys/unix
Use IoctlGetTermios/IoctlSetTermios from golang.org/x/sys/unix instead
of manually reimplementing them.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-17 10:37:42 +02:00
Tobias Klauser
6c9d715a8c sysinfo: use Prctl() from x/sys/unix
Use unix.Prctl() instead of manually reimplementing it using
unix.RawSyscall. Also use unix.SECCOMP_MODE_FILTER instead of locally
defining it.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-17 10:37:42 +02:00
Vincent Demeester
97e730ae57 Merge pull request #34106 from tonistiigi/fix-tar-prefix-header
Fix tar prefix header
2017-07-17 09:40:39 +02:00
Tonis Tiigi
4a3cfda45e archive: add test for prefix header
With docker-17.06.0 some images pulled do not extract properly. Some files don't appear in correct directories. This may or may not cause the pull to fail. These images can't be pushed or saved. 17.06 is the first version of Docker built with go1.8.

Cause

There are multiple updates to the tar package in go1.8.

https://go-review.googlesource.com/c/32234/ disables using "prefix" field when new tar archives are being written. Prefix field was previously set when a record in the archive used a path longer than 100 bytes.

Another change https://go-review.googlesource.com/c/31444/ makes the reader ignore the "prefix" field value if the record is in GNU format. GNU format defines that same area should be used for access and modified times. If the "prefix" field is not read, a file will only be extracted by the basename.

The problem is that with a previous version of the golang archive package headers could be written, that use the prefix field while at the same time setting the header format to GNU. This happens when numeric fields are big enough that they can not be written as octal strings and need to be written in binary. Usually, this shouldn't happen: uid, gid, devmajor, devminor can use up to 7 bytes, size and timestamp can use 11. If one of the records does overflow it switches the whole writer to GNU mode and all next files will be saved in GNU format.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-14 10:20:48 -07:00
Brian Goff
f7ce35f47b Merge pull request #34077 from justincormack/mount_remount_opts
In the case of remounting with changed data, need to call mount
2017-07-14 09:58:30 -04:00
Aaron Lehmann
1128fc1add Store container names in memdb
Currently, names are maintained by a separate system called "registrar".
This means there is no way to atomically snapshot the state of
containers and the names associated with them.

We can add this atomicity and simplify the code by storing name
associations in the memdb. This removes the need for pkg/registrar, and
makes snapshots a lot less expensive because they no longer need to copy
all the names. This change also avoids some problematic behavior from
pkg/registrar where it returns slices which may be modified later on.

Note that while this change makes the *snapshotting* atomic, it doesn't
yet do anything to make sure containers are named at the same time that
they are added to the database. We can do that by adding a transactional
interface, either as a followup, or as part of this PR.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-13 12:35:00 -07:00
Justin Cormack
3a1ab5b479 In the case of remounting with changed data, need to call mount
The case where we are trying to do a remount with changed filesystem specific options was missing,
we need to call `mount` as well here to change those options.

See #33844 for where we need this, as we change `tmpfs` options.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-13 15:48:58 +01:00
Sebastiaan van Stijn
96bf279c7d Merge pull request #33539 from tklauser/x-sys-unix-symlink-xattrs
Use symlink xattr functions from x/sys/unix
2017-07-12 18:04:33 -07:00
Sebastiaan van Stijn
00b218216d Merge pull request #33845 from cyphar/devicemapper-show-me-your-logs
devicemapper: rework logging and add --storage-opt dm.libdm_log_level
2017-07-12 17:46:14 -07:00
Sebastiaan van Stijn
e04dbe5ac2 Merge pull request #33877 from rhvgoyal/sync-removal
devicemapper: Wait for device removal if deferredRemoval=true and deferredDeletion=…
2017-07-12 17:35:45 -07:00
Tobias Klauser
4d966409bc system: Use symlink xattr functions from x/sys/unix
Use the symlink xattr syscall wrappers Lgetxattr and Lsetxattr from
x/sys/unix (introduced in golang/sys@b90f89a) instead of providing own
wrappers. Leave the functionality of system.Lgetxattr intact with
respect to the retry with a larger buffer, but switch it to use
unix.Lgetxattr. Also leave system.Lsetxattr intact (even though it's
just a wrapper around the corresponding function from unix) in order to
keep moby building for !linux.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-12 14:52:42 +02:00
Tobias Klauser
ef5252fc5d [pkg/term] temporarily use STD_*_HANDLE from syscall again
Due to the CL https://go-review.googlesource.com/c/39608/ in
x/sys/windows which changed the definitions of STD_INPUT_HANDLE,
STD_OUTPUT_HANDLE and STD_ERROR_HANDLE, we get the following failure
after re-vendoring x/sys/windows:

  07:47:01 # github.com/docker/docker/pkg/term
  07:47:01 pkg/term/term_windows.go:82: constant 4294967286 overflows int
  07:47:01 pkg/term/term_windows.go:88: constant 4294967285 overflows int
  07:47:01 pkg/term/term_windows.go:94: constant 4294967284 overflows int
  07:47:12 Build step 'Execute shell' marked build as failure

Temporarily switch back pkg/term to use these constants from the syscall
package and add a comment about it.

To really fix this, go-ansiterm should probably be switched to use
x/sys/windows.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-12 14:52:42 +02:00
Christopher Jones
069fdc8a08
[project] change syscall to /x/sys/unix|windows
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>

[s390x] switch utsname from unsigned to signed

per 33267e036f
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2017-07-11 08:00:32 -04:00
Sebastiaan van Stijn
9789530b46 Merge pull request #33956 from keloyang/fix-cp-with-hardlink
Fix docker cp dir with hard link
2017-07-10 14:11:26 -07:00
Justin Cormack
b7bd959294 Split homedir files by operating system
libcontainer/user does not build at all on Windows any more, and
this was breaking the client on Windows with upstream `runc`. As
these functions are not used anyway, just split out and stop
checking `runtime`.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-10 14:10:43 +01:00
Yong Tang
05c7c31139 Merge pull request #33575 from cpuguy83/plugins_on_swarm
Plugins on swarm
2017-07-07 02:23:07 -07:00
Tibor Vass
0fcd082d88 Merge pull request #33935 from AkihiroSuda/pkg-archive-go1.9
pkg/archive.FileInfoHeader: fill file type bits
2017-07-06 14:22:45 -07:00
Brian Goff
72c3bcf2a5 Make plugin emit strongly typed, consumable events
Enables other subsystems to watch actions for a plugin(s).

This will be used specifically for implementing plugins on swarm where a
swarm controller needs to watch the state of a plugin.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-07-06 14:26:06 -04:00
yangshukui
d58ffa0364 Fix docker cp dir with hard link
docker run --name=test ubuntu /bin/sh -c "cd /tmp && echo hi > a && ln a b" && docker cp test:/tmp tmp_
test
link /root/tmp/a /root/tmp_/b: no such file or directory

Signed-off-by: yangshukui <yangshukui@huawei.com>
2017-07-06 17:03:38 +08:00
Akihiro Suda
1a451d9a7b pkg/archive.FileInfoHeader: fill file type bits
Go 1.9 (golang/go@66b5a2f) removed file type bits from
archive/tar.FileInfoHeader().

This commit ensures file type bits are filled even on Go 1.9 for
compatibility.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-07-05 05:24:32 +00:00
Josh Soref
39bcaee47b
Spelling fixes
* additional
* ambiguous
* anonymous
* anything
* application
* because
* before
* building
* capabilities
* circumstances
* commit
* committer
* compresses
* concatenated
* config
* container
* container's
* current
* definition
* delimiter
* disassociates
* discovery
* distributed
* doesnotexist
* downloads
* duplicates
* either
* enhancing
* enumerate
* escapable
* exactly
* expect
* expectations
* expected
* explicitly
* false
* filesystem
* following
* forbidden
* git with
* healthcheck
* ignore
* independent
* inheritance
* investigating
* irrelevant
* it
* logging
* looking
* membership
* mimic
* minimum
* modify
* mountpoint
* multiline
* notifier
* outputting
* outside
* overridden
* override
* parsable
* plugins
* precedence
* propagation
* provided
* provides
* registries
* repositories
* returning
* settings
* should
* signals
* someone
* something
* specifically
* successfully
* synchronize
* they've
* thinking
* uninitialized
* unintentionally
* unmarshaling
* unnamed
* unreferenced
* verify

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-03 13:13:09 -07:00
Josh Soref
6284cf5e9a
gofmt: 5 files
gofmt from go1.8.3

hg locate '*.go' |xargs ~/go/bin/gofmt -s -w

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-03 10:32:47 -07:00
Aleksa Sarai
cfd39e8d6d
devicemapper: change LogInit and move all callbacks to pkg
LogInit used to act as a manual way of registering the *necessary*
pkg/devicemapper logging callbacks. In addition, it was used to split up
the logic of pkg/devicemapper into daemon/graphdriver/devmapper (such
that some things were logged from libdm).

The manual aspect of this API was completely non-sensical and was just
begging for incorrect usage of pkg/devicemapper, so remove that semantic
and always register our own libdm callbacks.

In addition, recombine the split out logging callbacks into
pkg/devicemapper so that the default logger is local to the library and
also shown to be the recommended logger. This makes the code
substantially easier to read. Also the new DefaultLogger now has
configurable upper-bound for the log level, which allows for dynamically
changing the logging level.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2017-07-04 02:04:26 +10:00
Aleksa Sarai
c654147f06
devicemapper: actually remove DmLogInitVerbose
e07d3cd9a ("devmapper: Fix libdm logging") removed all of the callers of
DmLogInitVerbose, but we still kept around the wrapper. However, the
libdm dm_log_init_verbose API changes the verbosity of the *default*
libdm logger. Because pkg/devicemapper internally *relies* on using
logging callbacks to understand what errors were encountered by libdm,
this wrapper is useless (it only makes sense for the default logger
which we do not user).

Any user not inside Docker of this function almost certainly was not
using this API correctly, because pkg/devicemapper will misbehave if our
logging callbacks were not registered.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2017-07-04 02:04:26 +10:00
Aleksa Sarai
63328c6882
devicemapper: remove 256 character limit of libdm logs
This limit is unecessary and can lead to the truncation of long libdm
logs (which is quite annoying).

Fixes: b440ec013 ("device-mapper: Move all devicemapper spew to log through utils.Debugf().")
Signed-off-by: Aleksa Sarai <asarai@suse.de>
2017-07-04 02:04:26 +10:00
Vivek Goyal
36cb6efebc Wait for device removal if deferredRemoval=true and deferredDeletion=false
There have been some cases where umount, a device can be busy for a very
short duration. Maybe its udev rules, or maybe it is runc related races
or probably it is something else. We don't know yet.

If deferred removal is enabled but deferred deletion is not, then for the
case of "docker run -ti --rm fedora bash", a container will exit, device
will be deferred removed and then immediately a call will come to delete
the device. It is possible that deletion will fail if device was busy
at that time.

A device can't be deleted if it can't be removed/deactivated first. There
is only one exception and that is when deferred deletion is on. In that
case graph driver will keep track of deleted device and try to delete it
later and return success to caller.

Always make sure that device deactivation is synchronous when device is
being deleted (except the case when deferred deletion is enabled).

This should also take care of small races when device is busy for a short
duration and it is being deleted.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2017-06-30 14:27:26 -04:00
Victor Vieux
48e625ccca Merge pull request #33690 from raja-sami-10p/pkg/authorization
Increase test Coverage of pkg/authorization
2017-06-27 10:09:13 -07:00
Sebastiaan van Stijn
d3d1aabcc6
Move IsGitTransport() to gitutils
This function was only used inside gitutils,
and is written specifically for the requirements
there.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-06-26 10:07:04 -07:00
Sebastiaan van Stijn
4964b09238 Merge pull request #33674 from raja-sami-10p/pkg/templates
Increase test Coverage of pkg/templates
2017-06-24 01:04:24 -07:00
Raja Sami
27e771b714 Increase test Coverage of pkg/templates
Signed-off-by: Raja Sami <raja.sami@tenpearls.com>
2017-06-23 12:38:33 +05:00
Raja Sami
f1eb0c0ebb Increase test Coverage of pkg/authorization
Signed-off-by: Raja Sami <raja.sami@tenpearls.com>
2017-06-23 12:35:46 +05:00
Brian Goff
db57b85fcb Merge pull request #33653 from raja-sami-10p/pkg/streamformatter
Increase the Coverage of pkg/streamformatter
2017-06-22 19:52:02 -07:00
Tonis Tiigi
5c3d2d552b Implement incremental file sync using client session
Also exposes shared cache and garbage collection/prune
for the source data.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-22 11:52:35 -07:00
Raja Sami
956c5b61ce Increase the Coverage of pkg/streamformatter
Signed-off-by: Raja Sami <raja.sami@tenpearls.com>
2017-06-22 12:29:10 +05:00
John Stephens
930e689668 Merge pull request #33241 from Microsoft/jjh/multi-layerstore
LCOW: Support most operations excluding remote filesystem
2017-06-21 15:45:23 -07:00
John Howard
ed10ac6ee9 LCOW: Create layer folders with correct ACL
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:50:12 -07:00
John Howard
f154588226 LCOW: OCI Spec and Environment for container start
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:50:11 -07:00
John Howard
ed4d2aa981 LCOW: Graphdriver
Signed-off-by: John Howard <jhoward@microsoft.com>

Adds the graphdriver for Linux containers on Windows
2017-06-20 19:49:24 -07:00
Brian Goff
b5f68d7ed3 Merge pull request #33097 from stevvooe/remove-random-pkg
pkg: remove random package
2017-06-20 13:43:38 -07:00
John Howard
fe5b34ba88 LCOW: Add environment variable to enable
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 08:55:46 -07:00
Vincent Demeester
99c72eb268 Merge pull request #33454 from dnephin/refactor-builder-remove-copy-on-build
[Builder] Move file coping from the daemon to the builder
2017-06-20 10:12:57 +02:00
Akihiro Suda
93c06b8013 Merge pull request #33725 from ripcurld0/deprecate_go16
Remove Go 1.6 code from moby
2017-06-20 08:54:01 +03:00
Sebastiaan van Stijn
16380b3ee6 Merge pull request #33522 from naveed-jamil-tenpearls/pkg/promise
Added Test Case Coverage for PKG/PROMISE
2017-06-19 13:53:42 -07:00