Commit graph

33283 commits

Author SHA1 Message Date
Liron Levin
7d45cafd57 pluggable secret backend
This commit extends SwarmKit secret management with pluggable secret
backends support.

Updating the work in
[swarmkit](docker/swarmkit@eebac27434) for
pluggable secret backend and adding the
driver parameter to `SecretSpec`.

Remaining work:
- [ ] CLI support (docker/cli)
- [ ] api in [plugin helpers](docker/go-plugins-helpers))
- [ ] Reference plugin
- [ ] Documenation (after cli work)

Signed-off-by: Liron Levin <liron@twistlock.com>
2017-07-15 16:50:02 +03:00
Sebastiaan van Stijn
d0a8e73e7b
Service privileges: API docs
This documents the Service privileges
API changes, that were added in:
091b5e68ea

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-14 17:04:26 -07:00
Brian Goff
15a538a627 Testing: Use local plugins, not from hub
Use the (new) plugin fixtures for plugin tests rather than pulling
plugins from hub.

This removes the restriction for platforms/archs since plugin binaries
get built in the test environment.

Future work would be to add test plugins for the various subsystems so
tests that are actually using plugins (e.g. volumes, networks) can be
ported to use the fixtures as well.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-07-14 13:24:14 -04:00
Tonis Tiigi
4a3cfda45e archive: add test for prefix header
With docker-17.06.0 some images pulled do not extract properly. Some files don't appear in correct directories. This may or may not cause the pull to fail. These images can't be pushed or saved. 17.06 is the first version of Docker built with go1.8.

Cause

There are multiple updates to the tar package in go1.8.

https://go-review.googlesource.com/c/32234/ disables using "prefix" field when new tar archives are being written. Prefix field was previously set when a record in the archive used a path longer than 100 bytes.

Another change https://go-review.googlesource.com/c/31444/ makes the reader ignore the "prefix" field value if the record is in GNU format. GNU format defines that same area should be used for access and modified times. If the "prefix" field is not read, a file will only be extracted by the basename.

The problem is that with a previous version of the golang archive package headers could be written, that use the prefix field while at the same time setting the header format to GNU. This happens when numeric fields are big enough that they can not be written as octal strings and need to be written in binary. Usually, this shouldn't happen: uid, gid, devmajor, devminor can use up to 7 bytes, size and timestamp can use 11. If one of the records does overflow it switches the whole writer to GNU mode and all next files will be saved in GNU format.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-14 10:20:48 -07:00
Brian Goff
f7ce35f47b Merge pull request #34077 from justincormack/mount_remount_opts
In the case of remounting with changed data, need to call mount
2017-07-14 09:58:30 -04:00
Sebastiaan van Stijn
ce0ef9568d Merge pull request #34076 from yastij/33679-renaming-id-meth
renaming id() to state clearly that it's related to the networkAttachment
2017-07-14 00:29:09 -07:00
Sebastiaan van Stijn
93494237df Merge pull request #34089 from tonistiigi/builder-report-2017-07-10
Add builder dev report for 2017-07-10
2017-07-13 23:25:57 -07:00
Akihiro Suda
1daa10301a Merge pull request #34063 from dnephin/fix-builder-no-such-layer
Fix multiple `COPY --from` in multistage builds
2017-07-14 12:19:19 +09:00
Tonis Tiigi
72df48d1ad vendor: add archive/tar
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-13 19:08:19 -07:00
Sebastiaan van Stijn
f22cecf929 Merge pull request #33969 from Microsoft/jjh/lifetime
LCOW: Service VM lifetime changes
2017-07-13 15:51:21 -07:00
Aaron Lehmann
0e57eb95c5 container: Use wrapper to ensure commit/abort happens
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-13 12:35:03 -07:00
Aaron Lehmann
bc3209bc15 container: Abort transactions when memdb calls fail
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-13 12:35:02 -07:00
Aaron Lehmann
1128fc1add Store container names in memdb
Currently, names are maintained by a separate system called "registrar".
This means there is no way to atomically snapshot the state of
containers and the names associated with them.

We can add this atomicity and simplify the code by storing name
associations in the memdb. This removes the need for pkg/registrar, and
makes snapshots a lot less expensive because they no longer need to copy
all the names. This change also avoids some problematic behavior from
pkg/registrar where it returns slices which may be modified later on.

Note that while this change makes the *snapshotting* atomic, it doesn't
yet do anything to make sure containers are named at the same time that
they are added to the database. We can do that by adding a transactional
interface, either as a followup, or as part of this PR.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-13 12:35:00 -07:00
John Howard
76b0d4819a LCOW: Service VM lifetime changes
Signed-off-by: John Howard <jhoward@microsoft.com>

This changes the LCOW driver to support both global SVM lifetime and
per-instance lifetime. It also corrects the scratch implementation.
2017-07-13 11:09:43 -07:00
Tonis Tiigi
2689f9cc0b Add builder dev report for 2017-07-10
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-13 09:30:13 -07:00
Justin Cormack
3a1ab5b479 In the case of remounting with changed data, need to call mount
The case where we are trying to do a remount with changed filesystem specific options was missing,
we need to call `mount` as well here to change those options.

See #33844 for where we need this, as we change `tmpfs` options.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-13 15:48:58 +01:00
Sebastiaan van Stijn
53a75ee050 Merge pull request #32881 from coolljt0725/fix_exec_faild
Enable inspect exec if container is pause/restarting/non-running
2017-07-12 19:29:27 -07:00
Lei Jitang
6fdb2fb069 Merge pull request #34027 from cpuguy83/15853_allow_stopping_paused_container
Allow stopping of paused container
2017-07-13 10:16:46 +08:00
Sebastiaan van Stijn
96bf279c7d Merge pull request #33539 from tklauser/x-sys-unix-symlink-xattrs
Use symlink xattr functions from x/sys/unix
2017-07-12 18:04:33 -07:00
Sebastiaan van Stijn
00b218216d Merge pull request #33845 from cyphar/devicemapper-show-me-your-logs
devicemapper: rework logging and add --storage-opt dm.libdm_log_level
2017-07-12 17:46:14 -07:00
Sebastiaan van Stijn
e04dbe5ac2 Merge pull request #33877 from rhvgoyal/sync-removal
devicemapper: Wait for device removal if deferredRemoval=true and deferredDeletion=…
2017-07-12 17:35:45 -07:00
Daniel Nephin
b50ade0bfb Fix multiple copy from
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-07-12 12:36:03 -04:00
Brian Goff
c3feb046b9 Allow stopping of paused container
When a container is paused, signals are sent once the container has been
unpaused.
Instead of forcing the user to unpause a container before they can ever
send a signal, allow the user to send the signals, and in the case of a
stop signal, automatically unpause the container afterwards.

This is much safer than unpausing the container first then sending a
signal (what a user is currently forced to do), as the container may be
paused for very good reasons and should not be unpaused except for
stopping.
Note that not even SIGKILL is possible while a process is paused,
but it is killed the instant it is unpaused.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-07-12 10:35:48 -04:00
Yassine TIJANI
27c0131a43 renaming id() to state clearly that it's related to the networkAttachement container target
Signed-off-by: Yassine TIJANI <yasstij11@gmail.com>
2017-07-12 15:58:31 +02:00
Victor Vieux
45cad73ea8 Merge pull request #33887 from thaJeztah/update-version-error
Fix NewVersionError() for clients using default version
2017-07-12 15:24:00 +02:00
Tobias Klauser
4d966409bc system: Use symlink xattr functions from x/sys/unix
Use the symlink xattr syscall wrappers Lgetxattr and Lsetxattr from
x/sys/unix (introduced in golang/sys@b90f89a) instead of providing own
wrappers. Leave the functionality of system.Lgetxattr intact with
respect to the retry with a larger buffer, but switch it to use
unix.Lgetxattr. Also leave system.Lsetxattr intact (even though it's
just a wrapper around the corresponding function from unix) in order to
keep moby building for !linux.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-12 14:52:42 +02:00
Tobias Klauser
ef5252fc5d [pkg/term] temporarily use STD_*_HANDLE from syscall again
Due to the CL https://go-review.googlesource.com/c/39608/ in
x/sys/windows which changed the definitions of STD_INPUT_HANDLE,
STD_OUTPUT_HANDLE and STD_ERROR_HANDLE, we get the following failure
after re-vendoring x/sys/windows:

  07:47:01 # github.com/docker/docker/pkg/term
  07:47:01 pkg/term/term_windows.go:82: constant 4294967286 overflows int
  07:47:01 pkg/term/term_windows.go:88: constant 4294967285 overflows int
  07:47:01 pkg/term/term_windows.go:94: constant 4294967284 overflows int
  07:47:12 Build step 'Execute shell' marked build as failure

Temporarily switch back pkg/term to use these constants from the syscall
package and add a comment about it.

To really fix this, go-ansiterm should probably be switched to use
x/sys/windows.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-12 14:52:42 +02:00
Tobias Klauser
e769da88e6 Windows: fix build after re-vendoring golang.org/x/sys
Due to the CL https://go-review.googlesource.com/c/39608/ in
x/sys/windows which changed the definitions of STD_INPUT_HANDLE,
STD_OUTPUT_HANDLE and STD_ERROR_HANDLE, we get the following failure
in cmd/dockerd/service_windows.go after re-vendoring x/sys/windows:

  06:29:57 # github.com/docker/docker/cmd/dockerd
  06:29:57 .\service_windows.go:400: cannot use sh (type int) as type uint32 in argument to windows.GetStdHandle

Fix it by adding an explicit type conversion when calling
windows.GetStdHandle.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-12 14:52:42 +02:00
Victor Vieux
94260ccb1d Merge pull request #34067 from thaJeztah/bump-image-spec
bump image-spec to 372ad780f63454fbbbbcc7cf80e5b90245c13e13
2017-07-12 12:19:39 +02:00
Fengtu Wang
977c4046fd Keep pause state when restoring container's status
Do not change pause state when restoring container's
status, or status in docker will be different with
status in runc.

Signed-off-by: Fengtu Wang <wangfengtu@huawei.com>
2017-07-12 16:25:17 +08:00
Tobias Klauser
bd13a5a9e0 vendor: re-vendor golang.org/x/sys
Update golang.org/x/sys to 739734461d1c916b6c72a63d7efda2b27edb369f in
order to get the Lgetxattr, Llistxattr, Lremovexattr and Lsetxattr
syscall wrappers. These will be used in a successive commit to replace
the wrappers in libcontainer/system/xattr_linux.go.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-12 08:26:51 +02:00
Sebastiaan van Stijn
9fc66ec869
bump image-spec to 372ad780f63454fbbbbcc7cf80e5b90245c13e13
this commit matches what's used in SwarmKit

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-11 20:52:17 -07:00
Sebastiaan van Stijn
c0e6da7637 Merge pull request #34061 from cyli/re-vendor-swarmkit
Re-vendor swarmkit.
2017-07-11 20:30:56 -07:00
Sebastiaan van Stijn
e5862d42b3 Merge pull request #34065 from allencloud/add-cluster-events-change
add cluster events change in version_history.md
2017-07-11 20:28:05 -07:00
Sebastiaan van Stijn
334702ab1f Merge pull request #34032 from allencloud/support-config-event-in-cluster
make engine support cluster config event
2017-07-11 20:25:19 -07:00
Tibor Vass
c8a2596d67 Merge pull request #34004 from yummypeng/fix-docker-stats-hang
Return an empty stats if "container not found"
2017-07-11 20:10:17 -07:00
allencloud
e9da15a660 add cluster events change in version_history.md
Signed-off-by: allencloud <allen.sun@daocloud.io>
2017-07-12 09:25:30 +08:00
allencloud
c8d6477e5a add config event in swagger.yml
Signed-off-by: allencloud <allen.sun@daocloud.io>
2017-07-12 08:48:14 +08:00
Phil Estes
54251b53d7 Merge pull request #33399 from tophj-ibm/move-all-of-pkg-to-syscall
[project] Switch most syscalls to golang.org/x/sys
2017-07-11 16:50:52 -04:00
Ying Li
4509a001df Re-vendor swarmkit. This includes the following fixes:
- https://github.com/docker/swarmkit/pull/2266 (support for templating Node.Hostname in docker executor)
- https://github.com/docker/swarmkit/pull/2281 (change restore action on objects to be update, not delete/create)
- https://github.com/docker/swarmkit/pull/2285 (extend watch queue with timeout and size limit)
- https://github.com/docker/swarmkit/pull/2253 (version-aware failure tracking in the scheduler)
- https://github.com/docker/swarmkit/pull/2275 (update containerd and port executor to container client library)
- https://github.com/docker/swarmkit/pull/2292 (rename some generic resources)
- https://github.com/docker/swarmkit/pull/2300 (limit the size of the external CA response)
- https://github.com/docker/swarmkit/pull/2301 (delete global tasks when the node running them is deleted)

Minor cleanups, dependency bumps, and vendoring:
- https://github.com/docker/swarmkit/pull/2271
- https://github.com/docker/swarmkit/pull/2279
- https://github.com/docker/swarmkit/pull/2283
- https://github.com/docker/swarmkit/pull/2282
- https://github.com/docker/swarmkit/pull/2274
- https://github.com/docker/swarmkit/pull/2296 (dependency bump of etcd, go-winio)

Signed-off-by: Ying Li <ying.li@docker.com>
2017-07-11 13:43:43 -07:00
Christopher Jones
069fdc8a08
[project] change syscall to /x/sys/unix|windows
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>

[s390x] switch utsname from unsigned to signed

per 33267e036f
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2017-07-11 08:00:32 -04:00
allencloud
ed916a233c make engine support cluster config event
Signed-off-by: allencloud <allen.sun@daocloud.io>
2017-07-11 17:21:15 +08:00
Victor Vieux
6978a6e25a Merge pull request #34035 from allencloud/add-cluster-event-details-in-swagger
add cluster events details in swagger.yml
2017-07-11 11:04:27 +02:00
Victor Vieux
177d281b5c Merge pull request #34047 from thaJeztah/revendor-dependencies
Bump vndr to 9909bb2b8a0b7ea464527b376dc50389c90df587
2017-07-11 11:04:05 +02:00
Sebastiaan van Stijn
031fd77702 Merge pull request #33816 from albers/enable-completion
Enable bash completion in development container
2017-07-10 21:39:43 -07:00
Sebastiaan van Stijn
5892a41c58 Merge pull request #34050 from thaJeztah/fix-api-version-docs
Fix api-version history
2017-07-10 21:21:57 -07:00
Sebastiaan van Stijn
dd5e818fab
Fix api-version history
Commit c79c16910c
inadvertently put these API changes under API 1.31,
but they were added in API 1.30.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-10 21:15:02 -07:00
allencloud
f596fb7683 add cluster events details in swagger.yml
Signed-off-by: allencloud <allen.sun@daocloud.io>
2017-07-11 11:54:22 +08:00
Brian Goff
73e8f56d6a Merge pull request #33979 from allencloud/return-prune-data-when-context-canceled
return prune data when context canceled
2017-07-10 21:17:18 -04:00
Sebastiaan van Stijn
9789530b46 Merge pull request #33956 from keloyang/fix-cp-with-hardlink
Fix docker cp dir with hard link
2017-07-10 14:11:26 -07:00