Commit graph

5280 commits

Author SHA1 Message Date
Flavio Crisciani
f9f25ca5e4
Allow to set the control plane MTU
Add daemon config to allow the user to specify the MTU of the control plane network.
The first user of this new parameter is actually libnetwork that can seed the
gossip with the proper MTU value allowing to pack multiple messages per UDP packet sent.
If the value is not specified or is lower than 1500 the logic will set it to the default.

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2017-07-28 13:52:03 -07:00
John Stephens
e9cd2fef80 Merge pull request #34192 from swernli/fixBuilderCopy
Fixing releaseableLayer handling of layer streams and mounts.
2017-07-27 16:15:11 -07:00
Tobias Klauser
01f70b028e Switch Stat syscalls to x/sys/unix
Switch some more usage of the Stat function and the Stat_t type from the
syscall package to golang.org/x/sys. Those were missing in PR #33399.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-27 10:09:02 +02:00
Stefan Wernli
1d457999c4 Fixing releaseableLayer handling of layer streams and mounts.
releaseableLayer includes automatic handling for creating a read/write layer and mounting it on a call to Mount(), but then does not correspondingly unmount the layer before trying to delete it, which will fail for some graphdrivers. Commit on a releaseable layer also leaks the tarstream for the layer. To fix this, the stream close is deferred in Commit and releaseRWLayer now correctly handles unmounting the layer before trying to delete it.  In addition, the changes include better error handling in Release() to make sure that errors are returned to the caller for failures on read/write layers instead of being ignored.# Please enter the commit message for your changes. Lines starting

Signed-off-by: Stefan Wernli <swernli@ntdev.microsoft.com>
2017-07-26 14:26:56 -07:00
Brian Goff
9319a8a2dd Merge pull request #33440 from RenaudWasTaken/genericresource
Added support for Generic Resources
2017-07-25 15:32:25 -04:00
Renaud Gaubert
87e1464c43 Added support for Generic Resources
Signed-off-by: Renaud Gaubert <rgaubert@nvidia.com>
2017-07-24 17:49:56 -07:00
Aaron Lehmann
bd4f66c8f1 cluster: Avoid recursive RLock
GetTasks can call GetService and GetNode with the read lock held. These
methods try to aquire the read side of the same lock. According to the
sync package documentation, this is not safe:

> If a goroutine holds a RWMutex for reading, it must not expect this or
> any other goroutine to be able to also take the read lock until the
> first read lock is released. In particular, this prohibits recursive
> read locking. This is to ensure that the lock eventually becomes
> available; a blocked Lock call excludes new readers from acquiring the
> lock.

Fix GetTasks to use the lower-level getService and getNode methods
instead. Also, use lockedManagerAction to simplify GetTasks.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-24 10:52:44 -07:00
Sebastiaan van Stijn
901fe35bd3 Merge pull request #34174 from aaronlehmann/logattributes
Avoid using a map for log attributes
2017-07-21 22:31:51 +02:00
Sebastiaan van Stijn
67eeb0490d Merge pull request #33960 from cpuguy83/ignore_not_exist_err
Fix error handling with not-exist errors on remove
2017-07-21 13:52:27 +02:00
Brian Goff
d42dbdd3d4 Fix error handling with not-exist errors on remove
Specifically, none of the graphdrivers are supposed to return a
not-exist type of error on remove (or at least that's how they are
currently handled).

Found that AUFS still had one case where a not-exist error could escape,
when checking if the directory is mounted we call a `Statfs` on the
path.

This fixes AUFS to not return an error in this case, but also
double-checks at the daemon level on layer remove that the error is not
a `not-exist` type of error.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-07-20 12:26:27 -04:00
Jérôme Petazzoni
84aefe8697 Add a log message when the storage driver is overriden through the environment
Signed-off-by: Jérôme Petazzoni <jerome.petazzoni@gmail.com>
2017-07-20 17:38:34 +02:00
Aaron Lehmann
b642b3f21f Avoid using a map for log attributes
Having a map per log entry seemed heavier than necessary. These
attributes end up being sorted and serialized, so storing them in a map
doesn't add anything (there's no random access element). In SwarmKit,
they originate as a slice, so there's an unnecessary conversion to a map
and back.

This also fixes the sort comparator, which used to inefficiently split
the string on each comparison.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-18 19:01:20 -07:00
Brian Goff
08f7cf0526 Merge pull request #34123 from twistlock/plugable_secrets_backend
pluggable secret backend
2017-07-17 19:00:30 -04:00
Evan Hazlett
458f6712d4 Merge pull request #33886 from aaronlehmann/names-in-memdb
Store container names in memdb
2017-07-17 14:16:41 -04:00
tim
cb972b1515 Replaces fluentd-address string by constant
Signed-off-by: tim <tim@fewagainstmany.com>
2017-07-16 23:06:57 -07:00
Liron Levin
7d45cafd57 pluggable secret backend
This commit extends SwarmKit secret management with pluggable secret
backends support.

Updating the work in
[swarmkit](docker/swarmkit@eebac27434) for
pluggable secret backend and adding the
driver parameter to `SecretSpec`.

Remaining work:
- [ ] CLI support (docker/cli)
- [ ] api in [plugin helpers](docker/go-plugins-helpers))
- [ ] Reference plugin
- [ ] Documenation (after cli work)

Signed-off-by: Liron Levin <liron@twistlock.com>
2017-07-15 16:50:02 +03:00
Sebastiaan van Stijn
ce0ef9568d Merge pull request #34076 from yastij/33679-renaming-id-meth
renaming id() to state clearly that it's related to the networkAttachment
2017-07-14 00:29:09 -07:00
Sebastiaan van Stijn
f22cecf929 Merge pull request #33969 from Microsoft/jjh/lifetime
LCOW: Service VM lifetime changes
2017-07-13 15:51:21 -07:00
Aaron Lehmann
1128fc1add Store container names in memdb
Currently, names are maintained by a separate system called "registrar".
This means there is no way to atomically snapshot the state of
containers and the names associated with them.

We can add this atomicity and simplify the code by storing name
associations in the memdb. This removes the need for pkg/registrar, and
makes snapshots a lot less expensive because they no longer need to copy
all the names. This change also avoids some problematic behavior from
pkg/registrar where it returns slices which may be modified later on.

Note that while this change makes the *snapshotting* atomic, it doesn't
yet do anything to make sure containers are named at the same time that
they are added to the database. We can do that by adding a transactional
interface, either as a followup, or as part of this PR.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-13 12:35:00 -07:00
John Howard
76b0d4819a LCOW: Service VM lifetime changes
Signed-off-by: John Howard <jhoward@microsoft.com>

This changes the LCOW driver to support both global SVM lifetime and
per-instance lifetime. It also corrects the scratch implementation.
2017-07-13 11:09:43 -07:00
Sebastiaan van Stijn
53a75ee050 Merge pull request #32881 from coolljt0725/fix_exec_faild
Enable inspect exec if container is pause/restarting/non-running
2017-07-12 19:29:27 -07:00
Lei Jitang
6fdb2fb069 Merge pull request #34027 from cpuguy83/15853_allow_stopping_paused_container
Allow stopping of paused container
2017-07-13 10:16:46 +08:00
Sebastiaan van Stijn
00b218216d Merge pull request #33845 from cyphar/devicemapper-show-me-your-logs
devicemapper: rework logging and add --storage-opt dm.libdm_log_level
2017-07-12 17:46:14 -07:00
Sebastiaan van Stijn
e04dbe5ac2 Merge pull request #33877 from rhvgoyal/sync-removal
devicemapper: Wait for device removal if deferredRemoval=true and deferredDeletion=…
2017-07-12 17:35:45 -07:00
Brian Goff
c3feb046b9 Allow stopping of paused container
When a container is paused, signals are sent once the container has been
unpaused.
Instead of forcing the user to unpause a container before they can ever
send a signal, allow the user to send the signals, and in the case of a
stop signal, automatically unpause the container afterwards.

This is much safer than unpausing the container first then sending a
signal (what a user is currently forced to do), as the container may be
paused for very good reasons and should not be unpaused except for
stopping.
Note that not even SIGKILL is possible while a process is paused,
but it is killed the instant it is unpaused.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-07-12 10:35:48 -04:00
Yassine TIJANI
27c0131a43 renaming id() to state clearly that it's related to the networkAttachement container target
Signed-off-by: Yassine TIJANI <yasstij11@gmail.com>
2017-07-12 15:58:31 +02:00
Sebastiaan van Stijn
c0e6da7637 Merge pull request #34061 from cyli/re-vendor-swarmkit
Re-vendor swarmkit.
2017-07-11 20:30:56 -07:00
Sebastiaan van Stijn
334702ab1f Merge pull request #34032 from allencloud/support-config-event-in-cluster
make engine support cluster config event
2017-07-11 20:25:19 -07:00
Tibor Vass
c8a2596d67 Merge pull request #34004 from yummypeng/fix-docker-stats-hang
Return an empty stats if "container not found"
2017-07-11 20:10:17 -07:00
Ying Li
4509a001df Re-vendor swarmkit. This includes the following fixes:
- https://github.com/docker/swarmkit/pull/2266 (support for templating Node.Hostname in docker executor)
- https://github.com/docker/swarmkit/pull/2281 (change restore action on objects to be update, not delete/create)
- https://github.com/docker/swarmkit/pull/2285 (extend watch queue with timeout and size limit)
- https://github.com/docker/swarmkit/pull/2253 (version-aware failure tracking in the scheduler)
- https://github.com/docker/swarmkit/pull/2275 (update containerd and port executor to container client library)
- https://github.com/docker/swarmkit/pull/2292 (rename some generic resources)
- https://github.com/docker/swarmkit/pull/2300 (limit the size of the external CA response)
- https://github.com/docker/swarmkit/pull/2301 (delete global tasks when the node running them is deleted)

Minor cleanups, dependency bumps, and vendoring:
- https://github.com/docker/swarmkit/pull/2271
- https://github.com/docker/swarmkit/pull/2279
- https://github.com/docker/swarmkit/pull/2283
- https://github.com/docker/swarmkit/pull/2282
- https://github.com/docker/swarmkit/pull/2274
- https://github.com/docker/swarmkit/pull/2296 (dependency bump of etcd, go-winio)

Signed-off-by: Ying Li <ying.li@docker.com>
2017-07-11 13:43:43 -07:00
Christopher Jones
069fdc8a08
[project] change syscall to /x/sys/unix|windows
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>

[s390x] switch utsname from unsigned to signed

per 33267e036f
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2017-07-11 08:00:32 -04:00
allencloud
ed916a233c make engine support cluster config event
Signed-off-by: allencloud <allen.sun@daocloud.io>
2017-07-11 17:21:15 +08:00
Brian Goff
73e8f56d6a Merge pull request #33979 from allencloud/return-prune-data-when-context-canceled
return prune data when context canceled
2017-07-10 21:17:18 -04:00
Vincent Demeester
e56e78976e Merge pull request #33377 from yastij/33371-matchevent-filtering
matching events using ID and name
2017-07-10 16:11:14 +02:00
Yuanhong Peng
4a6cbf9bcb Return an empty stats if "container not found"
If we get "container not found" error from containerd, it's possibly
because that this container has already been stopped. It will be ok to
ignore this error and just return an empty stats.

Signed-off-by: Yuanhong Peng <pengyuanhong@huawei.com>
2017-07-10 16:30:48 +08:00
allencloud
87b4dc2002 return prune data when context canceled
Signed-off-by: allencloud <allen.sun@daocloud.io>
2017-07-10 10:06:24 +08:00
Yassine TIJANI
be9aa09246 removing TODO, and explain why we can't filter by id on the matchevent method.
Signed-off-by: Yassine TIJANI <yasstij11@gmail.com>
2017-07-07 23:32:37 +02:00
Yong Tang
05c7c31139 Merge pull request #33575 from cpuguy83/plugins_on_swarm
Plugins on swarm
2017-07-07 02:23:07 -07:00
Yong Tang
9aecbbf9bf Merge pull request #33662 from madhanrm/sharedendpoint
Include Endpoint List for Shared Endpoints
2017-07-06 23:44:23 -07:00
Akihiro Suda
8e3555f853 Merge pull request #33940 from yaocaigen/fix_container_multi_names
Fix run `docker rename <container-id> new_name` concurrently, the container will have multi names
2017-07-07 10:58:01 +09:00
Aaron Lehmann
3be2273d03 Merge pull request #33713 from x1022as/error_fd
fd leak and error handling
2017-07-06 16:47:34 -07:00
Madhan Raj Mookkandy
349913ce9f Include Endpoint List for Shared Endpoints
Do not allow sharing of container network with hyperv containers

Signed-off-by: Madhan Raj Mookkandy <madhanm@microsoft.com>
2017-07-06 12:19:17 -07:00
Brian Goff
72c3bcf2a5 Make plugin emit strongly typed, consumable events
Enables other subsystems to watch actions for a plugin(s).

This will be used specifically for implementing plugins on swarm where a
swarm controller needs to watch the state of a plugin.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-07-06 14:26:06 -04:00
Sebastiaan van Stijn
db8c2651b8 Merge pull request #33897 from cpuguy83/jsonfile_log_blocking
Fix log readers can block writes indefinitely
2017-07-05 18:21:14 -07:00
Tibor Vass
490b55e8fa Merge pull request #33898 from crosbymichael/swappiness
Do not set -1 for swappiness
2017-07-05 17:04:08 -07:00
Brian Goff
e2209185ed Fix log readers can block writes indefinitely
Before this patch, a log reader is able to block all log writes
indefinitely (and other operations) by simply opening the log stream and
not consuming all the messages.

The reason for this is we protect the read stream from corruption by
ensuring there are no new writes while the log stream is consumed (and
caught up with the live entries).

We can get around this issue because log files are append only, so we
can limit reads to only the section of the file that was written to when
the log stream was first requested.

Now logs are only blocked until all files are opened, rather than
streamed to the client.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-07-05 14:04:52 -04:00
Yang Pengfei
cc2340689c Fix run docker rename <container-id> new_name concurrently, the container will have multi names
When run `docker rename <container-id> new_name` concurrently, every operation will release
container's old name. So container will have multi new names reserve in nameIndex.

Signed-off-by: Yang Pengfei <yangpengfei4@huawei.com>
2017-07-05 16:56:31 +08:00
Vincent Demeester
ff4f700f74 Merge pull request #33322 from jsoref/spelling
Spelling
2017-07-04 15:46:34 +02:00
Brian Goff
c2ad549533 Merge pull request #33213 from gkze/master
GCP Logs: send log text as jsonPayload.message instead of jsonPayload.data
2017-07-03 16:18:18 -04:00
Josh Soref
39bcaee47b
Spelling fixes
* additional
* ambiguous
* anonymous
* anything
* application
* because
* before
* building
* capabilities
* circumstances
* commit
* committer
* compresses
* concatenated
* config
* container
* container's
* current
* definition
* delimiter
* disassociates
* discovery
* distributed
* doesnotexist
* downloads
* duplicates
* either
* enhancing
* enumerate
* escapable
* exactly
* expect
* expectations
* expected
* explicitly
* false
* filesystem
* following
* forbidden
* git with
* healthcheck
* ignore
* independent
* inheritance
* investigating
* irrelevant
* it
* logging
* looking
* membership
* mimic
* minimum
* modify
* mountpoint
* multiline
* notifier
* outputting
* outside
* overridden
* override
* parsable
* plugins
* precedence
* propagation
* provided
* provides
* registries
* repositories
* returning
* settings
* should
* signals
* someone
* something
* specifically
* successfully
* synchronize
* they've
* thinking
* uninitialized
* unintentionally
* unmarshaling
* unnamed
* unreferenced
* verify

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-03 13:13:09 -07:00