Commit graph

288 commits

Author SHA1 Message Date
Ian Campbell
9b001c4f5f Correct CPU usage calculation in presence of offline CPUs and newer Linux
In https://github.com/torvalds/linux/commit/5ca3726 (released in v4.7-rc1) the
content of the `cpuacct.usage_percpu` file in sysfs was changed to include both
online and offline cpus. This broke the arithmetic in the stats helpers used by
`docker stats`, since it was using the length of the PerCPUUsage array as a
proxy for the number of online CPUs.

Add current number of online CPUs to types.StatsJSON and use it in the
calculation.

Keep a fallback to `len(v.CPUStats.CPUUsage.PercpuUsage)` so this code
continues to work when talking to an older daemon. An old client talking to a
new daemon will ignore the new field and behave as before.

Fixes #28941.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
(cherry picked from commit 115f91d757)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2017-03-13 16:16:05 -07:00
Brian Goff
96b0320d5c Add docker plugin upgrade
This allows a plugin to be upgraded without requiring to
uninstall/reinstall a plugin.
Since plugin resources (e.g. volumes) are tied to a plugin ID, this is
important to ensure resources aren't lost.

The plugin must be disabled while upgrading (errors out if enabled).
This does not add any convenience flags for automatically
disabling/re-enabling the plugin during before/after upgrade.

Since an upgrade may change requested permissions, the user is required
to accept permissions just like `docker plugin install`.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 03c6949739)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-02-03 19:14:56 -05:00
Daniel Nephin
732e19892d Move ConvertVolumes to composetransform package.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-01-25 15:42:06 -05:00
Tibor Vass
83b2782ea9 plugins: rename DeviceCreation to AllowAllDevices
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit f265727bdf)
Signed-off-by: Tibor Vass <tibor@docker.com>
2017-01-10 15:21:46 -08:00
Tonis Tiigi
38d914cc96 Implement content addressability for plugins
Move plugins to shared distribution stack with images.

Create immutable plugin config that matches schema2 requirements.

Ensure data being pushed is same as pulled/created.

Store distribution artifacts in a blobstore.

Run init layer setup for every plugin start.

Fix breakouts from unsafe file accesses.

Add support for `docker plugin install --alias`

Uses normalized references for default names to avoid collisions when using default hosts/tags.

Some refactoring of the plugin manager to support the change, like removing the singleton manager and adding manager config struct.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
(cherry picked from commit 3d86b0c79b)
2016-12-27 13:31:14 -08:00
Anusha Ragunathan
2039ea6adc Enforce zero plugin refcount during disable.
When plugins have a positive refcount, they were not allowed to be
removed. However, plugins could still be disabled when volumes
referenced it and containers using them were running.

This change fixes that by enforcing plugin refcount during disable.
A "force" disable option is also added to ignore reference refcounting.

Signed-off-by: Anusha Ragunathan <anusha@docker.com>
(cherry picked from commit 8cb2229cd1)
2016-12-27 13:19:25 -08:00
Doug Davis
64aac182d6 Fix processing of unset build-args during build
This reverts 26103.  26103 was trying to make it so that if someone did:
  docker build --build-arg FOO .
and FOO wasn't set as an env var then it would pick-up FOO from the
Dockerfile's ARG cmd.  However, it went too far and removed the ability
to specify a build arg w/o any value. Meaning it required the --build-arg
param to always be in the form "name=value", and not just "name".

This PR does the right fix - it allows just "name" and it'll grab the value
from the env vars if set. If "name" isn't set in the env then it still needs
to send "name" to the server so that a warning can be printed about an
unused --build-arg. And this is why buildArgs in the options is now a
*string instead of just a string - 'nil' == mentioned but no value.

Closes #29084

Signed-off-by: Doug Davis <dug@us.ibm.com>
(cherry picked from commit cdb8ea90b0)
Signed-off-by: Victor Vieux <vieux@docker.com>
2016-12-09 14:24:00 -08:00
Tibor Vass
3a571b72fd plugins: container-rootfs-relative paths
Legacy plugins expect host-relative paths (such as for Volume.Mount).
However, a containerized plugin cannot respond with a host-relative
path. Therefore, this commit modifies new volume plugins' paths in Mount
and List to prepend the container's rootfs path.

This introduces a new PropagatedMount field in the Plugin Config.
When it is set for volume plugins, RootfsPropagation is set to rshared
and the path specified by PropagatedMount is bind-mounted with rshared
prior to launching the container. This is so that the daemon code can
access the paths returned by the plugin from the host mount namespace.

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit c54b717caf)
Signed-off-by: Victor Vieux <vieux@docker.com>
2016-12-09 14:16:04 -08:00
Vincent Demeester
0f2364f73c Handle logging in compose to swarm
Logging configuration was completely ignore when deploy a compose file
to swarm. This fixes it.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit 806cc1e0f8)
Signed-off-by: Victor Vieux <vieux@docker.com>
2016-12-07 16:14:20 -08:00
Yong Tang
d1d6357beb Convert DanglingOnly to Filters for docker image prune
This fix convert DanglingOnly in ImagesPruneConfig to Filters,
so that it is possible to maintain API compatibility in the future.

Several integration tests have been added to cover changes.

This fix is related to 28497.

A follow up to this PR will be done once this PR is merged.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
(cherry picked from commit a6be56b54e)
Signed-off-by: Victor Vieux <vieux@docker.com>
2016-12-07 16:08:18 -08:00
Vincent Demeester
936499e1ff Fixes ImageList to be retro-compatible with older API
Make sure current client code can talk for ImageList can still talk to
older daemon.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit c6e31454ba)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-29 14:10:44 -08:00
cyli
aa6211e153 Do not display the digest or size of swarm secrets
Signed-off-by: cyli <cyli@twistedmatrix.com>
(cherry picked from commit 2c0613540a)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-22 16:54:50 -08:00
Ben Firshman
520e601dc2 Rename Remote API to Engine API
Implementation of https://github.com/docker/docker/issues/28319

Signed-off-by: Ben Firshman <ben@firshman.co.uk>
(cherry picked from commit f0d55cd081)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-22 16:49:58 -08:00
Tibor Vass
561f2b27cb plugins: support for devices
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 53b9b99e5c)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-22 16:16:33 -08:00
Tibor Vass
c28c63472a plugins: linux capabilities and device creation
In the plugin manifest, Capabilities has been moved to
Linux.Capabilities to avoid confusion with Interface.Types[i].Capability

A DeviceCreation boolean has also been added to the manifest. This could
be changed in the future to be specific to a major number.

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 9f239281b1)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-22 16:16:04 -08:00
Anusha Ragunathan
0403addc5f Add HTTP client timeout.
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
(cherry picked from commit 83ca993c15)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-22 11:23:20 -08:00
Evan Hazlett
01aab8baa3 do not force target type for secret references
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

use secret store interface instead of embedded secret data into container

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
(cherry picked from commit bebd472e40)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-21 14:45:39 -08:00
Antonio Murdaca
0a5732d1cf api: types: keep info.SecurityOptions a string slice
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
(cherry picked from commit 514ca09426)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-21 14:45:05 -08:00
Yanqiang Miao
4acf6aa456 The type of 'ExportdPorts' should be 'nat.PortSet'
Signed-off-by: Yanqiang Miao <miao.yanqiang@zte.com.cn>
(cherry picked from commit c44c542d95)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-21 13:23:16 -08:00
Victor Vieux
4edaaeb671 fix a few golint errors
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
(cherry picked from commit 9c559e6d0b)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-19 23:50:06 -08:00
Aaron Lehmann
d4392659f7 Return warnings from service create and service update when digest pinning fails
Modify the service update and create APIs to return optional warning
messages as part of the response. Populate these messages with an
informative reason when digest resolution fails.

This is a small API change, but significantly improves the UX. The user
can now get immediate feedback when they've specified a nonexistent
image or unreachable registry.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
(cherry picked from commit 948e60691e)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-18 13:57:11 -08:00
Aaron Lehmann
48c2da3cd5 api: Remove omitempty tag on Parallelism
It doesn't make sense to use omitempty here. 0 is a meaningful value and
it's different from the default. If someone sets Parallelism to 0, we
want to show that Parallelism is 0, not hide the field.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
(cherry picked from commit d8b8b12994)
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2016-11-18 13:35:33 -08:00
Brian Goff
de325535d4 Merge pull request #27872 from vdemeester/images-filter-filters
Add reference filter and deprecated filter param…
2016-11-11 11:24:19 -05:00
Vincent Demeester
820b809e70
Add reference filter and deprecated filter param…
… for `docker images`.

This deprecates the `filter` param for the `/images` endpoint and make a
new filter called `reference` to replace it. It does change the CLI
side (still possible to do `docker images busybox:musl`) but changes the
cli code to use the filter instead (so that `docker images --filter
busybox:musl` and `docker images busybox:musl` act the same).

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-11-11 15:34:01 +01:00
Vincent Demeester
84450b72cd Add support for stdin_open in composefile v3
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-11-11 15:19:00 +01:00
Victor Vieux
0427afa409 Merge pull request #27955 from mlaventure/runc-docker-info
Add external binaries version to docker info
2016-11-10 21:27:14 -08:00
Jana Radhakrishnan
14ac9f60d0 Add support for host port PublishMode in services
Add api/cli support for adding host port PublishMode in services.

Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
2016-11-10 18:07:55 -08:00
Victor Vieux
bed96ce922 Merge pull request #28031 from yongtang/27902-extra-hosts
Add flag `--host` to `service create` and `--host-add/rm` to `service update`
2016-11-10 16:55:29 -08:00
Michael Crosby
801230ce31 Merge pull request #27797 from vdemeester/stats-format-names-support
Add support for Names and ID in stats format
2016-11-10 13:26:00 -08:00
Yong Tang
ea9a23ccdb Add flag --host to service create and --host-add/--host-rm to service update
This fix tries to address 27902 by adding a flag `--host`
to `docker service create` and `--host-add/--host-rm` to
`docker service update`, so that it is possible to
specify extra `host:ip` settings in `/etc/hosts`.

This fix adds `Hosts` in swarmkit's `ContainerSpec` so that it
is possible to specify extra hosts during service creation.

Related docs has been updated.

An integration test has been added.

This fix fixes 27902.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-11-10 10:20:24 -08:00
Victor Vieux
49ca91fbeb rename plugin manifest
Signed-off-by: Victor Vieux <vieux@docker.com>
2016-11-10 00:09:23 -08:00
Victor Vieux
28a1ea342d Merge pull request #28164 from anusha-ragunathan/plugin-build
Add plugin create functionality.
2016-11-09 17:49:51 -08:00
Aaron Lehmann
0f9fc54df9 Revise swarm init/update flags, add unlocking capability
- Neither swarm init or swarm update should take an unlock key
- Add an autolock flag to turn on autolock
- Make the necessary docker api changes
- Add SwarmGetUnlockKey API call and use it when turning on autolock
- Add swarm unlock-key subcommand

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2016-11-09 16:09:00 -08:00
Tonis Tiigi
b4a667c8c4 Add support for swarm init lock and swarm unlock
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-11-09 16:09:00 -08:00
Jana Radhakrishnan
5ef5a4216a Merge pull request #28078 from sanimej/inspect
Show peer nodes in network inspect for swarm overlay networks
2016-11-09 15:33:13 -08:00
Anusha Ragunathan
5b6e1bc9e5 Add plugin create functionality.
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
2016-11-09 15:23:03 -08:00
Santhosh Manohar
dd9944aa64 Show peer nodes in network inspect for swarm overlay networks
Signed-off-by: Santhosh Manohar <santhosh@docker.com>
2016-11-09 13:58:04 -08:00
Evan Hazlett
83af60e623 fix lint issue from rebase
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:46 -05:00
Evan Hazlett
427c70d977 more review updates
- return err instead of wrap for update secret
- add omitempty for data in secret spec

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:45 -05:00
Evan Hazlett
5b2230a38b SecretRequestOptions -> SecretRequestOption
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:45 -05:00
Evan Hazlett
e9fd1c5918 review updates
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:45 -05:00
Evan Hazlett
eff36b7d66 more review updates
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:45 -05:00
Evan Hazlett
583c013a87 support labels for secrets upon creation; review updates
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:45 -05:00
Evan Hazlett
4d1fba0204 remove SecretRequestSpec
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:45 -05:00
Evan Hazlett
b2e4c7f3b5 review updates
- use Filters instead of Filter for secret list
- UID, GID -> string
- getSecrets -> getSecretsByName
- updated test case for secrets with better source
- use golang.org/x/context instead of context
- for grpc conversion allocate with make
- check for nil with task.Spec.GetContainer()

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:45 -05:00
Evan Hazlett
c00138748d move secretopt to opts pkg
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:45 -05:00
Evan Hazlett
88dea0e06e update to support new target in swarmkit
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:44 -05:00
Evan Hazlett
189f89301e more review updates
- use /secrets for swarm secret create route
- do not specify omitempty for secret and secret reference
- simplify lookup for secret ids
- do not use pointer for secret grpc conversion

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:43 -05:00
Evan Hazlett
857e60c2f9 review changes
- fix lint issues
- use errors pkg for wrapping errors
- cleanup on error when setting up secrets mount
- fix erroneous import
- remove unneeded switch for secret reference mode
- return single mount for secrets instead of slice

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:43 -05:00
Evan Hazlett
3716ec25b4 secrets: secret management for swarm
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

wip: use tmpfs for swarm secrets

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

wip: inject secrets from swarm secret store

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

secrets: use secret names in cli for service create

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

switch to use mounts instead of volumes

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

vendor: use ehazlett swarmkit

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>

secrets: finish secret update

Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
2016-11-09 14:27:43 -05:00