Enables other subsystems to watch actions for a plugin(s).
This will be used specifically for implementing plugins on swarm where a
swarm controller needs to watch the state of a plugin.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
COmmit 0307fe1a0b added
a new `DataPathAddr` property to the swarm/init and swarm/join
endpoints. This property was not yet added to the
documentation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
With debug logging turned on, we currently log the base64-encoded secret
payload.
Change the middleware code to redact this. Since the field is called
"Data", it requires some context-sensitivity. The URI path is examined
to see which route is being invoked.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
- DisplayablePorts is a `cli` function, moving to `docker/cli`
- Move MatchesContentType to the only package using it,
`api/server/httputils` (and remove the deps on logrus for `api` package)
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
This fix tries to add a `scope` in the query of `/networks/<id>`
(`NetworkInspect`) so that in case of duplicate network names,
it is possible to locate the network ID based on the network
scope (`local`, 'swarm', or `global`).
Multiple networks might exist in different scopes, which is a legitimate case.
For example, a network name `foo` might exists locally and in swarm network.
However, before this PR it was not possible to query a network name `foo`
in a specific scope like swarm.
This fix fixes the issue by allowing a `scope` query in `/networks/<id>`.
Additional test cases have been added to unit tests and integration tests.
This fix is related to docker/cli#167, moby/moby#30897, moby/moby#33561, moby/moby#30242
This fix fixesdocker/cli#167
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
With the Moby/Docker split, no decisions have been
made yet how, and when to bump the API version.
Although these decisions should not be lead
by Docker releases, I'm bumping the API version
to not complicate things for now; after this bump
we should make a plan how to handle this in future
(for example, using SemVer for the REST api, and
bump with every change).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit the rwLayer to get the correct DiffID
Refacator copy in thebuilder
move more code into exportImage
cleanup some windows tests
Release the newly commited layer.
Set the imageID on the buildStage after exporting a new image.
Move archiver to BuildManager.
Have ReleaseableLayer.Commit return a layer
and store the Image from exportImage in the local imageSources cache
Remove NewChild from image interface.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Closes#32663 by adding CreatedAt field when volume is created.
Displaying CreatedAt value when volume is inspected
Adding tests to verfiy the new field is correctly populated
Signed-off-by: Marianna <mtesselh@gmail.com>
Moving CreatedAt tests from the CLI
Moving the tests added for the newly added CreatedAt field for Volume, from CLI to API tests
Signed-off-by: Marianna <mtesselh@gmail.com>
Commit abd72d4008 added
a "FIXME" comment to the container "State", mentioning
that a container cannot be both "Running" and "Paused".
This comment was incorrect, because containers on
Linux actually _must_ be running in order to be
paused.
This patch adds additional information both in a
comment, and in the API documentation to clarify
that these booleans are not mutually exclusive.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The commit adds capability to accept csv parameters
for network option in service create/update commands.The change
includes name,alias driver options specific to the network.
With this the following will be supported
docker service create --name web --network name=docknet,alias=web1,driver-opt=field1=value1 nginx
docker service create --name web --network docknet nginx
docker service update web --network-add name=docknet,alias=web1,driver-opt=field1=value1
docker service update web --network-rm docknet
Signed-off-by: Abhinandan Prativadi <abhi@docker.com>
I noticed that we could return a Platform that has no information filled
in. This doesn't look like it would cause any problems, but it would be
confusing. Fix the handler to only append to this slice when the
Platform is not empty.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
- They are configuration-only networks which
can be used to supply the configuration
when creating regular networks.
- They do not get allocated and do net get plumbed.
Drivers do not get to know about them.
- They can be removed, once no other network is
using them.
- When user creates a network specifying a
configuration network for the config, no
other network specific configuration field
is are accepted. User can only specify
network operator fields (attachable, internal,...)
Signed-off-by: Alessandro Boch <aboch@docker.com>
This patch adds the untilRemoved option to the ContainerWait API which
allows the client to wait until the container is not only exited but
also removed.
This patch also adds some more CLI integration tests for waiting for a
created container and waiting with the new --until-removed flag.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Handle detach sequence in CLI
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Update Container Wait Conditions
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Apply container wait changes to API 1.30
The set of changes to the containerWait API missed the cut for the
Docker 17.05 release (API version 1.29). This patch bumps the version
checks to use 1.30 instead.
This patch also makes a minor update to a testfile which was added to
the builder/dockerfile package.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Remove wait changes from CLI
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Address minor nits on wait changes
- Changed the name of the tty Proxy wrapper to `escapeProxy`
- Removed the unnecessary Error() method on container.State
- Fixes a typo in comment (repeated word)
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Use router.WithCancel in the containerWait handler
This handler previously added this functionality manually but now uses
the existing wrapper which does it for us.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Add WaitCondition constants to api/types/container
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Address more ContainerWait review comments
- Update ContainerWait backend interface to not return pointer values
for container.StateStatus type.
- Updated container state's Wait() method comments to clarify that a
context MUST be used for cancelling the request, setting timeouts,
and to avoid goroutine leaks.
- Removed unnecessary buffering when making channels in the client's
ContainerWait methods.
- Renamed result and error channels in client's ContainerWait methods
to clarify that only a single result or error value would be sent
on the channel.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Move container.WaitCondition type to separate file
... to avoid conflict with swagger-generated code for API response
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Address more ContainerWait review comments
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This patch consolidates the two WaitStop and WaitWithContext methods
on the container.State type. Now there is a single method, Wait, which
takes a context and a bool specifying whether to wait for not just a
container exit but also removal.
The behavior has been changed slightly so that a wait call during a
Created state will not return immediately but instead wait for the
container to be started and then exited.
The interface has been changed to no longer block, but instead returns
a channel on which the caller can receive a *StateStatus value which
indicates the ExitCode or an error if there was one (like a context
timeout or state transition error).
These changes have been propagated through the rest of the deamon to
preserve all other existing behavior.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
The current use of the types from distribution brings in some
unfortunate dependencies, including other distribution packages and the
gorilla/mux and gorilla/context packages. Using the OCI types avoids
the extra dependencies for client users.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
The --allow-nondistributable-artifacts daemon option specifies
registries to which foreign layers should be pushed. (By default,
foreign layers are not pushed to registries.)
Additionally, to make this option effective, foreign layers are now
pulled from the registry if possible, falling back to the URLs in the
image manifest otherwise.
This option is useful when pushing images containing foreign layers to a
registry on an air-gapped network so hosts on that network can pull the
images without connecting to another server.
Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
URL query encode log details, so that characters like spaces don't make
log parsing ambiguous. Add a helper function to parse these details to a
map, if needed
Add support for details on service logs
Signed-off-by: Drew Erny <drew.erny@docker.com>
in the Docker REST APIs when viewing or updating the swarm spec info, and
also propagate the desired CA key in the Docker REST APIs when updating
swarm spec info only (it is not available for viewing).
Signed-off-by: Ying Li <ying.li@docker.com>
Fixes a warning that was shown;
Warning: Other properties are defined at the same level as $ref at
"#/paths/~1secrets~1{id}/get/responses/200/schema". They are IGNORED according
to the JsonSchema spec
That resulted in the example not being
rendered in the documentation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
buildStages now tracks the imageID and runConfig for a build stage
imageMounter tracks image mounts so they can released when the build ends.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Removes 3 methods from the builder.Backend interface
Remove the coupling between imageContexts, imageMounts and the builder.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
objects into the REST API responses. In the CLI, display only
whether the nodes' TLS info matches the cluster's TLS info, or
whether the node needs cert rotation.
Signed-off-by: Ying Li <ying.li@docker.com>
As of API 1.28, the network-list endpoint no longer
returns a list of containers attached to each network.
This patch updates the example response, and adds
a note to the description.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is synonymous with `docker run --cidfile=FILE` and writes the digest of
the newly built image to the named file. This is intended to be used by build
systems which want to avoid tagging (perhaps because they are in CI or
otherwise want to avoid fixed names which can clash) by enabling e.g. Makefile
constructs like:
image.id: Dockerfile
docker build --iidfile=image.id .
do-some-more-stuff: image.id
do-stuff-with <image.id
Currently the only way to achieve this is to use `docker build -q` and capture
the stdout, but at the expense of losing the build output.
In non-silent mode (without `-q`) with API >= v1.29 the caller will now see a
`JSONMessage` with the `Aux` field containing a `types.BuildResult` in the
output stream for each image/layer produced during the build, with the final
one being the end product. Having all of the intermediate images might be
interesting in some cases.
In silent mode (with `-q`) there is no change, on success the only output will
be the resulting image digest as it was previosuly.
There was no wrapper to just output an Aux section without enclosing it in a
Progress, so add one here.
Added some tests to integration cli tests.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
The `Log` field for plugins was added to `/info` in
17abacb894 but the swagger spec was not
updated.
This just updates the spec to match reality.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This field was missing a type, which meant that the example objects that Swagger tools generate did not work.
Signed-off-by: Wayne Song <wsong@docker.com>
Instead of mutating and reverting, just create a copy and pass the copy
around.
Add a unit test for builder dispatcher.run
Fix two test failures
Fix image history by adding a CreatedBy to commit options. Previously the
createdBy field was being created by modifying a reference to the runConfig that
was held from when the container was created.
Fix a test that expected a trailing slash. Previously the runConfig was being
modified by container create. Now that we're creating a copy of runConfig
instead of sharing a reference the runConfig retains the trailing slash.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Remove pathCache and replace it with syncmap
Cleanup NewBuilder
Create an api/server/backend/build
Extract BuildTagger
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Just a small grammatical correction, but it is unnecessary to repeat "to," after the colon, and deleting it makes the format of the list more consistent and readable.
Signed-off-by: Ryan Cooper <ryan.cooper7@gmail.com>
This new flag will allow the configuration of an interface that
can be used for data path traffic to be isolated from control
plane traffic. This flag is simply percolated down to libnetwork
and will be used by all the global scope drivers (today overlay)
Negative test added for invalid flag arguments
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
swarmkit's API type. Make sure this parameter gets propagated to
swarmkit, and also add an extra option to the CLI when providing
external CAs to parse the CA cert from a file.
Signed-off-by: Ying Li <ying.li@docker.com>
Redefine a better interface for remote context dependency.
Separate Dockerfile build instruction from remote context.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This changes the long-standing bug of copy operations not preserving the
UID/GID information after the files arrive to the container.
Signed-off-by: Erik Hollensbe <github@hollensbe.org>
This adds a new parameter insertDefaults to /services/{id}. When this is
set, an empty field (such as UpdateConfig) will be populated with
default values in the API response. Make "service inspect" use this, so
that empty fields do not result in missing information when inspecting a
service.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Service logs API is now stable. Service logs now support all features,
except retrieving details provided to the log driver.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Logging plugins use the same HTTP interface as other plugins for basic
command operations meanwhile actual logging operations are handled (on
Unix) via a fifo.
The plugin interface looks like so:
```go
type loggingPlugin interface {
StartLogging(fifoPath string, loggingContext Context) error
StopLogging(fifoPath)
```
This means a plugin must implement `LoggingDriver.StartLogging` and
`LoggingDriver.StopLogging` endpoints and be able to consume the passed
in fifo.
Logs are sent via stream encoder to the fifo encoded with protobuf.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
- Defined "normalized" type for Credential Spec and SELinux
- Added --credential-spec to docker service create & update
- SELinux is API only at the time
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
Refactor container logs system to make communicating log messages
internally much simpler. Move responsibility for marshalling log
messages into the REST server. Support TTY logs. Pave the way for fixing
the ambiguous bytestream format. Pave the way for fixing details.
Signed-off-by: Drew Erny <drew.erny@docker.com>
This parameter controls the order of operations when rolling out an
update task. Either the old task is stopped before starting the new one,
or the new task is started first, and the running tasks will briefly
overlap.
This commit adds Rollout to the API, and --update-order / --rollback-order
flags to the CLI.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fix tries to address the issue raised in 29999 where it was not
possible to mask these items (like important non-removable stuff)
from `docker system prune`.
This fix adds `label` and `label!` field for `--filter` in `system prune`,
so that it is possible to selectively prune items like:
```
$ docker container prune --filter label=foo
$ docker container prune --filter label!=bar
```
Additional unit tests and integration tests have been added.
This fix fixes 29999.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Refactored the API to more easily accept new endpoints. Added REST,
client, and CLI endpoints for getting logs from a specific task. All
that is needed after this commit to enable arbitrary service log
selectors is a REST endpoint and handler.
Task logs can be retrieved by putting in a task ID at the CLI instead of
a service ID.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Currently these fields are included in the response JSON with zero
values. It's better not to include them if the information is
unavailable (for example, on a worker node).
This turns Cluster into a pointer so that it can be left out.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fix tries to address the request in 31324 by adding
`--filter scope=swarm|local` for `docker network ls`.
As `docker network ls` has a `SCOPE` column by default,
it is natural to add the support of `--filter scope=swarm|local`.
This fix adds the `scope=swarm|local` support for
`docker network ls --filter`.
Related docs has been updated.
Additional unit test cases have been added.
This fix fixes 31324.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Embedding DockerVersion in plugin config when the plugin is created,
enables users to do a docker plugin inspect and know which version
the plugin was built on. This is helpful in cases where users are
running a new plugin on older docker releases and confused at
unexpected behavior.
By embedding DockerVersion in the config, we claim that there's no
guarantee that if the plugin config's DockerVersion is greater that
the version of the docker engine the plugin is executed against, the
plugin will work as expected.
For example, lets say:
- in 17.03, a plugin was released as johndoe/foo:v1
- in 17.05, the plugin uses the new ipchost config setting and author
publishes johndoe/foo:v2
In this case, johndoe/foo:v2 was built on 17.05 using ipchost, but is
running on docker-engine version 17.03. Since 17.05 > 17.03, there's
no guarantee that the plugin will work as expected. Ofcourse, if the
plugin did not use newly added config settings (ipchost in this case)
in 17.05, it would work fine in 17.03.
Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
Tested using global-net-plugin-ipc which sets PidHost in config.json.
Plugins might need access to host pid namespace. Add support for that.
Tested using aragunathan/global-net-plugin-ipc which sets "pidhost" in
config.json. Observed using `readlink /proc/self/ns/pid` that plugin and
host have the same ns.
Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
Plugins might need access to host ipc namespace. A good usecase is
a volume plugin running iscsi multipath commands that need access to
host kernel locks.
Tested with a custom plugin (aragunathan/global-net-plugin-full) that's
built with `"ipchost" : true` in config.json. Observed using
`readlink /proc/self/ns/ipc` that plugin and host have the same ns.
Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
This type is only used by CLI code. It duplicates SecretReference in the
types/swarm package. Change the CLI code to use that type instead.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This more (in spirit) mimics the handler usage in net/http/pprof.
It also makes sure that any new profiles that are added are
automatically supported (e.g. `mutex` profiles in go1.8).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This fix tries to address the request in 31325 by adding
`--filter mode=global|replicated` to `docker service ls`.
As `docker service ls` has a `MODE` column by default, it is natural
to support `--filter mode=global|replicated` for `docker service ls`.
There are multiple ways to address the issue. One way is to pass
the filter of mode to SwarmKit, another way is to process the filter
of mode in the daemon.
This fix process the filter in the daemon.
Related docs has been updated.
An integration test has been added.
This fix fixes 31325.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
In https://github.com/torvalds/linux/commit/5ca3726 (released in v4.7-rc1) the
content of the `cpuacct.usage_percpu` file in sysfs was changed to include both
online and offline cpus. This broke the arithmetic in the stats helpers used by
`docker stats`, since it was using the length of the PerCPUUsage array as a
proxy for the number of online CPUs.
Add current number of online CPUs to types.StatsJSON and use it in the
calculation.
Keep a fallback to `len(v.CPUStats.CPUUsage.PercpuUsage)` so this code
continues to work when talking to an older daemon. An old client talking to a
new daemon will ignore the new field and behave as before.
Fixes#28941.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Running the rm command on a paused/restarting container
will give an error message saying the container is running
which is incorrect.
To fix that, the error message will have the correct
container state and a procedure to remove it accordingly.
Notice: docker-py was bumped to:
4a08d04aef0595322e1b5ac7c52f28a931da85a5
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
Server-side rollback can take advantage of the rollback-specific update
parameters, instead of being treated as a normal update that happens to
go back to a previous version of the spec.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This adds 'consistency' mode flags to the mount command line argument.
Initially, the valid 'consistency' flags are 'consistent', 'cached',
'delegated', and 'default'.
Signed-off-by: David Sheets <dsheets@docker.com>
Signed-off-by: Jeremy Yallop <yallop@docker.com>
This adds support for placement preferences in Swarm services.
- Convert PlacementPreferences between GRPC API and HTTP API
- Add --placement-pref, --placement-pref-add and --placement-pref-rm to CLI
- Add support for placement preferences in service inspect --pretty
- Add integration test
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fix is an attempt to improve the documentation of
`Version` usage for Swarm API in Swagger docs.
As is shown in the docs updates, `Version` field in Swarm
API is a way to ensure safe concurrency and determinism
so that multiple updates to the same swarm related objects
(Node, Service, etc) will not unintentially overwrite each other.
This fix fixes 29448.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the issue raised in 25696 where
it was not possible to specify `--stop-signal` for `docker service create`
and `docker service update`, in order to use special signal to stop
the container.
This fix adds `--stop-signal` and update the `StopSignal` in `Config`
through `service create` and `service update`.
Related docs has been updated.
Integration test has been added.
This fix fixes 25696.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the issue in 31179 where
secrets fields is missing for `docker service create`
or `docker service update`.
This fix adds Secrets for `ContainerSpec` in Swagger docs.
This fix fixes 31179.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
PR #31197 made me think I needed to add more text explaing why
BuildArgs needs to be a *string instead of string.
Signed-off-by: Doug Davis <dug@us.ibm.com>
This fix adds the missing fields of `Hostname`, `OpenStdin`,
`Hosts`, `Groups`, and `Healthcheck` to `ContainerSpec`.
This fix also moves the `Label` in `ContainerSpec` to align with
the definition order.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Previously, `docker logs` (and by extension, `docker service logs`) had
an error in the way they returned errors that occurred after a stream
had already been started. Specifically, the errors were added verbatim
to the HTTP response, which caused StdCopy to fail with an esoteric
error. This change updates StdCopy to accept errors from the source
stream and then return those errors when copying to the destination
stream.
Signed-off-by: Drew Erny <drew.erny@docker.com>
This fix adds `Attachable` property for network object
in the documentation of Swagger API docs
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix is partially based on comment
https://github.com/docker/docker/issues/30242#issuecomment-273517205
Currently, `docker network inspect` relies on `FindNetwork()` which
does not take into consideration that multiple networks with the same
name might exist.
This fix propose to return `docker network inspect` in a similiar
fashion like other commands:
1. Lookup full ID
2. Lookup full name
3. Lookup partial ID
If multiple networks exist, an error will be returned.
NOTE: this fix is not a complete fix for the issue raised in
https://github.com/docker/docker/issues/30242#issuecomment-273517205
where SwarmKit is unable to update when multiple networks with the same
name exit.
To fix that issue requires multiple places when `FindNetwork()` is called.
Because of the impact of changing `FindNetwork()`, this fix focus on
the issue in `docker network inspect`.
A separate PR will be created to address
https://github.com/docker/docker/issues/30242#issuecomment-273517205
An integration test has been added.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.
Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
