This fix tries to address the request in 31324 by adding
`--filter scope=swarm|local` for `docker network ls`.
As `docker network ls` has a `SCOPE` column by default,
it is natural to add the support of `--filter scope=swarm|local`.
This fix adds the `scope=swarm|local` support for
`docker network ls --filter`.
Related docs has been updated.
Additional unit test cases have been added.
This fix fixes 31324.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This more (in spirit) mimics the handler usage in net/http/pprof.
It also makes sure that any new profiles that are added are
automatically supported (e.g. `mutex` profiles in go1.8).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Server-side rollback can take advantage of the rollback-specific update
parameters, instead of being treated as a normal update that happens to
go back to a previous version of the spec.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Previously, `docker logs` (and by extension, `docker service logs`) had
an error in the way they returned errors that occurred after a stream
had already been started. Specifically, the errors were added verbatim
to the HTTP response, which caused StdCopy to fail with an esoteric
error. This change updates StdCopy to accept errors from the source
stream and then return those errors when copying to the destination
stream.
Signed-off-by: Drew Erny <drew.erny@docker.com>
This fix is partially based on comment
https://github.com/docker/docker/issues/30242#issuecomment-273517205
Currently, `docker network inspect` relies on `FindNetwork()` which
does not take into consideration that multiple networks with the same
name might exist.
This fix propose to return `docker network inspect` in a similiar
fashion like other commands:
1. Lookup full ID
2. Lookup full name
3. Lookup partial ID
If multiple networks exist, an error will be returned.
NOTE: this fix is not a complete fix for the issue raised in
https://github.com/docker/docker/issues/30242#issuecomment-273517205
where SwarmKit is unable to update when multiple networks with the same
name exit.
To fix that issue requires multiple places when `FindNetwork()` is called.
Because of the impact of changing `FindNetwork()`, this fix focus on
the issue in `docker network inspect`.
A separate PR will be created to address
https://github.com/docker/docker/issues/30242#issuecomment-273517205
An integration test has been added.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.
Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
This allows a plugin to be upgraded without requiring to
uninstall/reinstall a plugin.
Since plugin resources (e.g. volumes) are tied to a plugin ID, this is
important to ensure resources aren't lost.
The plugin must be disabled while upgrading (errors out if enabled).
This does not add any convenience flags for automatically
disabling/re-enabling the plugin during before/after upgrade.
Since an upgrade may change requested permissions, the user is required
to accept permissions just like `docker plugin install`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Pulling all the endpoints is a very resource heavy operation especially
for Global-scoped networks with a backing KVStore. Such heavy operations
can be fetched for individual network inspect. These are unneccessary
for a simple network list operation.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
This fix tries to address the issue raised in 28176 where
text frame was used in websocket attach endpoint. In case
the data send out contains non utf8 data, the connection
will be closed in certain browsers, e.g., Safari.
This fix address the issue by change `PayloadType` to `BinaryFrame`.
This fix is tested manually with Safari. The docker daemon is inside a Linux Virtual Machine.
Create a container with:
```
docker run -itd --name websocket busybox sh -c "while true; do echo -e 'he\\xc3\\x28o'; sleep 5; done"
```
Use the following url (172.16.66.128:2375 is the tcp address of the daemon):
```
file:///websocket.html?url=ws://172.16.66.128:2375/v1.25/containers/websocket/attach/ws?logs=1&stderr=1&stdout=1&stream=1&stdin=1
```
and the following html:
```
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Websocket</title>
<script type="text/javascript">
function DockerWebSocket() {
if ("WebSocket" in window) {
console.log("WebSocket is supported by Browser...")
// Remove '?url=' prefix
url = window.location.search.replace(/^(\?url=)/,"");
console.log("URL ["+url+"]...");
var ws = new WebSocket(url);
ws.onopen = function() {
console.log("Connection is opened...");
};
ws.onclose = function() {
console.log("Connection is closed...");
};
ws.onmessage = function (e) {
if (typeof e.data === "string") {
alert("WebSocket received text message ["+e.data+"]!")
} else {
console.log("Message is received...")
var blobReader = new FileReader();
blobReader.onload = function(event) {
console.log(JSON.stringify(blobReader.result))
};
blobReader.readAsText(e.data)
console.log("Message complete...")
}
};
} else {
alert("WebSocket is not supported by Browser!");
}
}
</script>
</head>
<body>
<div>
<a href="javascript:DockerWebSocket()">Run DockerWebSocket</a>
</div>
</body>
</html>
```
This fix fixes 28176.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix adds `--filter enabled=true` to `docker plugin ls`,
as was specified in 28624.
The related API and docs has been updated.
An integration test has been added.
This fix fixes 28624.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the issue raised in 30242 where the `Scope`
field always changed to `swarm` in the ouput of `docker network ls/inspect`
when duplicate networks name exist.
The reason for the issue was that `buildNetworkResource()` use network name
(which may not be unique) to check for the scope.
This fix fixes the issue by always use network ID in `buildNetworkResource()`.
A test has been added. The test fails before the fix and passes after the fix.
This fix fixes 30242.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Docker 1.13 moves the `--rm` flag to the daemon,
through an AutoRemove option in HostConfig.
When using API 1.24 and under, AutoRemove should not be
used, even if the daemon is version 1.13 or above and
"supports" this feature.
This patch fixes a situation where an 1.13 client,
talking to an 1.13 daemon, but using the 1.24 API
version, still set the AutoRemove property.
As a result, both the client _and_ the daemon
were attempting to remove the container, resulting
in an error:
ERRO[0000] error removing container: Error response from daemon:
removal of container ce0976ad22495c7cbe9487752ea32721a282164862db036b2f3377bd07461c3a
is already in progress
In addition, the validation of conflicting options
is moved from `docker run` to `opts.parse()`, so
that conflicting options are also detected when
running `docker create` and `docker start` separately.
To resolve the issue, the `AutoRemove` option is now
always set to `false` both by the client and the
daemon, if API version 1.24 or under is used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fix tries to address the issue raised in 29946 where
listing networks from API will return `null` if the result of
network filter is empty.
The reason for the issue was that inside the `filterNetworks()`,
the return value was initialized as `nil`:
```
var typeNet []types.NetworkResource
```
This is inconsistent with other places where return value was
initialized with `[]`
```
displayNet := []types.NetworkResource{}
```
This fix addresses the issue by changing `typeNet` to `[]` as well.
This fix fixes 29946.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix is a follow up for comment
https://github.com/docker/docker/pull/28535#issuecomment-263215225
This fix provides `--filter until=<timestamp>` for `docker container/image prune`.
This fix adds `--filter until=<timestamp>` to `docker container/image prune`
so that it is possible to specify a timestamp and prune those containers/images
that are earlier than the timestamp.
Related docs has been updated
Several integration tests have been added to cover changes.
This fix fixes#28497.
This fix is related to #28535.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Signed-off-by: Ke Li <kel@splunk.com>
Add missing changes
Signed-off-by: Ke Li <kel@splunk.com>
User errors.New to create error
Signed-off-by: Ke Li <kel@splunk.com>
Move plugins to shared distribution stack with images.
Create immutable plugin config that matches schema2 requirements.
Ensure data being pushed is same as pulled/created.
Store distribution artifacts in a blobstore.
Run init layer setup for every plugin start.
Fix breakouts from unsafe file accesses.
Add support for `docker plugin install --alias`
Uses normalized references for default names to avoid collisions when using default hosts/tags.
Some refactoring of the plugin manager to support the change, like removing the singleton manager and adding manager config struct.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
When plugins have a positive refcount, they were not allowed to be
removed. However, plugins could still be disabled when volumes
referenced it and containers using them were running.
This change fixes that by enforcing plugin refcount during disable.
A "force" disable option is also added to ignore reference refcounting.
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
Previously, it doesn't allow creating such a network:
e.g.
$ docker network inspect -f '{{.Id}}' ingress
84xh9knigj6zyt00u31e26nj3
$ docker network create 84
Error response from daemon: network with name 84 already exists
Fix#27866
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
This reverts 26103. 26103 was trying to make it so that if someone did:
docker build --build-arg FOO .
and FOO wasn't set as an env var then it would pick-up FOO from the
Dockerfile's ARG cmd. However, it went too far and removed the ability
to specify a build arg w/o any value. Meaning it required the --build-arg
param to always be in the form "name=value", and not just "name".
This PR does the right fix - it allows just "name" and it'll grab the value
from the env vars if set. If "name" isn't set in the env then it still needs
to send "name" to the server so that a warning can be printed about an
unused --build-arg. And this is why buildArgs in the options is now a
*string instead of just a string - 'nil' == mentioned but no value.
Closes#29084
Signed-off-by: Doug Davis <dug@us.ibm.com>
This fix convert DanglingOnly in ImagesPruneConfig to Filters,
so that it is possible to maintain API compatibility in the future.
Several integration tests have been added to cover changes.
This fix is related to 28497.
A follow up to this PR will be done once this PR is merged.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Modify the service update and create APIs to return optional warning
messages as part of the response. Populate these messages with an
informative reason when digest resolution fails.
This is a small API change, but significantly improves the UX. The user
can now get immediate feedback when they've specified a nonexistent
image or unreachable registry.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Instead of not adding experimental routes at all, fail with an explicit
message if the daemon is not running in experimental mode.
Added the `router.Experimental` which does this automatically.
Signed-off-by: Andrea Luzzardi <aluzzardi@gmail.com>
When swarm-mode is disabled, we need to return an error indicating this.
406 was chosen for the "Not Acceptable" verbiage, but this code has
specific semantics in relation to the `Accept` header, which aren't
applicable here.
We now use a 503 for this case. While it is not a perfect match, it does
make it clear that the particular "service" (read: API endpoint) is not
available. The body of the message provides the user with enough
information to take action on it by enabling swarm-mode and ensuring the
service is available.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Error code resolution is powered by string matching. Not the greatest
thing in the world and I hope no one is proud of this code, but it
works. However, because a map is used, the iteration order of the map is
random, such that if an error matches two of the snippets, it may return
a different error code depending on the seed of the hashmap. This change
converts it to use a slice instead.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
… for `docker images`.
This deprecates the `filter` param for the `/images` endpoint and make a
new filter called `reference` to replace it. It does change the CLI
side (still possible to do `docker images busybox:musl`) but changes the
cli code to use the filter instead (so that `docker images --filter
busybox:musl` and `docker images busybox:musl` act the same).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
- Neither swarm init or swarm update should take an unlock key
- Add an autolock flag to turn on autolock
- Make the necessary docker api changes
- Add SwarmGetUnlockKey API call and use it when turning on autolock
- Add swarm unlock-key subcommand
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
- use Filters instead of Filter for secret list
- UID, GID -> string
- getSecrets -> getSecretsByName
- updated test case for secrets with better source
- use golang.org/x/context instead of context
- for grpc conversion allocate with make
- check for nil with task.Spec.GetContainer()
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
- use /secrets for swarm secret create route
- do not specify omitempty for secret and secret reference
- simplify lookup for secret ids
- do not use pointer for secret grpc conversion
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
Signed-off-by: Victor Vieux <vieux@docker.com>
update cobra and use Tags
Signed-off-by: Victor Vieux <vieux@docker.com>
allow client to talk to an older server
Signed-off-by: Victor Vieux <vieux@docker.com>
Allow built images to be squash to scratch.
Squashing does not destroy any images or layers, and preserves the
build cache.
Introduce a new CLI argument --squash to docker build
Introduce a new param to the build API endpoint `squash`
Once the build is complete, docker creates a new image loading the diffs
from each layer into a single new layer and references all the parent's
layers.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
In file `api/types/client.go`, some of the "*Options{}" structs own a
`Filters` field while some else have the name of `Filter`, this commit
will rename all `Filter` to `Filters` for consistency. Also `Filters`
is consistent with API with format `/xxx?filters=xxx`, that's why
`Filters` is the right name.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
`docker network prune` prunes unused networks, including overlay ones.
`docker system prune` also prunes unused networks.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
This adds support for two enhancements to swarm service rolling updates:
- Failure thresholds: In Docker 1.12, a service update could be set up
to either pause or continue after a single failure occurs. This adds
an --update-max-failure-ratio flag that controls how many tasks need to
fail to update for the update as a whole to be considered a failure. A
counterpart flag, --update-monitor, controls how long to monitor each
task for a failure after starting it during the update.
- Rollback flag: service update --rollback reverts the service to its
previous version. If a service update encounters task failures, or
fails to function properly for some other reason, the user can roll back
the update.
SwarmKit also has the ability to roll back updates automatically after
hitting the failure thresholds, but we've decided not to expose this in
the Docker API/CLI for now, favoring a workflow where the decision to
roll back is always made by an admin. Depending on user feedback, we may
add a "rollback" option to --update-failure-action in the future.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fix tries to address the issue raised in 27021 where
HTML strings like (`&, >, <, etc`) in environmental variables
are escaped for JSON output for `docker inspect`. For example,
`TEST_ENV="soanni&rtr"` has been escaped to `TEST_ENV="soanni\u0026rtr"`
This fix disabled HTML escaping with `SetEscapeHTML`, which is available
since golang 1.7.0. This changes will be applied to all JSON output
that utilize `httputils.WriteJSON`.
An integration test has been added to cover the changes.
This fix fixes 27021.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
These new endpoints request the daemon to delete all resources
considered "unused" in their respective category:
- all stopped containers
- all volumes not attached to any containers
- images with no associated containers
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
Those data include:
- size of data shared with other images
- size of data unique to a given image
- how many containers are using a given image
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
When calling the /networks/ endpoint with a trailing
slash, the default network was returned.
This changes the endpoint to return the list of networks
instead (same response as `/networks` without trailing
slash).
Also updated the description for GetNetworkByName to
explain that the "default" network is returned if
no name or id is provided.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This PR adds support for running regular containers to be connected to
swarm mode multi-host network so that:
- containers connected to the same network across the cluster can
discover and connect to each other.
- Get access to services(and their associated loadbalancers)
connected to the same network
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
There are cases such as migrating from classic overlay network to the
swarm-mode networking (without kv-store), such a mechanism to allow
disconnecting a container even when a network isnt available will be
useful.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
This fix tries to address the issue in raised #23367 where an out-of-band
volume driver deletion leaves some data in docker. This prevent the
reuse of deleted volume names (by out-of-band volume driver like flocker).
This fix adds a `--force` field in `docker volume rm` to forcefully purge
the data of the volume that has already been deleted.
Related documentations have been updated.
This fix is tested manually with flocker, as is specified in #23367.
An integration test has also been added for the scenario described.
This fix fixes#23367.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the issue raised in 25529 wehre the
image/load API returns `application/json` for quiet=0 and
`text/plain` for quite=1.
This fix makes the change so that `application/json` is returned
for both quiet=0 and quite=1.
This fix has been tested manually.
This fix fixes 25529.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
There's existing code to generate these
kind of errors, so make the errors added
in commit cc493a52a4
less DRY.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit cc493a52a4 added
a constraint to network connect/disconnect operations
on "Swarm scoped" networks.
This adds those errors to the API documentation. Also
changes the error to lowercase for consistency.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Swarm handles service updates quite differently and also it doesnt
support worker driver network operations. Hence prevent containers from
connecting to swarm scoped networks
Signed-off-by: Madhu Venugopal <madhu@docker.com>
This renames the `rotate_xxx` flags to camelBack, for
consistency with other API query-params, such as
`detachKeys`, `noOverwriteDirNonDir`, and `fromImage`.
Also makes this flag accept a wider range of boolean
values ("0", "1", "true", "false"), and throw an error
if an invalid value is passed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Implement the proposal from
https://github.com/docker/docker/issues/24430#issuecomment-233100121
Removes acceptance policy and secret in favor of an automatically
generated join token that combines the secret, CA hash, and
manager/worker role into a single opaque string.
Adds a docker swarm join-token subcommand to inspect and rotate the
tokens.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
In order to keep a little bit of "sanity" on the API side, validate
hostname only starting from v1.24 API version.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
* Detect name conflicts on network creation
* Detect and prevent network connect/disconnect for managed containers
Signed-off-by: Madhu Venugopal <madhu@docker.com>
This patch introduces a new experimental engine-level plugin management
with a new API and command line. Plugins can be distributed via a Docker
registry, and their lifecycle is managed by the engine.
This makes plugins a first-class construct.
For more background, have a look at issue #20363.
Documentation is in a separate commit. If you want to understand how the
new plugin system works, you can start by reading the documentation.
Note: backwards compatibility with existing plugins is maintained,
albeit they won't benefit from the advantages of the new system.
Signed-off-by: Tibor Vass <tibor@docker.com>
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
As described in our ROADMAP.md, introduce new Swarm management API
endpoints relying on swarmkit to deploy services. It currently vendors
docker/engine-api changes.
This PR is fully backward compatible (joining a Swarm is an optional
feature of the Engine, and existing commands are not impacted).
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Victor Vieux <vieux@docker.com>
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Signed-off-by: Madhu Venugopal <madhu@docker.com>
This fix tries to fix logrus formatting by removing `f` from
`logrus.[Error|Warn|Debug|Fatal|Panic|Info]f` when formatting string
is not present.
This fix fixes#23459.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This endpoint has been deprecated since 1.8. Return an error starting
from this API version (1.24) in order to make sure it's not used for the
next API version and so that we can remove it some times later.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
