URL query encode log details, so that characters like spaces don't make
log parsing ambiguous. Add a helper function to parse these details to a
map, if needed
Add support for details on service logs
Signed-off-by: Drew Erny <drew.erny@docker.com>
From Go 1.8 HTTP client redirect behaviour is changed:
When status code is 301, 307 or 308, the client
automatically converts it to a new HTTP request.
This behaviour change manifests in the client in that
before the 301 was not followed and the client did not generate
an error, but now results in an error message:
"Error response from daemon: page not found."
To fix that a new redirect policy is forced by setting
HTTP Client's CheckRedirect.
That policy is to return an error for any 301, 307 or 308
in the response's status code to a non-GET request.
The error message specifies that the daemon could not
process the request and it is probably due to bad
arguments that were provided by the user.
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
I noticed that we're using a homegrown package for assertions. The
functions are extremely similar to testify, but with enough slight
differences to be confusing (for example, Equal takes its arguments in a
different order). We already vendor testify, and it's used in a few
places by tests.
I also found some problems with pkg/testutil/assert. For example, the
NotNil function seems to be broken. It checks the argument against
"nil", which only works for an interface. If you pass in a nil map or
slice, the equality check will fail.
In the interest of avoiding NIH, I'm proposing replacing
pkg/testutil/assert with testify. The test code looks almost the same,
but we avoid the confusion of having two similar but slightly different
assertion packages, and having to maintain our own package instead of
using a commonly-used one.
In the process, I found a few places where the tests should halt if an
assertion fails, so I've made those cases (that I noticed) use "require"
instead of "assert", and I've vendored the "require" package from
testify alongside the already-present "assert" package.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This changes the long-standing bug of copy operations not preserving the
UID/GID information after the files arrive to the container.
Signed-off-by: Erik Hollensbe <github@hollensbe.org>
The source of a tag operation is allowed to be a 64-character hex
string. This means it should use ParseAnyReference for validation
instead of ParseNormalizedNamed.
This fixes a regression that happened in 17.04.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This adds a new parameter insertDefaults to /services/{id}. When this is
set, an empty field (such as UpdateConfig) will be populated with
default values in the API response. Make "service inspect" use this, so
that empty fields do not result in missing information when inspecting a
service.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fix tries to address the issue raised in 29999 where it was not
possible to mask these items (like important non-removable stuff)
from `docker system prune`.
This fix adds `label` and `label!` field for `--filter` in `system prune`,
so that it is possible to selectively prune items like:
```
$ docker container prune --filter label=foo
$ docker container prune --filter label!=bar
```
Additional unit tests and integration tests have been added.
This fix fixes 29999.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Refactored the API to more easily accept new endpoints. Added REST,
client, and CLI endpoints for getting logs from a specific task. All
that is needed after this commit to enable arbitrary service log
selectors is a REST endpoint and handler.
Task logs can be retrieved by putting in a task ID at the CLI instead of
a service ID.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Server-side rollback can take advantage of the rollback-specific update
parameters, instead of being treated as a normal update that happens to
go back to a previous version of the spec.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
When a plugin fails to start, we still incorrectly mark it as enabled.
This change verifies that we can dial to the plugin socket to confirm that
the plugin is functional and only then mark the plugin as enabled. Also,
dont delete the plugin on install, if only the enable fails.
Signed-off-by: Anusha Ragunathan <anusha.ragunathan@docker.com>
Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.
Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Docker 1.13 and up allows a client to communicate
with older daemons. As a result, flags may be
present that are not supported by the older daemon.
The client already _hides_ flags that are not
supported yet, but this doesn't present users
from using those flags.
This change shows an error if a flag is used
that is not supported by the daemon (either
based on the API version, or because experimental
features are not enabled).
Note that for some options, a check is already
in place in the API client. For those
options, this is just a minor enhancement
to more clearly indicate which _flag_ is
not supported.
Before this change;
DOCKER_API_VERSION=1.24 docker run -d --stop-timeout=30 busybox top
mjfyt3qpvnq0iwmun3sjwth9i
echo -e "FROM busybox\nRUN echo foo > bar" | DOCKER_API_VERSION=1.24 docker build --squash -
"squash" requires API version 1.25, but the Docker server is version 1.24
After this change;
DOCKER_API_VERSION=1.24 docker run -d --stop-timeout=30 busybox top
"--stop-timeout" requires API version 1.25, but the Docker daemon is version 1.24
echo -e "FROM busybox\nRUN echo foo > bar" | DOCKER_API_VERSION=1.24 docker build --squash -
"--squash" requires API version 1.25, but the Docker daemon is version 1.24
echo -e "FROM busybox\nRUN echo foo > bar" | docker build --squash -
"--squash" is only supported on a Docker daemon with experimental features enabled
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This allows a plugin to be upgraded without requiring to
uninstall/reinstall a plugin.
Since plugin resources (e.g. volumes) are tied to a plugin ID, this is
important to ensure resources aren't lost.
The plugin must be disabled while upgrading (errors out if enabled).
This does not add any convenience flags for automatically
disabling/re-enabling the plugin during before/after upgrade.
Since an upgrade may change requested permissions, the user is required
to accept permissions just like `docker plugin install`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This fix adds `--filter capability=[volumedriver|authz]` to `docker plugin ls`.
The related docs has been updated.
An integration test has been added.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix adds `--filter enabled=true` to `docker plugin ls`,
as was specified in 28624.
The related API and docs has been updated.
An integration test has been added.
This fix fixes 28624.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Docker 1.13 moves the `--rm` flag to the daemon,
through an AutoRemove option in HostConfig.
When using API 1.24 and under, AutoRemove should not be
used, even if the daemon is version 1.13 or above and
"supports" this feature.
This patch fixes a situation where an 1.13 client,
talking to an 1.13 daemon, but using the 1.24 API
version, still set the AutoRemove property.
As a result, both the client _and_ the daemon
were attempting to remove the container, resulting
in an error:
ERRO[0000] error removing container: Error response from daemon:
removal of container ce0976ad22495c7cbe9487752ea32721a282164862db036b2f3377bd07461c3a
is already in progress
In addition, the validation of conflicting options
is moved from `docker run` to `opts.parse()`, so
that conflicting options are also detected when
running `docker create` and `docker start` separately.
To resolve the issue, the `AutoRemove` option is now
always set to `false` both by the client and the
daemon, if API version 1.24 or under is used.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
At the "Build image from Dockerfile" section in the API docs
the Content-Type header is missing.
In addition, some parts in the code are still setting the
Content-Type header to application/tar while it was changed
to application/x-tar since 16th September 2015.
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
This fix is a follow up for comment
https://github.com/docker/docker/pull/28535#issuecomment-263215225
This fix provides `--filter until=<timestamp>` for `docker container/image prune`.
This fix adds `--filter until=<timestamp>` to `docker container/image prune`
so that it is possible to specify a timestamp and prune those containers/images
that are earlier than the timestamp.
Related docs has been updated
Several integration tests have been added to cover changes.
This fix fixes#28497.
This fix is related to #28535.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Move plugins to shared distribution stack with images.
Create immutable plugin config that matches schema2 requirements.
Ensure data being pushed is same as pulled/created.
Store distribution artifacts in a blobstore.
Run init layer setup for every plugin start.
Fix breakouts from unsafe file accesses.
Add support for `docker plugin install --alias`
Uses normalized references for default names to avoid collisions when using default hosts/tags.
Some refactoring of the plugin manager to support the change, like removing the singleton manager and adding manager config struct.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
When plugins have a positive refcount, they were not allowed to be
removed. However, plugins could still be disabled when volumes
referenced it and containers using them were running.
This change fixes that by enforcing plugin refcount during disable.
A "force" disable option is also added to ignore reference refcounting.
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
In cases where there is high latency (ie, not-local network)
`waitExitOrRemoved` was not receiving events for short-lived containers.
This caused the client to hang while waiting for a notification that the
container has stopped.
This happens because `client.Events()` returns immediately and spins a
goroutine up to process events. The problem here is it returns before
the request to the events endpoint is even made.
Even without high-latency issues, there is no guarantee that the
goroutine is even scheduled by the time the function returns.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This reverts 26103. 26103 was trying to make it so that if someone did:
docker build --build-arg FOO .
and FOO wasn't set as an env var then it would pick-up FOO from the
Dockerfile's ARG cmd. However, it went too far and removed the ability
to specify a build arg w/o any value. Meaning it required the --build-arg
param to always be in the form "name=value", and not just "name".
This PR does the right fix - it allows just "name" and it'll grab the value
from the env vars if set. If "name" isn't set in the env then it still needs
to send "name" to the server so that a warning can be printed about an
unused --build-arg. And this is why buildArgs in the options is now a
*string instead of just a string - 'nil' == mentioned but no value.
Closes#29084
Signed-off-by: Doug Davis <dug@us.ibm.com>
This fix convert DanglingOnly in ImagesPruneConfig to Filters,
so that it is possible to maintain API compatibility in the future.
Several integration tests have been added to cover changes.
This fix is related to 28497.
A follow up to this PR will be done once this PR is merged.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the proposal raised in 28946
to support plugins in `docker inspect`.
The command `docker inspect` already supports
"container", "image", "node", "network", "service", "volume", "task".
However, `--type plugin` is not supported yet at the moment.
This fix address this issue by adding the support of `--type plugin`
for `docker inspect`.
An additional integration test has been added to cover the changes.
This fix fixes 28946.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
If env var "DOCKER_API_VERSION" is specified by user, we'll set
`cli.manualOverride`, before this, this field is always true due to
wrong logic.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
Modify the service update and create APIs to return optional warning
messages as part of the response. Populate these messages with an
informative reason when digest resolution fails.
This is a small API change, but significantly improves the UX. The user
can now get immediate feedback when they've specified a nonexistent
image or unreachable registry.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
During error cases, we dont cleanup correctly. This commit takes care
of removing the plugin, if there are errors after the pull passed. It
also shuts down the plugin, if there are errors after the plugin in the
enable path.
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
… for `docker images`.
This deprecates the `filter` param for the `/images` endpoint and make a
new filter called `reference` to replace it. It does change the CLI
side (still possible to do `docker images busybox:musl`) but changes the
cli code to use the filter instead (so that `docker images --filter
busybox:musl` and `docker images busybox:musl` act the same).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
- Neither swarm init or swarm update should take an unlock key
- Add an autolock flag to turn on autolock
- Make the necessary docker api changes
- Add SwarmGetUnlockKey API call and use it when turning on autolock
- Add swarm unlock-key subcommand
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
- use Filters instead of Filter for secret list
- UID, GID -> string
- getSecrets -> getSecretsByName
- updated test case for secrets with better source
- use golang.org/x/context instead of context
- for grpc conversion allocate with make
- check for nil with task.Spec.GetContainer()
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
- use /secrets for swarm secret create route
- do not specify omitempty for secret and secret reference
- simplify lookup for secret ids
- do not use pointer for secret grpc conversion
Signed-off-by: Evan Hazlett <ejhazlett@gmail.com>
Signed-off-by: Victor Vieux <vieux@docker.com>
update cobra and use Tags
Signed-off-by: Victor Vieux <vieux@docker.com>
allow client to talk to an older server
Signed-off-by: Victor Vieux <vieux@docker.com>
In file `api/types/client.go`, some of the "*Options{}" structs own a
`Filters` field while some else have the name of `Filter`, this commit
will rename all `Filter` to `Filters` for consistency. Also `Filters`
is consistent with API with format `/xxx?filters=xxx`, that's why
`Filters` is the right name.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
`docker network prune` prunes unused networks, including overlay ones.
`docker system prune` also prunes unused networks.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
The docker client has historically used Transport.TLSClientConfig to set
the scheme for the API client. A recent moved the resolution to use the
http.Transport directly, rather than save the TLSClientConfig state on a
client struct. This caused issues when mutliple calls made with a single
client would have this field set in the http package on pre-1.7
installations. This fix detects the presence of the TLSClientConfig once
and sets the scheme accordingly.
We still don't know why this issue doesn't happen with Go 1.7 but it
must be more deterministic in the newer version.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
- Tightened up copy in README
- Make example in README a bit simpler
- Update README to point at GoDoc
Signed-off-by: Ben Firshman <ben@firshman.co.uk>
Signed-off-by: Sean Rodman <srodman7689@gmail.com>
Updated the check for the permission error to use os.IsPermission instead of checking the error string. Also, changed the PermissionDenied method to just a new error.
Fixed a typo in client/request.go
Fixed Error name as specified by Pull request builder output.
Worked on making changes to the permissiondenied error.
Fixed typo
Signed-off-by: Sean Rodman <srodman7689@gmail.com>
Updated error message as requested.
Fixed the error as requested
Signed-off-by: Sean Rodman <srodman7689@gmail.com>
Under the convoluted code path for the transport configuration,
TLSConfig was being set even though the socket type is unix. This caused
other code detecting the TLSConfig to assume https, rather than using
the http scheme. This led to a situation where if `DOCKER_CERT_PATH` is
set, unix sockets start reverting to https. There is other odd behavior
from go-connections that is also reproduced here.
For the most part, we try to reproduce the side-effecting behavior from
go-connections to retain the current docker behavior. This whole mess
needs to ripped out and fixed, as this pile spaghetti is unnacceptable.
This code is way to convoluted for an http client. We'll need to fix
this but the Go API will break to do it.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This package doesn't really seem to do anything of real interest.
Removing it and replacing with a few helper functions. Most of this was
maintaining a fork of ctxhttp to support a mock that was unnecessary.
We could probably do with a further refactor of the client interface.
There is a lot of confusion of between transport, http layer and
application layer that makes for some awkward code. This change
improves the situation to the point where no breaking changes are
introduced.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
Instead of reformatting error from the request action, we wrap it,
allowing the cause to be recovered. This is important for consumers that
need to be able to detect context errors, such as `Cancelled` and
`DeadlineExceeded`.
Signed-off-by: Stephen J Day <stephen.day@docker.com>
This change adds file version information to docker.exe and dockerd.exe by
adding a Windows version resource with the windres tool.
This change adds a dependency to binutils-mingw-w64 on Linux, but removes
a dependency on rsrc. Most Windows build environments should already have
windres if they have gcc (which is necessary to build dockerd).
Signed-off-by: John Starks <jostarks@microsoft.com>
Add a proxy to support 'docker daemon'
Fix configFile option, and remove a test that is no longer relevant.
Remove daemon build tag.
Remove DOCKER_CLIENTONLY from build scripts.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Change docker-daemon to dockerd.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
