This adds both a daemon-wide flag and a container creation property:
- Set the `CgroupnsMode: "host|private"` HostConfig property at
container creation time to control what cgroup namespace the container
is created in
- Set the `--default-cgroupns-mode=host|private` daemon flag to control
what cgroup namespace containers are created in by default
- Set the default if the daemon flag is unset to "host", for backward
compatibility
- Default to CgroupnsMode: "host" for client versions < 1.40
Signed-off-by: Rob Gulewich <rgulewich@netflix.com>
lxc-user-nic can eliminate slirp overhead but needs /etc/lxc/lxc-usernet to be configured for the current user.
To use lxc-user-nic, $DOCKERD_ROOTLESS_ROOTLESSKIT_NET=lxc-user-nic also needs to be set.
This commit also bumps up RootlessKit from v0.3.0 to v0.4.0:
70e0502f32...e92d5e772e
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
The `--rootless` flag had a couple of issues:
* #38702: euid=0, $USER="root" but no access to cgroup ("rootful" Docker in rootless Docker)
* #39009: euid=0 but $USER="docker" (rootful boot2docker)
To fix#38702, XDG dirs are ignored as in rootful Docker, unless the
dockerd is directly running under RootlessKit namespaces.
RootlessKit detection is implemented by checking whether `$ROOTLESSKIT_STATE_DIR` is set.
To fix#39009, the non-robust `$USER` check is now completely removed.
The entire logic can be illustrated as follows:
```
withRootlessKit := getenv("ROOTLESSKIT_STATE_DIR")
rootlessMode := withRootlessKit || cliFlag("--rootless")
honorXDG := withRootlessKit
useRootlessKitDockerProxy := withRootlessKit
removeCgroupSpec := rootlessMode
adjustOOMScoreAdj := rootlessMode
```
Close#39024Fix#38702#39009
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
Now `docker run -p` ports can be exposed to the host namespace automatically when `dockerd-rootless.sh` is launched with
`--userland-proxy --userland-proxy-path $(which rootlesskit-docker-proxy)`.
This is akin to how Docker for Mac/Win works with `--userland-proxy-path=/path/to/vpnkit-expose-port`.
The port number on the host namespace needs to be set to >= 1024.
SCTP ports are currently unsupported.
RootlessKit changes: 7bbbc48a6f...ed26714429
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
This patch hard-codes support for NVIDIA GPUs.
In a future patch it should move out into its own Device Plugin.
Signed-off-by: Tibor Vass <tibor@docker.com>
This changes the default ipc mode of daemon/engine to be private,
meaning the containers will not have their /dev/shm bind-mounted
from the host by default. The benefits of doing this are:
1. No leaked mounts. Eliminate a possibility to leak mounts into
other namespaces (and therefore unfortunate errors like "Unable to
remove filesystem for <ID>: remove /var/lib/docker/containers/<ID>/shm:
device or resource busy").
2. Working checkpoint/restore. Make `docker checkpoint`
not lose the contents of `/dev/shm`, but save it to
the dump, and be restored back upon `docker start --checkpoint`
(currently it is lost -- while CRIU handles tmpfs mounts,
the "shareable" mount is seen as external to container,
and thus rightfully ignored).
3. Better security. Currently any container is opened to share
its /dev/shm with any other container.
Obviously, this change will break the following usage scenario:
$ docker run -d --name donor busybox top
$ docker run --rm -it --ipc container:donor busybox sh
Error response from daemon: linux spec namespaces: can't join IPC
of container <ID>: non-shareable IPC (hint: use IpcMode:shareable
for the donor container)
The soution, as hinted by the (amended) error message, is to
explicitly enable donor sharing by using --ipc shareable:
$ docker run -d --name donor --ipc shareable busybox top
Compatibility notes:
1. This only applies to containers created _after_ this change.
Existing containers are not affected and will work fine
as their ipc mode is stored in HostConfig.
2. Old backward compatible behavior ("shareable" containers
by default) can be enabled by either using
`--default-ipc-mode shareable` daemon command line option,
or by adding a `"default-ipc-mode": "shareable"`
line in `/etc/docker/daemon.json` configuration file.
3. If an older client (API < 1.40) is used, a "shareable" container
is created. A test to check that is added.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The Swarmkit api specifies a target for configs called called "Runtime"
which indicates that the config is not mounted into the container but
has some other use. This commit updates the Docker api to reflect this.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Please refer to `docs/rootless.md`.
TLDR:
* Make sure `/etc/subuid` and `/etc/subgid` contain the entry for you
* `dockerd-rootless.sh --experimental`
* `docker -H unix://$XDG_RUNTIME_DIR/docker.sock run ...`
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
Monitoring systems and load balancers are usually configured to use HEAD
requests for health monitoring. The /_ping endpoint currently does not
support this type of request, which means that those systems have fallback
to GET requests.
This patch adds support for HEAD requests on the /_ping endpoint.
Although optional, this patch also returns `Content-Type` and `Content-Length`
headers in case of a HEAD request; Refering to RFC 7231, section 4.3.2:
The HEAD method is identical to GET except that the server MUST NOT
send a message body in the response (i.e., the response terminates at
the end of the header section). The server SHOULD send the same
header fields in response to a HEAD request as it would have sent if
the request had been a GET, except that the payload header fields
(Section 3.3) MAY be omitted. This method can be used for obtaining
metadata about the selected representation without transferring the
representation data and is often used for testing hypertext links for
validity, accessibility, and recent modification.
A payload within a HEAD request message has no defined semantics;
sending a payload body on a HEAD request might cause some existing
implementations to reject the request.
The response to a HEAD request is cacheable; a cache MAY use it to
satisfy subsequent HEAD requests unless otherwise indicated by the
Cache-Control header field (Section 5.2 of [RFC7234]). A HEAD
response might also have an effect on previously cached responses to
GET; see Section 4.3.5 of [RFC7234].
With this patch applied, either `GET` or `HEAD` requests work; the only
difference is that the body is empty in case of a `HEAD` request;
curl -i --unix-socket /var/run/docker.sock http://localhost/_ping
HTTP/1.1 200 OK
Api-Version: 1.40
Cache-Control: no-cache, no-store, must-revalidate
Docker-Experimental: false
Ostype: linux
Pragma: no-cache
Server: Docker/dev (linux)
Date: Mon, 14 Jan 2019 12:35:16 GMT
Content-Length: 2
Content-Type: text/plain; charset=utf-8
OK
curl --head -i --unix-socket /var/run/docker.sock http://localhost/_ping
HTTP/1.1 200 OK
Api-Version: 1.40
Cache-Control: no-cache, no-store, must-revalidate
Content-Length: 0
Content-Type: text/plain; charset=utf-8
Docker-Experimental: false
Ostype: linux
Pragma: no-cache
Server: Docker/dev (linux)
Date: Mon, 14 Jan 2019 12:34:15 GMT
The client is also updated to use `HEAD` by default, but fallback to `GET`
if the daemon does not support this method.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Add support for exact list of capabilities, support only OCI model
- Support OCI model on CapAdd and CapDrop but remain backward compatibility
- Create variable locally instead of declaring it at the top
- Use const for magic "ALL" value
- Rename `cap` variable as it overlaps with `cap()` built-in
- Normalize and validate capabilities before use
- Move validation for conflicting options to validateHostConfig()
- TweakCapabilities: simplify logic to calculate capabilities
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fix tries to address the issue raised in 37038 where
there were no memory.kernelTCP support for linux.
This fix add MemoryKernelTCP to HostConfig, and pass
the config to runtime-spec.
Additional test case has been added.
This fix fixes 37038.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This commit contains changes to configure DataPathPort
option. By default we use 4789 port number. But this commit
will allow user to configure port number during swarm init.
DataPathPort can't be modified after swarm init.
Signed-off-by: selansen <elango.siva@docker.com>
This allows non-recursive bind-mount, i.e. mount(2) with "bind" rather than "rbind".
Swarm-mode will be supported in a separate PR because of mutual vendoring.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
This feature was added in 514ce73391,
and was merged after API v1.39 shipped as part of the Docker 18.09
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This feature was added in 14da20f5e7,
and was merged after API v1.39 shipped as part of the Docker 18.09
release candidates.
This commit moves the feature to the correct API version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Looks like -i (together with DOCKER_INCREMENTAL_BINARY etc)
were used to get faster incremental builds.
Nowdays (since Go 1.10) this is no longer the case, as
go build cache is used [1]. Here's a quote:
> You do not have to use "go test -i" or "go build -i" or
> "go install" just to get fast incremental builds. We will
> not have to teach new users those workarounds anymore.
> Everything will just be fast.
To enable go cache between builds, add a volume for /root/.cache.
[1] https://groups.google.com/forum/#!msg/golang-dev/qfa3mHN4ZPA/X2UzjNV1BAAJ
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Adds support for sysctl options in docker services.
* Adds API plumbing for creating services with sysctl options set.
* Adds swagger.yaml documentation for new API field.
* Updates the API version history document.
* Changes executor package to make use of the Sysctls field on objects
* Includes integration test to verify that new behavior works.
Essentially, everything needed to support the equivalent of docker run's
`--sysctl` option except the CLI.
Includes a vendoring of swarmkit for proto changes to support the new
behavior.
Signed-off-by: Drew Erny <drew.erny@docker.com>
When requesting information about the daemon's configuration through the `/info`
endpoint, missing features (or non-recommended settings) may have to be presented
to the user.
Detecting these situations, and printing warnings currently is handled by the
cli, which results in some complications:
- duplicated effort: each client has to re-implement detection and warnings.
- it's not possible to generate warnings for reasons outside of the information
returned in the `/info` response.
- cli-side detection has to be updated for new conditions. This means that an
older cli connecting to a new daemon may not print all warnings (due to
it not detecting the new conditions)
- some warnings (in particular, warnings about storage-drivers) depend on
driver-status (`DriverStatus`) information. The format of the information
returned in this field is not part of the API specification and can change
over time, resulting in cli-side detection no longer being functional.
This patch adds a new `Warnings` field to the `/info` response. This field is
to return warnings to be presented by the user.
Existing warnings that are currently handled by the CLI are copied to the daemon
as part of this patch; This change is backward-compatible with existing
clients; old client can continue to use the client-side warnings, whereas new
clients can skip client-side detection, and print warnings that are returned by
the daemon.
Example response with this patch applied;
```bash
curl --unix-socket /var/run/docker.sock http://localhost/info | jq .Warnings
```
```json
[
"WARNING: bridge-nf-call-iptables is disabled",
"WARNING: bridge-nf-call-ip6tables is disabled"
]
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This feature allows user to specify list of subnets for global
default address pool. User can configure subnet list using
'swarm init' command. Daemon passes the information to swarmkit.
We validate the information in swarmkit, then store it in cluster
object. when IPAM init is called, we pass subnet list to IPAM driver.
Signed-off-by: selansen <elango.siva@docker.com>
* Expose license status in Info
This wires up a new field in the Info payload that exposes the license.
For moby this is hardcoded to always report a community edition.
Downstream enterprise dockerd will have additional licensing logic wired
into this function to report details about the current license status.
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
* Code review comments
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
* Add windows autogen support
Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
It said `TESTFLAGS='-test.run ^TestValidateIPAddress$' make test-unit`
runs `TestBuild` test, but actually runs `TestValidateIPAddress` test.
Signed-off-by: Donghwa Kim <shanytt@gmail.com>
In version-history.md, the link for `Docker Engine API v1.37`
was pointed to `v1.36`.
This fix fixes the incorrect link.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Further to 355cf9483c which caught some
of these. This should fix the remainder in the contributing docs.
Signed-off-by: Brett Randall <javabrett@gmail.com>
If there is a package-rename forthcoming, it isn't currently evident on master, but seems to show in doc.
Changed doc to show actual docker/docker paths as they occur now instead of moby/moby.
Signed-off-by: Brett Randall <javabrett@gmail.com>
This largely removes references to Docker where possible.
The HOWTO guides are still Docker docs and refer to Docker not
moby, so the next step is to rework these as Moby docs and put
them in `docs/` in this repo.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Commit d91c5f42eb added
support for "Isolation" mode for services, but didn't
get merged before API 1.34.
This patch moves the description in the API version
history to the correct API version (1.35), and does
a slight rewording of the functionality.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Currently, if a container removal has failed for some reason,
any client waiting for removal (e.g. `docker run --rm`) is
stuck, waiting for removal to succeed while it has failed already.
For more details and the reproducer, please check
https://github.com/moby/moby/issues/34945
This commit addresses that by allowing `ContainerWait()` with
`container.WaitCondition == "removed"` argument to return an
error in case of removal failure. The `ContainerWaitOKBody`
stucture returned to a client is amended with a pointer to `struct Error`,
containing an error message string, and the `Client.ContainerWait()`
is modified to return the error, if any, to the client.
Note that this feature is only available for API version >= 1.34.
In order for the old clients to be unstuck, we just close the connection
without writing anything -- this causes client's error.
Now, docker-cli would need a separate commit to bump the API to 1.34
and to show an error returned, if any.
[v2: recreate the waitRemove channel after closing]
[v3: document; keep legacy behavior for older clients]
[v4: convert Error from string to pointer to a struct]
[v5: don't emulate old behavior, send empty response in error case]
[v6: rename legacy* vars to include version suffix]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Describe more how host port allocation is done when
container is stopped/started in "PublishAllPorts".
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
If a network is created with a name that matches another
network's ID, the network with that name was masking the
other network's ID.
As a result, it was not possible to remove the network
with a given ID.
This patch changes the order in which networks are
matched to be what we use for other cases;
1. Match on full ID
2. Match on full Name
3. Match on Partial ID
Before this patch:
$ docker network create foo
336717eac9eaa3da6557042a04efc803f7e8862ce6cf96f6b9565265ba5c618b
$ docker network create 336717eac9eaa3da6557042a04efc803f7e8862ce6cf96f6b9565265ba5c618b
4a698333f1197f20224583abce14876d7f25fdfe416a8545927006c315915a2a
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
4a698333f119 336717eac9eaa3da6557042a04efc803f7e8862ce6cf96f6b9565265ba5c618b bridge local
d1e40d43a2c0 bridge bridge local
336717eac9ea foo bridge local
13cf280a1bbf host host local
d9e4c03728a0 none null local
$ docker network rm 336717eac9eaa3da6557042a04efc803f7e8862ce6cf96f6b9565265ba5c618b
4a698333f1197f20224583abce14876d7f25fdfe416a8545927006c315915a2a
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
d1e40d43a2c0 bridge bridge local
336717eac9ea foo bridge local
13cf280a1bbf host host local
d9e4c03728a0 none null local
After this patch:
$ docker network create foo
2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835
$ docker network create 2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835
6cbc749a529cd2d9d3b10566c84e56c4203dd88b67417437b5fc7a6e955dd48f
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
6cbc749a529c 2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835 bridge local
166c943dbeb5 bridge bridge local
2d1791a7def4 foo bridge local
6c45b8aa6d8e host host local
b11c96b51ea7 none null local
$ docker network rm 2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835
2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
6cbc749a529c 2d1791a7def4e2a1ef0f6b83c6add333df0bb4ced2f196c584cb64e6bd94b835 bridge local
166c943dbeb5 bridge bridge local
6c45b8aa6d8e host host local
b11c96b51ea7 none null local
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit e98e4a7111 implemented API version
negotiation using the `/_ping` endpoint. In that change, URL validation for the
maximum supported API version was removed from the API server (validation for
the _minimum_ version was kept in place).
With this feature, clients that support version negotiation would negotiate the
maximum version supported by the daemon, and downgrade to an older API version
if the client's default API version is not supported.
However, clients that do _not_ support version negotiation can call API versions
that are higher than the maximum supported version. Due to the missing version
check, this is silently ignored, and the daemon's default API version is used.
This is a problem, because the actual API version in use is non-deterministic;
for example, calling `/v9999.9999/version` on a daemon that runs API v1.34 will
use API v1.34, but calling the same URL on an older daemon may use API version
v1.24.
This patch reverts the removal of the API check for maximum supported versions.
The documentation has been updated accordingly
Before this patch is applied, the daemon returns a 200 (success):
$ curl -v --unix-socket /var/run/docker.sock http://localhost/v9999.9999/version
* Trying /var/run/docker.sock...
* Connected to localhost (/Users/sebastiaan/Library/Containers/com.dock) port 80 (#0)
> GET /v9999.9999/version HTTP/1.1
> Host: localhost
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Api-Version: 1.32
< Content-Length: 240
< Content-Type: application/json
< Date: Tue, 10 Oct 2017 09:11:29 GMT
< Docker-Experimental: true
< Ostype: linux
< Server: Docker/17.09.0-ce (linux)
<
{"Version":"17.09.0-ce","ApiVersion":"1.32","MinAPIVersion":"1.12","GitCommit":"afdb6d4","GoVersion":"go1.8.3","Os":"linux","Arch":"amd64","KernelVersion":"4.9.49-moby","Experimental":true,"BuildTime":"2017-09-26T22:45:38.000000000+00:00"}
* Connection #0 to host localhost left intact
After this patch is applied, a 400 (Bad Request) is returned:
$ curl -v --unix-socket /var/run/docker.sock http://localhost/v9999.9999/version
* Trying /var/run/docker.sock...
* Connected to localhost (/var/run/docker.sock) port 80 (#0)
> GET /v9999.9999/info HTTP/1.1
> Host: localhost
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 400 Bad Request
< Content-Type: application/json
< Date: Tue, 10 Oct 2017 08:08:34 GMT
< Content-Length: 89
<
{"message":"client version 9999.9999 is too new. Maximim supported API version is 1.34"}
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Since the commit d88fe447df ("Add support for sharing /dev/shm/ and
/dev/mqueue between containers") container's /dev/shm is mounted on the
host first, then bind-mounted inside the container. This is done that
way in order to be able to share this container's IPC namespace
(and the /dev/shm mount point) with another container.
Unfortunately, this functionality breaks container checkpoint/restore
(even if IPC is not shared). Since /dev/shm is an external mount, its
contents is not saved by `criu checkpoint`, and so upon restore any
application that tries to access data under /dev/shm is severily
disappointed (which usually results in a fatal crash).
This commit solves the issue by introducing new IPC modes for containers
(in addition to 'host' and 'container:ID'). The new modes are:
- 'shareable': enables sharing this container's IPC with others
(this used to be the implicit default);
- 'private': disables sharing this container's IPC.
In 'private' mode, container's /dev/shm is truly mounted inside the
container, without any bind-mounting from the host, which solves the
issue.
While at it, let's also implement 'none' mode. The motivation, as
eloquently put by Justin Cormack, is:
> I wondered a while back about having a none shm mode, as currently it is
> not possible to have a totally unwriteable container as there is always
> a /dev/shm writeable mount. It is a bit of a niche case (and clearly
> should never be allowed to be daemon default) but it would be trivial to
> add now so maybe we should...
...so here's yet yet another mode:
- 'none': no /dev/shm mount inside the container (though it still
has its own private IPC namespace).
Now, to ultimately solve the abovementioned checkpoint/restore issue, we'd
need to make 'private' the default mode, but unfortunately it breaks the
backward compatibility. So, let's make the default container IPC mode
per-daemon configurable (with the built-in default set to 'shareable'
for now). The default can be changed either via a daemon CLI option
(--default-shm-mode) or a daemon.json configuration file parameter
of the same name.
Note one can only set either 'shareable' or 'private' IPC modes as a
daemon default (i.e. in this context 'host', 'container', or 'none'
do not make much sense).
Some other changes this patch introduces are:
1. A mount for /dev/shm is added to default OCI Linux spec.
2. IpcMode.Valid() is simplified to remove duplicated code that parsed
'container:ID' form. Note the old version used to check that ID does
not contain a semicolon -- this is no longer the case (tests are
modified accordingly). The motivation is we should either do a
proper check for container ID validity, or don't check it at all
(since it is checked in other places anyway). I chose the latter.
3. IpcMode.Container() is modified to not return container ID if the
mode value does not start with "container:", unifying the check to
be the same as in IpcMode.IsContainer().
3. IPC mode unit tests (runconfig/hostconfig_test.go) are modified
to add checks for newly added values.
[v2: addressed review at https://github.com/moby/moby/pull/34087#pullrequestreview-51345997]
[v3: addressed review at https://github.com/moby/moby/pull/34087#pullrequestreview-53902833]
[v4: addressed the case of upgrading from older daemon, in this case
container.HostConfig.IpcMode is unset and this is valid]
[v5: document old and new IpcMode values in api/swagger.yaml]
[v6: add the 'none' mode, changelog entry to docs/api/version-history.md]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
This adds the new `CreatedAt` field to the API version history
and updates some examples to show this information.
The `CreatedAt` field was implemented in a46f757c40
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit c79c16910c
inadvertently put these API changes under API 1.31,
but they were added in API 1.30.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Enables other subsystems to watch actions for a plugin(s).
This will be used specifically for implementing plugins on swarm where a
swarm controller needs to watch the state of a plugin.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
COmmit 0307fe1a0b added
a new `DataPathAddr` property to the swarm/init and swarm/join
endpoints. This property was not yet added to the
documentation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This fix tries to add a `scope` in the query of `/networks/<id>`
(`NetworkInspect`) so that in case of duplicate network names,
it is possible to locate the network ID based on the network
scope (`local`, 'swarm', or `global`).
Multiple networks might exist in different scopes, which is a legitimate case.
For example, a network name `foo` might exists locally and in swarm network.
However, before this PR it was not possible to query a network name `foo`
in a specific scope like swarm.
This fix fixes the issue by allowing a `scope` query in `/networks/<id>`.
Additional test cases have been added to unit tests and integration tests.
This fix is related to docker/cli#167, moby/moby#30897, moby/moby#33561, moby/moby#30242
This fix fixesdocker/cli#167
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
With the Moby/Docker split, no decisions have been
made yet how, and when to bump the API version.
Although these decisions should not be lead
by Docker releases, I'm bumping the API version
to not complicate things for now; after this bump
we should make a plan how to handle this in future
(for example, using SemVer for the REST api, and
bump with every change).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add Casbin plugin to the list of Authorization plugins in docs.
(cherry picked from commit 220831d541)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch adds the untilRemoved option to the ContainerWait API which
allows the client to wait until the container is not only exited but
also removed.
This patch also adds some more CLI integration tests for waiting for a
created container and waiting with the new --until-removed flag.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Handle detach sequence in CLI
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Update Container Wait Conditions
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Apply container wait changes to API 1.30
The set of changes to the containerWait API missed the cut for the
Docker 17.05 release (API version 1.29). This patch bumps the version
checks to use 1.30 instead.
This patch also makes a minor update to a testfile which was added to
the builder/dockerfile package.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Remove wait changes from CLI
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Address minor nits on wait changes
- Changed the name of the tty Proxy wrapper to `escapeProxy`
- Removed the unnecessary Error() method on container.State
- Fixes a typo in comment (repeated word)
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Use router.WithCancel in the containerWait handler
This handler previously added this functionality manually but now uses
the existing wrapper which does it for us.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Add WaitCondition constants to api/types/container
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Address more ContainerWait review comments
- Update ContainerWait backend interface to not return pointer values
for container.StateStatus type.
- Updated container state's Wait() method comments to clarify that a
context MUST be used for cancelling the request, setting timeouts,
and to avoid goroutine leaks.
- Removed unnecessary buffering when making channels in the client's
ContainerWait methods.
- Renamed result and error channels in client's ContainerWait methods
to clarify that only a single result or error value would be sent
on the channel.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Move container.WaitCondition type to separate file
... to avoid conflict with swagger-generated code for API response
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Address more ContainerWait review comments
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
The --allow-nondistributable-artifacts daemon option specifies
registries to which foreign layers should be pushed. (By default,
foreign layers are not pushed to registries.)
Additionally, to make this option effective, foreign layers are now
pulled from the registry if possible, falling back to the URLs in the
image manifest otherwise.
This option is useful when pushing images containing foreign layers to a
registry on an air-gapped network so hosts on that network can pull the
images without connecting to another server.
Signed-off-by: Noah Treuhaft <noah.treuhaft@docker.com>
Allows for a plugin type that can be used to scrape metrics.
This is useful because metrics are not neccessarily at a standard
location... `--metrics-addr` must be set, and must currently be a TCP
socket.
Even if metrics are done via a unix socket, there's no guarentee where
the socket may be located on the system, making bind-mounting such a
socket into a container difficult (and racey, failure-prone on daemon
restart).
Metrics plugins side-step this issue by always listening on a unix
socket and then bind-mounting that into a known path in the plugin
container.
Note there has been similar work in the past (and ultimately punted at
the time) for consistent access to the Docker API from within a
container.
Why not add metrics to the Docker API and just provide a plugin with
access to the Docker API? Certainly this can be useful, but gives a lot
of control/access to a plugin that may only need the metrics. We can
look at supporting API plugins separately for this reason.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
in the Docker REST APIs when viewing or updating the swarm spec info, and
also propagate the desired CA key in the Docker REST APIs when updating
swarm spec info only (it is not available for viewing).
Signed-off-by: Ying Li <ying.li@docker.com>
objects into the REST API responses. In the CLI, display only
whether the nodes' TLS info matches the cluster's TLS info, or
whether the node needs cert rotation.
Signed-off-by: Ying Li <ying.li@docker.com>
Starting with this commit, integration tests should no longer rely on
the docker cli, they should be API tests instead. For the existing tests
the scripts will use a frozen version of the docker cli with a
DOCKER_API_VERSION frozen to 1.30, which should ensure that the CI remains
green at all times.
To help contributors develop and test manually with a modified docker
cli, this commit also adds a DOCKER_CLI_PATH environment variable to the
Makefile. This allows to set the path of a custom cli that will be
available inside the development container and used to run the
integration tests.
Signed-off-by: Arnaud Porterie (icecrime) <arnaud.porterie@docker.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
This is synonymous with `docker run --cidfile=FILE` and writes the digest of
the newly built image to the named file. This is intended to be used by build
systems which want to avoid tagging (perhaps because they are in CI or
otherwise want to avoid fixed names which can clash) by enabling e.g. Makefile
constructs like:
image.id: Dockerfile
docker build --iidfile=image.id .
do-some-more-stuff: image.id
do-stuff-with <image.id
Currently the only way to achieve this is to use `docker build -q` and capture
the stdout, but at the expense of losing the build output.
In non-silent mode (without `-q`) with API >= v1.29 the caller will now see a
`JSONMessage` with the `Aux` field containing a `types.BuildResult` in the
output stream for each image/layer produced during the build, with the final
one being the end product. Having all of the intermediate images might be
interesting in some cases.
In silent mode (with `-q`) there is no change, on success the only output will
be the resulting image digest as it was previosuly.
There was no wrapper to just output an Aux section without enclosing it in a
Progress, so add one here.
Added some tests to integration cli tests.
Signed-off-by: Ian Campbell <ian.campbell@docker.com>
Instead of forcing users to manually configure a block device to use
with devmapper, this gives the user the option to let the devmapper
driver configure a device for them.
Adds several new options to the devmapper storage-opts:
- dm.directlvm_device="" - path to the block device to configure for
direct-lvm
- dm.thinp_percent=95 - sets the percentage of space to use for
storage from the passed in block device
- dm.thinp_metapercent=1 - sets the percentage of space to for metadata
storage from the passed in block device
- dm.thinp_autoextend_threshold=80 - sets the threshold for when `lvm`
should automatically extend the thin pool as a percentage of the total
storage space
- dm.thinp_autoextend_percent=20 - sets the percentage to increase the
thin pool by when an autoextend is triggered.
Defaults are taken from
[here](https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/#/configure-direct-lvm-mode-for-production)
The only option that is required is `dm.directlvm_device` for docker to
set everything up.
Changes to these settings are not currently supported and will error
out.
Future work could support allowing changes to these values.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
added some clarification around why attach can appear hung to some. issue #1456 on docs
Signed-off-by: gary schaetz <gary@schaetzkc.com>
Updated the documentation to reflect what happens when you use a fragment in
the docker build url parameter.
Signed-off-by: Gary Schaetz <gary@schaetzkc.com>
added markup for commands
Signed-off-by: Gary Schaetz <gary@schaetzkc.com>
The `Log` field for plugins was added to `/info` in
17abacb894 but the swagger spec was not
updated.
This just updates the spec to match reality.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
The daemon config for defaulting to no-new-privileges for containers was
added in d7fda019bb, but somehow we
managed to omit the flag itself, but also documented the flag.
This just adds the actual flag.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This new flag will allow the configuration of an interface that
can be used for data path traffic to be isolated from control
plane traffic. This flag is simply percolated down to libnetwork
and will be used by all the global scope drivers (today overlay)
Negative test added for invalid flag arguments
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
This does some minor fix-ups in the CLI reference
for "history", and copies the formattting section to
the man-pages.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Simplified the docs on how to set environment variables in a
container. Makes it clear that you have three options, and how
to use them.
Signed-off-by: Joao Fernandes <joao.fernandes@docker.com>
Also fixed some examples of using `docker build` to clarify that the
positional argument is a directory, not a file.
Also fixed some terminology. Dockerfiles contain instructions, not directives or
commands.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
If no fields related to an update config or restart policy are
specified, these structs should not be created as part of the service,
to avoid hardcoding the current defaults.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Service logs API is now stable. Service logs now support all features,
except retrieving details provided to the log driver.
Signed-off-by: Drew Erny <drew.erny@docker.com>
Logging plugins use the same HTTP interface as other plugins for basic
command operations meanwhile actual logging operations are handled (on
Unix) via a fifo.
The plugin interface looks like so:
```go
type loggingPlugin interface {
StartLogging(fifoPath string, loggingContext Context) error
StopLogging(fifoPath)
```
This means a plugin must implement `LoggingDriver.StartLogging` and
`LoggingDriver.StopLogging` endpoints and be able to consume the passed
in fifo.
Logs are sent via stream encoder to the fifo encoded with protobuf.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Resolve networks IDs on the client side.
Avoid filling in deprecated Spec.Networks field.
Sort networks in the TaskSpec for update stability.
Add an integration test for changing service networks.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This parameter controls the order of operations when rolling out an
update task. Either the old task is stopped before starting the new one,
or the new task is started first, and the running tasks will briefly
overlap.
This commit adds Rollout to the API, and --update-order / --rollback-order
flags to the CLI.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This fix tries to address the issue raised in 31032 where it was
not possible to specify `--cpus` for `docker update`.
This fix adds `--cpus` support for `docker update`. In case both
`--cpus` and `--cpu-period/--cpu-quota` have been specified,
an error will be returned.
Related docs has been updated.
Integration tests have been added.
This fix fixes 31032.
This fix is related to 27921, 27958.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This commit adds a hidden placeholder of `.Self` for
`docker node ls --format` so that if the node is the same
as the current docker daemon, then a `*` is outputed.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to address the comment https://github.com/docker/docker/pull/30376#discussion_r97465334
where it was not possible to specify `--format` for `docker node ls`. The `--format` flag
is a quite useful flag that could be used in many places such as completion.
This fix implements `--format` for `docker node ls` and add `nodesFormat` in config.json
so that it is possible to specify the output when `docker node ls` is invoked.
Related documentations have been updated.
A set of unit tests have been added.
This fix is related to #30376.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix is related to 30232 wherw `docker service ls`
does not show `PORTS` information like `docker service ps`.
This fix adds `PORTS` fields for services that publish
ports in ingress mode.
Additional unit tests cases have been updated.
This fix is related to 30232.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Change "service create" and "service update" to wait until the creation
or update finishes, when --detach=false is specified. Show progress bars
for the overall operation and for each individual task (when there are a
small enough number of tasks), unless "-q" / "--quiet" is specified.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This allows graphdrivers to declare that they can reproduce the original
diff stream for a layer. If they do so, the layer store will not use
tar-split processing, but will still verify the digest on layer export.
This makes it easier to experiment with non-default diff formats.
Signed-off-by: Alfred Landrum <alfred.landrum@docker.com>
