Commit graph

7036 commits

Author SHA1 Message Date
Sebastiaan van Stijn
b645c8c70e
Merge pull request #449 from thaJeztah/19.03_backport_move_windows_gopath_out_of_goroot
[19.03 backport] Move GOPATH out from under the GO source tree
2020-01-23 20:21:56 +01:00
Felipe Ruhland
1942d3a8b1
Fix Engine API version history typo
Signed-off-by: Felipe Ruhland <felipe.ruhland@gmail.com>
(cherry picked from commit 8107d44852)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-17 11:23:54 +01:00
Vikram bir Singh
bbd22fb5d9
Move GOPATH out from under the GO source tree
Unlike Linux which uses a temp dir as GOPATH, Windows
uses c:\go. Among other things, this blocks go get.

Moving GOPATH to c:\gopath and updating references in
comments and documentation.

Currently the change is being scoped narrowly. In the
future GOPATH value could be passed as a parameter to
the ps1 scripts.

Signed-off-by: Vikram bir Singh <vikrambir.singh@docker.com>
(cherry picked from commit ecf91f0d7f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-17 10:47:18 +01:00
Akihiro Suda
a8b454a934
docs/rootless.md: update
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e76dea157e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-20 23:50:07 +02:00
Andrew Hsu
5787ef7e9c
Merge pull request #396 from thaJeztah/19.03_backport_update_moved_repositories
[19.03 backport] Update links/references to transferred repositories
2019-10-10 10:58:11 -07:00
Sebastiaan van Stijn
3472e441c5
hack/ci/windows.ps1 update references to repositories that were moved
Also updated the related docs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5175ed54e5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-03 15:37:56 +02:00
Sebastiaan van Stijn
923e849f28
API: update docs that /session left experimental in V1.39
The `/session` endpoint left experimental in API V1.39 through
239047c2d3 and
01c9e7082e, but the API reference
was not updated accordingly.

This updates the API documentation to match the change.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6756f5f378)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-02 12:33:14 +02:00
Tibor Vass
682a46189b
integration-cli: move each test suite to its own TestX testing function
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit f1c1cd436a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-26 23:52:51 +02:00
Tibor Vass
df569fd54c
hack: update scripts
- remove -check.* flags
- use (per-test) -timeout flag
- allow user to override TEST_SKIP_* regardless of TESTFLAGS
- remove test-imports validation

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7cd028f2d0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-26 23:52:44 +02:00
Akihiro Suda
509a793378 rootless: allow exposing dockerd TCP socket easily
eg.

  $ DOCKERD_ROOTLESS_ROOTLESSKIT_FLAGS="-p 0.0.0.0:2376:2376/tcp" \
   dockerd-rootless.sh --experimental \
   -H tcp://0.0.0.0:2376 \
   --tlsverify --tlscacert=ca.pem --tlscert=cert.pem --tlskey=key.pem

This commit bumps up RootlessKit from v0.4.1 to v0.6.0:
27a0c7a248...2fcff6ceae

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 34f4729bc0)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-07-18 13:03:33 +09:00
zhangyue
049a1090c3
fix: fix lack of copyUIDGID in swagger.yaml
Signed-off-by: Zhang Yue <zy675793960@yeah.net>
Signed-off-by: zhangyue <zy675793960@yeah.net>
(cherry picked from commit a4f828cb89)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-07 14:40:21 +02:00
Akihiro Suda
d575af39ac
rootless: update docker info docs
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit ca5aab19b4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-03 22:54:22 +02:00
Akihiro Suda
8493fb18ae
dockerd: fix rootless detection (alternative to #39024)
The `--rootless` flag had a couple of issues:
* #38702: euid=0, $USER="root" but no access to cgroup ("rootful" Docker in rootless Docker)
* #39009: euid=0 but $USER="docker" (rootful boot2docker)

To fix #38702, XDG dirs are ignored as in rootful Docker, unless the
dockerd is directly running under RootlessKit namespaces.

RootlessKit detection is implemented by checking whether `$ROOTLESSKIT_STATE_DIR` is set.

To fix #39009, the non-robust `$USER` check is now completely removed.

The entire logic can be illustrated as follows:

```
withRootlessKit := getenv("ROOTLESSKIT_STATE_DIR")
rootlessMode := withRootlessKit || cliFlag("--rootless")
honorXDG := withRootlessKit
useRootlessKitDockerProxy := withRootlessKit
removeCgroupSpec := rootlessMode
adjustOOMScoreAdj := rootlessMode
```

Close #39024
Fix #38702 #39009

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 3518383ed9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-13 15:31:31 -07:00
Akihiro Suda
e8b9a752d3
rootless: optional support for lxc-user-nic SUID binary
lxc-user-nic can eliminate slirp overhead but needs /etc/lxc/lxc-usernet to be configured for the current user.

To use lxc-user-nic, $DOCKERD_ROOTLESS_ROOTLESSKIT_NET=lxc-user-nic also needs to be set.

This commit also bumps up RootlessKit from v0.3.0 to v0.4.0:
70e0502f32...e92d5e772e

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 63a66b0eb0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-13 15:31:11 -07:00
WithLin
213e49b8f1 change WindowsContainers.md to README.md
Signed-off-by: Fu JinLin <withlin@yeah.net>
2019-04-09 16:11:54 +08:00
WithLin
806db1c663 fix link error.
Signed-off-by: Fu JinLin <withlin@yeah.net>
2019-04-09 15:31:56 +08:00
Akihiro Suda
f0b405fbda rootless: expose ports automatically
Now `docker run -p` ports can be exposed to the host namespace automatically when `dockerd-rootless.sh` is launched with
`--userland-proxy --userland-proxy-path $(which rootlesskit-docker-proxy)`.
This is akin to how Docker for Mac/Win works with `--userland-proxy-path=/path/to/vpnkit-expose-port`.

The port number on the host namespace needs to be set to >= 1024.
SCTP ports are currently unsupported.

RootlessKit changes: 7bbbc48a6f...ed26714429

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-03-21 02:44:08 +09:00
Tibor Vass
827cb09f87
Merge pull request #38898 from tonistiigi/build-outputs
builder-next: allow outputs configuration
2019-03-19 14:00:16 -07:00
Tonis Tiigi
768c6d7b29 builder-next: allow outputs configuration
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-03-19 10:28:30 -07:00
Tibor Vass
8f936ae8cf Add DeviceRequests to HostConfig to support NVIDIA GPUs
This patch hard-codes support for NVIDIA GPUs.
In a future patch it should move out into its own Device Plugin.

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-03-18 17:19:45 +00:00
Sebastiaan van Stijn
9d9d3fe4b4
minor touch-up in api/version-history.md
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-16 17:49:53 +01:00
Kir Kolyshkin
596ca142e0 daemon: use 'private' ipc mode by default
This changes the default ipc mode of daemon/engine to be private,
meaning the containers will not have their /dev/shm bind-mounted
from the host by default. The benefits of doing this are:

 1. No leaked mounts. Eliminate a possibility to leak mounts into
    other namespaces (and therefore unfortunate errors like "Unable to
    remove filesystem for <ID>: remove /var/lib/docker/containers/<ID>/shm:
    device or resource busy").

 2. Working checkpoint/restore. Make `docker checkpoint`
    not lose the contents of `/dev/shm`, but save it to
    the dump, and be restored back upon `docker start --checkpoint`
    (currently it is lost -- while CRIU handles tmpfs mounts,
    the "shareable" mount is seen as external to container,
    and thus rightfully ignored).

3. Better security. Currently any container is opened to share
   its /dev/shm with any other container.

Obviously, this change will break the following usage scenario:

 $ docker run -d --name donor busybox top
 $ docker run --rm -it --ipc container:donor busybox sh
 Error response from daemon: linux spec namespaces: can't join IPC
 of container <ID>: non-shareable IPC (hint: use IpcMode:shareable
 for the donor container)

The soution, as hinted by the (amended) error message, is to
explicitly enable donor sharing by using --ipc shareable:

 $ docker run -d --name donor --ipc shareable busybox top

Compatibility notes:

1. This only applies to containers created _after_ this change.
   Existing containers are not affected and will work fine
   as their ipc mode is stored in HostConfig.

2. Old backward compatible behavior ("shareable" containers
   by default) can be enabled by either using
   `--default-ipc-mode shareable` daemon command line option,
   or by adding a `"default-ipc-mode": "shareable"`
   line in `/etc/docker/daemon.json` configuration file.

3. If an older client (API < 1.40) is used, a "shareable" container
   is created. A test to check that is added.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-03-09 18:57:42 -08:00
Sebastiaan van Stijn
ca91918dd4
docs follow-ups for networks "dangling" filter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-02-28 17:54:31 +01:00
Sebastiaan van Stijn
fc77445147
Add new PidsLimit options to API version history
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-02-24 14:27:30 +01:00
Drew Erny
6f1d7ddfa4 Use Runtime target
The Swarmkit api specifies a target for configs called called "Runtime"
which indicates that the config is not mounted into the container but
has some other use. This commit updates the Docker api to reflect this.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2019-02-19 13:14:17 -06:00
Sebastiaan van Stijn
20383d504b Add support for using Configs as CredentialSpecs in services
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-02-04 15:29:33 -06:00
Akihiro Suda
ec87479b7e allow running dockerd in an unprivileged user namespace (rootless mode)
Please refer to `docs/rootless.md`.

TLDR:
 * Make sure `/etc/subuid` and `/etc/subgid` contain the entry for you
 * `dockerd-rootless.sh --experimental`
 * `docker -H unix://$XDG_RUNTIME_DIR/docker.sock run ...`

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-02-04 00:24:27 +09:00
Sebastiaan van Stijn
7e7e100be0
Add HEAD support for /_ping endpoint
Monitoring systems and load balancers are usually configured to use HEAD
requests for health monitoring. The /_ping endpoint currently does not
support this type of request, which means that those systems have fallback
to GET requests.

This patch adds support for HEAD requests on the /_ping endpoint.

Although optional, this patch also returns `Content-Type` and `Content-Length`
headers in case of a HEAD request; Refering to RFC 7231, section 4.3.2:

    The HEAD method is identical to GET except that the server MUST NOT
    send a message body in the response (i.e., the response terminates at
    the end of the header section).  The server SHOULD send the same
    header fields in response to a HEAD request as it would have sent if
    the request had been a GET, except that the payload header fields
    (Section 3.3) MAY be omitted.  This method can be used for obtaining
    metadata about the selected representation without transferring the
    representation data and is often used for testing hypertext links for
    validity, accessibility, and recent modification.

    A payload within a HEAD request message has no defined semantics;
    sending a payload body on a HEAD request might cause some existing
    implementations to reject the request.

    The response to a HEAD request is cacheable; a cache MAY use it to
    satisfy subsequent HEAD requests unless otherwise indicated by the
    Cache-Control header field (Section 5.2 of [RFC7234]).  A HEAD
    response might also have an effect on previously cached responses to
    GET; see Section 4.3.5 of [RFC7234].

With this patch applied, either `GET` or `HEAD` requests work; the only
difference is that the body is empty in case of a `HEAD` request;

    curl -i --unix-socket /var/run/docker.sock http://localhost/_ping
    HTTP/1.1 200 OK
    Api-Version: 1.40
    Cache-Control: no-cache, no-store, must-revalidate
    Docker-Experimental: false
    Ostype: linux
    Pragma: no-cache
    Server: Docker/dev (linux)
    Date: Mon, 14 Jan 2019 12:35:16 GMT
    Content-Length: 2
    Content-Type: text/plain; charset=utf-8

    OK

    curl --head -i --unix-socket /var/run/docker.sock http://localhost/_ping
    HTTP/1.1 200 OK
    Api-Version: 1.40
    Cache-Control: no-cache, no-store, must-revalidate
    Content-Length: 0
    Content-Type: text/plain; charset=utf-8
    Docker-Experimental: false
    Ostype: linux
    Pragma: no-cache
    Server: Docker/dev (linux)
    Date: Mon, 14 Jan 2019 12:34:15 GMT

The client is also updated to use `HEAD` by default, but fallback to `GET`
if the daemon does not support this method.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-31 18:18:24 +01:00
Yong Tang
393838ca5e
Merge pull request #38569 from thaJeztah/forget_about_it
Add Cache-Control headers to disable caching /_ping endpoint
2019-01-31 23:59:11 +08:00
Olli Janatuinen
80d7bfd54d Capabilities refactor
- Add support for exact list of capabilities, support only OCI model
- Support OCI model on CapAdd and CapDrop but remain backward compatibility
- Create variable locally instead of declaring it at the top
- Use const for magic "ALL" value
- Rename `cap` variable as it overlaps with `cap()` built-in
- Normalize and validate capabilities before use
- Move validation for conflicting options to validateHostConfig()
- TweakCapabilities: simplify logic to calculate capabilities

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-22 21:50:41 +02:00
Sebastiaan van Stijn
5f788fbf56
Add Cache-Control headers to disable caching /_ping endpoint
The result of this endpoint should not be cached, so it's better to
explicitly disable caching.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-14 22:03:23 +01:00
Olli Janatuinen
153171e9dd Added support for maximum replicas per node to services
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2018-12-24 02:04:15 +02:00
Mohammad Nasirifar
c46c3c1689 Update test.md doc to fix sample test regex
Remove the extra dollar sign from the test regex so the command
would find the desired test.

Signed-off-by: Mohammad Nasirifar <farnasirim@gmail.com>
2018-12-13 20:35:49 -05:00
Jonathan A. Schweder
126f371e47 Fix mistake in docs/contributing/test.md running specific unit test (#38096)
Signed-off-by: Jonathan A. Schweder <jonathanschweder@gmail.com>
2018-12-10 11:01:46 -02:00
Yong Tang
f023816608 Add memory.kernelTCP support for linux
This fix tries to address the issue raised in 37038 where
there were no memory.kernelTCP support for linux.

This fix add MemoryKernelTCP to HostConfig, and pass
the config to runtime-spec.

Additional test case has been added.

This fix fixes 37038.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-11-26 21:03:08 +00:00
selansen
32180ac0c7 VXLAN UDP Port configuration support
This commit contains changes to configure DataPathPort
option. By default we use 4789 port number. But this commit
will allow user to configure port number during swarm init.
DataPathPort can't be modified after swarm init.
Signed-off-by: selansen <elango.siva@docker.com>
2018-11-22 17:35:02 -05:00
Akihiro Suda
596cdffb9f mount: add BindOptions.NonRecursive (API v1.40)
This allows non-recursive bind-mount, i.e. mount(2) with "bind" rather than "rbind".

Swarm-mode will be supported in a separate PR because of mutual vendoring.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-11-06 17:51:58 +09:00
Sebastiaan van Stijn
c20aa781e6
Move support for filtering on node labels to API v1.40
This feature was added in 514ce73391,
and was merged after API v1.39 shipped as part of the Docker 18.09

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-26 15:51:23 +02:00
Sebastiaan van Stijn
cb9d2cb71b
Move support for sysctl options in services to API v1.40
This feature was added in 14da20f5e7,
and was merged after API v1.39 shipped as part of the Docker 18.09
release candidates.

This commit moves the feature to the correct API version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-26 15:39:36 +02:00
Sebastiaan van Stijn
e238713cc9
Update API version to v1.40
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-26 15:34:27 +02:00
Yong Tang
87e7930892
Merge pull request #37879 from kolyshkin/no-i
go {build,test}: rm -i option, add go cache volume
2018-09-26 12:52:22 -07:00
Kir Kolyshkin
bdcd81d330 go {build,test}: rm -i option, add go cache volume
Looks like -i (together with DOCKER_INCREMENTAL_BINARY etc)
were used to get faster incremental builds.

Nowdays (since Go 1.10) this is no longer the case, as
go build cache is used [1]. Here's a quote:

> You do not have to use "go test -i" or "go build -i" or
> "go install" just to get fast incremental builds. We will
> not have to teach new users those workarounds anymore.
> Everything will just be fast.

To enable go cache between builds, add a volume for /root/.cache.

[1] https://groups.google.com/forum/#!msg/golang-dev/qfa3mHN4ZPA/X2UzjNV1BAAJ

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-09-26 10:10:42 -07:00
Drew Erny
14da20f5e7 Add support for sysctl options in services
Adds support for sysctl options in docker services.

* Adds API plumbing for creating services with sysctl options set.
* Adds swagger.yaml documentation for new API field.
* Updates the API version history document.
* Changes executor package to make use of the Sysctls field on objects
* Includes integration test to verify that new behavior works.

Essentially, everything needed to support the equivalent of docker run's
`--sysctl` option except the CLI.

Includes a vendoring of swarmkit for proto changes to support the new
behavior.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2018-09-20 10:51:56 -05:00
Anshul Pundir
514ce73391
Changes to cluster/filter, swagger.yaml, version-history.md for filtering on node labels.
Signed-off-by: Anshul Pundir <anshul.pundir@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-08-21 22:17:23 +02:00
Sebastiaan van Stijn
a3d4238b9c
Add "Warnings" to /info endpoint, and move detection to the daemon
When requesting information about the daemon's configuration through the `/info`
endpoint, missing features (or non-recommended settings) may have to be presented
to the user.

Detecting these situations, and printing warnings currently is handled by the
cli, which results in some complications:

- duplicated effort: each client has to re-implement detection and warnings.
- it's not possible to generate warnings for reasons outside of the information
  returned in the `/info` response.
- cli-side detection has to be updated for new conditions. This means that an
  older cli connecting to a new daemon may not print all warnings (due to
  it not detecting the new conditions)
- some warnings (in particular, warnings about storage-drivers) depend on
  driver-status (`DriverStatus`) information. The format of the information
  returned in this field is not part of the API specification and can change
  over time, resulting in cli-side detection no longer being functional.

This patch adds a new `Warnings` field to the `/info` response. This field is
to return warnings to be presented by the user.

Existing warnings that are currently handled by the CLI are copied to the daemon
as part of this patch; This change is backward-compatible with existing
clients; old client can continue to use the client-side warnings, whereas new
clients can skip client-side detection, and print warnings that are returned by
the daemon.

Example response with this patch applied;

```bash
curl --unix-socket /var/run/docker.sock http://localhost/info | jq .Warnings
```

```json
[
  "WARNING: bridge-nf-call-iptables is disabled",
  "WARNING: bridge-nf-call-ip6tables is disabled"
]
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-08-21 11:36:15 +02:00
selansen
f7ad95cab9 Global Default Address Pool feature support
This feature allows user to specify list of subnets for global
default address pool. User can configure subnet list using
'swarm init' command. Daemon passes the information to swarmkit.
We validate the information in swarmkit, then store it in cluster
object. when IPAM init is called, we pass subnet list to IPAM driver.

Signed-off-by: selansen <elango.siva@docker.com>
2018-08-20 15:07:08 -04:00
Daniel Hiltgen
896d1b1c61 Expose license status in Info (#37612)
* Expose license status in Info

This wires up a new field in the Info payload that exposes the license.
For moby this is hardcoded to always report a community edition.
Downstream enterprise dockerd will have additional licensing logic wired
into this function to report details about the current license status.

Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>

* Code review comments

Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>

* Add windows autogen support

Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
2018-08-17 17:05:21 -07:00
Sebastiaan van Stijn
e6e8ab50fa
Do not return "<unknown>" in /info response
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-07-16 16:09:58 +02:00
Donghwa Kim
3048c3a768 Fix typo on test.md
It said `TESTFLAGS='-test.run ^TestValidateIPAddress$' make test-unit`
runs `TestBuild` test, but actually runs `TestValidateIPAddress` test.

Signed-off-by: Donghwa Kim <shanytt@gmail.com>
2018-07-08 18:22:21 +09:00
Benjamin Baker
5122aaf26f Updated path to be consistent w/ current Windows build process
Signed-off-by: Benjamin Baker <Benjamin.baker@utexas.edu>
2018-06-15 19:02:53 -04:00