Commit graph

2733 commits

Author SHA1 Message Date
Akihiro Suda
9499acc360
bump up rootlesskit to v0.14.4
Fixes `panic: tap2vif: read: read /dev/net/tun: not pollable` on early
start up of RootlessKit with VPNKit.

Changes:
- https://github.com/rootless-containers/rootlesskit/releases/tag/v0.14.4
- https://github.com/rootless-containers/rootlesskit/releases/tag/v0.14.3

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-08-04 16:34:09 +09:00
Roman Volosatovs
3af2217dc4
hack/test/unit: run libnetwork tests sequentially
Run all tests within `libnetwork` namespace with `-p=1`
in a separate `gotestsum` invocation.

Signed-off-by: Roman Volosatovs <roman.volosatovs@docker.com>
2021-08-03 12:19:49 +02:00
Sebastiaan van Stijn
bca9f60845
Merge pull request #42697 from thaJeztah/update_containerd_1.5.5
Update to containerd v1.5.5
2021-08-02 09:58:10 +02:00
Sebastiaan van Stijn
4a07b89e9a
update containerd binary to v1.5.5
Welcome to the v1.5.5 release of containerd!

The fifth patch release for containerd 1.5 updates runc to 1.0.1 and contains
other minor updates.

Notable Updates

- Update runc binary to 1.0.1
- Update pull logic to try next mirror on non-404 response
- Update pull authorization logic on redirect

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-29 19:53:26 +02:00
Sebastiaan van Stijn
79f89ed9b7
hack/ci: remove LCOW from examples, and fix minor linting issue
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-27 13:36:15 +02:00
Sebastiaan van Stijn
cf1328cd46
update containerd binary v1.4.8
Update to containerd 1.4.8 to address [CVE-2021-32760][1].

[1]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-32760

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-19 21:22:25 +02:00
Akihiro Suda
f50c7644cf
update runc binary to v1.0.1
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-07-18 20:19:33 +09:00
Brian Goff
b316cc059a
Merge pull request #42636 from thaJeztah/update_containerd
Update containerd binary and vendor to v1.5.3
2021-07-15 14:10:26 -07:00
Sebastiaan van Stijn
5ae2af41ee
Update containerd binary to v1.5.3
full diff: https://github.com/containerd/containerd/compare/v1.5.2...v1.5.3

Welcome to the v1.5.3 release of containerd!

The third patch release for containerd 1.5 updates runc to 1.0.0 and contains
various other fixes.

Notable Updates

- Update runc binary to 1.0.0
- Send pod UID to CNI plugins as K8S_POD_UID
- Fix invalid validation error checking
- Fix error on image pull resume
- Fix User Agent sent to registry authentication server
- Fix symlink resolution for disk mounts on Windows

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-07-13 23:17:29 +02:00
Kevin Parsons
3334fb9af1 Fix up vndr tooling
- Fix the error message in hack/validate/vendor to specify that
  hack/vendor.sh should be run instead of vndr.
- Fix hack/vendor.sh to also match on Windows paths for the whitelist.
  This allows the script to be run on Windows via Git Bash.

Signed-off-by: Kevin Parsons <kevpar@microsoft.com>
2021-07-13 09:45:07 -07:00
Sebastiaan van Stijn
2050e085f9
hack: remove proxy.installer
It's no longer used

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-25 18:02:51 +02:00
Brian Goff
e7cf711c02
Move proxy CLI to main cmd/
Since this command is part of the official distribution and even
required for tests, let's move this up to the main cmd's.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-25 18:01:55 +02:00
Akihiro Suda
159bad5332
Merge pull request #42450 from AkihiroSuda/runc-v1.0.0
update runc binary to v1.0.0 GA
2021-06-25 13:54:29 +09:00
Frédéric Dalleau
e93132e273 Add s390x to cross platforms target
Update build script and buid environment for supporting s390x
Signed-off-by: Frédéric Dalleau <frederic.dalleau@docker.com>
2021-06-24 18:52:55 +02:00
Frédéric Dalleau
3a208c0ff9 Add ppc64le to cross platforms target
debian uses ppc64el
go uses ppc64le
the cross compiler is powerpc64le
Signed-off-by: Frédéric Dalleau <frederic.dalleau@docker.com>
2021-06-24 18:51:49 +02:00
Akihiro Suda
64badfc018
update runc binary to v1.0.0 GA
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-06-22 15:07:48 +09:00
Sebastiaan van Stijn
2773f81aa5
Merge pull request #42445 from thaJeztah/bump_golang_ci
[testing] ~update~ fix linting issues found by golangci-lint v1.40.1
2021-06-16 22:15:01 +02:00
Tianon Gravi
1dac53860c Adjust buildmode comment to link to commit
This should make sure the link is still meaningful if that file drastically changes (which should make it easier to trace where the interesting block of code moved to and how it changes over time).

Also, add TODO items for Go 1.15+ and 1.16+ where we can "pie" more builds.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2021-06-11 12:34:50 -07:00
Georgy Yakovlev
2cd68d5268
don't use buildmode=pie on ppc64
It's already omitted for ppc64 in
hack/dockerfile/install/install.sh
not using wildcard, because GOARCH=ppc64le supports pie

Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>
2021-06-10 19:06:51 -07:00
Sebastiaan van Stijn
594c972fc5
golangci.yml: do not limit max reported issues
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:54 +02:00
Sebastiaan van Stijn
22ce0f8faa
golangci.yml: skip some tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:52 +02:00
Sebastiaan van Stijn
ea74765a58
golangci.yml: update regex for ignoring SA1019
The message changed from "is deprecated" to "has been deprecated":

    client/hijack.go:85:16: SA1019: httputil.NewClientConn has been deprecated since Go 1.0: Use the Client or Transport in package net/http instead. (staticcheck)
        clientconn := httputil.NewClientConn(conn, nil)
                      ^
    integration/plugin/authz/authz_plugin_test.go:180:7: SA1019: httputil.NewClientConn has been deprecated since Go 1.0: Use the Client or Transport in package net/http instead. (staticcheck)
        c := httputil.NewClientConn(conn, nil)
             ^
    integration/plugin/authz/authz_plugin_test.go:479:12: SA1019: httputil.NewClientConn has been deprecated since Go 1.0: Use the Client or Transport in package net/http instead. (staticcheck)
        client := httputil.NewClientConn(conn, nil)
                  ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-10 13:03:49 +02:00
Sebastiaan van Stijn
e047d984dc
Remove LCOW code (step 1)
The LCOW implementation in dockerd has been deprecated in favor of re-implementation
in containerd (in progress). Microsoft started removing the LCOW V1 code from the
build dependencies we use in Microsoft/opengcs (soon to be part of Microsoft/hcshhim),
which means that we need to start removing this code.

This first step removes the lcow graphdriver, the LCOW initialization code, and
some LCOW-related utilities.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-03 21:16:21 +02:00
Brian Goff
4b981436fe Fixup libnetwork lint errors
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 23:48:32 +00:00
Brian Goff
72c4a7b496 Fix issues running libnetwork tests.
libnetwork does different stuff depending on if you are running the
tests in a container or not... without telling it we are in a container
a bunch of the tests actually fail.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 22:14:41 +00:00
Brian Goff
4da0dc974c Ensure docker-proxy is installed in unit-test ci
(some) libnetwork tests require docker-proxy so we need to install that
in CI.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 22:14:31 +00:00
Brian Goff
86c8b8f1a0 Move proxy build into hack/make
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 22:14:06 +00:00
Sebastiaan van Stijn
454abc560d
Merge pull request #41212 from thaJeztah/archive_vendor
hack/vendor: add check for vendored archive/tar
2021-05-28 00:10:01 +02:00
Sebastiaan van Stijn
e4109b3b6b
vendor: github.com/docker/libnetwork 64b7a4574d1426139437d20e81c0b6d391130ec8
Update libnetwork to make `docker run -p 80:80` functional again on environments
with kernel boot parameter `ipv6.disable=1`.

full diff: b3507428be...64b7a4574d

- fix port forwarding with ipv6.disable=1
    - fixes moby/moby/42288 Docker 20.10.6: all containers stopped and cannot start if ipv6 is disabled on host
    - fixes docker/libnetwork/2629 Network issue with IPv6 following update to version 20.10.6
    - fixesdocker/for-linux/1233 Since 20.10.6 it's not possible to run docker on a machine with disabled IPv6 interfaces
- vendor: github.com/ishidawataru/sctp f2269e66cdee387bd321445d5d300893449805be
- Enforce order of lock acquisitions on network/controller, fixes #2632
    - fixes docker/libnetwork/2632 Name resolution stuck due to deadlock between different network struct methods
    - fixes moby/moby/42032 Docker deamon get's stuck, can't serve DNS requests

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-25 11:33:19 +02:00
Sebastiaan van Stijn
a6e51a7aee
Merge pull request #42391 from thaJeztah/nomod
Disable go modules when compiling binaries used in tests
2021-05-20 21:19:40 +02:00
Brian Goff
8858ecb2ac
Merge pull request #42389 from thaJeztah/update_certs
hack: add script to regenerate certificates and update test-certs
2021-05-20 11:07:32 -07:00
moby
31b2c3bbd9
hack/vendor: add check for vendored archive/tar
Also allow re-vendoring using `./hack/vendor.sh archive/tar`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-20 18:57:52 +02:00
Akihiro Suda
d086c52c9a
Merge pull request #42399 from thaJeztah/update_containerd_1.5.2 2021-05-20 11:36:02 +09:00
Sebastiaan van Stijn
9ce5b7552f
Merge pull request #42394 from thaJeztah/runc_rc95
update runc binary to v1.0.0-rc95
2021-05-19 20:49:15 +02:00
Sebastiaan van Stijn
8e3186fc8f
update containerd binary to v1.5.2
full diff: https://github.com/containerd/containerd/compare/v1.5.1...v1.5.2

The second patch release for containerd 1.5 is a security release to update
runc for CVE-2021-30465

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-19 20:41:16 +02:00
Sebastiaan van Stijn
efec2bb368
update runc binary to v1.0.0-rc95
full diff: https://github.com/opencontainers/runc/compare/v1.0.0-rc94...v1.0.0-rc95

Release notes:

This release of runc contains a fix for CVE-2021-30465, and users are
strongly recommended to update (especially if you are providing
semi-limited access to spawn containers to untrusted users).

Aside from this security fix, only a few other changes were made since
v1.0.0-rc94 (the only user-visible change was the addition of support
for defaultErrnoRet in seccomp profiles).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-19 13:10:20 +02:00
Sebastiaan van Stijn
c706213427
Merge pull request #42384 from thaJeztah/skip_dco
hack/validate: skip DCO in "default"
2021-05-19 12:06:54 +02:00
Sebastiaan van Stijn
8bae2278ba
hack/ci/windows.ps1: disable go modules
INFO: Running integration tests at 05/17/2021 12:54:50...
    INFO: DOCKER_HOST at tcp://127.0.0.1:2357
    INFO: Integration API tests being run from the host:
    INFO: make.ps1 starting at 05/17/2021 12:54:50
    powershell.exe : go: cannot find main module, but found vendor.conf in D:\gopath\src\github.com\docker\docker
    At D:\gopath\src\github.com\docker\docker@tmp\durable-1ed00396\powershellWrapper.ps1:3 char:1
    + & powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -Comm ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (go: cannot find...m\docker\docker:String) [], RemoteException
        + FullyQualifiedErrorId : NativeCommandError

    	to create a module there, run:
    	go mod init
    INFO: make.ps1 ended at 05/17/2021 12:54:51

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-18 09:51:31 +02:00
Sebastiaan van Stijn
2fea30f146
hack: add script to regenerate certificates
Certificates were originally added in c000cb6471,
but did not include a script to generate them. Current versions of Go expect
certificates to use SAN instead of Common Name fields, so updating the script
to include those;

    x509: certificate relies on legacy Common Name field, use SANs or temporarily
    enable Common Name matching with GODEBUG=x509ignoreCN=0

Some fields were updated to be a bit more descriptive (instead of "replaceme"),
and the `-text` option was used to include a human-readable variant of the
content.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-18 09:43:18 +02:00
Akihiro Suda
0f124aba2e
Merge pull request #42383 from thaJeztah/containerd_1.5.1
update containerd binary to v1.5.1
2021-05-18 12:30:16 +09:00
Sebastiaan van Stijn
466e21b273
hack/validate: skip DCO in "default"
We perform a DCO check before we run all other tests, so we can skip it
as part of the validate step.

Leaving the line in for visibility, and in case we switch from Jenkins
to (e.g.) GitHub actions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-17 15:10:50 +02:00
Sebastiaan van Stijn
22c0291333
update containerd binary to v1.5.1
full diff: https://github.com/containerd/containerd/compare/v1.5.0...v1.5.1

Notable Updates

- Update runc to rc94
- Fix registry mirror authorization logic in CRI plugin
- Fix regression in cri-cni-release to include cri tools

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-17 15:03:10 +02:00
Jintao Zhang
8c019e830a Update runc binary to v1.0.0-rc94
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2021-05-11 10:00:03 +08:00
Sebastiaan van Stijn
9b2f55bc1c
update containerd binary to v1.5.0
Welcome to the v1.5.0 release of containerd!

The sixth major release of containerd includes many stability improvements
and code organization changes to make contribution easier and make future
features cleaner to develop. This includes bringing CRI development into the
main containerd repository and switching to Go modules. This release also
brings support for the Node Resource Interface (NRI).

Highlights
--------------------------------------------------------------------------------

*Project Organization*

- Merge containerd/cri codebase into containerd/containerd
- Move to Go modules
- Remove selinux build tag
- Add json log format output option for daemon log

*Snapshots*

- Add configurable overlayfs path
- Separate overlay implementation from plugin
- Native snapshotter configuration and plugin separation
- Devmapper snapshotter configuration and plugin separation
- AUFS snapshotter configuration and plugin separation
- ZFS snapshotter configuration and plugin separation
- Pass custom snapshot labels when creating snapshot
- Add platform check for snapshotter support when unpacking
- Handle loopback mounts
- Support userxattr mount option for overlay in user namespace
- ZFS snapshotter implementation of usage

*Distribution*

- Improve registry response errors
- Improve image pull performance over HTTP 1.1
- Registry configuration package
- Add support for layers compressed with zstd
- Allow arm64 to fallback to arm (v8, v7, v6, v5)

*Runtime*

- Add annotations to containerd task update API
- Add logging binary support when terminal is true
- Runtime support on FreeBSD

*Windows*

- Implement windowsDiff.Compare to allow outputting OCI images
- Optimize WCOW snapshotter to commit writable layers as read-only parent layers
- Optimize LCOW snapshotter use of scratch layers

*CRI*

- Add NRI injection points cri#1552
- Add support for registry host directory configuration
- Update privileged containers to use current capabilities instead of known capabilities
- Add pod annotations to CNI call
- Enable ocicrypt by default
- Support PID NamespaceMode_TARGET

Impactful Client Updates
--------------------------------------------------------------------------------

This release has changes which may affect projects which import containerd.

*Switch to Go modules*

containerd and all containerd sub-repositories are now using Go modules. This
should help make importing easier for handling transitive dependencies. As of
this release, containerd still does not guarantee client library compatibility
for 1.x versions, although best effort is made to minimize impact from changes
to exported Go packages.

*CRI plugin moved to main repository*

With the CRI plugin moving into the main repository, imports under github.com/containerd/cri/
can now be found github.com/containerd/containerd/pkg/cri/.
There are no changes required for end users of CRI.

*Library changes*

oci

The WithAllCapabilities has been removed and replaced with WithAllCurrentCapabilities
and WithAllKnownCapabilities. WithAllKnownCapabilities has similar
functionality to the previous WithAllCapabilities with added support for newer
capabilities. WithAllCurrentCapabilities can be used to give privileged
containers the same set of permissions as the calling process, preventing errors
when privileged containers attempt to get more permissions than given to the
caller.

*Configuration changes*

New registry.config_path for CRI plugin

registry.config_path specifies a directory to look for registry hosts
configuration. When resolving an image name during pull operations, the CRI
plugin will look in the <registry.config_path>/<image hostname>/ directory
for host configuration. An optional hosts.toml file in that directory may be
used to configure which hosts will be used for the pull operation as well
host-specific configurations. Updates under that directory do not require
restarting the containerd daemon.

Enable registry.config_path in the containerd configuration file.

    [plugins."io.containerd.grpc.v1.cri".registry]
       config_path = "/etc/containerd/certs.d"
    Configure registry hosts, such as /etc/containerd/certs.d/docker.io/hosts.toml
    for any image under the docker.io namespace (any image on Docker Hub).

    server = "https://registry-1.docker.io"

    [host."https://public-mirror.example.com"]
      capabilities = ["pull"]
    [host."https://docker-mirror.internal"]
      capabilities = ["pull", "resolve"]
      ca = "docker-mirror.crt"

If no hosts.toml configuration exists in the host directory, it will fallback
to check certificate files based on Docker's certificate file
pattern (".crt" files for CA certificates and ".cert"/".key" files for client
certificates).

*Deprecation of registry.mirrors and registry.configs in CRI plugin*

Mirroring and TLS can now be configured using the new registry.config_path
option. Existing configurations may be migrated to new host directory
configuration. These fields are only deprecated with no planned removal,
however, these configurations cannot be used while registry.config_path is
defined.

*Version 1 schema is deprecated*

Version 2 of the containerd configuration toml is recommended format and the
default. Starting this version, a deprecation warning will be logged when
version 1 is used.

To check version, see the version value in the containerd toml configuration.

    version=2

FreeBSD Runtime Support (Experimental)
--------------------------------------------------------------------------------

This release includes changes that allow containerd to run on FreeBSD with a
compatible runtime, such as runj. This
support should be considered experimental and currently there are no official
binary releases for FreeBSD. The runtimes used by containerd are maintained
separately and have their own stability guarantees. The containerd project
strives to be compatible with any runtime which aims to implement containerd's
shim API and OCI runtime specification.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-04 15:45:01 +02:00
Akihiro Suda
42b1175eda
hack/dind: fix cgroup v2 evacuation with docker run --init
Evacuate all the processes in `/sys/fs/cgroup/cgroup.procs`, not just PID 1.

Before:
```console
$ docker run --rm --privileged --init $(docker build -q .) cat /sys/fs/cgroup/cgroup.subtree_control
sed: couldn't flush stdout: Device or resource busy
```

After:
```console
$ docker run --rm --privileged --init $(docker build -q .) cat /sys/fs/cgroup/cgroup.subtree_control
cpuset cpu io memory hugetlb pids rdma
```

Fix docker-library/docker issue 308

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-28 13:30:10 +09:00
Akihiro Suda
3cd025bc67
bump up rootlesskit to v0.14.2
Fix `Timed out proxy starting the userland proxy.` error with `DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns`.
(https://github.com/rootless-containers/rootlesskit/issues/250)

Full changes: https://github.com/rootless-containers/rootlesskit/compare/v0.14.1...v0.14.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-19 16:41:25 +09:00
Sebastiaan van Stijn
1cb7ee4cd9
Merge pull request #42246 from thaJeztah/replace_toml
Replace BurntSushi/toml with pelletier/go-toml
2021-04-08 21:13:11 +02:00
Sebastiaan van Stijn
257b4db186
testing: update docker-py 5.0.0
drops support for python 2.

full diff: https://github.com/docker/docker-py/compare/4.4.1...5.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-06 19:15:42 +02:00
Sebastiaan van Stijn
1d4a273487
validate/toml: switch to github.com/pelletier/go-toml
The github.com/BurntSushi/toml project is no longer maintained,
and containerd is switching to this project instead, so start
moving our code as well.

This patch only changes the binary used during validation (tbh,
we could probably remove this validation step, but leaving that
for now).

I manually verified that the hack/verify/toml still works by adding a commit
that makes the MAINTAINERS file invalid;

        diff --git a/MAINTAINERS b/MAINTAINERS
        index b739e7e20c..81ababd8de 100644
        --- a/MAINTAINERS
        +++ b/MAINTAINERS
        @@ -23,7 +23,7 @@
                # a subsystem, they are responsible for doing so and holding the
                # subsystem maintainers accountable. If ownership is unclear, they are the de facto owners.

        -               people = [
        +               people =
                                "akihirosuda",
                                "anusha",
                                "coolljt0725",

Running `hack/verify/toml` was able to detect the broken format;

        hack/validate/toml
        (27, 4): keys cannot contain , characterThese files are not valid TOML:
         - MAINTAINERS

        Please reformat the above files as valid TOML

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-02 16:08:17 +02:00
Akihiro Suda
45021ee354
bump up rootlesskit to v0.14.1
Fix `DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns` regression.

Full changes: https://github.com/rootless-containers/rootlesskit/compare/v0.14.0...v0.14.1

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-30 18:01:44 +09:00