Commit graph

2900 commits

Author SHA1 Message Date
Sebastiaan van Stijn
9ce5b7552f
Merge pull request #42394 from thaJeztah/runc_rc95
update runc binary to v1.0.0-rc95
2021-05-19 20:49:15 +02:00
Sebastiaan van Stijn
8e3186fc8f
update containerd binary to v1.5.2
full diff: https://github.com/containerd/containerd/compare/v1.5.1...v1.5.2

The second patch release for containerd 1.5 is a security release to update
runc for CVE-2021-30465

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-19 20:41:16 +02:00
Sebastiaan van Stijn
efec2bb368
update runc binary to v1.0.0-rc95
full diff: https://github.com/opencontainers/runc/compare/v1.0.0-rc94...v1.0.0-rc95

Release notes:

This release of runc contains a fix for CVE-2021-30465, and users are
strongly recommended to update (especially if you are providing
semi-limited access to spawn containers to untrusted users).

Aside from this security fix, only a few other changes were made since
v1.0.0-rc94 (the only user-visible change was the addition of support
for defaultErrnoRet in seccomp profiles).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-19 13:10:20 +02:00
Sebastiaan van Stijn
c706213427
Merge pull request #42384 from thaJeztah/skip_dco
hack/validate: skip DCO in "default"
2021-05-19 12:06:54 +02:00
Sebastiaan van Stijn
8bae2278ba
hack/ci/windows.ps1: disable go modules
INFO: Running integration tests at 05/17/2021 12:54:50...
    INFO: DOCKER_HOST at tcp://127.0.0.1:2357
    INFO: Integration API tests being run from the host:
    INFO: make.ps1 starting at 05/17/2021 12:54:50
    powershell.exe : go: cannot find main module, but found vendor.conf in D:\gopath\src\github.com\docker\docker
    At D:\gopath\src\github.com\docker\docker@tmp\durable-1ed00396\powershellWrapper.ps1:3 char:1
    + & powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -Comm ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (go: cannot find...m\docker\docker:String) [], RemoteException
        + FullyQualifiedErrorId : NativeCommandError

    	to create a module there, run:
    	go mod init
    INFO: make.ps1 ended at 05/17/2021 12:54:51

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-18 09:51:31 +02:00
Sebastiaan van Stijn
2fea30f146
hack: add script to regenerate certificates
Certificates were originally added in c000cb6471,
but did not include a script to generate them. Current versions of Go expect
certificates to use SAN instead of Common Name fields, so updating the script
to include those;

    x509: certificate relies on legacy Common Name field, use SANs or temporarily
    enable Common Name matching with GODEBUG=x509ignoreCN=0

Some fields were updated to be a bit more descriptive (instead of "replaceme"),
and the `-text` option was used to include a human-readable variant of the
content.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-18 09:43:18 +02:00
Akihiro Suda
0f124aba2e
Merge pull request #42383 from thaJeztah/containerd_1.5.1
update containerd binary to v1.5.1
2021-05-18 12:30:16 +09:00
Sebastiaan van Stijn
466e21b273
hack/validate: skip DCO in "default"
We perform a DCO check before we run all other tests, so we can skip it
as part of the validate step.

Leaving the line in for visibility, and in case we switch from Jenkins
to (e.g.) GitHub actions.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-17 15:10:50 +02:00
Sebastiaan van Stijn
22c0291333
update containerd binary to v1.5.1
full diff: https://github.com/containerd/containerd/compare/v1.5.0...v1.5.1

Notable Updates

- Update runc to rc94
- Fix registry mirror authorization logic in CRI plugin
- Fix regression in cri-cni-release to include cri tools

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-17 15:03:10 +02:00
Jintao Zhang
8c019e830a Update runc binary to v1.0.0-rc94
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2021-05-11 10:00:03 +08:00
Sebastiaan van Stijn
9b2f55bc1c
update containerd binary to v1.5.0
Welcome to the v1.5.0 release of containerd!

The sixth major release of containerd includes many stability improvements
and code organization changes to make contribution easier and make future
features cleaner to develop. This includes bringing CRI development into the
main containerd repository and switching to Go modules. This release also
brings support for the Node Resource Interface (NRI).

Highlights
--------------------------------------------------------------------------------

*Project Organization*

- Merge containerd/cri codebase into containerd/containerd
- Move to Go modules
- Remove selinux build tag
- Add json log format output option for daemon log

*Snapshots*

- Add configurable overlayfs path
- Separate overlay implementation from plugin
- Native snapshotter configuration and plugin separation
- Devmapper snapshotter configuration and plugin separation
- AUFS snapshotter configuration and plugin separation
- ZFS snapshotter configuration and plugin separation
- Pass custom snapshot labels when creating snapshot
- Add platform check for snapshotter support when unpacking
- Handle loopback mounts
- Support userxattr mount option for overlay in user namespace
- ZFS snapshotter implementation of usage

*Distribution*

- Improve registry response errors
- Improve image pull performance over HTTP 1.1
- Registry configuration package
- Add support for layers compressed with zstd
- Allow arm64 to fallback to arm (v8, v7, v6, v5)

*Runtime*

- Add annotations to containerd task update API
- Add logging binary support when terminal is true
- Runtime support on FreeBSD

*Windows*

- Implement windowsDiff.Compare to allow outputting OCI images
- Optimize WCOW snapshotter to commit writable layers as read-only parent layers
- Optimize LCOW snapshotter use of scratch layers

*CRI*

- Add NRI injection points cri#1552
- Add support for registry host directory configuration
- Update privileged containers to use current capabilities instead of known capabilities
- Add pod annotations to CNI call
- Enable ocicrypt by default
- Support PID NamespaceMode_TARGET

Impactful Client Updates
--------------------------------------------------------------------------------

This release has changes which may affect projects which import containerd.

*Switch to Go modules*

containerd and all containerd sub-repositories are now using Go modules. This
should help make importing easier for handling transitive dependencies. As of
this release, containerd still does not guarantee client library compatibility
for 1.x versions, although best effort is made to minimize impact from changes
to exported Go packages.

*CRI plugin moved to main repository*

With the CRI plugin moving into the main repository, imports under github.com/containerd/cri/
can now be found github.com/containerd/containerd/pkg/cri/.
There are no changes required for end users of CRI.

*Library changes*

oci

The WithAllCapabilities has been removed and replaced with WithAllCurrentCapabilities
and WithAllKnownCapabilities. WithAllKnownCapabilities has similar
functionality to the previous WithAllCapabilities with added support for newer
capabilities. WithAllCurrentCapabilities can be used to give privileged
containers the same set of permissions as the calling process, preventing errors
when privileged containers attempt to get more permissions than given to the
caller.

*Configuration changes*

New registry.config_path for CRI plugin

registry.config_path specifies a directory to look for registry hosts
configuration. When resolving an image name during pull operations, the CRI
plugin will look in the <registry.config_path>/<image hostname>/ directory
for host configuration. An optional hosts.toml file in that directory may be
used to configure which hosts will be used for the pull operation as well
host-specific configurations. Updates under that directory do not require
restarting the containerd daemon.

Enable registry.config_path in the containerd configuration file.

    [plugins."io.containerd.grpc.v1.cri".registry]
       config_path = "/etc/containerd/certs.d"
    Configure registry hosts, such as /etc/containerd/certs.d/docker.io/hosts.toml
    for any image under the docker.io namespace (any image on Docker Hub).

    server = "https://registry-1.docker.io"

    [host."https://public-mirror.example.com"]
      capabilities = ["pull"]
    [host."https://docker-mirror.internal"]
      capabilities = ["pull", "resolve"]
      ca = "docker-mirror.crt"

If no hosts.toml configuration exists in the host directory, it will fallback
to check certificate files based on Docker's certificate file
pattern (".crt" files for CA certificates and ".cert"/".key" files for client
certificates).

*Deprecation of registry.mirrors and registry.configs in CRI plugin*

Mirroring and TLS can now be configured using the new registry.config_path
option. Existing configurations may be migrated to new host directory
configuration. These fields are only deprecated with no planned removal,
however, these configurations cannot be used while registry.config_path is
defined.

*Version 1 schema is deprecated*

Version 2 of the containerd configuration toml is recommended format and the
default. Starting this version, a deprecation warning will be logged when
version 1 is used.

To check version, see the version value in the containerd toml configuration.

    version=2

FreeBSD Runtime Support (Experimental)
--------------------------------------------------------------------------------

This release includes changes that allow containerd to run on FreeBSD with a
compatible runtime, such as runj. This
support should be considered experimental and currently there are no official
binary releases for FreeBSD. The runtimes used by containerd are maintained
separately and have their own stability guarantees. The containerd project
strives to be compatible with any runtime which aims to implement containerd's
shim API and OCI runtime specification.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-04 15:45:01 +02:00
Akihiro Suda
42b1175eda
hack/dind: fix cgroup v2 evacuation with docker run --init
Evacuate all the processes in `/sys/fs/cgroup/cgroup.procs`, not just PID 1.

Before:
```console
$ docker run --rm --privileged --init $(docker build -q .) cat /sys/fs/cgroup/cgroup.subtree_control
sed: couldn't flush stdout: Device or resource busy
```

After:
```console
$ docker run --rm --privileged --init $(docker build -q .) cat /sys/fs/cgroup/cgroup.subtree_control
cpuset cpu io memory hugetlb pids rdma
```

Fix docker-library/docker issue 308

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-28 13:30:10 +09:00
Akihiro Suda
3cd025bc67
bump up rootlesskit to v0.14.2
Fix `Timed out proxy starting the userland proxy.` error with `DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns`.
(https://github.com/rootless-containers/rootlesskit/issues/250)

Full changes: https://github.com/rootless-containers/rootlesskit/compare/v0.14.1...v0.14.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-04-19 16:41:25 +09:00
Sebastiaan van Stijn
1cb7ee4cd9
Merge pull request #42246 from thaJeztah/replace_toml
Replace BurntSushi/toml with pelletier/go-toml
2021-04-08 21:13:11 +02:00
Sebastiaan van Stijn
257b4db186
testing: update docker-py 5.0.0
drops support for python 2.

full diff: https://github.com/docker/docker-py/compare/4.4.1...5.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-06 19:15:42 +02:00
Sebastiaan van Stijn
1d4a273487
validate/toml: switch to github.com/pelletier/go-toml
The github.com/BurntSushi/toml project is no longer maintained,
and containerd is switching to this project instead, so start
moving our code as well.

This patch only changes the binary used during validation (tbh,
we could probably remove this validation step, but leaving that
for now).

I manually verified that the hack/verify/toml still works by adding a commit
that makes the MAINTAINERS file invalid;

        diff --git a/MAINTAINERS b/MAINTAINERS
        index b739e7e20c..81ababd8de 100644
        --- a/MAINTAINERS
        +++ b/MAINTAINERS
        @@ -23,7 +23,7 @@
                # a subsystem, they are responsible for doing so and holding the
                # subsystem maintainers accountable. If ownership is unclear, they are the de facto owners.

        -               people = [
        +               people =
                                "akihirosuda",
                                "anusha",
                                "coolljt0725",

Running `hack/verify/toml` was able to detect the broken format;

        hack/validate/toml
        (27, 4): keys cannot contain , characterThese files are not valid TOML:
         - MAINTAINERS

        Please reformat the above files as valid TOML

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-02 16:08:17 +02:00
Akihiro Suda
45021ee354
bump up rootlesskit to v0.14.1
Fix `DOCKERD_ROOTLESS_ROOTLESSKIT_PORT_DRIVER=slirp4netns` regression.

Full changes: https://github.com/rootless-containers/rootlesskit/compare/v0.14.0...v0.14.1

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-03-30 18:01:44 +09:00
Sebastiaan van Stijn
0450728267
vendor: docker/libnetwork b3507428be5b458cb0e2b4086b13531fb0706e46
full diff: fa125a3512...b3507428be

- fixed IPv6 iptables rules for enabled firewalld (libnetwork#2609)
    - fixes "Docker uses 'iptables' instead of 'ip6tables' for IPv6 NAT rule, crashes"
- Fix regression in docker-proxy
    - introduced in "Fix IPv6 Port Forwarding for the Bridge Driver" (libnetwork#2604)
    - fixes/addresses: "IPv4 and IPv6 addresses are not bound by default anymore" (libnetwork#2607)
    - fixes/addresses "IPv6 is no longer proxied by default anymore" (moby#41858)
- Use hostIP to decide on Portmapper version
    - fixes docker-proxy not being stopped correctly

Port mapping of containers now contain separatet mappings for IPv4 and IPv6 addresses, when
listening on "any" IP address. Various tests had to be updated to take multiple mappings into
account.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-25 00:19:35 +01:00
Akihiro Suda
dea989ea2f
Merge pull request #42192 from tiborvass/rootlesskit_goproxy
hack: use GOPROXY for rootlesskit to workaround issue with old git on…
2021-03-24 14:50:00 +09:00
Tibor Vass
cbc6cefdcb hack: use GOPROXY for rootlesskit to workaround issue with old git on CentOS/RHEL 7
Since rootlesskit removed vendor folder, building it has to rely on go mod.

Dockerfile in docker-ce-packaging uses GOPROXY=direct, which makes "go mod"
commands use git to fetch modules. "go mod" in Go versions before 1.14.1 are
incompatible with older git versions, including the version of git that ships
with CentOS/RHEL 7 (which have git 1.8), see golang/go#38373

This patch switches rootlesskit install script to set GOPROXY to
https://proxy.golang.org so that git is not required for downloading modules.

Once all our code has upgraded to Go 1.14+, this workaround should be
removed.

Signed-off-by: Tibor Vass <tibor@docker.com>
2021-03-23 21:21:26 +00:00
Sebastiaan van Stijn
e166af959d
update rootlesskit to v0.14.0
full diff: https://github.com/rootless-containers/rootlesskit/compare/v0.13.1...v0.14.0

v0.14.0 Changes (since v0.13.2)
--------------------------------------

- CLI: improve --help output
- API: support GET /info
- Port API: support specifying IP version explicitly ("tcp4", "tcp6")
- rootlesskit-docker-proxy: support libnetwork >= 20201216 convention
- Allow vendoring with moby/sys/mountinfo@v0.1.3 as well as @v0.4.0
- Remove socat port driver
    - socat driver has been deprecated since v0.7.1 (Dec 2019)
- New experimental flag: --ipv6
    - Enables IPv6 routing (slirp4netns --enable-ipv6). Unrelated to port driver.

v0.13.2
--------------------------------------

- Fix cleaning up crashed state dir
- Update Go to 1.16
- Misc fixes

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-22 11:00:54 +01:00
Sebastiaan van Stijn
4648e8bc6f
Merge pull request #42067 from AkihiroSuda/vpnkit-arm64
Include VPNkit binary for arm64
2021-03-12 09:02:28 +01:00
Sebastiaan van Stijn
1a49393403
update containerd binary to v1.4.4
full diff: https://github.com/containerd/containerd/compare/v1.4.3...v1.4.4

Release notes:

The fourth patch release for `containerd` 1.4 contains a fix for CVE-2021-21334
along with various other minor issues.
See [GHSA-36xw-fx78-c5r4](https://github.com/containerd/containerd/security/advisories/GHSA-36xw-fx78-c5r4)
for more details related to CVE-2021-21334.

Notable Updates

- Fix container create in CRI to prevent possible environment variable leak between containers
- Update shim server to return grpc NotFound error
- Add bounds on max `oom_score_adj` value for shim's AdjustOOMScore
- Update task manager to use fresh context when calling shim shutdown
- Update Docker resolver to avoid possible concurrent map access panic
- Update shim's log file open flags to avoid containerd hang on syscall open
- Fix incorrect usage calculation

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-03-08 15:27:26 +01:00
Akihiro Suda
088e6ee790
Include VPNkit binary for arm64
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-25 18:16:00 +09:00
Sebastiaan van Stijn
e32ae1973a
Update rootlesskit to v0.13.1 to fix handling of IPv6 addresses
v0.13.1

- Refactor `ParsePortSpec` to handle IPv6 addresses, and improve validation

v0.13.0

- `rootlesskit --pidns`: fix propagating exit status
- Support cgroup2 evacuation, e.g., `systemd-run -p Delegate=yes --user -t rootlesskit --cgroupns --pidns --evacuate-cgroup2=evac --net=slirp4netns bash`

v0.12.0

- Port forwarding API now supports setting `ChildIP`
- The `vendor` directory is no longer included in this repo. Run `go mod vendor` if you need

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-15 09:49:52 +01:00
Sebastiaan van Stijn
28e5a3c5a4
update runc binary to v1.0.0-rc93
full diff: https://github.com/opencontainers/runc/compare/v1.0.0-rc92...v1.0.0-rc93
release notes: https://github.com/opencontainers/runc/releases/tag/v1.0.0-rc93

Release notes for runc v1.0.0-rc93
-------------------------------------------------

This is the last feature-rich RC release and we are in a feature-freeze until
1.0. 1.0.0~rc94 will be released in a few weeks with minimal bug fixes only,
and 1.0.0 will be released soon afterwards.

- runc's cgroupv2 support is no longer considered experimental. It is now
  believed to be fully ready for production deployments. In addition, runc's
  cgroup code has been improved:
    - The systemd cgroup driver has been improved to be more resilient and
      handle more systemd properties correctly.
    - We now make use of openat2(2) when possible to improve the security of
      cgroup operations (in future runc will be wholesale ported to libpathrs to
      get this protection in all codepaths).
- runc's mountinfo parsing code has been reworked significantly, making
  container startup times significantly faster and less wasteful in general.
- runc now has special handling for seccomp profiles to avoid making new
  syscalls unusable for glibc. This is done by installing a custom prefix to
  all seccomp filters which returns -ENOSYS for syscalls that are newer than
  any syscall in the profile (meaning they have a larger syscall number).

  This should not cause any regressions (because previously users would simply
  get -EPERM rather than -ENOSYS, and the rule applied above is the most
  conservative rule possible) but please report any regressions you find as a
  result of this change -- in particular, programs which have special fallback
  code that is only run in the case of -EPERM.
- runc now supports the following new runtime-spec features:
    - The umask of a container can now be specified.
    - The new Linux 5.9 capabilities (CAP_PERFMON, CAP_BPF, and
      CAP_CHECKPOINT_RESTORE) are now supported.
    - The "unified" cgroup configuration option, which allows users to explicitly
      specify the limits based on the cgroup file names rather than abstracting
      them through OCI configuration. This is currently limited in scope to
      cgroupv2.
- Various rootless containers improvements:
    - runc will no longer cause conflicts if a user specifies a custom device
      which conflicts with a user-configured device -- the user device takes
      precedence.
    - runc no longer panics if /sys/fs/cgroup is missing in rootless mode.
- runc --root is now always treated as local to the current working directory.
- The --no-pivot-root hardening was improved to handle nested mounts properly
  (please note that we still strongly recommend that users do not use
  --no-pivot-root -- it is still an insecure option).
- A large number of code cleanliness and other various cleanups, including
  fairly large changes to our tests and CI to make them all run more
  efficiently.

For packagers the following changes have been made which will have impact on
your packaging of runc:

- The "selinux" and "apparmor" buildtags have been removed, and now all runc
  builds will have SELinux and AppArmor support enabled. Note that "seccomp"
  is still optional (though we very highly recommend you enable it).
- make install DESTDIR= now functions correctly.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-02-11 21:46:33 +01:00
Sebastiaan van Stijn
14fb165085
testing: update docker-py 4.4.1
run docker-py integration tests of the latest release;

full diff: https://github.com/docker/docker-py/compare/4.3.0...4.4.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-01-26 11:30:58 +01:00
Sebastiaan van Stijn
f266f13965
Merge pull request #41636 from TBBle/37352-test-and-fix
Set 127GB default sandbox size for WCOW, and ensure storage-opts is honoured on all paths under WCOW and LCOW
2021-01-25 14:34:34 +01:00
Arko Dasgupta
78eafdd947 Vendor in Libnetwork changes
Brings in https://github.com/moby/libnetwork/pull/2604

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-12-15 10:22:03 -08:00
Arko Dasgupta
5fd8d701ce Vendor in libnetwork changes
Vendor in IP6Tables fixes

https://github.com/moby/libnetwork/pull/2600
https://github.com/moby/libnetwork/pull/2603

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-12-11 10:52:32 -08:00
Sebastiaan van Stijn
0a8c769268
update containerd binary to v1.4.3 (CVE-2020-15257)
full diff: https://github.com/containerd/containerd/compare/v1.4.2...v1.4.3

Release notes:

Welcome to the v1.4.3 release of containerd!

The third patch release for containerd 1.4 is a security release to
address CVE-2020-15257. See GHSA-36xw-fx78-c5r4 for more details:

https://github.com/containerd/containerd/security/advisories/GHSA-36xw-fx78-c5r4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-30 20:16:50 +01:00
Sebastiaan van Stijn
703951197c
update containerd binary to v1.4.2
NOTE: the Dockerfile currently uses a single version of Golang for all
      stages. This means that currently, all binaries are built with Go
      1.13.x, including the containerd binary; upstream containerd switched
      to use Go 1.15.

full diff: https://github.com/containerd/containerd/compare/v1.4.1...v1.4.2

Release notes:

Welcome to the v1.4.2 release of containerd!
------------------------------------------------------

The second patch release for containerd 1.4 includes multiple minor fixes
and updates.

Notable Updates

- Fix bug limiting the number of layers by default containerd/cri#1602
- Fix selinux shared memory issue by relabeling /dev/shm containerd/cri#1605
- Fix unknown state preventing removal of containers containerd/containerd#4656
- Fix nil pointer error when restoring checkpoint containerd/containerd#4754
- Improve image pull performance when using HTTP 1.1 containerd/containerd#4653
- Update default seccomp profile for pidfd containerd/containerd#4730
- Update Go to 1.15

Windows

- Fix integer overflow on Windows containerd/containerd#4589
- Fix lcow snapshotter to read trailing tar data containerd/containerd#4628

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-11-26 14:46:45 +01:00
Tibor Vass
22e31dbd9a vendor libnetwork to fix mix up between IPv4 and IPv6
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-11-24 23:22:31 +00:00
Ameya Gawde
2630994b35
Bump libnetwork
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2020-11-17 16:03:27 -08:00
Paul "TBBle" Hampson
142b2b785b Add TestBuildWCOWSandboxSize integration test
This test validates that `RUN` and `COPY` both target a read-write
sandbox on Windows that is configured according to the daemon's
`storage-opts` setting.

Sadly, this is a slow test, so we need to bump the timeout to 60 minutes
from the default of 10 minutes.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-11-10 19:51:46 +11:00
Paul "TBBle" Hampson
080d9df63e Don't clear the exit code from the Finally block
It turns out that the Finally block does not see the exit code from the
`exit` call that triggered it, but from an earlier state. And it seems
that actions take in the Finally block other than `exit` will not affect
the $LastErrorCode set by the `exit` that triggered the Finally block.

Signed-off-by: Paul "TBBle" Hampson <Paul.Hampson@Pobox.com>
2020-11-08 23:15:15 +11:00
Tianon Gravi
3e955cd531
Merge pull request #41634 from AkihiroSuda/rootlesskit-0.11.0
bump up rootlesskit to v0.11.0
2020-11-05 19:12:53 +00:00
Akihiro Suda
c6accc67f2
bump up rootlesskit to v0.11.0
Important fix: Lock state dir for preventing automatic clean-up by systemd-tmpfiles
(https://github.com/rootless-containers/rootlesskit/pull/188)

Full changes:https://github.com/rootless-containers/rootlesskit/compare/v0.10.0...v0.11.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-11-05 16:49:32 +09:00
Arko Dasgupta
1623e6b222 Vendor in Libnetwork changes
Bring in changes from https://github.com/moby/libnetwork/pull/2572 to moby

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-10-29 14:32:04 -07:00
Sebastiaan van Stijn
a66aa12f69
Merge pull request #41597 from cpuguy83/gocache
Pass through GOCACHE to test_env
2020-10-28 09:24:10 +01:00
Brian Goff
1a1f5cc37b Pass through GOCACHE to test_env
Otherwise the cache gets written to bundles instead of to the volume
that we setup in the Makefile as expected.

Found this because vscode did not like having these huge objects in
bundles/ (didn't harm anything, just that it noticed and said hey I don't
like that).

This enables cache-reuse when building the test binaries.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-10-27 19:50:03 +00:00
Sebastiaan van Stijn
a6129c57e6
Bump vndr v0.1.2
full diff: https://github.com/lk4d4/vndr/compare/v0.1.1...v0.1.2

- cleanVCS: prevent panic
- Consider '.syso' as a Go file for vendoring

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-16 17:53:56 +02:00
Sebastiaan van Stijn
5338478546
Dockerfile: fix golang-ci-lint installer
We were using "go get", whith would try to fetch some dependencies
that are not in the version we're installing,

    go get -d github.com/golangci/golangci-lint/cmd/golangci-lint

Causing the build to fail;

    package 4d63.com/gochecknoglobals/checknoglobals: unrecognized import path "4d63.com/gochecknoglobals/checknoglobals" (parse https://4d63.com/gochecknoglobals/checknoglobals?go-get=1: no go-import meta tags ())

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-10-16 13:39:22 +02:00
Sebastiaan van Stijn
0bad48279e
Merge pull request #41463 from StefanScherer/win-investigate-cidut-out
Windows CI: Fix collecting the logs of the daemon under test
2020-09-19 15:00:25 +02:00
Sebastiaan van Stijn
9fb324f00b
vendor: docker/libnetwork d0951081b35fa4216fc4f0064bf065beeb55a74b
full diff: 9e99af28df...d0951081b3

- docker/libnetwork#2560 types: remove some dead code
- docker/libnetwork#2562 client/mflag: remove use of docker/docker/pkg/homedir
- docker/libnetwork#2576 Skip redundant kernel version checks
- docker/libnetwork#2583 vendor: update docker to 7ca355652f and reduce dependency graph

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-18 01:08:20 +02:00
Stefan Scherer
89bec2e267
Fix collecting the daemon under test logs
Signed-off-by: Stefan Scherer <stefan.scherer@docker.com>
2020-09-17 21:12:58 +02:00
Brian Goff
3956a86b62
Merge pull request #41450 from thaJeztah/containerd_1.4.1
update containerd vendor and binary to v1.4.1
2020-09-17 08:01:44 -07:00
Olli Janatuinen
82b5ff8026 Windows CI: Make sure that CI fails on any error
- If unit tests fails
- If intergration tests fails

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2020-09-17 08:59:49 +03:00
Sebastiaan van Stijn
1371a629d5
update containerd binary to v1.4.1
full diff: https://github.com/containerd/containerd/compare/v1.4.0...v1.4.1

Welcome to the v1.4.1 release of containerd!

The first patch release for `containerd` 1.4 includes a fix for v1 shims hanging
on exit and exec when the log pipe fills up along with other minor changes.

Notable Updates:

* Always consume shim logs to prevent logs in the shim from blocking
* Fix error deleting v2 bundle directory when removing rootfs returns `ErrNotExist`
* Fix metrics monitoring of v2 runtime tasks
* Fix incorrect stat for Windows containers
* Fix devmapper device deletion on rollback
* Update seccomp default profile

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-09-16 12:33:45 +02:00
Akihiro Suda
15292d7abf
update containerd binary to v1.4.0
release note: https://github.com/containerd/containerd/releases/tag/v1.4.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-08-19 23:01:28 +09:00
Sebastiaan van Stijn
3ee1ab87b7
Update docker-py to 4.3.0
full diff: https://github.com/docker/docker-py/compare/4.2.2...4.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-13 10:38:32 +02:00
Sebastiaan van Stijn
31d83961e9
Update tini v0.19.0
full diff: https://github.com/krallin/tini/compare/v0.18.0...v0.19.0

Release notes:

- mipsel build
- include checksums in releases
- ppc64le alias for the ppc64el build

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-11 11:26:28 +02:00
Jintao Zhang
2c7b48decd Update runc binary to v1.0.0-rc92
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-08-06 13:37:18 +08:00
Akihiro Suda
43d13054c5
update containerd to v1.3.7
Release note: https://github.com/containerd/containerd/releases/tag/v1.3.7

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-08-04 14:47:57 +09:00
Sebastiaan van Stijn
6302dbbf46
Disable manifest v2 schema 1 push
For CI, a temporary `DOCKER_ALLOW_SCHEMA1_PUSH_DONOTUSE` environment
variable was added while we work out a solution for testing schema 1
pulls (which currently require pushing them to a local registry first
for testing).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-31 20:13:58 +02:00
Akihiro Suda
5bc41368d9
bump up rootlesskit to v0.10.0
Fix port forwarder resource leak (https://github.com/rootless-containers/rootlesskit/issues/153).

Changes: https://github.com/rootless-containers/rootlesskit/compare/v0.9.5...v0.10.0

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-07-28 16:33:30 +09:00
Sebastiaan van Stijn
bf4a5982f0
testing: bump docker-py 4.2.2
full diff: https://github.com/docker/docker-py/compare/4.2.1...4.2.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-17 14:45:28 +02:00
Sebastiaan van Stijn
06832a702d
Merge pull request #41065 from AkihiroSuda/cgroup2-ci
test-integration: support cgroup2
2020-07-16 16:55:47 +02:00
Tibor Vass
a13fd15686
Merge pull request #40499 from thaJeztah/docker_py_api_version
docker-py: override default API version to run all tests
2020-07-16 16:12:22 +02:00
Sebastiaan van Stijn
de5812c2a1
Merge pull request #40807 from wpjunior/plugin-feedback
Improve error feedback when plugin does not implement desired interface
2020-07-16 09:55:17 +02:00
Sebastiaan van Stijn
219e7e7ddc
vendor: libnetwork 9e99af28df21367340c95a3863e31808d689c92a
full diff: 2e24aed516...9e99af28df

- docker/libnetwork#2548 Add docker interfaces to firewalld docker zone
    - fixes docker/for-linux#957 DNS Not Resolving under Network [CentOS8]
    - fixes docker/libnetwork#2496 Port Forwarding does not work on RHEL 8 with Firewalld running with FirewallBackend=nftables
- store.getNetworksFromStore() remove unused error return
- docker/libnetwork#2554 Fix 'failed to get network during CreateEndpoint'
    - fixes/addresses docker/for-linux#888 failed to get network during CreateEndpoint
- docker/libnetwork#2558 [master] bridge: disable IPv6 router advertisements
- docker/libnetwork#2563 log error instead if disabling IPv6 router advertisement failed
    - fixes docker/for-linux#1033 Shouldn't be fatal: Unable to disable IPv6 router advertisement: open /proc/sys/net/ipv6/conf/docker0/accept_ra: read-only file system

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-08 16:12:22 +02:00
Akihiro Suda
91ca3e7a8d
update runc binary to v1.0.0-rc91
release note: https://github.com/opencontainers/runc/releases/tag/v1.0.0-rc91

vendored library isn't updated in this commit (waiting for containerd to vendor runc rc91)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-07-02 20:10:11 +09:00
Jintao Zhang
85e3dddccd update containerd to v1.3.6
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-07-02 00:12:56 +08:00
Brian Goff
a70842f9c8
Merge pull request #41151 from thaJeztah/fix_gotestsum_install_again 2020-06-29 09:38:42 -07:00
Jintao Zhang
0e915e5413 update containerd to v1.3.5
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-06-27 11:09:38 +08:00
Sebastiaan van Stijn
a9d22cad93
hack/install: build gotestsum without -buildmode=pie
No need for this binary as it's only used in tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-26 16:57:09 +02:00
Sebastiaan van Stijn
441aff3a17
fix gotestsum.installer installing wrong version
When using go modules, `go build` will always fetch the latest
version of the package, so ignores the version we previously `go get`'d.

Instead of running `go get` and `go build` separately, this patch uses
`go get` (without the `-d` option) to do it all in one step.

Given that this binary is only used for testing, and only used inside the
Dockerfile, we should consider inlining this step in the Dockerfile itself,
but keeping that separate for now.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-26 16:56:15 +02:00
Akihiro Suda
ed89041433
test-integration: support cgroup2
Usage: DOCKER_BUILD_ARGS="--build-arg CONTAINERD_COMMIT=master --build-arg RUNC_COMMIT=master" DOCKER_EXPERIMENTAL=1 TEST_SKIP_INTEGRATION_CLI=1 make test-integration

Depends on containerd master (v1.4) and runc master (v1.0.0-rc91).

Currently `TEST_SKIP_INTEGRATION_CLI=1` must be specified.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-06-22 14:17:45 +09:00
Tianon Gravi
5ead7ba14b
Merge pull request #41063 from thaJeztah/bump_docker_py
testing: bump docker-py 4.2.1
2020-06-17 16:30:53 -07:00
Akihiro Suda
2befe194af hack: remove useless constants
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-06-05 12:48:38 +09:00
Akihiro Suda
f758d598c5 make install: install containerd-shim-runc-v2
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-06-04 18:08:42 +09:00
Sebastiaan van Stijn
8cc8e26835
testing: bump docker-py 4.2.1
full diff: https://github.com/docker/docker-py/compare/4.2.0...4.2.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-06-03 10:17:08 +02:00
Tibor Vass
83199187ef vendor libnetwork to 2e24aed516bd5c836e11378bb457dd612aa868ed
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-05-28 03:28:51 +00:00
Tibor Vass
b280ea114f hack: add more debugging to understand exit codepath
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-05-27 23:40:11 +00:00
Sebastiaan van Stijn
2d762840f0
Merge pull request #40973 from XiaodongLoong/master
-buildmode=pie is not supported for the mips arch
2020-05-26 08:30:49 +02:00
Brian Goff
07e6b84359
Merge pull request #40950 from AkihiroSuda/dockerd-rootless-setuptool.sh
add dockerd-rootless-setuptool.sh
2020-05-21 11:52:07 -07:00
Xiaodong Liu
0c350e87a0 ldmode=pie is not supported for the mips arch
reference:
https://github.com/docker/cli/pull/2507
4c99c81326

Signed-off-by: Xiaodong Liu <liuxiaodong@loongson.cn>
2020-05-21 09:23:00 +08:00
Tianon Gravi
e573542078
Merge pull request #40892 from thaJeztah/cleanup_containerd_installer
containerd.installer: refactor
2020-05-19 12:21:58 -07:00
Sebastiaan van Stijn
aadc55c605
Dockerfile.windows: fix build of gotestsum
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-16 20:12:39 +02:00
Sebastiaan van Stijn
1d9da1b233
Fix bug in gotestsum installer causing dependencies to not be downloaded
Building gotestsum started to fail after the repository removed some
dependencies on master.

What happens is that first, we `go get` the package (with go modules disabled);

    GO111MODULE=off go get -d gotest.tools/gotestsum

Which gets the latest version from master, and fetches the dependencies used
on master. Then we checkout the version we want to install (for example `v0.3.5`)
and run go build.

However, `v0.3.5` depends on logrus, and given that we ran `go get` for `master`,
that dependency was not fetched, and build fails.

This patch modifies the installer to use go modules (alternatively we could
probably run `go get .` after checking out the `v0.3.5` version),

We need to modify all installers, as it looks like this is a standard pattern
we use, but other dependencies were not failing (yet), so this patch only
addresses the immediate failure.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-16 18:52:53 +02:00
Akihiro Suda
c696b95269 add dockerd-rootless-setuptool.sh
Usage: `dockerd-rootless-setuptool.sh install --force` .
See `--help` for further information.

Co-authored-by: Tianon Gravi <admwiggin@gmail.com>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-05-15 08:46:36 +09:00
Akihiro Suda
17bb5f4b15 bump up rootlesskit to v0.9.5
Supports numeric ID in /etc/subuid and /etc/subgid .
Fix #40926

Full changes: https://github.com/rootless-containers/rootlesskit/compare/v0.9.4...v0.9.5

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-05-11 10:25:00 +09:00
Akihiro Suda
cdf49f37ee
Merge pull request #40672 from tiborvass/19.03.8-forwardport-to-master
vendor: add local copy of archive/tar
2020-05-08 08:11:03 +09:00
Sebastiaan van Stijn
886c8385f5
containerd.installer: refactor
- add `set -e` to prevent linting warnings
- use `install` instead of `cp`
- use a subshell for the whole function, instead of starting one
  inside it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-02 18:44:43 +02:00
Sebastiaan van Stijn
f85bea4ee7
Make binary: install all containerd shims to bundles
The containerd.installer was updated to also copy `containerd-shim-runc-v2`,
but `hack/make/binary-daemon` did not copy it to the bundles directory.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-02 17:29:22 +02:00
Akihiro Suda
f6ac841633 bump up rootlesskit to v0.9.4
Now `rootlesskit-docker-proxy` returns detailed error message on
exposing privileged ports: https://github.com/rootless-containers/rootlesskit/pull/136

Full changes: https://github.com/rootless-containers/rootlesskit/compare/v0.9.2...v0.9.4

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-27 13:02:30 +09:00
Tonis Tiigi
7260adfff9 vendor: add local copy of archive/tar
This version avoids doing name lookups on creating tarball that
should be avoided in to not hit loading glibc shared libraries.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit aa6a9891b0)
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-04-24 11:22:24 -07:00
Wilson Júnior
964731e1d3
Improve error feedback when plugin does not implement desired interface
Signed-off-by: Wilson Júnior <wilsonpjunior@gmail.com>
2020-04-21 18:06:24 -03:00
Jintao Zhang
fbaaca6351 update containerd to v1.3.4
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-04-16 14:36:38 +08:00
Akihiro Suda
5ca47f5179 rootless: graduate from experimental
Close #40484

Note that the support for cgroup v2 isn't ready for production yet,
regardless to rootful or rootless.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-07 00:59:15 +09:00
Akihiro Suda
846b557cb8
Merge pull request #40664 from carlosedp/riscv64-build
Add riscv64 support to the build scripts
2020-04-04 07:04:28 +09:00
Carlos de Paula
7ac638f86a Add support to riscv64 to the build scripts
Added riscv64 architecture support to the scripts used to build Docker
and it's dependencies.

Signed-off-by: Carlos de Paula <me@carlosedp.com>
2020-04-03 14:33:32 -03:00
Sebastiaan van Stijn
97b742676b
hack/ci/windows.ps1: build busybox from local Dockerfile
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-03 09:32:30 +02:00
Sebastiaan van Stijn
7f8b4b621b
Merge pull request #40689 from AkihiroSuda/test-rootless2
test-integration: support more rootless tests
2020-03-30 14:13:34 +02:00
Jintao Zhang
ea44be4b8f vendor: bump libnetwork ef149a924dfde2e506ea3cb3f617d7d0fa96b8ee
fix https://github.com/moby/moby/issues/40715

Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2020-03-19 17:15:58 +08:00
Akihiro Suda
f310bd29bd rootless: support forwarding signals from RootlessKit to dockerd
See https://github.com/rootless-containers/rootlesskit/pull/127

RootlessKit changes: https://github.com/rootless-containers/rootlesskit/compare/v0.9.1...v0.9.2

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-15 12:24:23 +09:00
Sebastiaan van Stijn
e26e1cc5c1
vendor: vishvananda/netlink v1.1.0
- vishvananda/netlink: https://github.com/vishvananda/netlink/compare/v1.0.0...v1.1.0
- vishvananda/netns: 7109fa855b...0a2b9b5464
- libnetwork: bf2bd42abc...beab24292c
- github.com/moby/ipvs: new dependency (was previously part of libnetwork)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-12 22:57:23 +01:00
Tianon Gravi
87c2e65259
Merge pull request #40661 from thaJeztah/update_golangci_lint
ci: update golangci-lint v1.23.8
2020-03-10 14:09:41 -07:00
Sebastiaan van Stijn
51ea39c592
Merge pull request #40658 from AkihiroSuda/install-shim-v2
containerd.installer: install containerd-shim-runc-v2
2020-03-10 16:06:28 +01:00
Sebastiaan van Stijn
e36d222666
ci: update golangci-lint v1.23.8
full diff: https://github.com/golangci/golangci-lint/compare/v1.20.0...v1.23.8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-10 11:52:40 +01:00
Akihiro Suda
d949fddc6c containerd.installer: install containerd-shim-runc-v2
cgroup2 mode requires containerd-shim-runc-v2 (containerd v1.4).

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-10 10:35:21 +09:00
Akihiro Suda
5bf33adba2 hack: support $DOCKER_ROOTLESS for testing rootless
```
$ DOCKER_EXPERIMENTAL=1 DOCKER_ROOTLESS=1 TEST_SKIP_INTEGRATION_CLI=1 \
 make test-integration
```

test-integration-cli is unsupported currently.
Also, tests that spawn custom daemon (testutil/daemon) are skipped.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-10 05:49:45 +09:00
Sebastiaan van Stijn
d5595a5845
Merge pull request #40596 from thaJeztah/bump_libnetwork
vendor: bump libnetwork bf2bd42abc0a3734f12b5ec724e571434e42c669
2020-03-09 17:23:06 +01:00
Akihiro Suda
1ea3a2b7f5 rootless: launch rootlesskit with --propagation=rslave
The propagation was previously set to rprivate and didn't propagate
mounts from the host mount namespace into the daemon's mount namespace.

Further information about --propagation: https://github.com/rootless-containers/rootlesskit/tree/v0.9.1#mount-propagation
RootlessKit changes: https://github.com/rootless-containers/rootlesskit/compare/v0.8.0...v0.9.1

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-07 21:16:29 +09:00
Sebastiaan van Stijn
e1710b42d3
vendor: bump libnetwork bf2bd42abc0a3734f12b5ec724e571434e42c669
full diff: 264bffcb88...bf2bd42abc

relevant changes:

- docker/libnetwork#2407 Macvlan internal network should not change default gateway
    - fixes docker/libnetwork#2406 Internal macvlan network overrides default gateway
- vendor godbus/dbus v5
- Fix InhibitIPv4 nil panic
- Cleanup VFP during overlay network removal
    - fixes VFP leak in windows overlay network deletion

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-07 00:35:47 +01:00
Akihiro Suda
abf2404642 hack/make.sh: remove extra empty lines
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 12:36:06 +09:00
Akihiro Suda
3cf82748dd run shfmt
git grep --name-only '^#!' | egrep -v '(vendor|\.go|Jenkinsfile)' | xargs shfmt -w -bn -ci -sr

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 12:27:49 +09:00
Akihiro Suda
eb484fcb67 validate: add shfmt
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 12:25:51 +09:00
Brian Goff
76e3a49933
Merge pull request #40486 from AkihiroSuda/rootless-cgroup2-systemd
rootless: support `--exec-opt native.cgroupdriver=systemd`
2020-03-02 16:11:21 -08:00
Stefan Scherer
fdf42f7a59
Update Windows ico
Signed-off-by: Stefan Scherer <stefan.scherer@docker.com>
2020-03-02 12:52:10 +01:00
Sam Whited
8c0f6142a8 Update DNS library
This makes sure that we don't become vulnerable to CVE-2018-17419 or
CVE-2019-19794 in the future.

Signed-off-by: Sam Whited <sam@samwhited.com>
2020-02-25 17:05:57 -05:00
Sebastiaan van Stijn
79b130ac31
Revert "Fix containerutility compilation on gcc-mingw-w64 8.3"
This reverts commit 25a1bf53d2.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-14 23:31:42 +01:00
Sebastiaan van Stijn
5125f8b304
bump windows-container-utility aa1ba87e99b68e0113bd27ec26c60b88f9d4ccd9
full diff: e004a1415a...aa1ba87e99

changes:

- Use standard include paths instead of hard-coding

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-14 23:31:14 +01:00
Sebastiaan van Stijn
d8772509d1
Merge pull request #40489 from pricec/fix-license-check
Vendor check to succeed even if copyright information is not found
2020-02-14 23:19:43 +01:00
Akihiro Suda
ca4b51868a rootless: support --exec-opt native.cgroupdriver=systemd
Support cgroup as in Rootless Podman.

Requires cgroup v2 host with crun.
Tested with Ubuntu 19.10 (kernel 5.3, systemd 242), crun v0.12.1.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-14 15:32:31 +09:00
Chris Price
9f94cfdc57 Vendor check to succeed even if copyright information is not found
The documentation for validate_vendor_used in hack/validate/vendor states
that a warning will be emitted if license information cannot be found in
a vendored package. However, because the script is run with pipefail set
(owing to the inclusion of the common validation script .validate) and
`grep -c` is used, the entire script will fail whenever license information
cannot be found in a vendored package.

Signed-off-by: Chris Price <cprice@mirantis.com>
2020-02-13 06:08:44 -08:00
Sebastiaan van Stijn
da6c1429d0
docker-py: skip flaky AttachContainerTest::test_attach_no_stream (again)
This test was disabled in the past, but re-enabled when we upgraded
docker-py to 4.2.0.

The test looks to be still flaky though, so skipping it again:

```
[2020-02-10T23:40:44.429Z] =================================== FAILURES ===================================
[2020-02-10T23:40:44.429Z] __________________ AttachContainerTest.test_attach_no_stream ___________________
[2020-02-10T23:40:44.429Z] tests/integration/api_container_test.py:1250: in test_attach_no_stream
[2020-02-10T23:40:44.429Z]     assert output == 'hello\n'.encode(encoding='ascii')
[2020-02-10T23:40:44.429Z] E   AssertionError: assert b'' == b'hello\n'
[2020-02-10T23:40:44.429Z] E     Right contains more items, first extra item: 104
[2020-02-10T23:40:44.429Z] E     Use -v to get the full diff
[2020-02-10T23:40:44.429Z] ------- generated xml file: /src/bundles/test-docker-py/junit-report.xml -------
````

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-12 18:37:58 +01:00
Sebastiaan van Stijn
5a028d2736
docker-py: override default API version to run all tests
Otherwise some tests are skipped with the default API version
used:

    SKIPPED [1] tests/integration/api_service_test.py:882: API version is too low (< 1.38)
    SKIPPED [1] tests/integration/api_swarm_test.py:59: API version is too low (< 1.39)
    SKIPPED [1] tests/integration/api_swarm_test.py:38: API version is too low (< 1.39)
    SKIPPED [1] tests/integration/api_swarm_test.py:45: API version is too low (< 1.39)
    SKIPPED [1] tests/integration/api_swarm_test.py:52: API version is too low (< 1.39)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 15:50:49 +01:00
Brian Goff
3473ff2893
Merge pull request #40469 from thaJeztah/containerd_1.3.3
update containerd binary to v1.3.3
2020-02-07 12:16:40 -08:00
Sebastiaan van Stijn
486161a63a
bump vndr v0.1.1
full diff: https:/github.com/LK4D4/vndr/compare/v0.1.0...v0.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-07 13:37:45 +01:00
Sebastiaan van Stijn
27649ee44f
update containerd binary to v1.3.3
full diff: https://github.com/containerd/containerd/compare/v1.3.2...v1.3.3
release notes: https://github.com/containerd/containerd/releases/tag/v1.3.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-07 10:21:01 +01:00
Sebastiaan van Stijn
1cb4a60950
testing: bump docker-py to 4.2.0
full diff: https://github.com/docker/docker-py/compare/4.1.0...4.2.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-06 11:55:18 +01:00
Brian Goff
5dffe74df2 Fix windows cross builds.
Windows still writes to the autogen directory, but the source code is
mounted in as read-only.
In order to do enable this without taking a massive hit in doing an rw
mount (for the source code) we mount a tmpfs into the build at the
autogen dir.
In order for this to work the directory must alreay exist, so we create
it before entering the build.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-01-30 15:43:23 -08:00
Akihiro Suda
cd43c1d1ac update runc binary to v1.0.0-rc10 (CVE-2019-19921)
Notable changes:
* Fix CVE-2019-19921 (Volume mount race condition with shared mounts): https://github.com/opencontainers/runc/pull/2207
* Fix exec FIFO race: https://github.com/opencontainers/runc/pull/2185
* Basic support for cgroup v2.  Almost feature-complete, but still missing support for systemd mode in rootless.
  See also https://github.com/opencontainers/runc/issues/2209 for the known issues.

Full changes: https://github.com/opencontainers/runc/compare/v1.0.0-rc9...v1.0.0-rc10

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-01-25 03:55:39 +09:00
Sebastiaan van Stijn
572cb55c0e
hack/make/.go-autogen: fix LDFLAGS to accept values with spaces
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-18 00:40:06 +01:00
Sebastiaan van Stijn
e8122ed724
hack/make/.go-autogen: fix typo in LDFLAGS variable
Introduced in 675b414f56

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-17 12:39:56 +01:00
Arko Dasgupta
4c407caada Bump libnetwork to latest refpoint
Commits:
feeff4f0 Merge pull request #2380 from liskin/bridge-atomic-hwaddr
fec6476d Merge pull request #2489 from suwang48404/doc
8757597e Added document describing libnetwork traffic flow.
eaea5722 Merge pull request #2445 from kdomanski/ipv6-addr-in-hosts
1680ce71 Merge pull request #2462 from arkodg/fix-key-spi-panic
4420ee92 Fix panic in drivers/overlay/encryption.go
57178323 Merge pull request #2472 from thaJeztah/bump_golang_1.12.12
f741dc9c Update Golang 1.12.12 (CVE-2019-17596)
79c19d09 Merge pull request #2461 from suwang48404/master
94facacc Added API to set ephemeral port allocator range.

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-01-16 16:28:23 -08:00
Brian Goff
1d699d40f4
Merge pull request #40359 from thaJeztah/bump_vndr
bump vndr v0.1.0 to support versioned import paths
2020-01-10 09:21:39 -08:00
Sebastiaan van Stijn
d4f05c168d
bump vndr v0.1.0 to support versioned import paths
With this change, go packages/modules that use versioned
import paths (github.com/foo/bar/v2), but don't use a directory
in the repository, can now be supported.

For example:

```
github.com/coreos/go-systemd/v22 v22.0.0
```

will vendor the github.com/coreos/go-systemd repository
into `vendor/github.com/coreos/go-systemd/v22`.

full diff: f5ab8fc5fb...v0.1.0

- LK4D4/vndr#83 migrate bitbucket to api 2.0
    - fixes LK4D4/vndr#82 https://api.bitbucket.org/1.0/repositories/ww/goautoneg: 410 Gone
- LK4D4/vndr#86 Replace sort.Sort with sort.Strings
- LK4D4/vndr#87 support `github.com/coreos/go-systemd/v22`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-09 21:56:20 +01:00
Sebastiaan van Stijn
25a1bf53d2
Fix containerutility compilation on gcc-mingw-w64 8.3
The makefile for this binary has version 6.3 hardcoded,
which causes compilation on 8.3 to fail:

```
Building: bundles/cross/windows/amd64/containerutility.exe
In file included from /usr/x86_64-w64-mingw32/include/minwindef.h:163,
                 from /usr/x86_64-w64-mingw32/include/windef.h:8,
                 from /usr/x86_64-w64-mingw32/include/windows.h:69,
                 from containerutility.h:3,
                 from argumentstream.cpp:1:
/usr/x86_64-w64-mingw32/include/winnt.h:1554:11: fatal error: x86intrin.h: No such file or directory
 # include <x86intrin.h>
           ^~~~~~~~~~~~~
compilation terminated.
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-09 18:52:09 +01:00
Sebastiaan van Stijn
5dbfae6949
hack/make: ignore failure to stop apparmor
```
 ---> Making bundle: .integration-daemon-stop (in bundles/test-integration)
 ++++ cat bundles/test-integration/docker.pid
 +++ kill 13137
 +++ /etc/init.d/apparmor stop
 Leaving: AppArmorNo profiles have been unloaded.

 Unloading profiles will leave already running processes permanently
 unconfined, which can lead to unexpected situations.

 To set a process to complain mode, use the command line tool
 'aa-complain'. To really tear down all profiles, run 'aa-teardown'."

script returned exit code 255
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-09 18:52:03 +01:00
Sebastiaan van Stijn
f2b25e498f
docker-py: skip broken ImageCollectionTest::test_pull_multiple
The ImageCollectionTest.test_pull_multiple test performs a `docker pull` without
a `:tag` specified) to pull all tags of the given repository (image).

After pulling the image, the image(s) pulled are checked to verify if the list
of images contains the `:latest` tag.

However, the test assumes that all tags of the image are tags for the same
version of the image (same digest), and thus a *single* image is returned, which
is not always the case.

Currently, the `hello-world:latest` and `hello-world:linux` tags point to a
different digest, therefore the `client.images.pull()` returns multiple images:
one image for digest, making the test fail:

    =================================== FAILURES ===================================
    ____________________ ImageCollectionTest.test_pull_multiple ____________________
    tests/integration/models_images_test.py:90: in test_pull_multiple
        assert len(images) == 1
    E   AssertionError: assert 2 == 1
    E    +  where 2 = len([<Image: 'hello-world:linux'>, <Image: 'hello-world:latest'>])

This patch temporarily skips the broken test until it is fixed upstream.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-03 15:47:39 +01:00
Sebastiaan van Stijn
6bc45b09e7
docker-py: re-enable tests that were fixed in v4.1.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-03 15:46:28 +01:00
Sebastiaan van Stijn
bbcabf69c8
Merge pull request #40050 from thaJeztah/ci_force_validate
Jenkinsfile: force validate on master, and some fixes/improvements
2019-12-23 13:39:43 +01:00
Vikram bir Singh
8d2e1ee546
Build and use gotestsum for running all tests
1. Dockerfile.Windows modified to build gotestsum.exe

2. Use gotestsum.exe in invoking the execution of:

   (a) Unit tests (run in containers),
   (b) Integration tests (run outside containers)
   (c) Integration-cli (run outside containers)

No changes made to other categories of tests (e.g.
LCOW).

3. Copy .xml files produced by gotestsum in
   appropriate paths where Jenkins can ingest them

4. Modify Jenkinsfile to mark results output as
   being jUnit "type" as well as to archive the
   .xml test result files as artifacts.

Signed-off-by: Vikram bir Singh <vikrambir.singh@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 20:31:22 +01:00
Sebastiaan van Stijn
d3c0261876
hack/validate/deprecate-integration-cli: add integration-cli/*_api_*.go
The intent of this check is to not add integration tests that use the
CLI to test API functionality, so check for changes in "API" tests as
well, now that the `integration/` suite has matured enough to develop
new tests, or migrate existing ones.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 18:07:12 +01:00
Sebastiaan van Stijn
1f0fdcc6f5
hack/validate: clean-up output, move golang-ci-lint to end
The golang-ci-lint takes longest to run, so do the other
checks first; this also makes the output a bit more readable.

While at it, fix some consistency issues in the output of the
other checks.

Before this change:

    TEST_FORCE_VALIDATE=1 hack/validate/default
    Congratulations!  All commits are properly signed with the DCO!
    Congratulations! Seccomp profile generation is done correctly.
    INFO [config_reader] Used config file hack/validate/golangci-lint.yml
    INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
    ...
    INFO Execution took 2m12.4287302s
    Congratulations!  "./pkg/..." is safely isolated from internal code.
    The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
    Congratulations! All api changes are done the right way.
    Congratulations!  All toml source files changed here have valid syntax.
    Congratulations!  Changelog CHANGELOG.md is well-formed.
    Congratulations!  Changelog CHANGELOG.md dates are in descending order.

After this change:

    TEST_FORCE_VALIDATE=1 hack/validate/default
    Congratulations!  All commits are properly signed with the DCO!
    Congratulations!  Seccomp profile generation is done correctly.
    Congratulations!  Packages in "./pkg/..." are safely isolated from internal code.
    Congratulations!  The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
    Congratulations!  All API changes are done the right way.
    Congratulations!  All TOML source files changed here have valid syntax.
    Congratulations!  Changelog CHANGELOG.md is well-formed.
    Congratulations!  Changelog CHANGELOG.md dates are in descending order.
    Congratulations!  No new tests were added to integration-cli.

    INFO Start validation with golang-ci-lint
    INFO [config_reader] Used config file hack/validate/golangci-lint.yml
    INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
    ...
    INFO Execution took 2m12.4287302s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 18:07:07 +01:00
Sebastiaan van Stijn
545e81714b
hack/validate: allow forcing validation
This introduces a FORCE_VALIDATE environment variable, which allows
forcing some validation steps, even if no changes were detected.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 18:03:55 +01:00
Sebastiaan van Stijn
9dde8f2b46
hack/validate/deprecate-integration-cli: don't exit on success
The `hack/validate/deprecate-integration-cli` script exited on
success. As a result, validation steps to run afterwards would
not be executed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 17:59:23 +01:00
Sebastiaan van Stijn
e21019fa51
hack/validate/default: fix minor linting issue
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 17:58:03 +01:00
Sebastiaan van Stijn
c8ecedbde9
Revert "hack: temporarily disable deprecate-integration-cli"
This reverts commit 5f67568bfd.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 17:57:56 +01:00
Tianon Gravi
3f2d1eb95d
Merge pull request #40317 from AkihiroSuda/bump-up-rootlesskit
rootless: fix proxying UDP packets
2019-12-18 14:49:49 -08:00
Akihiro Suda
8b8c01dd87
Merge pull request #40074 from yedamao/fix-integration-cli
integration-cli: Fix `SA1019: httputil.ClientConn is deprecated`
2019-12-19 03:30:34 +09:00
Akihiro Suda
658723badd rootless: fix proxying UDP packets
UDP reply packets were not proxied: https://github.com/rootless-containers/rootlesskit/issues/86

The issue was fixed in RootlessKit v0.7.1: https://github.com/rootless-containers/rootlesskit/pull/87

Full changes since v0.7.0: https://github.com/rootless-containers/rootlesskit/compare/v0.7.0...v0.7.1

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-12-18 19:33:27 +09:00
Tõnis Tiigi
550d5f9d43
Merge pull request #40180 from cpuguy83/no_more_dev_tools
Build binaries with minimal deps and remove autogen code
2019-12-12 11:29:30 -08:00
Jintao Zhang
7f809e1080 Update containerd to v1.3.2
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-12-09 01:23:40 +08:00
Brian Goff
79a52bb1ba Add support for outputing binaries to custom dir
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-12-02 10:21:46 -08:00
Brian Goff
675b414f56 Use -X ldflags to set dockerversion package vars
This eliminates the need to lay down an auto-generated file.
IIRC this was originally hadded for gccgo which we no longer support.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-12-02 10:21:41 -08:00
Brian Goff
3152f94362
Merge pull request #40048 from tao12345666333/update-to-runc-1.0.0-rc9
Update containerd to v1.3.1, runc v1.0.0-rc9
2019-11-27 14:20:17 -08:00
Jintao Zhang
517946eb47 Update containerd to v1.3.1
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-11-20 23:23:07 +08:00
Sebastiaan van Stijn
9bcbc66032
Merge pull request #40169 from cpuguy83/windows_version_quad_err
Windows: Only set VERSION_QUAD if unset
2019-11-11 17:14:12 -08:00
Sebastiaan van Stijn
9cf349d0f8
bump libnetwork 90afbb01e1d8acacb505a092744ea42b9f167377
full diff: 0025177e3d...90afbb01e1

includes:

- docker/libnetwork#/2459 Fix Error Check in NewNetwork
- docker/libnetwork#/2466 Revert "Merge pull request #2339 from phyber/iptables-check"
    - reverts docker/libnetwork#/2339 controller: Check if IPTables is enabled for arrangeUserFilterRule
    - re-opens docker/libnetwork#2158 dockerd when run with --iptables=false modifies iptables by adding DOCKER-USER
    - re-opens moby/moby#35777 With iptables=false dockerd still creates DOCKER-USER chain and rules
    - re-opens docker/for-linux#136 dockerd --iptables=false adds DOCKER-USER chain and modify FORWARD chain anyway

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-07 13:43:42 -08:00
Sebastiaan van Stijn
e9bd017b68
Merge pull request #40148 from kolyshkin/vendor-diff
hack/validate/vendor: print diff for modified files
2019-11-07 12:42:03 -08:00
Kirill Kolyshkin
c36460c437
Merge pull request #40077 from thaJeztah/fix_autogen_detection
Update "auto-generate" comments to improve detection by linters
2019-11-05 13:04:14 -08:00
Brian Goff
47c5c67ed8
Merge pull request #40032 from jmartin84/fix-grpc-withdialer-deprecation-warning
Fix grpc withdialer deprecation warning
2019-11-05 12:20:33 -08:00
Sebastiaan van Stijn
6186e9fe87
hack/make/.go-autogen: fix "generated code" comment not in correct format
As described in https://golang.org/s/generatedcode, Go has
a formalized format that should be used to indicate that a
file is generated.

Matching that format helps linters to skip generated files;

From https://golang.org/s/generatedcode (https://github.com/golang/go/issues/13560#issuecomment-288457920);

> Generated files are marked by a line of text that matches the regular expression, in Go syntax:
>
> ^// Code generated .* DO NOT EDIT\.$
>
> The `.*` means the tool can put whatever folderol it wants in there, but the comment
> must be a single line and must start with `Code generated` and end with `DO NOT EDIT.`,
> with a period.
>
> The text may appear anywhere in the file.

This patch updates the autogenerated code to match that format.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-05 11:32:43 -08:00
Brian Goff
ce931f28ea Windows: Only set VERSION_QUAD if unset
When trying to build with some pretty typical version strings this was
causing failures trying to generate the windows resource file.

The resource file is already gated by an `ifdef` for this var, so
instead of blindly setting based on "VERSION", which can contain some
characters which are incompatible (e.g. 1.2.3.rc.0 will fail due to the
".rc").

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-11-04 17:16:06 -08:00
Kir Kolyshkin
5791019028 hack/validate/vendor: simplify looking for license
It was suggested that we use '.*\(COPYING\|LICENSE\|COPYRIGHT\).*'
as an argument to `find -iregex`, and this is how it all started.

Next thing, there is no COPYRIGHT in any of the vendored packages,
so it can be removed for good.

Next, we should not look too deep inside the package directory, as
the license should be in its root directory, so add `-maxdepth 1`
to `find`. This should also speed things up.

Finally, since we're not using the recursion feature of `find`,
it can be replaced with `echo | grep`.

While at it,
* avoid temporary $pkgs variable as it is only used once;
* replace `ls -d "vendor/$f"  > /dev/null 2>&1` with `test -d`.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-10-29 10:59:05 -07:00
Kir Kolyshkin
4be12ad3d0 hack/validate/vendor: shellcheck fixes
The export statement is definitely not needed. The rest is obvious.

> In hack/validate/vendor line 3:
> export SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
>        ^-- SC2155: Declare and assign separately to avoid masking return values.
>
>
> In hack/validate/vendor line 43:
>	if ls -d vendor/$f  > /dev/null 2>&1; then
>                       ^-- SC2086: Double quote to prevent globbing and word splitting.
>
>
> In hack/validate/vendor line 44:
> 		found=$(find vendor/$f -iregex '.*LICENSE.*' -or -iregex '.*COPYRIGHT.*' -or -iregex '.*COPYING.*' | wc -l)
>                                   ^-- SC2086: Double quote to prevent globbing and word splitting.
>
>
> In hack/validate/vendor line 45:
>		if [ $found -eq 0 ]; then
>                    ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-10-28 19:27:55 -07:00
Kir Kolyshkin
9d4e81e8bf hack/validate/vendor: print diff for modified files
In case some files were modified (rather than merely added
or removed), we're curious to see the diff for those.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-10-28 19:22:28 -07:00
Jintao Zhang
5ce9df2a40 Update to runc v1.0.0-rc9
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-10-29 01:59:24 +08:00
Brian Goff
221012c92c
Merge pull request #40110 from thaJeztah/fix_test_filter
Fix TEST_FILTER to work for both "integration" and "integration-cli"
2019-10-25 09:00:13 -07:00
HuanHuan Ye
203ba72fc5 integration-cli: Fix SA1019: httputil.ClientConn is deprecated
Rewrite sockRequestHijack to requestHijack which use writable
Transport's Response.Body to replace deprecated hijacked httputil.ClientConn.
```
// As of Go 1.12, the Body will also implement io.Writer
// on a successful "101 Switching Protocols" response,
// as used by WebSockets and HTTP/2's "h2c" mode.
Body io.ReadCloser
```.

TestPostContainersAttach and TestExecResizeImmediatelyAfterExecStart
replace all sockRequestHijack to requestHijack.

Signed-off-by: HuanHuan Ye <logindaveye@gmail.com>
2019-10-22 10:55:29 +08:00
Sebastiaan van Stijn
e7805653b8
Fix TEST_FILTER to work for both "integration" and "integration-cli"
The TEST_FILTER variable allows running a single integration or integration-cli
test. However, it failed to work properly for integration-cli tests.

Before:
-----------

    # Filtering "integration" tests works:
    make TEST_FILTER=TestInspectCpusetInConfigPre120 test-integration
    ...
    DONE 1 tests in 18.331s

    # But running a single test in "integration-cli" did not:

    make TEST_FILTER=TestSwarmNetworkCreateIssue27866 test-integration
    ...
    DONE 0 tests in 17.314s

Trying to manually add the `/` prefix, didn't work either, because that made the
"grep" fail to find which test-suites to run/skip:

    make TEST_FILTER=/TestSwarmNetworkCreateIssue27866 test-integration
    ---> Making bundle: test-integration (in bundles/test-integration)
    make: *** [test-integration] Error 1

After:
-----------

    make TEST_FILTER=TestInspectCpusetInConfigPre120 test-integration
    ...
    DONE 1 tests in 18.331s

    make TEST_FILTER=TestSwarmNetworkCreateIssue27866 test-integration
    ...
    DONE 12 tests in 26.527s

Note that the `12` tests is still a bit misleading, because every _suite_ is
started (which is counted as a test), but no tests are run. This is still
something that could be improved on.

This patch also makes a small modification to the code that's setting
`integration_api_dirs`, and no longer runs `go list` if not needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-21 14:03:58 +02:00
Sebastiaan van Stijn
9cb9c6eba4
bump golangci-lint v1.20.0
release notes:

- v1.20.0: https://github.com/golangci/golangci-lint/releases/tag/v1.20.0
    - Add WSL linter (#771)
    - Add gocognit linter (#756)
    - govet: add more analyzers
    - dramatically reduce memory usage
    - reduce 1.5x memory usage on large repos on repeated runs
    - Rename deadline option to timeout and mark deadline as deprecated
- v1.19.1: https://github.com/golangci/golangci-lint/releases/tag/v1.19.1
- v1.19.0: https://github.com/golangci/golangci-lint/releases/tag/v1.19.0
    - Add dogsled linter.
    - Add whitespace linter
    - Add Godox linter support
    - Various performance optimizations
- v1.18.0: https://github.com/golangci/golangci-lint/releases/tag/v1.18.0
    - Support go1.13
    - Update format of junit xml output to mark failures as such
    - Speed up linting: use deduplicated packages
    - Add user supplied error messages in depguard issues
    - Add funlen linter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-18 00:46:15 +02:00
Sebastiaan van Stijn
a06d7f5a3e
golangci-lint: include all enabled linters in config
This makes it clearer from the configuration what linters
are enabled.

Also disables the `gofmt` linter, because it's superseded
by the `goimports` linter.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-18 00:46:13 +02:00
Sebastiaan van Stijn
086b4541cf
golangci-lint: disable default exclusion rules
The default exclusion rules were too permissive; disable
them, but copy the relevant ones to the configuation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-18 00:46:11 +02:00
Sebastiaan van Stijn
fe3c14d5ba
golangci-lint: fix "golint" not ignoring generated files
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-18 00:46:08 +02:00
Sebastiaan van Stijn
47502344b9
golangci-lint: update exclusion rules for todo's
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-18 00:46:04 +02:00
Grant Millar
df7b8f458a daemon: Use short libnetwork ID in exec-root & update libnetwork
Signed-off-by: Grant Millar <rid@cylo.io>
2019-10-15 11:40:24 +01:00
Brian Goff
28b645755a
Merge pull request #40062 from thaJeztah/testutil_improvements
Various testutil improvements
2019-10-11 15:37:48 -07:00
Sebastiaan van Stijn
9407a57522
hack/make: don't attempt to unmount non-existing daemon root-dir
Before:

    DONE 2 tests in 12.272s
    ---> Making bundle: .integration-daemon-stop (in bundles/test-integration)
    umount: bundles/test-integration/root: mountpoint not found

After:

    DONE 2 tests in 14.650s
    ---> Making bundle: .integration-daemon-stop (in bundles/test-integration)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 00:34:53 +02:00
Justen Martin
3b49bd1d84
replaced call to deprecated grpc method WithDialer with WithContextDialer
Signed-off-by: Justen Martin <jmart@the-coder.com>
2019-10-10 15:34:42 -05:00
Justen Martin
095ca77f48
Use build args to override binary commits in dockerfile
Signed-off-by: Justen Martin <jmart@the-coder.com>
2019-10-10 14:52:57 -05:00
Tibor Vass
d31633be7d
Merge pull request #39678 from thaJeztah/carry_39595_tailor_arm_ci
[Carry 39595] Tailor CI for ARM, skip legacy integration test.
2019-10-10 11:22:02 -07:00
Michael Zhao
48b06a2561
Tailor CI for ARM, skip legacy integration test.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 18:37:13 +02:00
Derek McGowan
6c94a50f41
update containerd binary v1.3.0
full diff: https://github.com/containerd/containerd/compare/v1.2.8..v1.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-10-04 15:05:32 -07:00
Sebastiaan van Stijn
5a703ccb46
bump docker-py to 4.1.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-03 16:51:03 +02:00
Sebastiaan van Stijn
48353e16fe
Temporarily switch docker-py to "master"
The docker-py tests were broken, because the version of
py-test that was used, used a dependency that had a new
major release with a breaking change.

Unfortunately, it was not pinned to a specific version,
so when the dependency did the release, py-test broke;

```
22:16:47  Traceback (most recent call last):
22:16:47    File "/usr/local/bin/pytest", line 10, in <module>
22:16:47      sys.exit(main())
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 61, in main
22:16:47      config = _prepareconfig(args, plugins)
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 182, in _prepareconfig
22:16:47      config = get_config()
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 156, in get_config
22:16:47      pluginmanager.import_plugin(spec)
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 530, in import_plugin
22:16:47      __import__(importspec)
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/tmpdir.py", line 25, in <module>
22:16:47      class TempPathFactory(object):
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/tmpdir.py", line 35, in TempPathFactory
22:16:47      lambda p: Path(os.path.abspath(six.text_type(p)))
22:16:47  TypeError: attrib() got an unexpected keyword argument 'convert'
```

docker-py master has a fix for this (bumping the version of
`py-test`), but it's not in a release yet, and the docker cli that's used
in our CI is pinned to 17.06, which doesn't support building from a remote
git repository from a specific git commit.

To fix the immediate situation, this patch switches the docker-py
tests to run from the master branch.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-02 17:54:48 +02:00
Sebastiaan van Stijn
bc9a7ec898
update runc to v1.0.0-rc8-92-g84373aaa (CVE-2019-16884)
full diff: https://github.com/opencontainers/runc/compare/v1.0.0-rc8...3e425f80a8c931f88e6d94a8c831b9d5aa481657

  - opencontainers/runc#2010 criu image path permission error when checkpoint rootless container
  - opencontainers/runc#2028 Update to Go 1.12 and drop obsolete versions
  - opencontainers/runc#2029 Update dependencies
  - opencontainers/runc#2034 Support for logging from children processes
  - opencontainers/runc#2035 specconv: always set "type: bind" in case of MS_BIND
  - opencontainers/runc#2038 `r.destroy` can defer exec in `runner.run` method
  - opencontainers/runc#2041 Change the permissions of the notify listener socket to rwx for everyone
  - opencontainers/runc#2042 libcontainer: intelrdt: add missing destroy handler in defer func
  - opencontainers/runc#2047 Move systemd.Manager initialization into a function in that module
  - opencontainers/runc#2057 main: not reopen /dev/stderr
      - closes opencontainers/runc#2056 Runc + podman|cri-o + systemd issue with stderr
      - closes kubernetes/kubernetes#77615 kubelet fails starting CRI-O containers (Ubuntu 18.04 + systemd cgroups driver)
      - closes cri-o/cri-o#2368 Joining worker node not starting flannel or kube-proxy / CRI-O error "open /dev/stderr: no such device or address"
  - opencontainers/runc#2061 libcontainer: fix TestGetContainerState to check configs.NEWCGROUP
  - opencontainers/runc#2065 Fix cgroup hugetlb size prefix for kB
  - opencontainers/runc#2067 libcontainer: change seccomp test for clone syscall
  - opencontainers/runc#2074 Update dependency libseccomp-golang
  - opencontainers/runc#2081 Bump CRIU to 3.12
  - opencontainers/runc#2089 doc: First process in container needs `Init: true`
  - opencontainers/runc#2094 Skip searching /dev/.udev for device nodes
      - closes opencontainers/runc#2093 HostDevices() race with older udevd versions
  - opencontainers/runc#2098 man: fix man-pages
  - opencontainers/runc#2103 cgroups/fs: check nil pointers in cgroup manager
  - opencontainers/runc#2107 Make get devices function public
  - opencontainers/runc#2113 libcontainer: initial support for cgroups v2
  - opencontainers/runc#2116 Avoid the dependency on cgo through go-systemd/util package
      - removes github.com/coreos/pkg as dependency
  - opencontainers/runc#2117 Remove libcontainer detection for systemd features
      - fixes opencontainers/runc#2117 Cache the systemd detection results
  - opencontainers/runc#2119 libcontainer: update masked paths of /proc
      - relates to moby/moby#36368 Add /proc/keys to masked paths
      - relates to moby/moby#38299 Masked /proc/asound
      - relates to moby/moby#37404 Add /proc/acpi to masked paths (CVE-2018-10892)
  - opencontainers/runc#2122 nsenter: minor fixes
  - opencontainers/runc#2123 Bump x/sys and update syscall for initial Risc-V support
  - opencontainers/runc#2125 cgroup: support mount of cgroup2
  - opencontainers/runc#2126 libcontainer/nsenter: Don't import C in non-cgo file
  - opencontainers/runc#2129 Only allow proc mount if it is procfs
      - addresses opencontainers/runc#2129 AppArmor can be bypassed by a malicious image that specifies a volume at /proc (CVE-2019-16884)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-27 16:22:34 +02:00
Jintao Zhang
c4ec02b0af Update containerd to v1.2.10
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-09-27 11:47:20 +08:00
Sebastiaan van Stijn
5175ed54e5
hack/ci/windows.ps1 update references to repositories that were moved
Also updated the related docs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-26 12:14:38 +02:00
Akihiro Suda
1dd9260e47
Merge pull request #39956 from jmartin84/fix-closenotifier-deprecation-warning
Removed deprecated CloseNotifier logic
2019-09-24 02:37:15 +09:00
Sebastiaan van Stijn
9b07a3e0a5
Merge pull request #39917 from StefanScherer/windows-zap-fixed-subdir
Zap a fixed windows folder, add build number to folder inside
2019-09-23 13:12:29 +02:00
Sebastiaan van Stijn
7d573c7b61
Merge pull request #39967 from thaJeztah/bump_libnetwork
bump libnetwork to 96bcc0dae898308ed659c5095526788a602f4726
2019-09-23 10:56:14 +02:00
Sebastiaan van Stijn
6c3113e599
Merge pull request #39951 from yedamao/fix-base-Dial-is-deprecated
Fix base.Dial is deprecated: Use DialContext instead
2019-09-21 13:01:40 +02:00
Sebastiaan van Stijn
75477f0b3c
bump libnetwork to 96bcc0dae898308ed659c5095526788a602f4726
full diff: 92d1fbe1eb...96bcc0dae8

changes included:

- docker/libnetwork#2429 Updating IPAM config with results from HNS create network call
  - addresses moby/moby#38358
- docker/libnetwork#2450 Always configure iptables forward policy
  - related to moby/moby#14041 and docker/libnetwork#1526

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-20 19:48:49 +02:00
Akihiro Suda
4e8cf9f5dd
Merge pull request #39960 from thaJeztah/lint_integration_cli
integration-cli: enable golangci-lint
2019-09-20 13:26:29 +09:00
HuanHuan Ye
a57fd5488d Fix base.Dial is deprecated: Use DialContext instead
1.Change base.Dial to base.DailContext.
2.Remove proxyDialer that was previously used to configure a
net.Dialer to route proxy.Dialer which will route the connections
through the proxy using the connections through a SOCKS proxy.
SOCKS proxies are now supported by configuring only http.Transport.Proxy,
and no longer require changing http.Transport.Dial.

Signed-off-by: HuanHuan Ye <logindaveye@gmail.com>
2019-09-20 10:42:13 +08:00
Brian Goff
0f18e434b5
Merge pull request #39911 from tiborvass/gotestsum-integration-cli
hack: have integration-cli use gotestsum codepath
2019-09-19 08:01:25 -07:00
Stefan Scherer
4866207543
Zap a fixed folder, add build number to folder inside
Signed-off-by: Stefan Scherer <stefan.scherer@docker.com>
2019-09-19 15:52:07 +02:00
Sebastiaan van Stijn
851b000641
integration-cli: enable golangci-lint
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-19 10:50:31 +02:00
Tibor Vass
3acf0cc795
Merge pull request #39930 from thaJeztah/fix_TESTFLAGS
hack/test/unit: fix custom TESTFLAGS not working
2019-09-18 17:56:23 -07:00
Justen Martin
23ab331979 Removed deprecated CloseNotifier logic
Signed-off-by: Justen Martin <jmart@the-coder.com>
2019-09-18 16:14:30 -05:00
Tibor Vass
f1c1cd436a integration-cli: move each test suite to its own TestX testing function
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-18 18:26:36 +00:00
Tibor Vass
84928be605 hack: have integration-cli use gotestsum codepath
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-18 18:26:36 +00:00
Sebastiaan van Stijn
5f67568bfd
Revert "Revert "hack: temporarily disable deprecate-integration-cli""
This reverts commit 01332b54e0.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:58:02 +02:00
Sebastiaan van Stijn
6f0dfe44eb
golangci-lint: suppress remaining deprecation errors for now (staticcheck)
```
distribution/registry.go:84:3: SA1019: base.Dial is deprecated: Use DialContext instead, which allows the transport to cancel dials as soon as they are no longer needed. If both are set, DialContext takes priority.  (staticcheck)
registry/registry.go:188:3:    SA1019: base.Dial is deprecated: Use DialContext instead, which allows the transport to cancel dials as soon as they are no longer needed. If both are set, DialContext takes priority.  (staticcheck)

client/hijack.go:85:16:                               SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead.  (staticcheck)
integration-cli/docker_api_attach_test.go:245:12:     SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead.  (staticcheck)
integration/plugin/authz/authz_plugin_test.go:180:7:  SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead.  (staticcheck)
integration/plugin/authz/authz_plugin_test.go:479:12: SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead.  (staticcheck)

integration-cli/docker_api_attach_test.go:239:134:    SA1019: httputil.ClientConn is deprecated: Use Client or Transport in package net/http instead.  (staticcheck)

daemon/daemon.go:885:3:                               SA1019: grpc.WithDialer is deprecated: use WithContextDialer instead.  Will be supported throughout 1.x.  (staticcheck)

pkg/authorization/response.go:18:2:                   SA1019: http.CloseNotifier is deprecated: the CloseNotifier interface predates Go's context package. New code should use Request.Context instead.  (staticcheck)
pkg/authorization/response.go:160:30:                 SA1019: http.CloseNotifier is deprecated: the CloseNotifier interface predates Go's context package. New code should use Request.Context instead.  (staticcheck)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:58:01 +02:00
Sebastiaan van Stijn
df7d719e4c
golangci-lint: suppress Xattrs is deprecated: Use PAXRecords instead
```
pkg/archive/archive.go:407:3:       SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead.  (staticcheck)
pkg/archive/archive.go:408:3:       SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead.  (staticcheck)
pkg/archive/archive.go:661:26:      SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead.  (staticcheck)
pkg/archive/archive_linux.go:47:7:  SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead.  (staticcheck)
pkg/archive/archive_linux.go:48:12: SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead.  (staticcheck)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:58:00 +02:00
Sebastiaan van Stijn
f41712c4be
golangci-lint: enable misspell, tweak configuration
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:59 +02:00
Sebastiaan van Stijn
c9553897f3
golangci-lint: enable gosec linter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:58 +02:00
Sebastiaan van Stijn
d1363f5a90
golangci-lint: set correct build-tags to lint journald if supported
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:58 +02:00
Sebastiaan van Stijn
85b89c9e50
Replace gometalinter with golangci-lint
Replacing gometalinter, because it has been deprecated,
and the repository was archived.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:57 +02:00
Sebastiaan van Stijn
0620990307
hack/test/unit: fix custom TESTFLAGS not working
The `-test.timeout=5m` was glued directly after the current `TESTFLAGS`,
causing them to be non-functional;

Before:

    make TESTDEBUG=1 TESTDIRS='github.com/docker/docker/pkg/filenotify' TESTFLAGS='-test.run TestPollerEvent' test-unit
    + mkdir -p bundles
    + gotestsum --format=standard-quiet --jsonfile=bundles/go-test-report.json --junitfile=bundles/junit-report.xml -- -tags 'netgo seccomp libdm_no_deferred_remove' -cover -coverprofile=bundles/profile.out -covermode=atomic -test.run TestPollerEvent-test.timeout=5m github.com/docker/docker/pkg/filenotify
    testing: warning: no tests to run
    ok  	github.com/docker/docker/pkg/filenotify	0.003s	coverage: 0.0% of statements [no tests to run]

    DONE 0 tests in 0.298s

After:

    make TESTDEBUG=1 TESTDIRS='github.com/docker/docker/pkg/filenotify' TESTFLAGS='-test.run TestPollerEvent' test-unit
    + mkdir -p bundles
    + gotestsum --format=standard-quiet --jsonfile=bundles/go-test-report.json --junitfile=bundles/junit-report.xml -- -tags 'netgo seccomp libdm_no_deferred_remove' -cover -coverprofile=bundles/profile.out -covermode=atomic -test.run TestPollerEvent -test.timeout=5m github.com/docker/docker/pkg/filenotify
    ok  	github.com/docker/docker/pkg/filenotify	0.608s	coverage: 44.7% of statements

    DONE 1 tests in 0.922s

This was introduced in 42f0a0db75

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-16 09:15:02 +02:00
Brian Goff
4b55f3cf8b
Merge pull request #39854 from thaJeztah/hack_windows_explicit_exit
hack/ci/windows.ps1: explicitly set exit code to result of tests
2019-09-12 11:34:44 -07:00
Brian Goff
15d3310f72
Merge pull request #39877 from thaJeztah/ci_windows_storageopt
hack/ci/windows.ps1: add support for DOCKER_STORAGE_OPTS
2019-09-12 11:13:03 -07:00
Kirill Kolyshkin
7bcffeef6e
Merge pull request #39901 from thaJeztah/remove_docker_cli_autogen
hack/make: remove autogen resources for Docker CLI
2019-09-12 20:21:51 +03:00
Tibor Vass
2900a919f4
Merge pull request #39879 from thaJeztah/fix_go_version_check_again
hack/ci/windows.ps1: fix Go version check (due to trailing .0)
2019-09-11 17:34:45 -07:00
Sebastiaan van Stijn
f007b0150a
integration: improve package- and filename for junit.xml
Generate more unique names, based on architecture and test-suite name.

Clean up the path to this integration test to create a useful package name.
"$dir" can be either absolute (/go/src/github.com/docker/docker/integration/foo)
or relative (./integration/foo). To account for both, first we strip the
absolute path, then any leading periods and slashes.

For the package-name, we use periods as separator instead of slashes, to be more
in-line with Java package names (which is what junit.xml was originally designed
for).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 22:31:56 +02:00
Sebastiaan van Stijn
d2e00d62e2
integration: test2json: enable timestamps to fix zero-time test durations
Without these options set, test2json does not include a `Time`
field in the generated JSON;

    {"Action":"run","Test":"TestCgroupNamespacesBuild"}
    {"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"=== RUN   TestCgroupNamespacesBuild\n"}
    {"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (1.70s)\n"}
    ...
    {"Action":"pass","Test":"TestCgroupNamespacesBuild"}

As a result, `gotestsum` was not able to calculate test-duration, and
reported `time="0.000000"` for all tests;

    <testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="0.000000"></testcase>

With this patch applied:

    {"Time":"2019-08-23T22:42:41.644361357Z","Action":"run","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild"}
    {"Time":"2019-08-23T22:42:41.644367647Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"=== RUN   TestCgroupNamespacesBuild\n"}
    {"Time":"2019-08-23T22:42:44.926933252Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (3.28s)\n"}
    ...
    {"Time":"2019-08-23T22:42:44.927003836Z","Action":"pass","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Elapsed":3.28}

Which now correctly reports the test's duration:

    <testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="3.280000"></testcase>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 22:31:54 +02:00
Sebastiaan van Stijn
f3be6b346f
integration: use gotestsum to generate junit.xml and go-test-report.json
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 22:31:52 +02:00
Tibor Vass
553df774d4
Merge pull request #39802 from thaJeztah/integration_parallel_simplify
integration: simplify parallel run destination
2019-09-11 11:09:54 -07:00
Tibor Vass
6d8c5bc181
Merge pull request #39878 from thaJeztah/cut_the_tail
hack/ci/windows.ps1: stop tailing logs after stopping the daemon
2019-09-11 10:03:11 -07:00
Sebastiaan van Stijn
3262a69be6
integration: simplify parallel run destination
'Namespace' parallel runs by bind-mounting a different directory
in the container, instead of making the tests running inside
the container aware of the namespaced location.

This makes it transparent to the tests, and slightly reduces
complexity.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 17:18:38 +02:00
Sebastiaan van Stijn
f2498e21c4
hack/make: remove autogen resources for Docker CLI
the files used by the docker cli were moved to the docker/cli repository,
so are no longer needed here.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 15:16:07 +02:00
Sebastiaan van Stijn
a464a3d51f
Dockerfile: remove GOMETALINTER_OPTS
This `ENV` was added to the Dockerfile in b96093fa56,
when the repository used per-architecture Dockerfiles, and some architectures needed
a different configuration.

Now that we use a multi-arch Dockerfile, and CI uses a Jenkinsfile, we can remove
this `ENV` from the Dockerfile, and set it in CI instead if needed.

Also updated the wording and fixed linting issues in hack/validate/gometalinter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 08:58:38 +02:00
Tibor Vass
6754976015
Merge pull request #39888 from vikramhh/move_windows_gopath_out_of_goroot
Move GOPATH out from under the GO source tree
2019-09-10 14:33:12 -07:00
Sebastiaan van Stijn
01332b54e0
Revert "hack: temporarily disable deprecate-integration-cli"
This reverts commit beadc92e07.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-10 23:01:46 +02:00
Sebastiaan van Stijn
1fd0107d7f
Merge pull request #39799 from tiborvass/bye-bye-gocheck
Remove go-check
2019-09-10 22:58:57 +02:00
Vikram bir Singh
ecf91f0d7f Move GOPATH out from under the GO source tree
Unlike Linux which uses a temp dir as GOPATH, Windows
uses c:\go. Among other things, this blocks go get.

Moving GOPATH to c:\gopath and updating references in
comments and documentation.

Currently the change is being scoped narrowly. In the
future GOPATH value could be passed as a parameter to
the ps1 scripts.

Signed-off-by: Vikram bir Singh <vikrambir.singh@docker.com>
2019-09-10 16:57:33 +00:00
Sebastiaan van Stijn
61450a651b
hack/ci/windows.ps1: fix Go version check (due to trailing .0)
The Windows Dockerfile downloads the Go binaries, which (unlike
the Golang images) do not have a trailing `.0` in their version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-10 11:40:31 +02:00
Sebastiaan van Stijn
536e26c81a
Merge pull request #39874 from tao12345666333/update-containerd-v1.2.9
Update containerd to v1.2.9
2019-09-10 00:10:47 +02:00
Tibor Vass
8bffe9524d remove per-test -timeout logic because it does not work
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-09 21:09:57 +00:00
Tibor Vass
beadc92e07 hack: temporarily disable deprecate-integration-cli
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-09 21:09:57 +00:00
Tibor Vass
7cd028f2d0 hack: update scripts
- remove -check.* flags
- use (per-test) -timeout flag
- allow user to override TEST_SKIP_* regardless of TESTFLAGS
- remove test-imports validation

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-09 21:09:57 +00:00
Sebastiaan van Stijn
e1636ad5fa
hack/ci/windows.ps1: stop tailing logs after stopping the daemon
There's already a step in  "Nuke Everything", but lets' stop it
after stopping the daemon as well

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-07 17:50:01 +02:00
Sebastiaan van Stijn
b6f596c411
hack/ci/windows.ps1: add support for DOCKER_STORAGE_OPTS
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-07 17:46:23 +02:00
Sebastiaan van Stijn
7eb522a235
hack/ci/windows.ps1 print all environment variables to check how Jenkins runs this script
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-06 23:32:57 +02:00
Sebastiaan van Stijn
8e8c52c4ab
hack/ci/windows.ps1: explicitly set exit code to result of tests
Trying to see if this helps with the cleanup step exiting in CI, but
Jenkins continuing to wait for the script to end afterwards.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-06 23:31:07 +02:00
Tibor Vass
ef2890c295
Merge pull request #39848 from thaJeztah/docker_py_disable_flaky
docker-py: skip flaky AttachContainerTest::test_attach_no_stream
2019-09-06 13:46:32 -07:00
Jintao Zhang
9ef9a337f6 Update containerd to v1.2.9
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-09-06 10:07:08 +08:00
Tibor Vass
30166b9a4b
Merge pull request #39858 from thaJeztah/Jenkinsfile_windows_bundles
Jenkinsfile: create bundles for Windows stages
2019-09-05 11:19:56 -07:00
Sebastiaan van Stijn
edaf292c6e
Merge pull request #39840 from AkihiroSuda/slirp4netns-sandbox
rootless: harden slirp4netns with mount namespace and seccomp
2019-09-04 20:45:19 +02:00
Sebastiaan van Stijn
6ee61f5493
Jenkinsfile: create bundles for Windows stages
CI already stores the logs of the test daemon, so we might as well
store them as artifacts

```
[2019-09-03T12:49:39.835Z] INFO: Tidying up at end of run
[2019-09-03T12:49:39.835Z] INFO: Saving daemon under test log (d:\CI-2\CI-3593e7622\dut.out) to C:\windows\TEMP\CIDUT.out
[2019-09-03T12:49:39.835Z] INFO: Saving daemon under test log (d:\CI-2\CI-3593e7622\dut.err) to C:\windows\TEMP\CIDUT.err
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-03 18:48:27 +02:00
Sebastiaan van Stijn
bab58c1924
bump lib network to 92d1fbe1eb0883cf11d283cea8e658275146411d
full diff: 09cdcc8c0e...92d1fbe1eb

relevant changes included (omitting some changes that were added _and_ reverted in this bump):

- docker/libnetwork#2433 Fix parseIP error when parseIP before get AddressFamily
  - fixes docker/libnetwork#2431 parseIP Error ip=[172 17 0 2 0 0 0 0 0 0 0 0 0 0 0 0]
  - https://github.com/docker/libnetwork/issues/2289
  - this was a regression introduced in docker/libnetwork#2416 Fix hardcoded AF_INET for IPv6 address handling
- docker/libnetwork#2440 Bump hashicorp go-sockaddr v1.0.2, go-multierror v1.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-03 11:49:06 +02:00
Kirill Kolyshkin
1a117b8b5c
Merge pull request #39804 from thaJeztah/defender_warning_only
integration: windows.ps1: turn defender error into a warning
2019-09-02 18:34:48 +03:00
Kirill Kolyshkin
d99b0302d3
Merge pull request #39836 from thaJeztah/hack_linting_nits
hack/make/binary-daemon: fix some linting issues
2019-09-02 16:45:24 +03:00
Sebastiaan van Stijn
ce77a804b8
docker-py: skip flaky AttachContainerTest::test_attach_no_stream
Seen failing a couple of times:

```
[2019-09-02T08:40:15.796Z] =================================== FAILURES ===================================
[2019-09-02T08:40:15.796Z] __________________ AttachContainerTest.test_attach_no_stream ___________________
[2019-09-02T08:40:15.796Z] tests/integration/api_container_test.py:1250: in test_attach_no_stream
[2019-09-02T08:40:15.796Z]     assert output == 'hello\n'.encode(encoding='ascii')
[2019-09-02T08:40:15.796Z] E   AssertionError: assert b'' == b'hello\n'
[2019-09-02T08:40:15.796Z] E     Right contains more items, first extra item: 104
[2019-09-02T08:40:15.796Z] E     Use -v to get the full diff
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-02 15:36:32 +02:00
Sebastiaan van Stijn
c0ef45e90c
Merge pull request #39831 from StefanScherer/fix-dutimgVersion
Fix docker inspect for dutimgVersion
2019-09-02 10:41:03 +02:00
Akihiro Suda
e20b7323fb rootless: harden slirp4netns with mount namespace and seccomp
When slirp4netns v0.4.0+ is used, now slirp4netns is hardened using
mount namespace ("sandbox") and seccomp to mitigate potential
vulnerabilities.

bump up rootlesskit: 2fcff6ceae...791ac8cb20

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-09-02 14:58:58 +09:00
Sebastiaan van Stijn
70d3677825
hack/make/binary-daemon: fix some linting issues
- Add quotes to prevent word splitting in `cp` statement (SC2046)
- Replace legacy back tics with `$()`
- Replace `which` with `command -v` (SC2230)
- Fix incorrect (`==`) comparison

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-31 12:12:25 +02:00
Stefan Scherer
52a53e2587
Fix docker inspect for dutimgVersion
Signed-off-by: Stefan Scherer <stefan.scherer@docker.com>
2019-08-30 12:44:20 +02:00
Sebastiaan van Stijn
156ad54fb7
windows.ps1: fix leaked NdisAdapters not being cleaned up on RS1
Windows RS1 has problems with leaking NdisAdapters during the integration
tests; the windows.ps1 script has a cleanup stesp to remove those
leaked adapters.

For internal testing at Microsoft on internal builds, this cleanup step
was skipped, and only ran on the CI machines in our Jenkins.

Due to the move to our new Jenkins, the names of Windows machines changed,
and because of that, the cleanup step was never executed, resulting in the
leaked adapters not being cleaned up:

```
20:32:23  WARNING: There are 608 NdisAdapters leaked under Psched\Parameters
20:32:23  WARNING: Not cleaning as not a production RS1 server
20:32:24  WARNING: There are 608 NdisAdapters leaked under WFPLWFS\Parameters
20:32:24  WARNING: Not cleaning as not a production RS1 server
```

```
22:01:31  WARNING: There are 1209 NdisAdapters leaked under Psched\Parameters
22:01:31  WARNING: Not cleaning as not a production RS1 server
22:01:31  WARNING: There are 1209 NdisAdapters leaked under WFPLWFS\Parameters
22:01:31  WARNING: Not cleaning as not a production RS1 server
```

This patch removes the check for non-production builds, and unconditionally
cleans up leaked adapters if they are found.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-29 23:47:23 +02:00
Sebastiaan van Stijn
31885181fc
integration: windows.ps1: turn defender error into a warning
Some integration tests are known to fail if Windows Defender is
enabled. On the machines that run our CI, defender is disabled
for that reason.

Contributors likely will have defender enabled, and because of
that are currently not able to run the integration tests.

This patch changes the ERROR into a WARNING, so that contributors
can still run (a limited set of) the integration tests, but get
informed that some may fail.

We should make this requirement more specific, and only skip
tests that are known to require defender to be disabled, but
while that's not yet in place, let's print a warning instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-28 16:08:53 +02:00
Tianon Gravi
7a8a9c186b
Merge pull request #39604 from seemethere/in_container_question_mark
hack: Remove inContainer check, it wasn't useful
2019-08-27 16:10:47 -07:00
Sebastiaan van Stijn
001b78bffe
Merge pull request #39790 from thaJeztah/jenkinsfile_linting_and_skipping
integration: fix some linting issues, and don't fail build on artifact uploads
2019-08-27 15:34:04 +02:00
Kirill Kolyshkin
5c0d487416
Merge pull request #39795 from thaJeztah/remove_bundles_latest_symlink
hack/make.sh remove "latest" symlink
2019-08-26 23:14:20 -07:00
Sebastiaan van Stijn
38ab9da003
Merge pull request #39747 from tiborvass/use-new-windows-labels
Jenkinsfile: Use new windows labels
2019-08-26 09:45:03 +02:00
Sebastiaan van Stijn
917b0dcd3d
hack/make: fix some linting issues reported by shellcheck
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-25 17:02:20 +02:00
Sebastiaan van Stijn
dde1fd78c7
hack/make.sh remove "latest" symlink
This symlink was added in d42753485b,
to allow finding the path to the latest built binary, because at the time,
those paths were prefixed with the version or commit (e.g. `bundles/1.5.0-dev`).

Commit bac2447964 removed the version-prefix in
paths, but kept the old symlink for backward compatiblity. However, many
things were moved since then (e.g. paths were renamed to `binary-daemon`,
and various other changes). With the symlink pointing to the symlink's parent
directory, following the symlink may result into an infinite recursion,
which can happen if scripts using wildcards / globbing to find files.

With this symlink no longer serving a real purpose, we can probably safely
remove this symlink now.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-25 16:26:14 +02:00