Commit graph

2533 commits

Author SHA1 Message Date
Sebastiaan van Stijn
086b4541cf
golangci-lint: disable default exclusion rules
The default exclusion rules were too permissive; disable
them, but copy the relevant ones to the configuation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-18 00:46:11 +02:00
Sebastiaan van Stijn
fe3c14d5ba
golangci-lint: fix "golint" not ignoring generated files
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-18 00:46:08 +02:00
Sebastiaan van Stijn
47502344b9
golangci-lint: update exclusion rules for todo's
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-18 00:46:04 +02:00
Grant Millar
df7b8f458a daemon: Use short libnetwork ID in exec-root & update libnetwork
Signed-off-by: Grant Millar <rid@cylo.io>
2019-10-15 11:40:24 +01:00
Brian Goff
28b645755a
Merge pull request #40062 from thaJeztah/testutil_improvements
Various testutil improvements
2019-10-11 15:37:48 -07:00
Sebastiaan van Stijn
9407a57522
hack/make: don't attempt to unmount non-existing daemon root-dir
Before:

    DONE 2 tests in 12.272s
    ---> Making bundle: .integration-daemon-stop (in bundles/test-integration)
    umount: bundles/test-integration/root: mountpoint not found

After:

    DONE 2 tests in 14.650s
    ---> Making bundle: .integration-daemon-stop (in bundles/test-integration)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 00:34:53 +02:00
Justen Martin
095ca77f48
Use build args to override binary commits in dockerfile
Signed-off-by: Justen Martin <jmart@the-coder.com>
2019-10-10 14:52:57 -05:00
Tibor Vass
d31633be7d
Merge pull request #39678 from thaJeztah/carry_39595_tailor_arm_ci
[Carry 39595] Tailor CI for ARM, skip legacy integration test.
2019-10-10 11:22:02 -07:00
Michael Zhao
48b06a2561
Tailor CI for ARM, skip legacy integration test.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-08 18:37:13 +02:00
Derek McGowan
6c94a50f41
update containerd binary v1.3.0
full diff: https://github.com/containerd/containerd/compare/v1.2.8..v1.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2019-10-04 15:05:32 -07:00
Sebastiaan van Stijn
5a703ccb46
bump docker-py to 4.1.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-03 16:51:03 +02:00
Sebastiaan van Stijn
48353e16fe
Temporarily switch docker-py to "master"
The docker-py tests were broken, because the version of
py-test that was used, used a dependency that had a new
major release with a breaking change.

Unfortunately, it was not pinned to a specific version,
so when the dependency did the release, py-test broke;

```
22:16:47  Traceback (most recent call last):
22:16:47    File "/usr/local/bin/pytest", line 10, in <module>
22:16:47      sys.exit(main())
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 61, in main
22:16:47      config = _prepareconfig(args, plugins)
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 182, in _prepareconfig
22:16:47      config = get_config()
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 156, in get_config
22:16:47      pluginmanager.import_plugin(spec)
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 530, in import_plugin
22:16:47      __import__(importspec)
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/tmpdir.py", line 25, in <module>
22:16:47      class TempPathFactory(object):
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/tmpdir.py", line 35, in TempPathFactory
22:16:47      lambda p: Path(os.path.abspath(six.text_type(p)))
22:16:47  TypeError: attrib() got an unexpected keyword argument 'convert'
```

docker-py master has a fix for this (bumping the version of
`py-test`), but it's not in a release yet, and the docker cli that's used
in our CI is pinned to 17.06, which doesn't support building from a remote
git repository from a specific git commit.

To fix the immediate situation, this patch switches the docker-py
tests to run from the master branch.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-02 17:54:48 +02:00
Sebastiaan van Stijn
bc9a7ec898
update runc to v1.0.0-rc8-92-g84373aaa (CVE-2019-16884)
full diff: https://github.com/opencontainers/runc/compare/v1.0.0-rc8...3e425f80a8c931f88e6d94a8c831b9d5aa481657

  - opencontainers/runc#2010 criu image path permission error when checkpoint rootless container
  - opencontainers/runc#2028 Update to Go 1.12 and drop obsolete versions
  - opencontainers/runc#2029 Update dependencies
  - opencontainers/runc#2034 Support for logging from children processes
  - opencontainers/runc#2035 specconv: always set "type: bind" in case of MS_BIND
  - opencontainers/runc#2038 `r.destroy` can defer exec in `runner.run` method
  - opencontainers/runc#2041 Change the permissions of the notify listener socket to rwx for everyone
  - opencontainers/runc#2042 libcontainer: intelrdt: add missing destroy handler in defer func
  - opencontainers/runc#2047 Move systemd.Manager initialization into a function in that module
  - opencontainers/runc#2057 main: not reopen /dev/stderr
      - closes opencontainers/runc#2056 Runc + podman|cri-o + systemd issue with stderr
      - closes kubernetes/kubernetes#77615 kubelet fails starting CRI-O containers (Ubuntu 18.04 + systemd cgroups driver)
      - closes cri-o/cri-o#2368 Joining worker node not starting flannel or kube-proxy / CRI-O error "open /dev/stderr: no such device or address"
  - opencontainers/runc#2061 libcontainer: fix TestGetContainerState to check configs.NEWCGROUP
  - opencontainers/runc#2065 Fix cgroup hugetlb size prefix for kB
  - opencontainers/runc#2067 libcontainer: change seccomp test for clone syscall
  - opencontainers/runc#2074 Update dependency libseccomp-golang
  - opencontainers/runc#2081 Bump CRIU to 3.12
  - opencontainers/runc#2089 doc: First process in container needs `Init: true`
  - opencontainers/runc#2094 Skip searching /dev/.udev for device nodes
      - closes opencontainers/runc#2093 HostDevices() race with older udevd versions
  - opencontainers/runc#2098 man: fix man-pages
  - opencontainers/runc#2103 cgroups/fs: check nil pointers in cgroup manager
  - opencontainers/runc#2107 Make get devices function public
  - opencontainers/runc#2113 libcontainer: initial support for cgroups v2
  - opencontainers/runc#2116 Avoid the dependency on cgo through go-systemd/util package
      - removes github.com/coreos/pkg as dependency
  - opencontainers/runc#2117 Remove libcontainer detection for systemd features
      - fixes opencontainers/runc#2117 Cache the systemd detection results
  - opencontainers/runc#2119 libcontainer: update masked paths of /proc
      - relates to moby/moby#36368 Add /proc/keys to masked paths
      - relates to moby/moby#38299 Masked /proc/asound
      - relates to moby/moby#37404 Add /proc/acpi to masked paths (CVE-2018-10892)
  - opencontainers/runc#2122 nsenter: minor fixes
  - opencontainers/runc#2123 Bump x/sys and update syscall for initial Risc-V support
  - opencontainers/runc#2125 cgroup: support mount of cgroup2
  - opencontainers/runc#2126 libcontainer/nsenter: Don't import C in non-cgo file
  - opencontainers/runc#2129 Only allow proc mount if it is procfs
      - addresses opencontainers/runc#2129 AppArmor can be bypassed by a malicious image that specifies a volume at /proc (CVE-2019-16884)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-27 16:22:34 +02:00
Jintao Zhang
c4ec02b0af Update containerd to v1.2.10
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-09-27 11:47:20 +08:00
Sebastiaan van Stijn
5175ed54e5
hack/ci/windows.ps1 update references to repositories that were moved
Also updated the related docs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-26 12:14:38 +02:00
Akihiro Suda
1dd9260e47
Merge pull request #39956 from jmartin84/fix-closenotifier-deprecation-warning
Removed deprecated CloseNotifier logic
2019-09-24 02:37:15 +09:00
Sebastiaan van Stijn
9b07a3e0a5
Merge pull request #39917 from StefanScherer/windows-zap-fixed-subdir
Zap a fixed windows folder, add build number to folder inside
2019-09-23 13:12:29 +02:00
Sebastiaan van Stijn
7d573c7b61
Merge pull request #39967 from thaJeztah/bump_libnetwork
bump libnetwork to 96bcc0dae898308ed659c5095526788a602f4726
2019-09-23 10:56:14 +02:00
Sebastiaan van Stijn
6c3113e599
Merge pull request #39951 from yedamao/fix-base-Dial-is-deprecated
Fix base.Dial is deprecated: Use DialContext instead
2019-09-21 13:01:40 +02:00
Sebastiaan van Stijn
75477f0b3c
bump libnetwork to 96bcc0dae898308ed659c5095526788a602f4726
full diff: 92d1fbe1eb...96bcc0dae8

changes included:

- docker/libnetwork#2429 Updating IPAM config with results from HNS create network call
  - addresses moby/moby#38358
- docker/libnetwork#2450 Always configure iptables forward policy
  - related to moby/moby#14041 and docker/libnetwork#1526

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-20 19:48:49 +02:00
Akihiro Suda
4e8cf9f5dd
Merge pull request #39960 from thaJeztah/lint_integration_cli
integration-cli: enable golangci-lint
2019-09-20 13:26:29 +09:00
HuanHuan Ye
a57fd5488d Fix base.Dial is deprecated: Use DialContext instead
1.Change base.Dial to base.DailContext.
2.Remove proxyDialer that was previously used to configure a
net.Dialer to route proxy.Dialer which will route the connections
through the proxy using the connections through a SOCKS proxy.
SOCKS proxies are now supported by configuring only http.Transport.Proxy,
and no longer require changing http.Transport.Dial.

Signed-off-by: HuanHuan Ye <logindaveye@gmail.com>
2019-09-20 10:42:13 +08:00
Brian Goff
0f18e434b5
Merge pull request #39911 from tiborvass/gotestsum-integration-cli
hack: have integration-cli use gotestsum codepath
2019-09-19 08:01:25 -07:00
Stefan Scherer
4866207543
Zap a fixed folder, add build number to folder inside
Signed-off-by: Stefan Scherer <stefan.scherer@docker.com>
2019-09-19 15:52:07 +02:00
Sebastiaan van Stijn
851b000641
integration-cli: enable golangci-lint
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-19 10:50:31 +02:00
Tibor Vass
3acf0cc795
Merge pull request #39930 from thaJeztah/fix_TESTFLAGS
hack/test/unit: fix custom TESTFLAGS not working
2019-09-18 17:56:23 -07:00
Justen Martin
23ab331979 Removed deprecated CloseNotifier logic
Signed-off-by: Justen Martin <jmart@the-coder.com>
2019-09-18 16:14:30 -05:00
Tibor Vass
f1c1cd436a integration-cli: move each test suite to its own TestX testing function
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-18 18:26:36 +00:00
Tibor Vass
84928be605 hack: have integration-cli use gotestsum codepath
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-18 18:26:36 +00:00
Sebastiaan van Stijn
5f67568bfd
Revert "Revert "hack: temporarily disable deprecate-integration-cli""
This reverts commit 01332b54e0.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:58:02 +02:00
Sebastiaan van Stijn
6f0dfe44eb
golangci-lint: suppress remaining deprecation errors for now (staticcheck)
```
distribution/registry.go:84:3: SA1019: base.Dial is deprecated: Use DialContext instead, which allows the transport to cancel dials as soon as they are no longer needed. If both are set, DialContext takes priority.  (staticcheck)
registry/registry.go:188:3:    SA1019: base.Dial is deprecated: Use DialContext instead, which allows the transport to cancel dials as soon as they are no longer needed. If both are set, DialContext takes priority.  (staticcheck)

client/hijack.go:85:16:                               SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead.  (staticcheck)
integration-cli/docker_api_attach_test.go:245:12:     SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead.  (staticcheck)
integration/plugin/authz/authz_plugin_test.go:180:7:  SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead.  (staticcheck)
integration/plugin/authz/authz_plugin_test.go:479:12: SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead.  (staticcheck)

integration-cli/docker_api_attach_test.go:239:134:    SA1019: httputil.ClientConn is deprecated: Use Client or Transport in package net/http instead.  (staticcheck)

daemon/daemon.go:885:3:                               SA1019: grpc.WithDialer is deprecated: use WithContextDialer instead.  Will be supported throughout 1.x.  (staticcheck)

pkg/authorization/response.go:18:2:                   SA1019: http.CloseNotifier is deprecated: the CloseNotifier interface predates Go's context package. New code should use Request.Context instead.  (staticcheck)
pkg/authorization/response.go:160:30:                 SA1019: http.CloseNotifier is deprecated: the CloseNotifier interface predates Go's context package. New code should use Request.Context instead.  (staticcheck)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:58:01 +02:00
Sebastiaan van Stijn
df7d719e4c
golangci-lint: suppress Xattrs is deprecated: Use PAXRecords instead
```
pkg/archive/archive.go:407:3:       SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead.  (staticcheck)
pkg/archive/archive.go:408:3:       SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead.  (staticcheck)
pkg/archive/archive.go:661:26:      SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead.  (staticcheck)
pkg/archive/archive_linux.go:47:7:  SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead.  (staticcheck)
pkg/archive/archive_linux.go:48:12: SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead.  (staticcheck)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:58:00 +02:00
Sebastiaan van Stijn
f41712c4be
golangci-lint: enable misspell, tweak configuration
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:59 +02:00
Sebastiaan van Stijn
c9553897f3
golangci-lint: enable gosec linter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:58 +02:00
Sebastiaan van Stijn
d1363f5a90
golangci-lint: set correct build-tags to lint journald if supported
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:58 +02:00
Sebastiaan van Stijn
85b89c9e50
Replace gometalinter with golangci-lint
Replacing gometalinter, because it has been deprecated,
and the repository was archived.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:57 +02:00
Sebastiaan van Stijn
0620990307
hack/test/unit: fix custom TESTFLAGS not working
The `-test.timeout=5m` was glued directly after the current `TESTFLAGS`,
causing them to be non-functional;

Before:

    make TESTDEBUG=1 TESTDIRS='github.com/docker/docker/pkg/filenotify' TESTFLAGS='-test.run TestPollerEvent' test-unit
    + mkdir -p bundles
    + gotestsum --format=standard-quiet --jsonfile=bundles/go-test-report.json --junitfile=bundles/junit-report.xml -- -tags 'netgo seccomp libdm_no_deferred_remove' -cover -coverprofile=bundles/profile.out -covermode=atomic -test.run TestPollerEvent-test.timeout=5m github.com/docker/docker/pkg/filenotify
    testing: warning: no tests to run
    ok  	github.com/docker/docker/pkg/filenotify	0.003s	coverage: 0.0% of statements [no tests to run]

    DONE 0 tests in 0.298s

After:

    make TESTDEBUG=1 TESTDIRS='github.com/docker/docker/pkg/filenotify' TESTFLAGS='-test.run TestPollerEvent' test-unit
    + mkdir -p bundles
    + gotestsum --format=standard-quiet --jsonfile=bundles/go-test-report.json --junitfile=bundles/junit-report.xml -- -tags 'netgo seccomp libdm_no_deferred_remove' -cover -coverprofile=bundles/profile.out -covermode=atomic -test.run TestPollerEvent -test.timeout=5m github.com/docker/docker/pkg/filenotify
    ok  	github.com/docker/docker/pkg/filenotify	0.608s	coverage: 44.7% of statements

    DONE 1 tests in 0.922s

This was introduced in 42f0a0db75

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-16 09:15:02 +02:00
Brian Goff
4b55f3cf8b
Merge pull request #39854 from thaJeztah/hack_windows_explicit_exit
hack/ci/windows.ps1: explicitly set exit code to result of tests
2019-09-12 11:34:44 -07:00
Brian Goff
15d3310f72
Merge pull request #39877 from thaJeztah/ci_windows_storageopt
hack/ci/windows.ps1: add support for DOCKER_STORAGE_OPTS
2019-09-12 11:13:03 -07:00
Kirill Kolyshkin
7bcffeef6e
Merge pull request #39901 from thaJeztah/remove_docker_cli_autogen
hack/make: remove autogen resources for Docker CLI
2019-09-12 20:21:51 +03:00
Tibor Vass
2900a919f4
Merge pull request #39879 from thaJeztah/fix_go_version_check_again
hack/ci/windows.ps1: fix Go version check (due to trailing .0)
2019-09-11 17:34:45 -07:00
Sebastiaan van Stijn
f007b0150a
integration: improve package- and filename for junit.xml
Generate more unique names, based on architecture and test-suite name.

Clean up the path to this integration test to create a useful package name.
"$dir" can be either absolute (/go/src/github.com/docker/docker/integration/foo)
or relative (./integration/foo). To account for both, first we strip the
absolute path, then any leading periods and slashes.

For the package-name, we use periods as separator instead of slashes, to be more
in-line with Java package names (which is what junit.xml was originally designed
for).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 22:31:56 +02:00
Sebastiaan van Stijn
d2e00d62e2
integration: test2json: enable timestamps to fix zero-time test durations
Without these options set, test2json does not include a `Time`
field in the generated JSON;

    {"Action":"run","Test":"TestCgroupNamespacesBuild"}
    {"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"=== RUN   TestCgroupNamespacesBuild\n"}
    {"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (1.70s)\n"}
    ...
    {"Action":"pass","Test":"TestCgroupNamespacesBuild"}

As a result, `gotestsum` was not able to calculate test-duration, and
reported `time="0.000000"` for all tests;

    <testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="0.000000"></testcase>

With this patch applied:

    {"Time":"2019-08-23T22:42:41.644361357Z","Action":"run","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild"}
    {"Time":"2019-08-23T22:42:41.644367647Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"=== RUN   TestCgroupNamespacesBuild\n"}
    {"Time":"2019-08-23T22:42:44.926933252Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (3.28s)\n"}
    ...
    {"Time":"2019-08-23T22:42:44.927003836Z","Action":"pass","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Elapsed":3.28}

Which now correctly reports the test's duration:

    <testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="3.280000"></testcase>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 22:31:54 +02:00
Sebastiaan van Stijn
f3be6b346f
integration: use gotestsum to generate junit.xml and go-test-report.json
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 22:31:52 +02:00
Tibor Vass
553df774d4
Merge pull request #39802 from thaJeztah/integration_parallel_simplify
integration: simplify parallel run destination
2019-09-11 11:09:54 -07:00
Tibor Vass
6d8c5bc181
Merge pull request #39878 from thaJeztah/cut_the_tail
hack/ci/windows.ps1: stop tailing logs after stopping the daemon
2019-09-11 10:03:11 -07:00
Sebastiaan van Stijn
3262a69be6
integration: simplify parallel run destination
'Namespace' parallel runs by bind-mounting a different directory
in the container, instead of making the tests running inside
the container aware of the namespaced location.

This makes it transparent to the tests, and slightly reduces
complexity.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 17:18:38 +02:00
Sebastiaan van Stijn
f2498e21c4
hack/make: remove autogen resources for Docker CLI
the files used by the docker cli were moved to the docker/cli repository,
so are no longer needed here.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 15:16:07 +02:00
Sebastiaan van Stijn
a464a3d51f
Dockerfile: remove GOMETALINTER_OPTS
This `ENV` was added to the Dockerfile in b96093fa56,
when the repository used per-architecture Dockerfiles, and some architectures needed
a different configuration.

Now that we use a multi-arch Dockerfile, and CI uses a Jenkinsfile, we can remove
this `ENV` from the Dockerfile, and set it in CI instead if needed.

Also updated the wording and fixed linting issues in hack/validate/gometalinter

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 08:58:38 +02:00
Tibor Vass
6754976015
Merge pull request #39888 from vikramhh/move_windows_gopath_out_of_goroot
Move GOPATH out from under the GO source tree
2019-09-10 14:33:12 -07:00