This change caused a regression, causing the DOCKER-USER chain
to not be created, despite iptables being enabled on the daemon.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The patch made in docker/libnetwork#2450 caused a breaking change in the
networking behaviour, causing Kubernetes installations on Docker Desktop
(and possibly other setups) to fail.
Rolling back this change in the 19.03 branch while we investigate if there
are alternatives.
diff: 45c710223c...96bcc0dae8
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When slirp4netns v0.4.0+ is used, now slirp4netns is hardened using
mount namespace ("sandbox") and seccomp to mitigate potential
vulnerabilities.
bump up rootlesskit: 2fcff6ceae...791ac8cb20
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e20b7323fb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also updated the related docs.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5175ed54e5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The docker-py tests were broken, because the version of
py-test that was used, used a dependency that had a new
major release with a breaking change.
Unfortunately, it was not pinned to a specific version,
so when the dependency did the release, py-test broke;
```
22:16:47 Traceback (most recent call last):
22:16:47 File "/usr/local/bin/pytest", line 10, in <module>
22:16:47 sys.exit(main())
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 61, in main
22:16:47 config = _prepareconfig(args, plugins)
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 182, in _prepareconfig
22:16:47 config = get_config()
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 156, in get_config
22:16:47 pluginmanager.import_plugin(spec)
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 530, in import_plugin
22:16:47 __import__(importspec)
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/tmpdir.py", line 25, in <module>
22:16:47 class TempPathFactory(object):
22:16:47 File "/usr/local/lib/python3.6/site-packages/_pytest/tmpdir.py", line 35, in TempPathFactory
22:16:47 lambda p: Path(os.path.abspath(six.text_type(p)))
22:16:47 TypeError: attrib() got an unexpected keyword argument 'convert'
```
docker-py master has a fix for this (bumping the version of
`py-test`), but it's not in a release yet, and the docker cli that's used
in our CI is pinned to 17.06, which doesn't support building from a remote
git repository from a specific git commit.
To fix the immediate situation, this patch switches the docker-py
tests to run from the master branch.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 48353e16fe)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
The `-test.timeout=5m` was glued directly after the current `TESTFLAGS`,
causing them to be non-functional;
Before:
make TESTDEBUG=1 TESTDIRS='github.com/docker/docker/pkg/filenotify' TESTFLAGS='-test.run TestPollerEvent' test-unit
+ mkdir -p bundles
+ gotestsum --format=standard-quiet --jsonfile=bundles/go-test-report.json --junitfile=bundles/junit-report.xml -- -tags 'netgo seccomp libdm_no_deferred_remove' -cover -coverprofile=bundles/profile.out -covermode=atomic -test.run TestPollerEvent-test.timeout=5m github.com/docker/docker/pkg/filenotify
testing: warning: no tests to run
ok github.com/docker/docker/pkg/filenotify 0.003s coverage: 0.0% of statements [no tests to run]
DONE 0 tests in 0.298s
After:
make TESTDEBUG=1 TESTDIRS='github.com/docker/docker/pkg/filenotify' TESTFLAGS='-test.run TestPollerEvent' test-unit
+ mkdir -p bundles
+ gotestsum --format=standard-quiet --jsonfile=bundles/go-test-report.json --junitfile=bundles/junit-report.xml -- -tags 'netgo seccomp libdm_no_deferred_remove' -cover -coverprofile=bundles/profile.out -covermode=atomic -test.run TestPollerEvent -test.timeout=5m github.com/docker/docker/pkg/filenotify
ok github.com/docker/docker/pkg/filenotify 0.608s coverage: 44.7% of statements
DONE 1 tests in 0.922s
This was introduced in 42f0a0db75
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0620990307)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- remove -check.* flags
- use (per-test) -timeout flag
- allow user to override TEST_SKIP_* regardless of TESTFLAGS
- remove test-imports validation
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 7cd028f2d0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This `ENV` was added to the Dockerfile in b96093fa56,
when the repository used per-architecture Dockerfiles, and some architectures needed
a different configuration.
Now that we use a multi-arch Dockerfile, and CI uses a Jenkinsfile, we can remove
this `ENV` from the Dockerfile, and set it in CI instead if needed.
Also updated the wording and fixed linting issues in hack/validate/gometalinter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a464a3d51f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Windows Dockerfile downloads the Go binaries, which (unlike
the Golang images) do not have a trailing `.0` in their version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 61450a651b)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
There's already a step in "Nuke Everything", but lets' stop it
after stopping the daemon as well
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e1636ad5fa)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
CI already stores the logs of the test daemon, so we might as well
store them as artifacts
```
[2019-09-03T12:49:39.835Z] INFO: Tidying up at end of run
[2019-09-03T12:49:39.835Z] INFO: Saving daemon under test log (d:\CI-2\CI-3593e7622\dut.out) to C:\windows\TEMP\CIDUT.out
[2019-09-03T12:49:39.835Z] INFO: Saving daemon under test log (d:\CI-2\CI-3593e7622\dut.err) to C:\windows\TEMP\CIDUT.err
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6ee61f5493)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Trying to see if this helps with the cleanup step exiting in CI, but
Jenkins continuing to wait for the script to end afterwards.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 8e8c52c4ab)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Seen failing a couple of times:
```
[2019-09-02T08:40:15.796Z] =================================== FAILURES ===================================
[2019-09-02T08:40:15.796Z] __________________ AttachContainerTest.test_attach_no_stream ___________________
[2019-09-02T08:40:15.796Z] tests/integration/api_container_test.py:1250: in test_attach_no_stream
[2019-09-02T08:40:15.796Z] assert output == 'hello\n'.encode(encoding='ascii')
[2019-09-02T08:40:15.796Z] E AssertionError: assert b'' == b'hello\n'
[2019-09-02T08:40:15.796Z] E Right contains more items, first extra item: 104
[2019-09-02T08:40:15.796Z] E Use -v to get the full diff
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ce77a804b8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Add quotes to prevent word splitting in `cp` statement (SC2046)
- Replace legacy back tics with `$()`
- Replace `which` with `command -v` (SC2230)
- Fix incorrect (`==`) comparison
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 70d3677825)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Some integration tests are known to fail if Windows Defender is
enabled. On the machines that run our CI, defender is disabled
for that reason.
Contributors likely will have defender enabled, and because of
that are currently not able to run the integration tests.
This patch changes the ERROR into a WARNING, so that contributors
can still run (a limited set of) the integration tests, but get
informed that some may fail.
We should make this requirement more specific, and only skip
tests that are known to require defender to be disabled, but
while that's not yet in place, let's print a warning instead.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 31885181fc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Generate more unique names, based on architecture and test-suite name.
Clean up the path to this integration test to create a useful package name.
"$dir" can be either absolute (/go/src/github.com/docker/docker/integration/foo)
or relative (./integration/foo). To account for both, first we strip the
absolute path, then any leading periods and slashes.
For the package-name, we use periods as separator instead of slashes, to be more
in-line with Java package names (which is what junit.xml was originally designed
for).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f007b0150a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Without these options set, test2json does not include a `Time`
field in the generated JSON;
{"Action":"run","Test":"TestCgroupNamespacesBuild"}
{"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"=== RUN TestCgroupNamespacesBuild\n"}
{"Action":"output","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (1.70s)\n"}
...
{"Action":"pass","Test":"TestCgroupNamespacesBuild"}
As a result, `gotestsum` was not able to calculate test-duration, and
reported `time="0.000000"` for all tests;
<testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="0.000000"></testcase>
With this patch applied:
{"Time":"2019-08-23T22:42:41.644361357Z","Action":"run","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild"}
{"Time":"2019-08-23T22:42:41.644367647Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"=== RUN TestCgroupNamespacesBuild\n"}
{"Time":"2019-08-23T22:42:44.926933252Z","Action":"output","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Output":"--- PASS: TestCgroupNamespacesBuild (3.28s)\n"}
...
{"Time":"2019-08-23T22:42:44.927003836Z","Action":"pass","Package":"amd64.integration.build","Test":"TestCgroupNamespacesBuild","Elapsed":3.28}
Which now correctly reports the test's duration:
<testcase classname="amd64.integration.build" name="TestCgroupNamespacesBuild" time="3.280000"></testcase>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d2e00d62e2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
'Namespace' parallel runs by bind-mounting a different directory
in the container, instead of making the tests running inside
the container aware of the namespaced location.
This makes it transparent to the tests, and slightly reduces
complexity.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3262a69be6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The inContainer check isn't really useful anymore.
Even though it was said that we shouldn't rely on its existence back in
2016, we're now in 2019 and this thing still exists so we should just
rely on it now to check whether or not we're in a container.
Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
(cherry picked from commit f5cd8fdd44)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diff: 09cdcc8c0e...92d1fbe1eb
relevant changes included (omitting some changes that were added _and_ reverted in this bump):
- docker/libnetwork#2433 Fix parseIP error when parseIP before get AddressFamily
- fixesdocker/libnetwork#2431 parseIP Error ip=[172 17 0 2 0 0 0 0 0 0 0 0 0 0 0 0]
- https://github.com/docker/libnetwork/issues/2289
- this was a regression introduced in docker/libnetwork#2416 Fix hardcoded AF_INET for IPv6 address handling
- docker/libnetwork#2440 Bump hashicorp go-sockaddr v1.0.2, go-multierror v1.0.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit bab58c1924)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
full diffs:
- fc5a7d91d5...62a13ae87c
- b2de5d10e3...v1.0.0
- 604eaf189e...13995c7128ccc8e51e9a6bd2b551020a27180abd
notable changes in libnetwork:
- docker/libnetwork#2366 Bump vishvananda/netlink to 1.0.0
- docker/libnetwork#2339 controller: Check if IPTables is enabled for arrangeUserFilterRule
- addresses docker/libnetwork#2158 dockerd when run with --iptables=false modifies iptables by adding DOCKER-USER
- addresses moby/moby#35777 With iptables=false dockerd still creates DOCKER-USER chain and rules
- addresses docker/for-linux#136 dockerd --iptables=false adds DOCKER-USER chain and modify FORWARD chain anyway
- docker/libnetwork#2394 Make DNS records and queries case-insensitive
- addresses moby/moby#28689 Embedded DNS is case-sensitive
- addresses moby/moby#21169 hostnames with new networking are case-sensitive
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 344b093258)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
From the release notes: https://github.com/containerd/containerd/releases/tag/v1.2.7
> Welcome to the v1.2.7 release of containerd!
>
> The seventh patch release for containerd 1.2 introduces OCI image
> descriptor annotation support and contains fixes for containerd shim logs,
> container stop/deletion, cri plugin and selinux.
>
> It also contains several important bug fixes for goroutine and file
> descriptor leakage in containerd and containerd shims.
>
> Notable Updates
>
> - Support annotations in the OCI image descriptor, and filtering image by annotations. containerd/containerd#3254
> - Support context timeout in ttrpc which can help avoid containerd hangs when a shim is unresponsive. containerd/ttrpc#31
> - Fix a bug that containerd shim leaks goroutine and file descriptor after containerd restarts. containerd/ttrpc#37
> - Fix a bug that a container can't be deleted if first deletion attempt is canceled or timeout. containerd/containerd#3264
> - Fix a bug that containerd leaks file descriptor when using v2 containerd shims, e.g. containerd-shim-runc-v1. containerd/containerd#3273
> - Fix a bug that a container with lingering processes can't terminate when it shares pid namespace with another container. moby/moby#38978
> - Fix a bug that containerd can't read shim logs after restart. containerd/containerd#3282
> - Fix a bug that shim_debug option is not honored for existing containerd shims after containerd restarts. containerd/containerd#3283
> - cri: Fix a bug that a container can't be stopped when the exit event is not successfully published by the containerd shim. containerd/containerd#3125, containerd/containerd#3177
> - cri: Fix a bug that exec process is not cleaned up if grpc context is canceled or timeout. contaienrd/cri#1159
> - Fix a selinux keyring labeling issue by updating runc to v1.0.0-rc.8 and selinux library to v1.2.2. opencontainers/selinux#50
> - Update ttrpc to f82148331ad2181edea8f3f649a1f7add6c3f9c2. containerd/containerd#3316
> - Update cri to 49ca74043390bc2eeea7a45a46005fbec58a3f88. containerd/containerd#3330
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d5669ec1c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Windows RS1 has problems with leaking NdisAdapters during the integration
tests; the windows.ps1 script has a cleanup stesp to remove those
leaked adapters.
For internal testing at Microsoft on internal builds, this cleanup step
was skipped, and only ran on the CI machines in our Jenkins.
Due to the move to our new Jenkins, the names of Windows machines changed,
and because of that, the cleanup step was never executed, resulting in the
leaked adapters not being cleaned up:
```
20:32:23 WARNING: There are 608 NdisAdapters leaked under Psched\Parameters
20:32:23 WARNING: Not cleaning as not a production RS1 server
20:32:24 WARNING: There are 608 NdisAdapters leaked under WFPLWFS\Parameters
20:32:24 WARNING: Not cleaning as not a production RS1 server
```
```
22:01:31 WARNING: There are 1209 NdisAdapters leaked under Psched\Parameters
22:01:31 WARNING: Not cleaning as not a production RS1 server
22:01:31 WARNING: There are 1209 NdisAdapters leaked under WFPLWFS\Parameters
22:01:31 WARNING: Not cleaning as not a production RS1 server
```
This patch removes the check for non-production builds, and unconditionally
cleans up leaked adapters if they are found.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 156ad54fb7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The "new test" detector in test-integration-flaky was a bit flaky since
it would detect function signatures that are not new tests.
In addition, the test calls `return` outside of a function which is not
allowed.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit e2b24490e4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This symlink was added in d42753485b,
to allow finding the path to the latest built binary, because at the time,
those paths were prefixed with the version or commit (e.g. `bundles/1.5.0-dev`).
Commit bac2447964 removed the version-prefix in
paths, but kept the old symlink for backward compatiblity. However, many
things were moved since then (e.g. paths were renamed to `binary-daemon`,
and various other changes). With the symlink pointing to the symlink's parent
directory, following the symlink may result into an infinite recursion,
which can happen if scripts using wildcards / globbing to find files.
With this symlink no longer serving a real purpose, we can probably safely
remove this symlink now.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dde1fd78c7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is basically taking some stuff that make a custom shell function
for.
This takes a test filter, builds the appropriate TESTFLAGS, and sets the
integration API test dirs that match the given filter to avoid building
all test dirs.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 13064b155e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Attempting to fix;
```
21:16:00 Traceback (most recent call last):
21:16:00 File "/usr/local/bin/yamllint", line 11, in <module>
21:16:00 sys.exit(run())
21:16:00 File "/usr/local/lib/python3.5/dist-packages/yamllint/cli.py", line 170, in run
21:16:00 problems = linter.run(f, conf, filepath)
21:16:00 File "/usr/local/lib/python3.5/dist-packages/yamllint/linter.py", line 233, in run
21:16:00 content = input.read()
21:16:00 File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
21:16:00 return codecs.ascii_decode(input, self.errors)[0]
21:16:00 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 123522: ordinal not in range(128)
21:16:00 Build step 'Execute shell' marked build as failure
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b5e5cac0f5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
