Commit graph

37634 commits

Author SHA1 Message Date
Arko Dasgupta
a65dee30fc Move defer method to the top right after New is called
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-08-15 14:09:47 -07:00
Tibor Vass
f18ad28874
Merge pull request #39745 from tonistiigi/update-buildkit
vendor: update buildkit to v0.6.1
2019-08-15 09:33:37 -07:00
Tonis Tiigi
e59b26087f vendor: update buildkit to v0.6.1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-08-14 18:55:30 -07:00
Sebastiaan van Stijn
8b6da9d82f
Jenkinsfile: save docker-py artifacts
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-15 02:52:37 +02:00
Sebastiaan van Stijn
4ddb40ee8a
Jenkinsfile: build dynamic binary for docker-py, to match makefile
This also makes sure that we can test all functionality of the
daemon, because some features are not available on static binaries.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-15 02:45:04 +02:00
Sebastiaan van Stijn
5969bbee79
docker-py: output junit.xml for test-results
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-15 02:45:02 +02:00
Sebastiaan van Stijn
535e29da05
docker-py: use --mount for bind-mounting docker.sock
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-15 02:45:00 +02:00
Sebastiaan van Stijn
b04cbf1072
docker-py: run without tty to disable color output
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-15 02:44:57 +02:00
Sebastiaan van Stijn
0b3d201892
docker-py: fix linting issues reported by shellcheck
- SC2006: use $(...) notation instead of legacy backticked `...`
- SC2086: double quote to prevent globbing and word splitting

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-15 02:44:54 +02:00
Brian Goff
d5f607bd0f
Merge pull request #39743 from thaJeztah/bump_criu
Dockerfile: update CRIU to v3.12
2019-08-14 15:54:31 -07:00
Sebastiaan van Stijn
00ad0222ce
Dockerfile: update CRIU to v3.12
New features

- build CRIU with Android NDK
- C/R of
  - IP RAW sockets
  - lsm: dump and restore any SELinux process label
  - support restoring ghost files on readonly mounts

Bugfixes

 - Do not lock network if running in the host network namespace
- Fix RPC configuration file handling
- util: don't leak file descriptors to third-party tools
- small fixes here and there

Improvements

- travis: switch to the Ubuntu Xenial
- travis-ci: Enable ia32 tests
- Many improvements and bug fixes in the libcriu
  - Changes in the API and ABI (SONAME increased from 1 to 2)

full diff: https://github.com/checkpoint-restore/criu/compare/v3.11...v3.12

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-14 20:43:52 +02:00
Arko Dasgupta
f3a3ea0d3c Fix flaky TestServiceWithDefaultAddressPoolInit
1.This commit replaces serviceRunningCount with
swarm.RunningTasksCount to accurately check if the
service is running with the accurate number of instances
or not. serviceRunningCount was only checking the ServiceList
and was not checking if the tasks were running or not

This adds a safe barrier to execute docker network inspect
commands for overlay networks which get created
asynchronously via Swarm

2. Make sure client connections are closed

3. Make sure every service and network name is unique

4. Make sure services and networks are cleaned up

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-08-14 08:02:15 -07:00
Justin Cormack
934cae7962
Merge pull request #39738 from thaJeztah/bump_golang_1.12.8
Bump golang 1.12.8 (CVE-2019-9512, CVE-2019-9514)
2019-08-14 13:42:24 +01:00
Sebastiaan van Stijn
683766613a
Adjust tests for changes in Go 1.12.8 / 1.11.13
```
00:38:11 === Failed
00:38:11 === FAIL: opts TestParseDockerDaemonHost (0.00s)
00:38:11     hosts_test.go:87: tcp tcp:a.b.c.d address expected error "Invalid bind address format: tcp:a.b.c.d" return, got "parse tcp://tcp:a.b.c.d: invalid port \":a.b.c.d\" after host" and addr
00:38:11     hosts_test.go:87: tcp tcp:a.b.c.d/path address expected error "Invalid bind address format: tcp:a.b.c.d/path" return, got "parse tcp://tcp:a.b.c.d/path: invalid port \":a.b.c.d\" after host" and addr
00:38:11
00:38:11 === FAIL: opts TestParseTCP (0.00s)
00:38:11     hosts_test.go:129: tcp tcp:a.b.c.d address expected error Invalid bind address format: tcp:a.b.c.d return, got parse tcp://tcp:a.b.c.d: invalid port ":a.b.c.d" after host and addr
00:38:11     hosts_test.go:129: tcp tcp:a.b.c.d/path address expected error Invalid bind address format: tcp:a.b.c.d/path return, got parse tcp://tcp:a.b.c.d/path: invalid port ":a.b.c.d" after host and addr
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-14 02:51:08 +02:00
Sebastiaan van Stijn
73b0e4c589
Bump golang 1.12.8 (CVE-2019-9512, CVE-2019-9514)
go1.12.8 (released 2019/08/13) includes security fixes to the net/http and net/url packages.
See the Go 1.12.8 milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.12.8

- net/http: Denial of Service vulnerabilities in the HTTP/2 implementation
  net/http and golang.org/x/net/http2 servers that accept direct connections from untrusted
  clients could be remotely made to allocate an unlimited amount of memory, until the program
  crashes. Servers will now close connections if the send queue accumulates too many control
  messages.
  The issues are CVE-2019-9512 and CVE-2019-9514, and Go issue golang.org/issue/33606.
  Thanks to Jonathan Looney from Netflix for discovering and reporting these issues.
  This is also fixed in version v0.0.0-20190813141303-74dc4d7220e7 of golang.org/x/net/http2.
  net/url: parsing validation issue
- url.Parse would accept URLs with malformed hosts, such that the Host field could have arbitrary
  suffixes that would appear in neither Hostname() nor Port(), allowing authorization bypasses
  in certain applications. Note that URLs with invalid, not numeric ports will now return an error
  from url.Parse.
  The issue is CVE-2019-14809 and Go issue golang.org/issue/29098.
  Thanks to Julian Hector and Nikolai Krein from Cure53, and Adi Cohen (adico.me) for discovering
  and reporting this issue.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-14 02:27:05 +02:00
Yong Tang
fade624f16
Merge pull request #39720 from thaJeztah/dockerfile_non_interactive
Dockerfile: use DEBIAN_FRONTEND=noninteractive
2019-08-13 16:48:19 -07:00
Sebastiaan van Stijn
d6f7909c76
Jenkinsfile: run DCO check before everything else
This will run the DCO check in a lightweight alpine container, before
running other stages, and before building the development image/container
(which can take a long time).

A Jenkins parameter was added to optionally skip the DCO check (skip_dco)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-14 00:57:36 +02:00
Tibor Vass
a176d30c54
Merge pull request #39719 from thaJeztah/junit_all_stages
Jenkinsfile: collect junit.xml for all architectures
2019-08-13 15:09:24 -07:00
Sebastiaan van Stijn
e2f5b78e78
Jenkinsfile: collect junit.xml for all architectures
Jenkins groups them per stage, so collecting them for all architectures
is possible (without them conflicting or becoming ambiguous)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-13 19:58:45 +02:00
Sebastiaan van Stijn
7f9328ad2e
Jenkinsfile: send junit.xml in the stage that produced it
This will send the results directly after the tests complete,
and make the stage more atomic.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-13 19:58:41 +02:00
Brian Goff
9b25c0fb25
Merge pull request #39682 from tiborvass/jenkinsfile_reduce_time
Reduce time of integration tests
2019-08-13 10:21:41 -07:00
Sebastiaan van Stijn
80cbcc854e
Merge pull request #39734 from SataQiu/fix-20190813
Fix some spelling mistakes
2019-08-13 19:08:35 +02:00
SataQiu
f6226a2a56 fix some spelling mistakes
Signed-off-by: SataQiu <qiushida@beyondcent.com>
2019-08-13 22:57:55 +08:00
Andrew Hsu
eb30f0ad84 fix bundles filenames in Jenkinsfile
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-08-12 20:49:01 +00:00
Andrew Hsu
ad29f9e471 rename powerpc bundles in Jenkinsfile
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-08-12 20:43:13 +00:00
Andrew Hsu
a049ea1a93 rename z bundles in Jenkinsfile
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-08-12 20:42:19 +00:00
Andrew Hsu
0cfc1ec2bd be more lenient on junit report gathering in Jenkinsfile
In case a job fails before even generating a report file.

Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-08-12 20:41:03 +00:00
Andrew Hsu
4e2f39cf14 use environment for z jobs in Jenkinsfile
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-08-12 20:41:03 +00:00
Andrew Hsu
3564b03fbc use environment for power jobs in Jenkinsfile
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-08-12 20:41:03 +00:00
Andrew Hsu
bf70a5975d set timeouts in Jenkinsfile to 2 hrs
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-08-12 20:41:03 +00:00
Andrew Hsu
bdc1c1a02a add z-master stage to Jenkinsfile
The z-master stage will just run the integration-cli tests. The
existing z stage will run the unit tests and the integration
tests. In this way, PR check jobs will be shorter, but all
integration tests will run after PR is merged to master.

Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-08-12 20:41:03 +00:00
Andrew Hsu
c2f9d58375 add powerpc-master stage to Jenkinsfile
The powerpc-master stage will just run the integration-cli tests. The
existing powerpc stage will run the unit tests and the integration
tests. In this way, PR check jobs will be shorter, but all integration
tests will run after PR is merged to master.

Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-08-12 20:41:03 +00:00
Tibor Vass
e554fb23c8 Jenkinsfile: reduce time of integration tests by dividing tests into 3 parallel runs
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-08-12 20:41:03 +00:00
Tibor Vass
13df617d4c hack: unmount leftover daemon root folders
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-08-12 20:41:03 +00:00
Tibor Vass
251c8dca28 Jenkinsfile: move static and cross compilation to unit-validate stage
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-08-12 20:41:03 +00:00
Sebastiaan van Stijn
2ff9ac4de5
Dockerfile: use DEBIAN_FRONTEND=noninteractive
Using a build-arg so that we don't have to specify it for each
`apt-get install`, and to preserve that the `DEBIAN_FRONTEND` is
preserved in the image itself (which changes the default behavior,
and can be surprising if the image is run interactively).`

With this patch, some (harmless, but possibly confusing) errors
are no longer printed during build, for example:

```patch
 Unpacking libgcc1:armhf (1:6.3.0-18+deb9u1) ...
 Selecting previously unselected package libc6:armhf.
 Preparing to unpack .../04-libc6_2.24-11+deb9u4_armhf.deb ...
-debconf: unable to initialize frontend: Dialog
-debconf: (TERM is not set, so the dialog frontend is not usable.)
-debconf: falling back to frontend: Readline
 Unpacking libc6:armhf (2.24-11+deb9u4) ...
 Selecting previously unselected package libgcc1:arm64.
 Preparing to unpack .../05-libgcc1_1%3a6.3.0-18+deb9u1_arm64.deb ...
 Unpacking libgcc1:arm64 (1:6.3.0-18+deb9u1) ...
 Selecting previously unselected package libc6:arm64.
 Preparing to unpack .../06-libc6_2.24-11+deb9u4_arm64.deb ...
-debconf: unable to initialize frontend: Dialog
-debconf: (TERM is not set, so the dialog frontend is not usable.)
-debconf: falling back to frontend: Readline

```

Looks like some output is now also printed on stdout instead of stderr

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-11 17:14:21 +02:00
Sebastiaan van Stijn
90af4ba5e7
Merge pull request #39714 from thaJeztah/fix_docker_py_deselects
docker-py: skip PullImageTest::test_pull_invalid_platform
2019-08-11 03:36:03 +02:00
Olli Janatuinen
8660330173 Unit test for getOrphan
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-08-10 14:48:47 +03:00
Sebastiaan van Stijn
6f5c377ddc
docker-py: skip PullImageTest::test_pull_invalid_platform
and remove `PullImageTest::test_build_invalid_platform` from the list,
which was a copy/paste error in f8cde0b32d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-10 13:18:13 +02:00
Sebastiaan van Stijn
901d30b6b4
Merge pull request #39685 from kolyshkin/jenkins-find
Jenkinsfile: avoid errors from find
2019-08-10 13:00:54 +02:00
Kir Kolyshkin
b283dff3ff Jenkinsfile: avoid errors from find
There are many errors like this one:

> 01:39:28.750 find: ‘bundles/test-integration/dbc77018d39a5/root/overlay2/f49953a883daceee60a481dd8e1e37b0f806d309258197d6ba0f6871236d3d47/work/work’: Permission denied

(probably caused by bad permissions)

These directories are not to be looked at when we search for logs, so
let's exclude them. It's not super easy to do in find, here is some
kind of an explanation for find arguments

```
PATTERN ACTION OR PATTERN                           ACTION
-path X -prune -o -type f [AND] (-name A -o name B) -print
```

(here -o means OR, while AND is implicit)

While at it,
 - let the find know we're only looking for files, not directories
 - remove a subshell and || true
 - remove `-name integration.test` (there are no such files)

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-08-09 15:18:41 -07:00
Tibor Vass
c5c11f9cef
Merge pull request #39709 from thaJeztah/remove_experimental
Jenkinsfile: remove "experimental" stage
2019-08-09 14:48:34 -07:00
Sebastiaan van Stijn
e856b46cfb
Jenkinsfile: remove "experimental" stage
All tests that require experimental either spin up a separate daemon,
or use the main daemon if experimental is enabled.

This patch

- allows enabling "experimental" for stages through an environment variable
- enables experimental by default on all stages, so that some of these tests
  don't have to start a new daemon.
- removes the seaprate "experimental" stage, because it was running exactly
  the same tests as the "janky" stage.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-09 20:25:54 +02:00
Sebastiaan van Stijn
a43123cab1
Consistently use DOCKER_EXPERIMENTAL=1 instead or =y
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-09 20:24:00 +02:00
Brian Goff
c2f70da793
Merge pull request #38859 from kolyshkin/journald
Fixes for reading journald logs
2019-08-09 10:22:40 -07:00
Sebastiaan van Stijn
f8cde0b32d
docker-py: deselect broken experimental tests
These tests are fixed upstream, but those fixes are not yet in a
released version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-09 18:55:08 +02:00
Sebastiaan van Stijn
1ea8b413d1
initBridgeDriver: minor cleanup and linting fixes
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-09 18:34:35 +02:00
Sebastiaan van Stijn
4cc9dc73ba
Merge pull request #39703 from ddebroy/fix-39623
Fix regression in handling of NotFound err during startup ENGCORE-929
2019-08-09 14:42:14 +02:00
Deep Debroy
4d5b6260bc Fix regression in handling of NotFound err during startup
Signed-off-by: Deep Debroy <ddebroy@docker.com>
2019-08-08 16:58:52 -07:00
Kirill Kolyshkin
150530564a
Merge pull request #39695 from thaJeztah/fix_copy_on_windows
Builder: fix "COPY --from" to non-existing directory on Windows
2019-08-08 10:25:31 -07:00