Commit graph

37557 commits

Author SHA1 Message Date
Tibor Vass
d4e12315cd hack: add more debugging to understand exit codepath
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit b280ea114f)
Signed-off-by: Tibor Vass <tibor@docker.com>
2020-05-28 03:32:24 +00:00
Tibor Vass
ad0f0b3970
Merge pull request #40978 from thaJeztah/19.03_backport_bump_golang_1.13.11
[19.03 backport] Bump Golang 1.13.11
2020-05-20 14:35:26 -07:00
Sebastiaan van Stijn
29796375c9
Bump Golang 1.13.11
full diff: https://github.com/golang/go/compare/go1.13.10...go1.13.11

go1.13.11 (released 2020/05/14) includes fixes to the compiler. See the Go 1.13.11
milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.11+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 90758fb028)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-19 20:18:02 +02:00
Tibor Vass
c1cc6ec81a
Merge pull request #40988 from thaJeztah/19.03_backport_fix_gotestsum_install
[19.03 backport] Fix bug in gotestsum installer causing dependencies to not be downloaded
2020-05-19 10:41:05 -07:00
Sebastiaan van Stijn
8f1ab4e612
Fix bug in gotestsum installer causing dependencies to not be downloaded
Building gotestsum started to fail after the repository removed some
dependencies on master.

What happens is that first, we `go get` the package (with go modules disabled);

    GO111MODULE=off go get -d gotest.tools/gotestsum

Which gets the latest version from master, and fetches the dependencies used
on master. Then we checkout the version we want to install (for example `v0.3.5`)
and run go build.

However, `v0.3.5` depends on logrus, and given that we ran `go get` for `master`,
that dependency was not fetched, and build fails.

This patch modifies the installer to use go modules (alternatively we could
probably run `go get .` after checking out the `v0.3.5` version),

We need to modify all installers, as it looks like this is a standard pattern
we use, but other dependencies were not failing (yet), so this patch only
addresses the immediate failure.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 1d9da1b233)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-17 22:52:54 +02:00
Tibor Vass
811a247d06
Merge pull request #40970 from AkihiroSuda/archive-40939-1903
[19.03 backport] pkg/archive: escape ":" symbol in overlay lowerdir
2020-05-14 16:03:53 -07:00
Tibor Vass
4d1885fb94
Merge pull request #40964 from AkihiroSuda/rootless-requires-slirp4netns-040-1903
[19.03 backport] dockerd-rootless.sh: bump up slirp4netns requirement to v0.4.0
2020-05-14 15:37:08 -07:00
Akihiro Suda
0a3b2bda34 pkg/archive: escape ":" symbol in overlay lowerdir
lowerdir needs escaping:
https://github.com/torvalds/linux/blob/v5.4/fs/overlayfs/super.c#L835-L853

Fix #40939

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 6a5e3547fb)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-05-15 06:57:49 +09:00
Akihiro Suda
9057ddf37c dockerd-rootless.sh: bump up slirp4netns requirement to v0.4.0
slirp4netns v0.3.X turned out not to work with RootlessKit >= v0.7.1:
https://github.com/rootless-containers/rootlesskit/issues/143

As slirp4netns v0.3.X reached EOL on Mar 31, 2020, RootlessKit is not
going to fix support for slirp4netns v0.3.X.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit c86abee1a4)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-05-13 12:22:27 +09:00
Akihiro Suda
ab567a4327
Merge pull request #40955 from tonistiigi/19.03-buildkit-update
[19.03] vendor: update buildkit to a7d7b7f1
2020-05-12 13:56:06 +09:00
Akihiro Suda
ee3f3ece72
Merge pull request #40951 from AkihiroSuda/rootlesskit-095-1903
[19.03 backport] bump up rootlesskit to v0.9.5
2020-05-12 13:39:56 +09:00
Tonis Tiigi
a76633684b vendor: update buildkit to a7d7b7f1
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2020-05-11 16:28:55 -07:00
Akihiro Suda
0803200be9
Merge pull request #40946 from thaJeztah/19.03_backport_fix_selinux_enotsup
[19.03 backport] SELinux: fix ENOTSUP errors not being detected when relabeling
2020-05-12 00:33:11 +09:00
Akihiro Suda
706008a1da bump up rootlesskit to v0.9.5
Supports numeric ID in /etc/subuid and /etc/subgid .
Fix #40926

Full changes: https://github.com/rootless-containers/rootlesskit/compare/v0.9.4...v0.9.5

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 17bb5f4b15)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-05-11 23:53:21 +09:00
Sebastiaan van Stijn
57f6c9a0ef
SELinux: fix ENOTSUP errors not being detected when relabeling
Commit 12c7541f1f updated the
opencontainers/selinux dependency to v1.3.1, which had a breaking
change in the errors that were returned.

Before v1.3.1, the "raw" `syscall.ENOTSUP` was returned if the
underlying filesystem did not support xattrs, but later versions
wrapped the error, which caused our detection to fail.

This patch uses `errors.Is()` to check for the underlying error.
This requires github.com/pkg/errors v0.9.1 or above (older versions
could use `errors.Cause()`, but are not compatible with "native"
wrapping of errors in Go 1.13 and up, and could potentially cause
these errors to not being detected again.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 49f8a4224c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-10 17:08:42 +02:00
Sebastiaan van Stijn
c4c6cf6b6a
Merge pull request #40921 from cpuguy83/19.03_log_rotate_error_handling
19.03: logfile: Check if log is closed on close error during rotate
2020-05-08 01:13:30 +02:00
Brian Goff
7d4dd91a52 logfile: Check if log is closed on close error during rotate
This prevents getting into a situation where a container log cannot make
progress because we tried to rotate a file, got an error, and now the
file is closed. The next time we try to write a log entry it will try
and rotate again but error that the file is already closed.

I wonder if there is more we can do to beef up this rotation logic.
Found this issue while investigating missing logs with errors in the
docker daemon logs like:

```
Failed to log message for json-file: error closing file: close <file>:
file already closed
```

I'm not sure why the original rotation failed since the data was no
longer available.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 3989f91075)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-05-07 12:22:58 -07:00
Sebastiaan van Stijn
edf2c49410 vendor: pkg/errors v0.9.1
full diff: https://github.com/pkg/errors/compare/v0.8.1...v0.9.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit dc089c22ce)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-05-07 12:22:58 -07:00
Tibor Vass
1adcc64f40
Merge pull request #40877 from thaJeztah/19.03_update_buildkit
[19.03] vendor: buildkit v0.6.4-5-g59e305aa
2020-05-01 15:51:47 -07:00
Tibor Vass
e7349349fd
Merge pull request #40850 from thaJeztah/19.03_backport_criu_3.13
[19.03 backport] Update CRIU to v3.13 "Silicon Willet"
2020-04-30 08:59:55 -07:00
Tibor Vass
3677003554
Merge pull request #40782 from thaJeztah/19.03_backport_switch_to_s390x_ubuntu_1804
[19.03 backport] Switch to s390x Ubuntu 18.04
2020-04-30 08:26:41 -07:00
Sebastiaan van Stijn
63841af153
[19.03] vendor: buildkit v0.6.4-5-g59e305aa
full diff: b26cff2413...59e305aa33

- moby/buildkit#1469 Avoid creation of irrelevant temporary files on Windows
    - backport of moby/buildkit#1462 for the docker-19.03/v0.6 branch

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-30 17:13:00 +02:00
Tianon Gravi
2fbb374ab7
Merge pull request #40863 from AkihiroSuda/rootlesskit-094-1903
[19.03 backport] bump up rootlesskit to v0.9.4
2020-04-28 23:23:49 -07:00
Akihiro Suda
946d0ff67e bump up rootlesskit to v0.9.4
Now `rootlesskit-docker-proxy` returns detailed error message on
exposing privileged ports: https://github.com/rootless-containers/rootlesskit/pull/136

Full changes: https://github.com/rootless-containers/rootlesskit/compare/v0.7.1...v0.9.4

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit f6ac841633)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-04-28 05:18:11 +09:00
Sebastiaan van Stijn
70e7d6fe4a
Update CRIU to v3.13 "Silicon Willet"
full diff: https://github.com/checkpoint-restore/criu/compare/v3.12...v3.13

Here we have some bugfixes, huuuge *.py patch for coding style
and nice set of new features like 32bit for ARM, TLS for page
server and new mode for CGroups.

New features

- VDSO: arm32 support
- Add TLS support for page server communications
- "Ignore" mode for --manage-cgroups
- Restore SO_BROADCAST option for inet sockets

Bugfixes

- Auxiliary events were left in inotify queues
- Lazy-pages daemon didn't detect stack pages and surrounders properly and marked them as "lazy"
- Memory and resource leakage were detected by coverity, cppcheck and clang

Improvements

- Use gettimeofday() directly from vdso for restore timings
- Reformat all .py code into pep8 style

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f508db4833)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-22 13:55:32 +02:00
Akihiro Suda
f432f71595
Merge pull request #40563 from thaJeztah/19.03_backport_fix_windows_file_handles
[19.03 backport] Use FILE_SHARE_DELETE for log files on Windows.
2020-04-17 17:00:19 +09:00
Akihiro Suda
47a6d9b54f
Merge pull request #40565 from thaJeztah/19.03_backport_fix_bip_subnet_config
[19.03 backport] Set the bip network value as the subnet
2020-04-17 16:59:34 +09:00
Akihiro Suda
6a0995e0d8
Merge pull request #40831 from thaJeztah/19.03_bump_swarmkit
[19.03] vendor: swarmkit 0b8364e7d08aa0e972241eb59ae981a67a587a0e
2020-04-17 16:35:05 +09:00
Sebastiaan van Stijn
e4f239d68e
[19.03] vendor: swarmkit 0b8364e7d08aa0e972241eb59ae981a67a587a0e
full diff: 062b694b46...0b8364e7d0

- Fix leaking tasks.db

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-16 21:55:43 +02:00
Brian Goff
25b82fa9b8
Merge pull request #40801 from thaJeztah/19.03_backport_update_go_events
[19.03 backport] vendor: update go-events to fix alignment for 32bit systems
2020-04-15 14:38:17 -07:00
Sebastiaan van Stijn
e149ff62fe
vendor: update go-events to fix alignment for 32bit systems
- relates to moby/buildkit 1111
- relates to moby/buildkit 1079
- relates to docker/buildx 129

full diff: 9461782956...e31b211e4f

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e7183dbfe9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-15 21:17:11 +02:00
Brian Goff
90a31c4829
Merge pull request #40809 from thaJeztah/19.03_update_libnetwork
[19.03] update libnetwork b9bcf0c3fba9ef8897c9676c5b70ba0345b84b17
2020-04-15 06:19:44 -07:00
Brian Goff
aa98b4f5d6
Merge pull request #40803 from thaJeztah/19.03_backport_bump_golang_1.13.10
[19.03 backport] Bump Golang 1.13.10
2020-04-13 10:59:40 -07:00
Sebastiaan van Stijn
860e7e273d
Merge pull request #40800 from thaJeztah/19.03_backport_api_docs_fix_link
[19.03 backport] api docs: fix broken link on GitHub
2020-04-12 15:47:12 +02:00
Tianon Gravi
a58b52b037
Merge pull request #40799 from thaJeztah/19.03_backport_fix_test_filter
[19.03 backport] Fix TEST_FILTER to work for both "integration" and "integration-cli"
2020-04-10 12:35:59 -07:00
Sebastiaan van Stijn
a6beb24dc5
[19.03] update libnetwork b9bcf0c3fba9ef8897c9676c5b70ba0345b84b17
full diff: 0941c3f409...b9bcf0c3fb

- docker/libnetwork#2545 Fix NPE due to null value returned by ep.Iface()
    - backport of docker/libnetwork#2544
    - addresses docker/docker#37506

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-10 20:34:31 +02:00
Sebastiaan van Stijn
282567a58d
Bump Golang 1.13.10
go1.13.10 (released 2020/04/08) includes fixes to the go command, the runtime,
os/exec, and time packages. See the Go 1.13.10 milestone on the issue tracker
for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.10+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.13.9...go1.13.10

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7cb13d4d85)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-09 22:00:49 +02:00
Sebastiaan van Stijn
b66813eb45
api docs: fix broken link on GitHub
The pages that were linked to have moved, so changing the
links to point to docs.docker.com instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e9348898d3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-09 21:36:10 +02:00
Brian Goff
edbb1d9e95
Merge pull request #40784 from thaJeztah/19.03_update_buildkit
[19.03] vendor buildkit b26cff2413cc6a466f8739262efa13bd126f8fc7
2020-04-09 12:04:55 -07:00
Sebastiaan van Stijn
9d8eccec8e
Fix TEST_FILTER to work for both "integration" and "integration-cli"
The TEST_FILTER variable allows running a single integration or integration-cli
test. However, it failed to work properly for integration-cli tests.

Before:
-----------

    # Filtering "integration" tests works:
    make TEST_FILTER=TestInspectCpusetInConfigPre120 test-integration
    ...
    DONE 1 tests in 18.331s

    # But running a single test in "integration-cli" did not:

    make TEST_FILTER=TestSwarmNetworkCreateIssue27866 test-integration
    ...
    DONE 0 tests in 17.314s

Trying to manually add the `/` prefix, didn't work either, because that made the
"grep" fail to find which test-suites to run/skip:

    make TEST_FILTER=/TestSwarmNetworkCreateIssue27866 test-integration
    ---> Making bundle: test-integration (in bundles/test-integration)
    make: *** [test-integration] Error 1

After:
-----------

    make TEST_FILTER=TestInspectCpusetInConfigPre120 test-integration
    ...
    DONE 1 tests in 18.331s

    make TEST_FILTER=TestSwarmNetworkCreateIssue27866 test-integration
    ...
    DONE 12 tests in 26.527s

Note that the `12` tests is still a bit misleading, because every _suite_ is
started (which is counted as a test), but no tests are run. This is still
something that could be improved on.

This patch also makes a small modification to the code that's setting
`integration_api_dirs`, and no longer runs `go list` if not needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit e7805653b8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-09 13:46:15 +02:00
Akihiro Suda
4275aec641
Merge pull request #40592 from thaJeztah/19.03_backport_bump_golang_1.13
[19.03 backport] Update Golang 1.13.9
2020-04-09 05:43:14 +09:00
Akihiro Suda
4b040147cf
Merge pull request #40417 from thaJeztah/19.03_backport_test_fixes
[19.03 backport] Testing changes
2020-04-07 09:50:27 +09:00
Sebastiaan van Stijn
08a2fe0d56
[19.03] vendor buildkit b26cff2413cc6a466f8739262efa13bd126f8fc7
full diff: https://github.com/moby/buildkit/compare/v0.6.4...b26cff2413cc6a466f8739262efa13bd126f8fc7

- solver: avoid looping over same keys in loadwithparents

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-06 19:06:25 +02:00
Brian Goff
1e1caccb13
Merge pull request #40780 from thaJeztah/19.03_backport_map_sync
[19.03 backport] builder: fix concurrent map write
2020-04-06 08:56:01 -07:00
Sebastiaan van Stijn
5ba2bf37a8
Bump Golang 1.13.9
go1.13.9 (released 2020/03/19) includes fixes to the go command, tools, the
runtime, the toolchain, and the crypto/cypher package. See the Go 1.13.9
milestone on the issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.9+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.13.8...go1.13.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6ee9a1ad29)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-06 15:38:14 +02:00
Sebastiaan van Stijn
f432a04243
Update Golang 1.13.8
full diff: https://github.com/golang/go/compare/go1.13.7...go1.13.8

go1.13.8 (released 2020/02/12) includes fixes to the runtime, the crypto/x509,
and net/http packages. See the Go 1.13.8 milestone on the issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.13.8+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3f7503f98a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-06 15:36:28 +02:00
Sebastiaan van Stijn
81458b3144
Update Golang 1.13.7 (CVE-2020-0601, CVE-2020-7919)
full diff: https://github.com/golang/go/compare/go1.13.6...go1.13.7

go1.13.7 (released 2020/01/28) includes two security fixes. One mitigates
the CVE-2020-0601 certificate verification bypass on Windows. The other affects
only 32-bit architectures.

https://github.com/golang/go/issues?q=milestone%3AGo1.13.7+label%3ACherryPickApproved

- X.509 certificate validation bypass on Windows 10
  A Windows vulnerability allows attackers to spoof valid certificate chains when
  the system root store is in use. These releases include a mitigation for Go
  applications, but it’s strongly recommended that affected users install the
  Windows security update to protect their system.
  This issue is CVE-2020-0601 and Go issue golang.org/issue/36834.
- Panic in crypto/x509 certificate parsing and golang.org/x/crypto/cryptobyte
  On 32-bit architectures, a malformed input to crypto/x509 or the ASN.1 parsing
  functions of golang.org/x/crypto/cryptobyte can lead to a panic.
  The malformed certificate can be delivered via a crypto/tls connection to a
  client, or to a server that accepts client certificates. net/http clients can
  be made to crash by an HTTPS server, while net/http servers that accept client
  certificates will recover the panic and are unaffected.
  Thanks to Project Wycheproof for providing the test cases that led to the
  discovery of this issue. The issue is CVE-2020-7919 and Go issue golang.org/issue/36837.
  This is also fixed in version v0.0.0-20200124225646-8b5121be2f68 of golang.org/x/crypto/cryptobyte.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 878db479be)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-06 15:36:26 +02:00
Sebastiaan van Stijn
6e1d159680
Update Golang 1.13.6
full diff: https://github.com/golang/go/compare/go1.13.5...go1.13.6

go1.13.6 (released 2020/01/09) includes fixes to the runtime and the net/http
package. See the Go 1.13.6 milestone on the issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.13.6+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit d68385b861)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-06 15:36:24 +02:00
Sebastiaan van Stijn
4241093b63
Update Golang 1.13.5
go1.13.5 (released 2019/12/04) includes fixes to the go command, the runtime, the
linker, and the net/http package. See the Go 1.13.5 milestone on our issue tracker
for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.13.5+label%3ACherryPickApproved

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a218e9b7b0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-06 15:36:22 +02:00
Jintao Zhang
162fd8b856
Bump Golang 1.13.4
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
(cherry picked from commit cf86eeaf96)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-06 15:36:21 +02:00