Commit graph

37356 commits

Author SHA1 Message Date
Tonis Tiigi
c0fd6556f2
builder-next: close build context upload on cancel
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
(cherry picked from commit 2c2cd9b86a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-17 10:27:47 +01:00
Sebastiaan van Stijn
9077436e6e
Merge pull request #424 from thaJeztah/19.03_backport_39608_short_libnetwork_id
[19.03 backport] daemon: Use short libnetwork ID in exec-root & update libnetwork
2020-01-16 22:15:04 +01:00
Sebastiaan van Stijn
1a451ca6e0
Merge pull request #423 from thaJeztah/19.03_backport_win_restore_no_parallelism
[19.03 backport] Windows: Use system specific parallelism value on containers restart
2020-01-16 21:13:10 +01:00
Sebastiaan van Stijn
cf14fa7a23
Merge pull request #427 from thaJeztah/19.03_backport_40232-comply_with_gelf_spec
[19.03 backport] logger/gelf: Skip empty lines to comply with spec
2020-01-16 21:09:12 +01:00
Sebastiaan van Stijn
f6398c1f07
Merge pull request #425 from cpuguy83/backport_40169_windows_version_quad
[19.03] Windows: Only set VERSION_QUAD if unset
2020-01-16 21:06:58 +01:00
Sebastiaan van Stijn
71e07f9130
Merge pull request #435 from thaJeztah/19.03_bump_golang_1.12.14
[19.03] Bump Golang 1.12.14
2020-01-16 20:58:52 +01:00
Sebastiaan van Stijn
c4dbf36951
Merge pull request #428 from thaJeztah/19.03_bump_containerd_1.2.11
[19.03] Update containerd to v1.2.11, runc v1.0.0-rc9
2020-01-16 20:58:28 +01:00
Sebastiaan van Stijn
077f093988
Merge pull request #437 from thaJeztah/19.03_backport_skip_broken_docker_py_test
[19.03 backport] docker-py: skip broken ImageCollectionTest::test_pull_multiple, and re-enable fixed tests
2020-01-16 20:56:16 +01:00
Sebastiaan van Stijn
96582ab4ba
Merge pull request #438 from ydcool/19.03_backport_fix_compiling_errors_on_mips
[19.03 backport] cast Dev and Rdev of Stat_t to uint64 for mips
2020-01-16 20:54:12 +01:00
Dominic
16f503c048
cast Dev and Rdev of Stat_t to uint64 for mips
Signed-off-by: Dominic <yindongchao@inspur.com>
Signed-off-by: Dominic Yin <yindongchao@inspur.com>
(cherry picked from commit 5f0231bca1)
Signed-off-by: Dominic Yin <yindongchao@inspur.com>
2020-01-13 09:25:13 +08:00
Sebastiaan van Stijn
8e57214487
docker-py: skip broken ImageCollectionTest::test_pull_multiple
The ImageCollectionTest.test_pull_multiple test performs a `docker pull` without
a `:tag` specified) to pull all tags of the given repository (image).

After pulling the image, the image(s) pulled are checked to verify if the list
of images contains the `:latest` tag.

However, the test assumes that all tags of the image are tags for the same
version of the image (same digest), and thus a *single* image is returned, which
is not always the case.

Currently, the `hello-world:latest` and `hello-world:linux` tags point to a
different digest, therefore the `client.images.pull()` returns multiple images:
one image for digest, making the test fail:

    =================================== FAILURES ===================================
    ____________________ ImageCollectionTest.test_pull_multiple ____________________
    tests/integration/models_images_test.py:90: in test_pull_multiple
        assert len(images) == 1
    E   AssertionError: assert 2 == 1
    E    +  where 2 = len([<Image: 'hello-world:linux'>, <Image: 'hello-world:latest'>])

This patch temporarily skips the broken test until it is fixed upstream.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f2b25e498f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-06 13:43:37 +01:00
Sebastiaan van Stijn
b617355190
docker-py: re-enable tests that were fixed in v4.1.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6bc45b09e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-06 13:43:28 +01:00
Sebastiaan van Stijn
8dbc7420ed
[19.03] Bump Golang 1.12.14
go1.12.14 (released 2019/12/04) includes a fix to the runtime. See the Go 1.12.14
milestone on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.12.14+label%3ACherryPickApproved

Update Golang 1.12.13
------------------------

go1.12.13 (released 2019/10/31) fixes an issue on macOS 10.15 Catalina where the
non-notarized installer and binaries were being rejected by Gatekeeper. Only macOS
users who hit this issue need to update.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-30 10:21:06 +01:00
Sebastiaan van Stijn
cfcf25bb54
[19.03] Update containerd binary to v1.2.11
full diff: https://github.com/containerd/containerd/compare/v1.2.10...v1.2.11

The eleventh patch release for containerd 1.2 includes an updated runc with
an additional fix for CVE-2019-16884 and a Golang update.

Notable Updates
-----------------------

- Update the runc vendor to v1.0.0-rc9 which includes an additional mitigation
  for CVE-2019-16884.
  More details on the runc CVE in opencontainers/runc#2128, and the additional
  mitigations in opencontainers/runc#2130.
- Add local-fs.target to service file to fix corrupt image after unexpected host
  reboot. Reported in containerd/containerd#3671, and fixed by containerd/containerd#3746.
- Update Golang runtime to 1.12.13, which includes security fixes to the crypto/dsa
  package made in Go 1.12.11 (CVE-2019-17596), and fixes to the go command, runtime,
  syscall and net packages (Go 1.12.12).

CRI fixes:
-----------------------

- Fix shim delete error code to avoid unnecessary retries in the CRI plugin. Discovered
  in containerd/cri#1309, and fixed by containerd/containerd#3732 and containerd/containerd#3739.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-28 10:56:28 +01:00
Sebastiaan van Stijn
efcd84e47c
[19.03] Update to runc v1.0.0-rc9
full diff: 3e425f80a8...v1.0.0-rc9

- opencontainers/runc#1951 Add SCMP_ACT_LOG as a valid Seccomp action
- opencontainers/runc#2130 *: verify operations on /proc/... are on procfs
  This is an additional mitigation for CVE-2019-16884. The primary problem
  is that Docker can be coerced into bind-mounting a file system on top of
  /proc (resulting in label-related writes to /proc no longer happening).

  While we are working on mitigations against permitting the mounts, this
  helps avoid our code from being tricked into writing to non-procfs
  files. This is not a perfect solution (after all, there might be a
  bind-mount of a different procfs file over the target) but in order to
  exploit that you would need to be able to tweak a config.json pretty
  specifically (which thankfully Docker doesn't allow).

  Specifically this stops AppArmor from not labeling a process silently
  due to /proc/self/attr/... being incorrectly set, and stops any
  accidental fd leaks because /proc/self/fd/... is not real.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-28 10:56:06 +01:00
Jonas Heinrich
449b60fcd0
logger/gelf: Skip empty lines to comply with spec
The [gelf payload specification](http://docs.graylog.org/en/2.4/pages/gelf.html#gelf-payload-specification)
demands that the field `short_message` *MUST* be set by the client library.
Since docker logging via the gelf driver sends messages line by line, it can happen that messages with an empty
`short_message` are passed on. This causes strict downstream processors (like graylog) to raise an exception.

The logger now skips messages with an empty line.

Resolves: #40232
See also: #37572

Signed-off-by: Jonas Heinrich <Jonas@JonasHeinrich.com>
(cherry picked from commit 5c6b913ff1)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-26 12:54:18 +01:00
Grant Millar
d3d724e45a
daemon: Use short libnetwork ID in exec-root & update libnetwork
also updates libnetwork to d9a6682a4dbb13b1f0d8216c425fe9ae010a0f23
full diff:

3eb39382bf...d9a6682a4d

- docker/libnetwork#2482 [19.03 backport] Shorten controller ID in exec-root to not hit UNIX_PATH_MAX
- docker/libnetwork#2483 [19.03 backport] Fix panic in drivers/overlay/encryption.go

Signed-off-by: Grant Millar <rid@cylo.io>
(cherry picked from commit df7b8f458a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-21 14:30:33 +01:00
Brian Goff
d699e3de12 Windows: Only set VERSION_QUAD if unset
When trying to build with some pretty typical version strings this was
causing failures trying to generate the windows resource file.

The resource file is already gated by an `ifdef` for this var, so
instead of blindly setting based on "VERSION", which can contain some
characters which are incompatible (e.g. 1.2.3.rc.0 will fail due to the
".rc").

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit ce931f28ea)
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-11-15 16:16:00 -08:00
Olli Janatuinen
e1cae011e2
Windows: Use system specific parallelism value on containers restart
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
(cherry picked from commit 447a840254)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-13 14:54:41 -08:00
Andrew Hsu
ea84732a77
Merge pull request #422 from tonistiigi/1903-update-buildkit
[19.03] vendor: update buildkit to 928f3b48
2019-11-12 20:22:39 -08:00
Tonis Tiigi
33b2719488 vendor: update buildkit to 928f3b48
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-11-12 18:17:50 -08:00
Andrew Hsu
075a0201b9
Merge pull request #374 from thaJeztah/19.03_backport_add_tc_dynamic_ingress_network
[19.03 backport] Add TC to check dynamic subnet for ingress network
2019-11-05 20:12:14 -08:00
Andrew Hsu
5d5083a57a
Merge pull request #420 from tonistiigi/1903-buildkit-update
[19.03] vendor: update buildkit to ff93519ee
2019-11-04 17:23:40 -08:00
Tonis Tiigi
25162d4a4e vendor: update buildkit to ff93519ee
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-11-04 16:06:35 -08:00
Andrew Hsu
35913e58c2
Merge pull request #419 from andrewhsu/xit
[19.03] Windows: disable flaky test TestStartReturnCorrectExitCode
2019-11-01 08:41:33 -07:00
Arko Dasgupta
12e7d99439
Add TC to check dyanmic subnet for ingress network
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
(cherry picked from commit e2b5ac75a393f6942c37efdd888fc3bc761de244)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-31 17:46:56 +01:00
Sebastiaan van Stijn
0c38d56a6d
Revert "Revert "[19.03] bump swarmkit to f35d9100f2c6ac810cc8d7de6e8f93dcc7a42d29""
This reverts commit ef4366ee89.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-31 17:46:54 +01:00
Andrew Hsu
031ef2dc8e Windows: disable flaky test TestStartReturnCorrectExitCode
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit 1be272ef76)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-10-31 16:05:07 +00:00
Andrew Hsu
ddb60aa6d1
Merge pull request #418 from kolyshkin/19.03-go1.12.12
[19.03] Bump golang 1.12.12
2019-10-30 09:47:17 -07:00
Kir Kolyshkin
92a8618ddc Bump golang 1.12.12
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-10-28 14:01:45 -07:00
Andrew Hsu
370def6b30
Merge pull request #412 from thaJeztah/19.03_backport_builder_entitilement_confg
[19.03 backport] builder entitlements configuration added.
2019-10-28 10:53:19 -07:00
Andrew Hsu
e2e3abec71
Merge pull request #410 from thaJeztah/19.03_backport_fix_buildkit_prunegc_filter_config
[19.03 backport] daemon/config: fix filter type in BuildKit GC config
2019-10-28 10:52:31 -07:00
Andrew Hsu
0e8949a003
Merge pull request #407 from thaJeztah/19.03_backport_better_container_error
[19.03 backport] Propagate GetContainer error from event processor
2019-10-28 10:50:46 -07:00
Andrew Hsu
967aa3a9ef
Merge pull request #405 from thaJeztah/19.03_backport_oci_regression
[19.03 backport] Use ocischema package instead of custom handler
2019-10-28 10:50:08 -07:00
Andrew Hsu
83bcde8f60
Merge pull request #408 from thaJeztah/19.03_backport_update_rootless_docs
[19.03 backport] docs/rootless.md: update
2019-10-28 10:46:27 -07:00
Andrew Hsu
d91a85a9b5
Merge pull request #397 from thaJeztah/19.03_backport_slirp4netns_sandbox
[19.03 backport] rootless: harden slirp4netns with mount namespace and seccomp
2019-10-28 10:45:18 -07:00
Sebastiaan van Stijn
e5a0bc6a50
Add GoDoc to fix linting validation
The validate step in CI was broken, due to a combination of
086b4541cf, fbdd437d29,
and 85733620eb being merged to master.

```
api/types/filters/parse.go:39:1: exported method `Args.Keys` should have comment or be unexported (golint)
func (args Args) Keys() []string {
^
daemon/config/builder.go:19:6: exported type `BuilderGCFilter` should have comment or be unexported (golint)
type BuilderGCFilter filters.Args
     ^
daemon/config/builder.go:21:1: exported method `BuilderGCFilter.MarshalJSON` should have comment or be unexported (golint)
func (x *BuilderGCFilter) MarshalJSON() ([]byte, error) {
^
daemon/config/builder.go:35:1: exported method `BuilderGCFilter.UnmarshalJSON` should have comment or be unexported (golint)
func (x *BuilderGCFilter) UnmarshalJSON(data []byte) error {
^
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9d726f1c18)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 10:28:25 +02:00
Tibor Vass
dae4436d1c
daemon/config: add MarshalJSON for future proofing
If anything marshals the daemon config now or in the future
this commit ensures the correct canonical form for the builder
GC policies' filters.

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 85733620eb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 10:28:11 +02:00
Tibor Vass
1e26b431c9
daemon/config: fix filter type in BuildKit GC config
For backwards compatibility, the old incorrect object format for
builder.GC.Rule.Filter still works but is deprecated in favor of array of
strings akin to what needs to be passed on the CLI.

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit fbdd437d29)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 10:28:09 +02:00
Kunal Kushwaha
ce74774c09
builder entitlements configutation added.
buildkit supports entitlements like network-host and security-insecure.
this patch aims to make it configurable through daemon.json file.
by default network-host is enabled & secuirty-insecure is disabled.

Signed-off-by: Kunal Kushwaha <kunal.kushwaha@gmail.com>
(cherry picked from commit 8b7bbf180f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 10:06:46 +02:00
Tibor Vass
645f559352
Merge pull request #411 from thaJeztah/19.03_backport_fix_dco_branch
[19.03 backport] Jenkinsfile: set repo and branch for DCO check as well
2019-10-21 16:22:02 -07:00
Sebastiaan van Stijn
9c388fb119
Jenkinsfile: set repo and branch for DCO check as well
Commit 7019b60d0d added these
env-vars to other stages, but forgot to update the DCO stage,
which also does a diff to validate commits that are in a PR.

Also adding openssh-client, for situations where the upstream
needs to be accessed through an ssh connection.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7c5fd83c22)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-21 23:32:48 +02:00
Akihiro Suda
a8b454a934
docs/rootless.md: update
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e76dea157e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-20 23:50:07 +02:00
Brian Goff
fd169c00bf
Propagate GetContainer error from event processor
Before this change we just accept that any error is "not found" and it
could be something else, but even if it it is just a "not found" kind of
error this should be dealt with from the container store and not the
event processor.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 54e30a62d3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-17 02:49:24 +02:00
Brian Goff
e037bade8c
Use ocischema package instead of custom handler
Previously we were re-using schema2.DeserializedManifest to handle oci
manifests. The issue lies in the fact that distribution started
validating the media type string during json deserialization. This
change broke our usage of that type.

Instead distribution now provides direct support for oci schemas, so use
that instead of our custom handlers.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit e443512ce4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-14 23:06:05 +02:00
Andrew Hsu
adfac697dc
Merge pull request #404 from thaJeztah/19.03_revert_iptables_check2
[19.03 backport] revert controller: Check if IPTables is enabled for arrangeUserFilterRule ENGCORE-1114
2019-10-11 14:19:53 -07:00
Sebastiaan van Stijn
54a58760b6
[19.03 backport] revert controller: Check if IPTables is enabled for arrangeUserFilterRule
This change caused a regression, causing the DOCKER-USER chain
to not be created, despite iptables being enabled on the daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 21:10:48 +02:00
Andrew Hsu
5787ef7e9c
Merge pull request #396 from thaJeztah/19.03_backport_update_moved_repositories
[19.03 backport] Update links/references to transferred repositories
2019-10-10 10:58:11 -07:00
Andrew Hsu
9a21cf7e55
Merge pull request #399 from thaJeztah/19.03_backport_do_the_right_diff_do_the_right_diff
[19.03 backport] Jenkinsfile: set repo and branch, to assist validate_diff()
2019-10-10 10:56:41 -07:00
Sebastiaan van Stijn
abbc956ac8
Jenkinsfile: set repo and branch, to assist validate_diff()
This is a continuation of 2a08f33166247da9d4c09d4c6c72cbb8119bf8df;

When running CI in other repositories (e.g. Docker's downstream
docker/engine repository), or other branches, the validation
scripts were calculating the list of changes based on the wrong
information.

This lead to weird failures in CI in a branch where these values
were not updated ':-) (CI on a pull request failed because it detected
that new tests were added to the deprecated `integration-cli` test-suite,
but the pull request did not actually make changes in that area).

This patch uses environment variables set by Jenkins to sets the
correct target repository (and branch) to compare to.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7019b60d0d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-07 23:52:55 +02:00