Commit graph

37345 commits

Author SHA1 Message Date
Sebastiaan van Stijn
c4dbf36951
Merge pull request #428 from thaJeztah/19.03_bump_containerd_1.2.11
[19.03] Update containerd to v1.2.11, runc v1.0.0-rc9
2020-01-16 20:58:28 +01:00
Sebastiaan van Stijn
077f093988
Merge pull request #437 from thaJeztah/19.03_backport_skip_broken_docker_py_test
[19.03 backport] docker-py: skip broken ImageCollectionTest::test_pull_multiple, and re-enable fixed tests
2020-01-16 20:56:16 +01:00
Sebastiaan van Stijn
96582ab4ba
Merge pull request #438 from ydcool/19.03_backport_fix_compiling_errors_on_mips
[19.03 backport] cast Dev and Rdev of Stat_t to uint64 for mips
2020-01-16 20:54:12 +01:00
Dominic
16f503c048
cast Dev and Rdev of Stat_t to uint64 for mips
Signed-off-by: Dominic <yindongchao@inspur.com>
Signed-off-by: Dominic Yin <yindongchao@inspur.com>
(cherry picked from commit 5f0231bca1)
Signed-off-by: Dominic Yin <yindongchao@inspur.com>
2020-01-13 09:25:13 +08:00
Sebastiaan van Stijn
8e57214487
docker-py: skip broken ImageCollectionTest::test_pull_multiple
The ImageCollectionTest.test_pull_multiple test performs a `docker pull` without
a `:tag` specified) to pull all tags of the given repository (image).

After pulling the image, the image(s) pulled are checked to verify if the list
of images contains the `:latest` tag.

However, the test assumes that all tags of the image are tags for the same
version of the image (same digest), and thus a *single* image is returned, which
is not always the case.

Currently, the `hello-world:latest` and `hello-world:linux` tags point to a
different digest, therefore the `client.images.pull()` returns multiple images:
one image for digest, making the test fail:

    =================================== FAILURES ===================================
    ____________________ ImageCollectionTest.test_pull_multiple ____________________
    tests/integration/models_images_test.py:90: in test_pull_multiple
        assert len(images) == 1
    E   AssertionError: assert 2 == 1
    E    +  where 2 = len([<Image: 'hello-world:linux'>, <Image: 'hello-world:latest'>])

This patch temporarily skips the broken test until it is fixed upstream.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f2b25e498f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-06 13:43:37 +01:00
Sebastiaan van Stijn
b617355190
docker-py: re-enable tests that were fixed in v4.1.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6bc45b09e7)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-06 13:43:28 +01:00
Sebastiaan van Stijn
cfcf25bb54
[19.03] Update containerd binary to v1.2.11
full diff: https://github.com/containerd/containerd/compare/v1.2.10...v1.2.11

The eleventh patch release for containerd 1.2 includes an updated runc with
an additional fix for CVE-2019-16884 and a Golang update.

Notable Updates
-----------------------

- Update the runc vendor to v1.0.0-rc9 which includes an additional mitigation
  for CVE-2019-16884.
  More details on the runc CVE in opencontainers/runc#2128, and the additional
  mitigations in opencontainers/runc#2130.
- Add local-fs.target to service file to fix corrupt image after unexpected host
  reboot. Reported in containerd/containerd#3671, and fixed by containerd/containerd#3746.
- Update Golang runtime to 1.12.13, which includes security fixes to the crypto/dsa
  package made in Go 1.12.11 (CVE-2019-17596), and fixes to the go command, runtime,
  syscall and net packages (Go 1.12.12).

CRI fixes:
-----------------------

- Fix shim delete error code to avoid unnecessary retries in the CRI plugin. Discovered
  in containerd/cri#1309, and fixed by containerd/containerd#3732 and containerd/containerd#3739.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-28 10:56:28 +01:00
Sebastiaan van Stijn
efcd84e47c
[19.03] Update to runc v1.0.0-rc9
full diff: 3e425f80a8...v1.0.0-rc9

- opencontainers/runc#1951 Add SCMP_ACT_LOG as a valid Seccomp action
- opencontainers/runc#2130 *: verify operations on /proc/... are on procfs
  This is an additional mitigation for CVE-2019-16884. The primary problem
  is that Docker can be coerced into bind-mounting a file system on top of
  /proc (resulting in label-related writes to /proc no longer happening).

  While we are working on mitigations against permitting the mounts, this
  helps avoid our code from being tricked into writing to non-procfs
  files. This is not a perfect solution (after all, there might be a
  bind-mount of a different procfs file over the target) but in order to
  exploit that you would need to be able to tweak a config.json pretty
  specifically (which thankfully Docker doesn't allow).

  Specifically this stops AppArmor from not labeling a process silently
  due to /proc/self/attr/... being incorrectly set, and stops any
  accidental fd leaks because /proc/self/fd/... is not real.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-28 10:56:06 +01:00
Andrew Hsu
ea84732a77
Merge pull request #422 from tonistiigi/1903-update-buildkit
[19.03] vendor: update buildkit to 928f3b48
2019-11-12 20:22:39 -08:00
Tonis Tiigi
33b2719488 vendor: update buildkit to 928f3b48
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-11-12 18:17:50 -08:00
Andrew Hsu
075a0201b9
Merge pull request #374 from thaJeztah/19.03_backport_add_tc_dynamic_ingress_network
[19.03 backport] Add TC to check dynamic subnet for ingress network
2019-11-05 20:12:14 -08:00
Andrew Hsu
5d5083a57a
Merge pull request #420 from tonistiigi/1903-buildkit-update
[19.03] vendor: update buildkit to ff93519ee
2019-11-04 17:23:40 -08:00
Tonis Tiigi
25162d4a4e vendor: update buildkit to ff93519ee
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-11-04 16:06:35 -08:00
Andrew Hsu
35913e58c2
Merge pull request #419 from andrewhsu/xit
[19.03] Windows: disable flaky test TestStartReturnCorrectExitCode
2019-11-01 08:41:33 -07:00
Arko Dasgupta
12e7d99439
Add TC to check dyanmic subnet for ingress network
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
(cherry picked from commit e2b5ac75a393f6942c37efdd888fc3bc761de244)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-31 17:46:56 +01:00
Sebastiaan van Stijn
0c38d56a6d
Revert "Revert "[19.03] bump swarmkit to f35d9100f2c6ac810cc8d7de6e8f93dcc7a42d29""
This reverts commit ef4366ee89.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-31 17:46:54 +01:00
Andrew Hsu
031ef2dc8e Windows: disable flaky test TestStartReturnCorrectExitCode
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
(cherry picked from commit 1be272ef76)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-10-31 16:05:07 +00:00
Andrew Hsu
ddb60aa6d1
Merge pull request #418 from kolyshkin/19.03-go1.12.12
[19.03] Bump golang 1.12.12
2019-10-30 09:47:17 -07:00
Kir Kolyshkin
92a8618ddc Bump golang 1.12.12
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-10-28 14:01:45 -07:00
Andrew Hsu
370def6b30
Merge pull request #412 from thaJeztah/19.03_backport_builder_entitilement_confg
[19.03 backport] builder entitlements configuration added.
2019-10-28 10:53:19 -07:00
Andrew Hsu
e2e3abec71
Merge pull request #410 from thaJeztah/19.03_backport_fix_buildkit_prunegc_filter_config
[19.03 backport] daemon/config: fix filter type in BuildKit GC config
2019-10-28 10:52:31 -07:00
Andrew Hsu
0e8949a003
Merge pull request #407 from thaJeztah/19.03_backport_better_container_error
[19.03 backport] Propagate GetContainer error from event processor
2019-10-28 10:50:46 -07:00
Andrew Hsu
967aa3a9ef
Merge pull request #405 from thaJeztah/19.03_backport_oci_regression
[19.03 backport] Use ocischema package instead of custom handler
2019-10-28 10:50:08 -07:00
Andrew Hsu
83bcde8f60
Merge pull request #408 from thaJeztah/19.03_backport_update_rootless_docs
[19.03 backport] docs/rootless.md: update
2019-10-28 10:46:27 -07:00
Andrew Hsu
d91a85a9b5
Merge pull request #397 from thaJeztah/19.03_backport_slirp4netns_sandbox
[19.03 backport] rootless: harden slirp4netns with mount namespace and seccomp
2019-10-28 10:45:18 -07:00
Sebastiaan van Stijn
e5a0bc6a50
Add GoDoc to fix linting validation
The validate step in CI was broken, due to a combination of
086b4541cf, fbdd437d29,
and 85733620eb being merged to master.

```
api/types/filters/parse.go:39:1: exported method `Args.Keys` should have comment or be unexported (golint)
func (args Args) Keys() []string {
^
daemon/config/builder.go:19:6: exported type `BuilderGCFilter` should have comment or be unexported (golint)
type BuilderGCFilter filters.Args
     ^
daemon/config/builder.go:21:1: exported method `BuilderGCFilter.MarshalJSON` should have comment or be unexported (golint)
func (x *BuilderGCFilter) MarshalJSON() ([]byte, error) {
^
daemon/config/builder.go:35:1: exported method `BuilderGCFilter.UnmarshalJSON` should have comment or be unexported (golint)
func (x *BuilderGCFilter) UnmarshalJSON(data []byte) error {
^
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9d726f1c18)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 10:28:25 +02:00
Tibor Vass
dae4436d1c
daemon/config: add MarshalJSON for future proofing
If anything marshals the daemon config now or in the future
this commit ensures the correct canonical form for the builder
GC policies' filters.

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 85733620eb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 10:28:11 +02:00
Tibor Vass
1e26b431c9
daemon/config: fix filter type in BuildKit GC config
For backwards compatibility, the old incorrect object format for
builder.GC.Rule.Filter still works but is deprecated in favor of array of
strings akin to what needs to be passed on the CLI.

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit fbdd437d29)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 10:28:09 +02:00
Kunal Kushwaha
ce74774c09
builder entitlements configutation added.
buildkit supports entitlements like network-host and security-insecure.
this patch aims to make it configurable through daemon.json file.
by default network-host is enabled & secuirty-insecure is disabled.

Signed-off-by: Kunal Kushwaha <kunal.kushwaha@gmail.com>
(cherry picked from commit 8b7bbf180f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-22 10:06:46 +02:00
Tibor Vass
645f559352
Merge pull request #411 from thaJeztah/19.03_backport_fix_dco_branch
[19.03 backport] Jenkinsfile: set repo and branch for DCO check as well
2019-10-21 16:22:02 -07:00
Sebastiaan van Stijn
9c388fb119
Jenkinsfile: set repo and branch for DCO check as well
Commit 7019b60d0d added these
env-vars to other stages, but forgot to update the DCO stage,
which also does a diff to validate commits that are in a PR.

Also adding openssh-client, for situations where the upstream
needs to be accessed through an ssh connection.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7c5fd83c22)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-21 23:32:48 +02:00
Akihiro Suda
a8b454a934
docs/rootless.md: update
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e76dea157e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-20 23:50:07 +02:00
Brian Goff
fd169c00bf
Propagate GetContainer error from event processor
Before this change we just accept that any error is "not found" and it
could be something else, but even if it it is just a "not found" kind of
error this should be dealt with from the container store and not the
event processor.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 54e30a62d3)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-17 02:49:24 +02:00
Brian Goff
e037bade8c
Use ocischema package instead of custom handler
Previously we were re-using schema2.DeserializedManifest to handle oci
manifests. The issue lies in the fact that distribution started
validating the media type string during json deserialization. This
change broke our usage of that type.

Instead distribution now provides direct support for oci schemas, so use
that instead of our custom handlers.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit e443512ce4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-14 23:06:05 +02:00
Andrew Hsu
adfac697dc
Merge pull request #404 from thaJeztah/19.03_revert_iptables_check2
[19.03 backport] revert controller: Check if IPTables is enabled for arrangeUserFilterRule ENGCORE-1114
2019-10-11 14:19:53 -07:00
Sebastiaan van Stijn
54a58760b6
[19.03 backport] revert controller: Check if IPTables is enabled for arrangeUserFilterRule
This change caused a regression, causing the DOCKER-USER chain
to not be created, despite iptables being enabled on the daemon.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-11 21:10:48 +02:00
Andrew Hsu
5787ef7e9c
Merge pull request #396 from thaJeztah/19.03_backport_update_moved_repositories
[19.03 backport] Update links/references to transferred repositories
2019-10-10 10:58:11 -07:00
Andrew Hsu
9a21cf7e55
Merge pull request #399 from thaJeztah/19.03_backport_do_the_right_diff_do_the_right_diff
[19.03 backport] Jenkinsfile: set repo and branch, to assist validate_diff()
2019-10-10 10:56:41 -07:00
Sebastiaan van Stijn
abbc956ac8
Jenkinsfile: set repo and branch, to assist validate_diff()
This is a continuation of 2a08f33166247da9d4c09d4c6c72cbb8119bf8df;

When running CI in other repositories (e.g. Docker's downstream
docker/engine repository), or other branches, the validation
scripts were calculating the list of changes based on the wrong
information.

This lead to weird failures in CI in a branch where these values
were not updated ':-) (CI on a pull request failed because it detected
that new tests were added to the deprecated `integration-cli` test-suite,
but the pull request did not actually make changes in that area).

This patch uses environment variables set by Jenkins to sets the
correct target repository (and branch) to compare to.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7019b60d0d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-07 23:52:55 +02:00
Sebastiaan van Stijn
646e7a5239
Jenkinsfile: remove redundant -f Dockerfile
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 64b3d12686)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-07 23:52:53 +02:00
Sebastiaan van Stijn
3e077fc866
Merge pull request #398 from thaJeztah/19.03_rollback_libnetwork
[19.03] roll-back libnetwork iptables forward policy change [DESKTOP-1934]
2019-10-07 23:12:15 +02:00
Sebastiaan van Stijn
fb0fca8607
[19.03] roll-back libnetwork iptables forward policy change
The patch made in  docker/libnetwork#2450 caused a breaking change in the
networking behaviour, causing Kubernetes installations on Docker Desktop
(and possibly other setups) to fail.

Rolling back this change in the 19.03 branch while we investigate if there
are alternatives.

diff: 45c710223c...96bcc0dae8

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-07 18:11:13 +02:00
Akihiro Suda
5bd4233d7b
rootless: harden slirp4netns with mount namespace and seccomp
When slirp4netns v0.4.0+ is used, now slirp4netns is hardened using
mount namespace ("sandbox") and seccomp to mitigate potential
vulnerabilities.

bump up rootlesskit: 2fcff6ceae...791ac8cb20

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e20b7323fb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-05 10:54:26 +02:00
Andrew Hsu
2ae5cbcf05
Merge pull request #391 from thaJeztah/19.03_backport_session_endpoint_docs_updates
[19.03 backport] API: update docs that /session left experimental in V1.39
2019-10-03 10:49:04 -07:00
Sebastiaan van Stijn
3472e441c5
hack/ci/windows.ps1 update references to repositories that were moved
Also updated the related docs.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5175ed54e5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-03 15:37:56 +02:00
Sebastiaan van Stijn
a2a4576c61
Dockerfile.windows: update references to repositories that were moved
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 83fd212f2c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-03 15:37:54 +02:00
Sebastiaan van Stijn
ac62fa7a61
Jenkinsfile: update references to repositories that were moved
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b323c6e9ae)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-03 15:37:52 +02:00
Andrew Hsu
d9fba87f5a
Merge pull request #392 from andrewhsu/bump_docker_py
[19.03 backport] Temporarily switch docker-py to "master"
2019-10-02 15:56:00 -07:00
Sebastiaan van Stijn
ec0e20a9eb Temporarily switch docker-py to "master"
The docker-py tests were broken, because the version of
py-test that was used, used a dependency that had a new
major release with a breaking change.

Unfortunately, it was not pinned to a specific version,
so when the dependency did the release, py-test broke;

```
22:16:47  Traceback (most recent call last):
22:16:47    File "/usr/local/bin/pytest", line 10, in <module>
22:16:47      sys.exit(main())
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 61, in main
22:16:47      config = _prepareconfig(args, plugins)
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 182, in _prepareconfig
22:16:47      config = get_config()
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 156, in get_config
22:16:47      pluginmanager.import_plugin(spec)
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/config/__init__.py", line 530, in import_plugin
22:16:47      __import__(importspec)
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/tmpdir.py", line 25, in <module>
22:16:47      class TempPathFactory(object):
22:16:47    File "/usr/local/lib/python3.6/site-packages/_pytest/tmpdir.py", line 35, in TempPathFactory
22:16:47      lambda p: Path(os.path.abspath(six.text_type(p)))
22:16:47  TypeError: attrib() got an unexpected keyword argument 'convert'
```

docker-py master has a fix for this (bumping the version of
`py-test`), but it's not in a release yet, and the docker cli that's used
in our CI is pinned to 17.06, which doesn't support building from a remote
git repository from a specific git commit.

To fix the immediate situation, this patch switches the docker-py
tests to run from the master branch.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 48353e16fe)
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2019-10-02 17:42:41 +00:00
Sebastiaan van Stijn
923e849f28
API: update docs that /session left experimental in V1.39
The `/session` endpoint left experimental in API V1.39 through
239047c2d3 and
01c9e7082e, but the API reference
was not updated accordingly.

This updates the API documentation to match the change.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6756f5f378)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-02 12:33:14 +02:00