Commit graph

2594 commits

Author SHA1 Message Date
Akihiro Suda
abf2404642 hack/make.sh: remove extra empty lines
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 12:36:06 +09:00
Akihiro Suda
3cf82748dd run shfmt
git grep --name-only '^#!' | egrep -v '(vendor|\.go|Jenkinsfile)' | xargs shfmt -w -bn -ci -sr

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 12:27:49 +09:00
Akihiro Suda
eb484fcb67 validate: add shfmt
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-03 12:25:51 +09:00
Brian Goff
76e3a49933
Merge pull request #40486 from AkihiroSuda/rootless-cgroup2-systemd
rootless: support `--exec-opt native.cgroupdriver=systemd`
2020-03-02 16:11:21 -08:00
Stefan Scherer
fdf42f7a59
Update Windows ico
Signed-off-by: Stefan Scherer <stefan.scherer@docker.com>
2020-03-02 12:52:10 +01:00
Sam Whited
8c0f6142a8 Update DNS library
This makes sure that we don't become vulnerable to CVE-2018-17419 or
CVE-2019-19794 in the future.

Signed-off-by: Sam Whited <sam@samwhited.com>
2020-02-25 17:05:57 -05:00
Sebastiaan van Stijn
79b130ac31
Revert "Fix containerutility compilation on gcc-mingw-w64 8.3"
This reverts commit 25a1bf53d2.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-14 23:31:42 +01:00
Sebastiaan van Stijn
5125f8b304
bump windows-container-utility aa1ba87e99b68e0113bd27ec26c60b88f9d4ccd9
full diff: e004a1415a...aa1ba87e99

changes:

- Use standard include paths instead of hard-coding

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-14 23:31:14 +01:00
Sebastiaan van Stijn
d8772509d1
Merge pull request #40489 from pricec/fix-license-check
Vendor check to succeed even if copyright information is not found
2020-02-14 23:19:43 +01:00
Akihiro Suda
ca4b51868a rootless: support --exec-opt native.cgroupdriver=systemd
Support cgroup as in Rootless Podman.

Requires cgroup v2 host with crun.
Tested with Ubuntu 19.10 (kernel 5.3, systemd 242), crun v0.12.1.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-14 15:32:31 +09:00
Chris Price
9f94cfdc57 Vendor check to succeed even if copyright information is not found
The documentation for validate_vendor_used in hack/validate/vendor states
that a warning will be emitted if license information cannot be found in
a vendored package. However, because the script is run with pipefail set
(owing to the inclusion of the common validation script .validate) and
`grep -c` is used, the entire script will fail whenever license information
cannot be found in a vendored package.

Signed-off-by: Chris Price <cprice@mirantis.com>
2020-02-13 06:08:44 -08:00
Sebastiaan van Stijn
da6c1429d0
docker-py: skip flaky AttachContainerTest::test_attach_no_stream (again)
This test was disabled in the past, but re-enabled when we upgraded
docker-py to 4.2.0.

The test looks to be still flaky though, so skipping it again:

```
[2020-02-10T23:40:44.429Z] =================================== FAILURES ===================================
[2020-02-10T23:40:44.429Z] __________________ AttachContainerTest.test_attach_no_stream ___________________
[2020-02-10T23:40:44.429Z] tests/integration/api_container_test.py:1250: in test_attach_no_stream
[2020-02-10T23:40:44.429Z]     assert output == 'hello\n'.encode(encoding='ascii')
[2020-02-10T23:40:44.429Z] E   AssertionError: assert b'' == b'hello\n'
[2020-02-10T23:40:44.429Z] E     Right contains more items, first extra item: 104
[2020-02-10T23:40:44.429Z] E     Use -v to get the full diff
[2020-02-10T23:40:44.429Z] ------- generated xml file: /src/bundles/test-docker-py/junit-report.xml -------
````

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-12 18:37:58 +01:00
Brian Goff
3473ff2893
Merge pull request #40469 from thaJeztah/containerd_1.3.3
update containerd binary to v1.3.3
2020-02-07 12:16:40 -08:00
Sebastiaan van Stijn
486161a63a
bump vndr v0.1.1
full diff: https:/github.com/LK4D4/vndr/compare/v0.1.0...v0.1.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-07 13:37:45 +01:00
Sebastiaan van Stijn
27649ee44f
update containerd binary to v1.3.3
full diff: https://github.com/containerd/containerd/compare/v1.3.2...v1.3.3
release notes: https://github.com/containerd/containerd/releases/tag/v1.3.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-07 10:21:01 +01:00
Sebastiaan van Stijn
1cb4a60950
testing: bump docker-py to 4.2.0
full diff: https://github.com/docker/docker-py/compare/4.1.0...4.2.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-06 11:55:18 +01:00
Brian Goff
5dffe74df2 Fix windows cross builds.
Windows still writes to the autogen directory, but the source code is
mounted in as read-only.
In order to do enable this without taking a massive hit in doing an rw
mount (for the source code) we mount a tmpfs into the build at the
autogen dir.
In order for this to work the directory must alreay exist, so we create
it before entering the build.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-01-30 15:43:23 -08:00
Akihiro Suda
cd43c1d1ac update runc binary to v1.0.0-rc10 (CVE-2019-19921)
Notable changes:
* Fix CVE-2019-19921 (Volume mount race condition with shared mounts): https://github.com/opencontainers/runc/pull/2207
* Fix exec FIFO race: https://github.com/opencontainers/runc/pull/2185
* Basic support for cgroup v2.  Almost feature-complete, but still missing support for systemd mode in rootless.
  See also https://github.com/opencontainers/runc/issues/2209 for the known issues.

Full changes: https://github.com/opencontainers/runc/compare/v1.0.0-rc9...v1.0.0-rc10

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-01-25 03:55:39 +09:00
Sebastiaan van Stijn
572cb55c0e
hack/make/.go-autogen: fix LDFLAGS to accept values with spaces
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-18 00:40:06 +01:00
Sebastiaan van Stijn
e8122ed724
hack/make/.go-autogen: fix typo in LDFLAGS variable
Introduced in 675b414f56

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-17 12:39:56 +01:00
Arko Dasgupta
4c407caada Bump libnetwork to latest refpoint
Commits:
feeff4f0 Merge pull request #2380 from liskin/bridge-atomic-hwaddr
fec6476d Merge pull request #2489 from suwang48404/doc
8757597e Added document describing libnetwork traffic flow.
eaea5722 Merge pull request #2445 from kdomanski/ipv6-addr-in-hosts
1680ce71 Merge pull request #2462 from arkodg/fix-key-spi-panic
4420ee92 Fix panic in drivers/overlay/encryption.go
57178323 Merge pull request #2472 from thaJeztah/bump_golang_1.12.12
f741dc9c Update Golang 1.12.12 (CVE-2019-17596)
79c19d09 Merge pull request #2461 from suwang48404/master
94facacc Added API to set ephemeral port allocator range.

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-01-16 16:28:23 -08:00
Brian Goff
1d699d40f4
Merge pull request #40359 from thaJeztah/bump_vndr
bump vndr v0.1.0 to support versioned import paths
2020-01-10 09:21:39 -08:00
Sebastiaan van Stijn
d4f05c168d
bump vndr v0.1.0 to support versioned import paths
With this change, go packages/modules that use versioned
import paths (github.com/foo/bar/v2), but don't use a directory
in the repository, can now be supported.

For example:

```
github.com/coreos/go-systemd/v22 v22.0.0
```

will vendor the github.com/coreos/go-systemd repository
into `vendor/github.com/coreos/go-systemd/v22`.

full diff: f5ab8fc5fb...v0.1.0

- LK4D4/vndr#83 migrate bitbucket to api 2.0
    - fixes LK4D4/vndr#82 https://api.bitbucket.org/1.0/repositories/ww/goautoneg: 410 Gone
- LK4D4/vndr#86 Replace sort.Sort with sort.Strings
- LK4D4/vndr#87 support `github.com/coreos/go-systemd/v22`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-09 21:56:20 +01:00
Sebastiaan van Stijn
25a1bf53d2
Fix containerutility compilation on gcc-mingw-w64 8.3
The makefile for this binary has version 6.3 hardcoded,
which causes compilation on 8.3 to fail:

```
Building: bundles/cross/windows/amd64/containerutility.exe
In file included from /usr/x86_64-w64-mingw32/include/minwindef.h:163,
                 from /usr/x86_64-w64-mingw32/include/windef.h:8,
                 from /usr/x86_64-w64-mingw32/include/windows.h:69,
                 from containerutility.h:3,
                 from argumentstream.cpp:1:
/usr/x86_64-w64-mingw32/include/winnt.h:1554:11: fatal error: x86intrin.h: No such file or directory
 # include <x86intrin.h>
           ^~~~~~~~~~~~~
compilation terminated.
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-09 18:52:09 +01:00
Sebastiaan van Stijn
5dbfae6949
hack/make: ignore failure to stop apparmor
```
 ---> Making bundle: .integration-daemon-stop (in bundles/test-integration)
 ++++ cat bundles/test-integration/docker.pid
 +++ kill 13137
 +++ /etc/init.d/apparmor stop
 Leaving: AppArmorNo profiles have been unloaded.

 Unloading profiles will leave already running processes permanently
 unconfined, which can lead to unexpected situations.

 To set a process to complain mode, use the command line tool
 'aa-complain'. To really tear down all profiles, run 'aa-teardown'."

script returned exit code 255
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-09 18:52:03 +01:00
Sebastiaan van Stijn
f2b25e498f
docker-py: skip broken ImageCollectionTest::test_pull_multiple
The ImageCollectionTest.test_pull_multiple test performs a `docker pull` without
a `:tag` specified) to pull all tags of the given repository (image).

After pulling the image, the image(s) pulled are checked to verify if the list
of images contains the `:latest` tag.

However, the test assumes that all tags of the image are tags for the same
version of the image (same digest), and thus a *single* image is returned, which
is not always the case.

Currently, the `hello-world:latest` and `hello-world:linux` tags point to a
different digest, therefore the `client.images.pull()` returns multiple images:
one image for digest, making the test fail:

    =================================== FAILURES ===================================
    ____________________ ImageCollectionTest.test_pull_multiple ____________________
    tests/integration/models_images_test.py:90: in test_pull_multiple
        assert len(images) == 1
    E   AssertionError: assert 2 == 1
    E    +  where 2 = len([<Image: 'hello-world:linux'>, <Image: 'hello-world:latest'>])

This patch temporarily skips the broken test until it is fixed upstream.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-03 15:47:39 +01:00
Sebastiaan van Stijn
6bc45b09e7
docker-py: re-enable tests that were fixed in v4.1.0
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-01-03 15:46:28 +01:00
Sebastiaan van Stijn
bbcabf69c8
Merge pull request #40050 from thaJeztah/ci_force_validate
Jenkinsfile: force validate on master, and some fixes/improvements
2019-12-23 13:39:43 +01:00
Vikram bir Singh
8d2e1ee546
Build and use gotestsum for running all tests
1. Dockerfile.Windows modified to build gotestsum.exe

2. Use gotestsum.exe in invoking the execution of:

   (a) Unit tests (run in containers),
   (b) Integration tests (run outside containers)
   (c) Integration-cli (run outside containers)

No changes made to other categories of tests (e.g.
LCOW).

3. Copy .xml files produced by gotestsum in
   appropriate paths where Jenkins can ingest them

4. Modify Jenkinsfile to mark results output as
   being jUnit "type" as well as to archive the
   .xml test result files as artifacts.

Signed-off-by: Vikram bir Singh <vikrambir.singh@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 20:31:22 +01:00
Sebastiaan van Stijn
d3c0261876
hack/validate/deprecate-integration-cli: add integration-cli/*_api_*.go
The intent of this check is to not add integration tests that use the
CLI to test API functionality, so check for changes in "API" tests as
well, now that the `integration/` suite has matured enough to develop
new tests, or migrate existing ones.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 18:07:12 +01:00
Sebastiaan van Stijn
1f0fdcc6f5
hack/validate: clean-up output, move golang-ci-lint to end
The golang-ci-lint takes longest to run, so do the other
checks first; this also makes the output a bit more readable.

While at it, fix some consistency issues in the output of the
other checks.

Before this change:

    TEST_FORCE_VALIDATE=1 hack/validate/default
    Congratulations!  All commits are properly signed with the DCO!
    Congratulations! Seccomp profile generation is done correctly.
    INFO [config_reader] Used config file hack/validate/golangci-lint.yml
    INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
    ...
    INFO Execution took 2m12.4287302s
    Congratulations!  "./pkg/..." is safely isolated from internal code.
    The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
    Congratulations! All api changes are done the right way.
    Congratulations!  All toml source files changed here have valid syntax.
    Congratulations!  Changelog CHANGELOG.md is well-formed.
    Congratulations!  Changelog CHANGELOG.md dates are in descending order.

After this change:

    TEST_FORCE_VALIDATE=1 hack/validate/default
    Congratulations!  All commits are properly signed with the DCO!
    Congratulations!  Seccomp profile generation is done correctly.
    Congratulations!  Packages in "./pkg/..." are safely isolated from internal code.
    Congratulations!  The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
    Congratulations!  All API changes are done the right way.
    Congratulations!  All TOML source files changed here have valid syntax.
    Congratulations!  Changelog CHANGELOG.md is well-formed.
    Congratulations!  Changelog CHANGELOG.md dates are in descending order.
    Congratulations!  No new tests were added to integration-cli.

    INFO Start validation with golang-ci-lint
    INFO [config_reader] Used config file hack/validate/golangci-lint.yml
    INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
    ...
    INFO Execution took 2m12.4287302s

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 18:07:07 +01:00
Sebastiaan van Stijn
545e81714b
hack/validate: allow forcing validation
This introduces a FORCE_VALIDATE environment variable, which allows
forcing some validation steps, even if no changes were detected.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 18:03:55 +01:00
Sebastiaan van Stijn
9dde8f2b46
hack/validate/deprecate-integration-cli: don't exit on success
The `hack/validate/deprecate-integration-cli` script exited on
success. As a result, validation steps to run afterwards would
not be executed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 17:59:23 +01:00
Sebastiaan van Stijn
e21019fa51
hack/validate/default: fix minor linting issue
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 17:58:03 +01:00
Sebastiaan van Stijn
c8ecedbde9
Revert "hack: temporarily disable deprecate-integration-cli"
This reverts commit 5f67568bfd.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-19 17:57:56 +01:00
Tianon Gravi
3f2d1eb95d
Merge pull request #40317 from AkihiroSuda/bump-up-rootlesskit
rootless: fix proxying UDP packets
2019-12-18 14:49:49 -08:00
Akihiro Suda
8b8c01dd87
Merge pull request #40074 from yedamao/fix-integration-cli
integration-cli: Fix `SA1019: httputil.ClientConn is deprecated`
2019-12-19 03:30:34 +09:00
Akihiro Suda
658723badd rootless: fix proxying UDP packets
UDP reply packets were not proxied: https://github.com/rootless-containers/rootlesskit/issues/86

The issue was fixed in RootlessKit v0.7.1: https://github.com/rootless-containers/rootlesskit/pull/87

Full changes since v0.7.0: https://github.com/rootless-containers/rootlesskit/compare/v0.7.0...v0.7.1

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2019-12-18 19:33:27 +09:00
Tõnis Tiigi
550d5f9d43
Merge pull request #40180 from cpuguy83/no_more_dev_tools
Build binaries with minimal deps and remove autogen code
2019-12-12 11:29:30 -08:00
Jintao Zhang
7f809e1080 Update containerd to v1.3.2
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-12-09 01:23:40 +08:00
Brian Goff
79a52bb1ba Add support for outputing binaries to custom dir
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-12-02 10:21:46 -08:00
Brian Goff
675b414f56 Use -X ldflags to set dockerversion package vars
This eliminates the need to lay down an auto-generated file.
IIRC this was originally hadded for gccgo which we no longer support.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-12-02 10:21:41 -08:00
Brian Goff
3152f94362
Merge pull request #40048 from tao12345666333/update-to-runc-1.0.0-rc9
Update containerd to v1.3.1, runc v1.0.0-rc9
2019-11-27 14:20:17 -08:00
Jintao Zhang
517946eb47 Update containerd to v1.3.1
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-11-20 23:23:07 +08:00
Sebastiaan van Stijn
9bcbc66032
Merge pull request #40169 from cpuguy83/windows_version_quad_err
Windows: Only set VERSION_QUAD if unset
2019-11-11 17:14:12 -08:00
Sebastiaan van Stijn
9cf349d0f8
bump libnetwork 90afbb01e1d8acacb505a092744ea42b9f167377
full diff: 0025177e3d...90afbb01e1

includes:

- docker/libnetwork#/2459 Fix Error Check in NewNetwork
- docker/libnetwork#/2466 Revert "Merge pull request #2339 from phyber/iptables-check"
    - reverts docker/libnetwork#/2339 controller: Check if IPTables is enabled for arrangeUserFilterRule
    - re-opens docker/libnetwork#2158 dockerd when run with --iptables=false modifies iptables by adding DOCKER-USER
    - re-opens moby/moby#35777 With iptables=false dockerd still creates DOCKER-USER chain and rules
    - re-opens docker/for-linux#136 dockerd --iptables=false adds DOCKER-USER chain and modify FORWARD chain anyway

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-07 13:43:42 -08:00
Sebastiaan van Stijn
e9bd017b68
Merge pull request #40148 from kolyshkin/vendor-diff
hack/validate/vendor: print diff for modified files
2019-11-07 12:42:03 -08:00
Kirill Kolyshkin
c36460c437
Merge pull request #40077 from thaJeztah/fix_autogen_detection
Update "auto-generate" comments to improve detection by linters
2019-11-05 13:04:14 -08:00
Brian Goff
47c5c67ed8
Merge pull request #40032 from jmartin84/fix-grpc-withdialer-deprecation-warning
Fix grpc withdialer deprecation warning
2019-11-05 12:20:33 -08:00
Sebastiaan van Stijn
6186e9fe87
hack/make/.go-autogen: fix "generated code" comment not in correct format
As described in https://golang.org/s/generatedcode, Go has
a formalized format that should be used to indicate that a
file is generated.

Matching that format helps linters to skip generated files;

From https://golang.org/s/generatedcode (https://github.com/golang/go/issues/13560#issuecomment-288457920);

> Generated files are marked by a line of text that matches the regular expression, in Go syntax:
>
> ^// Code generated .* DO NOT EDIT\.$
>
> The `.*` means the tool can put whatever folderol it wants in there, but the comment
> must be a single line and must start with `Code generated` and end with `DO NOT EDIT.`,
> with a period.
>
> The text may appear anywhere in the file.

This patch updates the autogenerated code to match that format.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-11-05 11:32:43 -08:00