Commit graph

688 commits

Author SHA1 Message Date
Sebastiaan van Stijn
88301d8f6c
hack/make: fix some linting issues reported by shellcheck
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 917b0dcd3d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-31 13:27:26 +02:00
Brian Goff
78abff3e39
Add support for setting a test filter
This is basically taking some stuff that make a custom shell function
for.
This takes a test filter, builds the appropriate TESTFLAGS, and sets the
integration API test dirs that match the given filter to avoid building
all test dirs.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 13064b155e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-31 13:27:11 +02:00
Sebastiaan van Stijn
0873c3b57d
hack: fix mixed tabs/spaces for indentation
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2cffe9be3d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 19:53:47 +02:00
Sebastiaan van Stijn
70303ded8e
docker-py: output junit.xml for test-results
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 5969bbee79)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:58:42 +02:00
Sebastiaan van Stijn
16639f549e
docker-py: use --mount for bind-mounting docker.sock
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 535e29da05)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:58:40 +02:00
Sebastiaan van Stijn
34fa29e8d4
docker-py: run without tty to disable color output
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit b04cbf1072)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:58:38 +02:00
Sebastiaan van Stijn
16d0807e7e
docker-py: fix linting issues reported by shellcheck
- SC2006: use $(...) notation instead of legacy backticked `...`
- SC2086: double quote to prevent globbing and word splitting

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0b3d201892)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:58:35 +02:00
Tibor Vass
eee3f67571
Jenkinsfile: reduce time of integration tests by dividing tests into 3 parallel runs
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit e554fb23c8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:58:00 +02:00
Tibor Vass
78207d5380
hack: unmount leftover daemon root folders
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit 13df617d4c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:57:58 +02:00
Sebastiaan van Stijn
3dd11dd0b5
docker-py: skip PullImageTest::test_pull_invalid_platform
and remove `PullImageTest::test_build_invalid_platform` from the list,
which was a copy/paste error in f8cde0b32d

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 6f5c377ddc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:57:35 +02:00
Sebastiaan van Stijn
52ec660936
docker-py: deselect broken experimental tests
These tests are fixed upstream, but those fixes are not yet in a
released version.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit f8cde0b32d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:57:22 +02:00
Brian Goff
da9289fb54
Improvements to the test runners
1. Use `go list` to get list of integration dirs to build. This means we
   do not need to have a valid `.go` in every subdirectory and also
   filters out other dirs like "bundles" which may have been created.
2. Add option to specify custom flags for integration and
   integration-cli. This is needed so both suites can be run AND set
   custom flags... since the cli suite does not support standard go
   flags.
3. Add options to skip an entire integration suite.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit abece9b562)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:55:57 +02:00
Michael Zhao
0b274cf18f
Set TIMEOUT according to os/arch.
Signed-off-by: Michael Zhao <michael.zhao@arm.com>
(cherry picked from commit 790da6c223)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:55:52 +02:00
Andrew Hsu
55fc016efc
allow running of single integration test
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit c222c5ac6f)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:55:45 +02:00
Sebastiaan van Stijn
ab68b5dd9a
docker-py: skip flaky tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 980f2813b4)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:54:53 +02:00
Sebastiaan van Stijn
9ad75d26fc
docker-py: use host-network for nested build of docker-py
When building this image docker-in-docker, the DNS in the environment
may not be usable for the build-container, causing resolution to fail:

```
02:35:31 W: Failed to fetch http://deb.debian.org/debian/dists/jessie/Release.gpg  Temporary failure resolving 'deb.debian.org'
```

This patch detects if we're building from within a container, and if
so, skips creating a networking namespace for the build by using
`--network=host`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 3c15cea650)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:54:51 +02:00
Sebastiaan van Stijn
c3a7556f73
docker-py: don't build --quiet is TESTDEBUG is set
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit ba8f4c7994)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:54:49 +02:00
Sebastiaan van Stijn
e092ff3f74
Bump docker-py to 4.0.2, and run tests from upstream repository
This removes all the installation steps for docker-py from the
Dockerfile, and instead builds the upstream Dockerfile, and runs
docker-py tests in a container.

To test;

```
make test-docker-py

...

Removing bundles/

---> Making bundle: dynbinary (in bundles/dynbinary)
Building: bundles/dynbinary-daemon/dockerd-dev
Created binary: bundles/dynbinary-daemon/dockerd-dev

---> Making bundle: test-docker-py (in bundles/test-docker-py)
---> Making bundle: .integration-daemon-start (in bundles/test-docker-py)
Using test binary docker
Starting dockerd
INFO: Waiting for daemon to start...
.
INFO: Building docker-sdk-python3:3.7.0...
sha256:686428ae28479e9b5c8fdad1cadc9b7a39b462e66bd13a7e35bd79c6a152a402
INFO: Starting docker-py tests...
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.1.0, py-1.8.0, pluggy-0.9.0
rootdir: /src, inifile: pytest.ini
plugins: timeout-1.3.3, cov-2.6.1
collected 359 items

tests/integration/api_build_test.py .......s....
....
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 7bfe48cc00)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:54:45 +02:00
Akihiro Suda
2b33fe3512
hack: remove integration-cli-on-swarm
integration-on-swarm had unnecessary complexity and was too hard to
maintain. Also, it didn't support the new non-CLI integration test suite.

I'm now doing some experiments out of the repo using Kubernetes:
https://github.com/AkihiroSuda/kube-moby-integration

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit e7fbe8e457)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-19 18:54:38 +02:00
Brian Goff
fbb001d1f9 Add support for setting GOARM in cross target.
This adds to the existing format of `<GOOS>/<GOARCH>` to allow for
`<GOOS>/arm/v<GOARM>`

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-04-17 13:19:14 -07:00
Brian Goff
61a3285864
Support cross-compile for arm
Pretty much cross-compile doesn't work because  of this:

> profiles/seccomp/seccomp.go:13:2: build constraints exclude all Go files in /go/src/github.com/docker/docker/vendor/github.com/seccomp/libseccomp-golang

This changes adds a new Dockerfile target for cross compilation with the
neccesary arch specific libseccomp packages and CC toolchains.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-10 14:13:32 +02:00
Akihiro Suda
f0b405fbda rootless: expose ports automatically
Now `docker run -p` ports can be exposed to the host namespace automatically when `dockerd-rootless.sh` is launched with
`--userland-proxy --userland-proxy-path $(which rootlesskit-docker-proxy)`.
This is akin to how Docker for Mac/Win works with `--userland-proxy-path=/path/to/vpnkit-expose-port`.

The port number on the host namespace needs to be set to >= 1024.
SCTP ports are currently unsupported.

RootlessKit changes: 7bbbc48a6f...ed26714429

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-03-21 02:44:08 +09:00
Kir Kolyshkin
228bc35e82 make test-integration: use correct dockerd binary
Here's what happens:
1. One runs `make binary` once
2. Days go by...
3. One makes changes to dockerd sources
4. One runs `make test-integration` to test the changes
5. One spends a long time figuring out why on Earth
   those changes in step 3 are ignored by step 4.
6. One writes this patch
7. ...
8. PROFIT!!

OK, so `make test-integration` builds a dockerd binary
in bundles/dynbinary-daemon/, when starts a daemon instance
for testing. The problem is, the script that starts the
daemon sets PATH to try `bundles/binary-daemon/` first,
and `bundles/dynbinary-daemon/` second.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2019-02-18 22:20:49 -08:00
Tibor Vass
42dcfc894a hack: Have TIMEOUT take -test.count into account when testing for flakiness
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-02-08 18:36:31 +00:00
Tonis Tiigi
20e8572a4c hack: restore bundling vpnkit on amd64
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-02-05 18:21:30 -08:00
Akihiro Suda
ec87479b7e allow running dockerd in an unprivileged user namespace (rootless mode)
Please refer to `docs/rootless.md`.

TLDR:
 * Make sure `/etc/subuid` and `/etc/subgid` contain the entry for you
 * `dockerd-rootless.sh --experimental`
 * `docker -H unix://$XDG_RUNTIME_DIR/docker.sock run ...`

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2019-02-04 00:24:27 +09:00
Sebastiaan van Stijn
ad2765b35e
Merge pull request #38523 from olljanat/flaky-test-finder
CI: Introduce flaky test finder
2019-01-12 11:16:18 +01:00
Olli Janatuinen
8a8fd37f6f CI: Introduce flaky test finder
comparing PR commit(s) to HEAD of moby/moby master branch and if founds
new (or renamed) integration tests will run stress tests for them.

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-01-11 01:11:07 +02:00
Sebastiaan van Stijn
37498f009d
Shell scripts: fix bare variables
This makes my IDE a bit more silent :-)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-10 02:50:47 +01:00
Sebastiaan van Stijn
c3650770cc
Revert "Bash scripts; use double brackets, fix bare variables, add quotes"
This reverts commit 297b30df5f.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-10 02:23:38 +01:00
Sebastiaan van Stijn
f9dbd383bb
Merge pull request #38418 from thaJeztah/mega_power
PowerShell: various cleanups / fixes
2019-01-08 00:03:23 +01:00
Sebastiaan van Stijn
297b30df5f
Bash scripts; use double brackets, fix bare variables, add quotes
These scripts explicitly use Bash, so we should be able to use
`[[` instead of `[` (which seems to be recommended).

Also added curly brackets to some bare variables, and quoted some paths.

This makes my IDE a bit more silent :-)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-24 00:05:14 +01:00
Sebastiaan van Stijn
6130c89cce
PowerShell: remove aliases, use their real commands instead
This patch replaces PowerShell aliases for their real commands, see https://blogs.technet.microsoft.com/heyscriptingguy/2012/04/21/when-you-should-use-powershell-aliases/

For example;

- use `Get-Location` instead of `pwd`
- use `Set-Location` instead of `cd`
- use `ForEach-Object` instead of the `%` shorthand
- use `Write-Output` instead of `echo`

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-22 21:34:39 +01:00
Sebastiaan van Stijn
0f8b616c0c
PowerShell: fix mixed tabs/spaces
Fixed some mixed/tabs spaces for indentation, and used
tabs for auto-generated Go code.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-22 20:53:52 +01:00
Sebastiaan van Stijn
c65f0bd13c
Remove version-checks for containerd and runc
With containerd reaching 1.0, the runtime now
has a stable API, so there's no need to do a check
if the installed version matches the expected version.

Current versions of Docker now also package containerd
and runc separately, and can be _updated_ separately.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-04 23:17:13 +02:00
Tibor Vass
361412c79e hack/make: remove 'docker-' prefix when copying binaries
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-09-25 16:58:28 +00:00
Tibor Vass
34eede0296 Remove 'docker-' prefix for containerd and runc binaries
This allows to run the daemon in environments that have upstream containerd installed.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-09-24 21:49:03 +00:00
jliao
7427fe12d8 fix typo
fix typo

Signed-off-by: jian liao <jliao@alauda.io>
2018-09-08 08:13:30 +08:00
Daniel Hiltgen
896d1b1c61 Expose license status in Info (#37612)
* Expose license status in Info

This wires up a new field in the Info payload that exposes the license.
For moby this is hardcoded to always report a community edition.
Downstream enterprise dockerd will have additional licensing logic wired
into this function to report details about the current license status.

Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>

* Code review comments

Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>

* Add windows autogen support

Signed-off-by: Daniel Hiltgen <daniel.hiltgen@docker.com>
2018-08-17 17:05:21 -07:00
Salahuddin Khan
763d839261 Add ADD/COPY --chown flag support to Windows
This implements chown support on Windows. Built-in accounts as well
as accounts included in the SAM database of the container are supported.

NOTE: IDPair is now named Identity and IDMappings is now named
IdentityMapping.

The following are valid examples:
ADD --chown=Guest . <some directory>
COPY --chown=Administrator . <some directory>
COPY --chown=Guests . <some directory>
COPY --chown=ContainerUser . <some directory>

On Windows an owner is only granted the permission to read the security
descriptor and read/write the discretionary access control list. This
fix also grants read/write and execute permissions to the owner.

Signed-off-by: Salahuddin Khan <salah@docker.com>
2018-08-13 21:59:11 -07:00
Kasper Fabæch Brandt
daba5daf4f -buildmode=pie is not supported on Linux on MIPS either
Signed-off-by: Kasper Fabæch Brandt <poizan@poizan.dk>
2018-07-31 13:02:01 +02:00
Kir Kolyshkin
70cdb1c664 Add osusergo build tar for static binaries
Go 1.11 includes a fix to os/user to be working in a static binary
(fixing https://github.com/golang/go/issues/23265). The fix requires
`osusergo` build tag to be set for static binaries, which is what
this commit adds (also for containerd).

[v2: sort build tags alphabetically]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-07-19 12:25:46 +03:00
Tibor Vass
195919d9d6 builder: set buildkit's exported product variable via PRODUCT
This introduces a PRODUCT environment variable that is used to set a constant
at dockerversion.ProductName.

That is then used to set BuildKit's ExportedProduct variable in order to show
useful error messages to users when a certain version of the product doesn't
support a BuildKit feature.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-07-16 21:41:54 +00:00
Tibor Vass
f6e58ca3c8 builder: pass DOCKER_BUILDKIT to enable buildkit in tests
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-06-10 10:05:28 -07:00
Vincent Demeester
d2af0d96e3
Allow to add any args when doing a make run
`make DOCKERD_ARGS=--init binary run` should start the daemon with
`--init` as flags (with any other "automagically" added ones).

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-01 19:19:20 +02:00
Dennis Chen
8eb7ed673b Remove arch-suffix detection of Dockerfile
Since now we have only one Dockerfile, so the arch-specific suffix
of the Dockerfile is not needed anymore.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-03-12 02:05:37 +00:00
Yong Tang
49ad3d69a7
Merge pull request #36336 from cpuguy83/split_installers
Split binary installers/commit scripts
2018-02-21 19:20:29 -08:00
Brian Goff
b529d1b093 Split binary installers/commit scripts
Originally I worked on this for the multi-stage build Dockerfile
changes. Decided to split this out as we are still waiting for
multi-stage to be available on CI and rebasing these is pretty annoying.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-21 15:39:24 -05:00
Daniel Nephin
e73d742cd7 Remove explicit DOCKER_API_VERSION from integration env setup
Use the default version because it is used by the client package

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-20 17:27:28 -05:00
Vincent Demeester
af306d149e
Rename integration/util to integration/internal
Both names have no real sense, but one allows to make sure these packages
aren't used outside of `integration`.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-10 09:16:32 +01:00