Commit graph

108 commits

Author SHA1 Message Date
Sebastiaan van Stijn
0fc914de5c
Merge pull request #40595 from sfzhu93/GL_2_25
[integration] add buffer to prevent goroutine leak
2020-05-08 16:00:52 +02:00
Brian Goff
d7ba1f85ef Use containerd dist libs for plugin pull/pull
This removes the use of the old distribution code in the plugin packages
and replaces it with containerd libraries for plugin pushes and pulls.

Additionally it uses a content store from containerd which seems like
it's compatible with the old "basicBlobStore" in the plugin package.
This is being used locally isntead of through the containerd client for
now.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-04-02 15:03:06 -07:00
Brian Goff
7a9cb29fb9 Accept platform spec on container create
This enables image lookup when creating a container to fail when the
reference exists but it is for the wrong platform. This prevents trying
to run an image for the wrong platform, as can be the case with, for
example binfmt_misc+qemu.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-03-20 16:10:36 -07:00
Akihiro Suda
5e1b246b9a test-integration: support more rootless tests
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-03-16 17:37:38 +09:00
Ziheng Liu
c322af8019 test: add buffer to prevent goroutine leak
Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>
2020-02-27 21:37:26 -05:00
Sam Whited
a6f99232f5 Always execute tests and skip in tests themselves
See issue #40193

Signed-off-by: Sam Whited <sam@samwhited.com>
2020-02-21 14:31:01 -05:00
Brian Goff
750f0d1648 Support configuration of log cacher.
Configuration over the API per container is intentionally left out for
the time being, but is supported to configure the default from the
daemon config.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit cbecf48bc352e680a5390a7ca9cff53098cd16d7)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2020-02-19 17:02:34 -05:00
Brian Goff
e2ceb83a53 Support reads for all log drivers.
This supplements any log driver which does not support reads with a
custom read implementation that uses a local file cache.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit d675e2bf2b75865915c7a4552e00802feeb0847f)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2020-02-19 17:01:44 -05:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
vikrambirsingh
25c5a55261 Check for OS Type and skip within the test
Premature check for OS type means that the test
will never even get to run on other OS types. This
will cause it to be always flagged as a failure on
such OS types.

Signed-off-by: vikrambirsingh <vikrambir.singh@docker.com>
2019-11-07 22:42:20 +00:00
Sebastiaan van Stijn
9ed58987ce
integration: use constants for http methods
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-13 17:30:13 +02:00
Sebastiaan van Stijn
554d9cec25
testutil: update WithExperimental signature to be a daemon.Option
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-30 17:38:24 +02:00
Sebastiaan van Stijn
5358c95a76
TestGraphdriverPluginV2: SA5001: should check returned error before deferring (staticcheck)
```
integration/plugin/graphdriver/external_test.go:427:2: SA5001: should check returned error before deferring responseReader.Close() (staticcheck)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:55 +02:00
Sebastiaan van Stijn
d948306255
integration: remove unused constants and fields (unused)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:10 +02:00
Sam Whited
b37c214e3c testutil: make testing packages public
This was done with something along the lines of:

```
mv internal/test testutil
pushd testutil/; grep -IRl "package test" | xargs -I '{}' sed -i -e 's|package test|package testutil|g' {}; popd
mv internal/testutil/*.go testutil/ && rm -rf internal/
grep -IRl "github.com\/docker\/docker\/internal\/test" | xargs -I '{}' sed -i -e 's|github.com/docker/docker/internal/test|github.com/docker/docker/test|g' {}
goimports .
```

I also modified the basic plugin path in testutil/fixtures/plugin.

Signed-off-by: Sam Whited <sam@samwhited.com>
2019-09-11 07:47:23 -05:00
Brian Goff
abece9b562 Improvements to the test runners
1. Use `go list` to get list of integration dirs to build. This means we
   do not need to have a valid `.go` in every subdirectory and also
   filters out other dirs like "bundles" which may have been created.
2. Add option to specify custom flags for integration and
   integration-cli. This is needed so both suites can be run AND set
   custom flags... since the cli suite does not support standard go
   flags.
3. Add options to skip an entire integration suite.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-31 16:37:17 -07:00
Sebastiaan van Stijn
dac5710b68
integration: change testGraphDriver signature to fix linting
Line 441: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-07 13:05:51 +02:00
Sebastiaan van Stijn
9f9b4290b9
integration: change container.Run signature to fix linting
Line 59: warning: context.Context should be the first parameter of a function (golint)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-07 13:05:26 +02:00
Sebastiaan van Stijn
3449b12cc7
Use assert.NilError() instead of assert.Assert()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-21 13:16:02 +01:00
Yong Tang
52475f8dd5 Replace t.Fatal(err) with assert.NilError(t, err)
So that they are consistent with integration tests style

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2019-01-15 04:58:30 +00:00
Sebastiaan van Stijn
b84bff7f8a
Fix: plugin-tests discarding current environment
By default, exec uses the environment of the current process, however,
if `exec.Env` is not `nil`, the environment is discarded:

e73f489494/src/os/exec/exec.go (L57-L60)

> If Env is nil, the new process uses the current process's environment.

When adding a new environment variable, prepend the current environment,
to make sure it is not discarded.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-12 13:06:25 +01:00
Sebastiaan van Stijn
3f2ecb5452
Merge pull request #38515 from yongtang/01082019-oll.WaitOn
Use poll.WaitOn in authz_plugin_test.go
2019-01-09 16:10:52 +01:00
Yong Tang
0492b0997b Use poll.WaitOn in authz_plugin_test.go
This fix uses poll.WaitOn to replace customerized
implementation in authz_plugin_test.go

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2019-01-09 03:08:47 +00:00
Sebastiaan van Stijn
a3948d17d3
Improve consistency in "skip"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-07 13:56:46 +01:00
Yong Tang
b958b430aa
Merge pull request #38417 from thaJeztah/replace_newclient
Test: Replace NewClient() with NewClientT()
2019-01-04 23:37:00 -08:00
Sebastiaan van Stijn
8d3feccfa9
Replace deprecated client.WithDialer()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-28 16:10:32 +01:00
Sebastiaan van Stijn
2cb7b73a1b
Test: Replace NewClient() with NewClientT()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-22 15:53:02 +01:00
Sebastiaan van Stijn
c7b488fbc8
API: properly handle invalid JSON to return a 400 status
The API did not treat invalid JSON payloads as a 400 error, as a result
returning a 500 error;

Before this change, an invalid JSON body would return a 500 error;

```bash
curl -v \
  --unix-socket /var/run/docker.sock \
  -X POST \
  "http://localhost/v1.30/networks/create" \
  -H "Content-Type: application/json" \
  -d '{invalid json'
```

```
> POST /v1.30/networks/create HTTP/1.1
> Host: localhost
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 13
>
* upload completely sent off: 13 out of 13 bytes
< HTTP/1.1 500 Internal Server Error
< Api-Version: 1.40
< Content-Type: application/json
< Docker-Experimental: false
< Ostype: linux
< Server: Docker/dev (linux)
< Date: Mon, 05 Nov 2018 11:55:20 GMT
< Content-Length: 79
<
{"message":"invalid character 'i' looking for beginning of object key string"}
```

Empty request:

```bash
curl -v \
  --unix-socket /var/run/docker.sock \
  -X POST \
  "http://localhost/v1.30/networks/create" \
  -H "Content-Type: application/json"
```

```
> POST /v1.30/networks/create HTTP/1.1
> Host: localhost
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type: application/json
>
< HTTP/1.1 500 Internal Server Error
< Api-Version: 1.38
< Content-Length: 18
< Content-Type: application/json
< Date: Mon, 05 Nov 2018 12:00:18 GMT
< Docker-Experimental: true
< Ostype: linux
< Server: Docker/18.06.1-ce (linux)
<
{"message":"EOF"}
```

After this change, a 400 is returned;

```bash
curl -v \
  --unix-socket /var/run/docker.sock \
  -X POST \
  "http://localhost/v1.30/networks/create" \
  -H "Content-Type: application/json" \
  -d '{invalid json'
```

```
> POST /v1.30/networks/create HTTP/1.1
> Host: localhost
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 13
>
* upload completely sent off: 13 out of 13 bytes
< HTTP/1.1 400 Bad Request
< Api-Version: 1.40
< Content-Type: application/json
< Docker-Experimental: false
< Ostype: linux
< Server: Docker/dev (linux)
< Date: Mon, 05 Nov 2018 11:57:15 GMT
< Content-Length: 79
<
{"message":"invalid character 'i' looking for beginning of object key string"}
```

Empty request:

```bash
curl -v \
  --unix-socket /var/run/docker.sock \
  -X POST \
  "http://localhost/v1.30/networks/create" \
  -H "Content-Type: application/json"
```

```
> POST /v1.30/networks/create HTTP/1.1
> Host: localhost
> User-Agent: curl/7.52.1
> Accept: */*
> Content-Type: application/json
>
< HTTP/1.1 400 Bad Request
< Api-Version: 1.40
< Content-Type: application/json
< Docker-Experimental: false
< Ostype: linux
< Server: Docker/dev (linux)
< Date: Mon, 05 Nov 2018 11:59:22 GMT
< Content-Length: 49
<
{"message":"got EOF while reading request body"}
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-11-06 21:30:44 +01:00
Salahuddin Khan
4c8b1fd5a2 Enabling Windows integration tests
Signed-off-by: Salahuddin Khan <salah@docker.com>
2018-09-26 15:57:40 -07:00
Vincent Demeester
d3cc071bb9 Windows: Start of enabling tests under integration/
- Add windows CI entrypoint script.

Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-09-26 12:28:22 -07:00
Sebastiaan van Stijn
352db26d5f
Fix flaky TestExternalGraphDriver/pull test
This test occassionally fails on s390x and Power;

    03:16:04 --- FAIL: TestExternalGraphDriver/pull (1.08s)
    03:16:04 external_test.go:402: assertion failed: error is not nil: Error: No such image: busybox:latest

Most likely these failures are caused due to Docker Hub updating
the busybox:latest image, but not all architectures yet being
available.

Instead of using `:latest`, pull an image by digest, so that
the test doesn't depend on Docker Hub having all architectures
available for `:latest`.

I selected the same digest as is currently used as "frozen image"
in the Dockerfile.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-07-17 10:45:33 +02:00
Sebastiaan van Stijn
35f7e7c50a
Fix golint issues
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-07-11 22:19:03 +02:00
Vincent Demeester
3845728524
Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-13 09:04:30 +02:00
Anda Xu
b9b4f88870 more fixes on integration
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-05-22 11:25:25 -07:00
Vincent Demeester
25494e4c74
Fix logging test type
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-05-15 15:04:46 +02:00
Sebastiaan van Stijn
6821ffd1a1
Merge pull request #37028 from cpuguy83/log_plugin_broken_pipe
Fix logging plugin crash unrecoverable
2018-05-15 13:57:01 +02:00
Brian Goff
e7479e3ab8 Fix logging plugin crash unrecoverable
In cases where a logging plugin has crashed when the daemon tries to
copy the container stdio to the logging plugin it returns a broken pipe
error and any log entries that occurr while the plugin is down are lost.

Fix this by opening read+write in the daemon so logs are not lost while
the plugin is down.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-14 16:51:56 -04:00
Brian Goff
b16b125bb4 Fix swagger volume type generation
This was broken by bf6a790f00

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-14 13:46:20 -04:00
Vincent Demeester
ef2c2040c2 Skip some test on remote daemon for e2e run(s)
We really need to run those on the CI too at some point.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-04-26 16:25:52 +00:00
Vincent Demeester
69bab3832c
Migrate test-integration-cli experimental plugin tests to integration
All `plugins` tests that require an `ExperimentalDaemon` are migrated
to `integration/plugin/*` and start an experimental daemon to test on
it.

The end goal being to remove the `experimental` build.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-04-19 11:57:28 +02:00
Vincent Demeester
5f56503f58
Clean some integration-cli/fixtures package/files
- Move go package used by both `integration-cli` and `integration` to
  `internal/test/fixtures`.
- Remove fixtures that are not used anymore (moved to `docker/cli` a
  while ago) : deploy, notary, secrets.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-04-16 10:48:58 +02:00
Brian Goff
74f8e47352 Limit authz response buffer
When the authz response buffer limit is hit, perform a flush.
This prevents excessive buffer sizes, especially on large responses
(e.g. `/containers/<id>/archive` or `/containers/<id>/export`).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-04-11 15:36:36 -04:00
Vincent Demeester
f0d277fe84
Move integration-cli daemon package to internal/test…
… and do not use the `docker` cli in it. One of the reason of this
move is to not make `integration` package using legacy
`integration-cli` package.

Next move will be to support swarm within this package *and* provide
some helper function using the api (compared to the one using cli in
`integration-cli/daemon` package).

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-04-10 16:29:48 +02:00
Brian Goff
ec90839ca3 Don't sort plugin mounts slice
This was added as part of a53930a04f with
the intent to sort the mounts in the plugin config, but this was sorting
*all* the mounts from the default OCI spec which is problematic.

In reality we don't need to sort this because we are only adding a
self-binded mount to flag it as rshared.

We may want to look at sorting the plugin mounts before they are added
to the OCI spec in the future, but for now I think the existing behavior
is fine since the plugin author has control of the order (except for the
propagated mount).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-03-28 09:10:43 -04:00
Vincent Demeester
6016e79d25
integration/*: make e2e run without failure
… mainly by skipping if daemon is remote.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-03-19 09:47:17 +01:00
Daniel Nephin
6be0f70983 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:43 -04:00
Daniel Nephin
a68ae4a2d9 Improve docstrings and small cleanup in client
Use client instead of helpers for TLS in integration test

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-20 15:15:02 -05:00
Brian Goff
b0b9a25e7e Move log validator logic after plugins are loaded
This ensures that all log plugins are registered when the log validator
is run.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-15 11:53:11 -05:00
Yong Tang
9fcd2a0510 Update api tests to use container.Run/Create in helper package
This fix is a sync up with 36266 so that relevant api tests
use the newly added container.Run/Create in helper package

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-02-13 14:54:31 +00:00
Vincent Demeester
af306d149e
Rename integration/util to integration/internal
Both names have no real sense, but one allows to make sure these packages
aren't used outside of `integration`.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-10 09:16:32 +01:00