Commit graph

112 commits

Author SHA1 Message Date
Paweł Gronowski
73814c48df
internal: Add compatcontext.WithoutCancel
Copy the implementation of `context.WithoutCancel` introduced in Go 1.21
to be able to use it when building with older versions.
This will use the stdlib directly when building with Go 1.21+.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit f6e44bc0e8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-10-20 16:43:03 +02:00
Cory Snider
60ee6f739f Add reusable chroot and unshare utilities
Refactor pkg/chrootarchive in terms of those utilities.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-10-26 12:06:31 -04:00
Tibor Vass
f1c1cd436a integration-cli: move each test suite to its own TestX testing function
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-18 18:26:36 +00:00
Sam Whited
b37c214e3c testutil: make testing packages public
This was done with something along the lines of:

```
mv internal/test testutil
pushd testutil/; grep -IRl "package test" | xargs -I '{}' sed -i -e 's|package test|package testutil|g' {}; popd
mv internal/testutil/*.go testutil/ && rm -rf internal/
grep -IRl "github.com\/docker\/docker\/internal\/test" | xargs -I '{}' sed -i -e 's|github.com/docker/docker/internal/test|github.com/docker/docker/test|g' {}
goimports .
```

I also modified the basic plugin path in testutil/fixtures/plugin.

Signed-off-by: Sam Whited <sam@samwhited.com>
2019-09-11 07:47:23 -05:00
Sebastiaan van Stijn
ead3f4e7c8
integration-cli: add daemon.StartNodeWithBusybox function
Starting the daemon should not load the busybox image again
in most cases, so add a new `StartNodeWithBusybox` function
to be clear that this one loads the busybox image, and use
`StartNode()` for cases where loading the busybox image is
not needed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 00:07:49 +02:00
Sebastiaan van Stijn
8fc23588f1
integration-cli: swarm.RestartNode(); don't load busybox again
The daemon was already created and started with the busybox
image loaded, so there's no need to load the image again.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-11 00:06:57 +02:00
Tibor Vass
d32e6bbde8 suite: put suite setup inside test run
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-10 00:25:01 +00:00
Tibor Vass
8bffe9524d remove per-test -timeout logic because it does not work
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-09 21:09:57 +00:00
Tibor Vass
925e407c7b cleanup
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-09 21:09:57 +00:00
Tibor Vass
fd0ed80ff2 internal/test/suite
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-09 21:09:57 +00:00
Tibor Vass
931edfe5e9 prepare for rm-gocheck script
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-09-09 20:49:53 +00:00
Tibor Vass
e554fb23c8 Jenkinsfile: reduce time of integration tests by dividing tests into 3 parallel runs
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-08-12 20:41:03 +00:00
Brian Goff
a1885bd655
Merge pull request #39603 from cpuguy83/test_daemon_logging
Better logging for swarm tests
2019-07-25 12:58:53 -07:00
Brian Goff
b0fe0dff7a Better logging for swarm tests
Call helper for starting swarm agents and add some logging with daemon
id's when joining the swarm.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-24 16:38:11 -07:00
Brian Goff
1381956499 Add FromClient to test env execution
While working on other tests I noticed that environment.Execution cannot
be used for anything but the pre-configured daemon, however this can
come in handy for being able share daemons across multiple tests that
currently spin up a new daemon.
The execution env also seems to be misused in some of these cases.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-24 12:27:41 -07:00
Tonis Tiigi
3df1095bbd integration-cli: allow temporary errors on leader switch
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-07-17 11:52:58 -07:00
Akihiro Suda
84edfb216a
Merge pull request #39507 from thaJeztah/cleanup_socket_addr
Cleanup "address" when connecting over a UNIX socket
2019-07-16 02:57:30 +09:00
Sebastiaan van Stijn
6a64a4deec
integration: fix cleanup of raft data
The directory used for storage was either changed or new directories
were added.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-13 13:23:57 +02:00
Brian Goff
5d818213ff Fix Microsecond -> Milisecond.
A bit too quick on the trigger on some text completion I think...

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-12 18:43:08 -07:00
Sebastiaan van Stijn
0e041d68d8
Merge pull request #39506 from thaJeztah/group_test_subdirs
integration: organize bundle directory per test
2019-07-12 21:50:15 +02:00
Kirill Kolyshkin
5cd635b80d
Merge pull request #39510 from cpuguy83/test_daemon_ping_logs
Don't log test initial test daemon ping failures
2019-07-12 10:42:40 -07:00
Brian Goff
15675e28f1 Don't log test initial test daemon ping failures
This is just noise due to timing. I picked `> 2` just based on
logs from tests I've seen there's always 1 or 2.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-12 09:51:32 -07:00
Sebastiaan van Stijn
92e6e7dd5f
Cleanup "address" when connecting over a UNIX socket
When connecting with the daemon using a UNIX socket, the HTTP hostname was set, based
on the socket location, which was generating some noise in the test-logs.

Given that the actual hostname is not important (the URL just has to be well-formed),
the hostname/address can be cleaned up to reduce the noise.

This patch strips the path from the `addr`, and keeps `<random-id>.sock` as address.

Before:

    daemon.go:329: [d15d31ba75501] error pinging daemon on start: Get http://%2Ftmp%2Fdocker-integration%2Fd15d31ba75501.sock/_ping: dial unix /tmp/docker-integration/d15d31ba75501.sock: connect: no such file or directory

After:

    daemon.go:329: [d15d31ba75501] error pinging daemon on start: Get http://d15d31ba75501.sock/_ping: dial unix /tmp/docker-integration/d15d31ba75501.sock: connect: no such file or directory

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-12 14:25:28 +02:00
Sebastiaan van Stijn
9b5e78888d
integration: organize bundle directory per test
The test-integration/test=integration-cli directory contains
a directory for each daemon that was created during the integration
tests, which makes it a long list to browse through. In addition,
some tests spin up multiple daemons, and when debugging test-failures,
the daemon-logs often have to be looked at together.

This patch organizes the bundl directory to group daemon storage
locationos per test, making it easier to find information about
all the daemons that were used in a specific test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-12 12:13:02 +02:00
Brian Goff
24ad2f486d Add (hidden) flags to set containerd namespaces
This allows our tests, which all share a containerd instance, to be a
bit more isolated by setting the containerd namespaces to the generated
daemon ID's rather than the default namespaces.

This came about because I found in some cases we had test daemons
failing to start (really very slow to start) because it was (seemingly)
processing events from other tests.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-07-11 17:27:48 -07:00
Michael Crosby
fb459f6671
Merge pull request #38441 from sirlatrom/swarm_plugin_env
Allow specifying environment variables when installing an engine plugin as a Swarm service
2019-07-08 15:26:55 -04:00
Michael Crosby
402433a5e4 Improve select for daemon restart tests
This improves the select logic for the restart tests or starting the
daemon in general.  With the way the ticker and select was setup, it was
possible for only the timeout to be displayed and not the wait errors.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-07-02 16:16:33 -04:00
Olli Janatuinen
213681b66a First step to implement full garbage collector for image layers
Refactored exiting logic on way that layers are first marked to be under
removal so if actual removal fails they can be found from disk and
cleaned up.

Full garbage collector will be implemented as part of containerd
migration.

Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2019-06-05 22:02:42 +03:00
Tibor Vass
8ff4ec98cf build: buildkit now also uses systemd's resolv.conf
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-04 16:04:10 +00:00
Kirill Kolyshkin
f0fe0d6c4e
Merge pull request #39105 from cpuguy83/optimize_test_daemon_startup
Optimize test daemon startup
2019-05-23 11:58:47 -07:00
Deep Debroy
15419d7ba0 Consider WINDOWS_BASE_IMAGE_TAG override when setting Windows base image for tests
Signed-off-by: Deep Debroy <ddebroy@docker.com>
2019-05-14 13:27:18 -07:00
Yong Tang
3042254a87
Merge pull request #38377 from rgulewich/38332-cgroup-ns
Start containers in their own cgroup namespaces
2019-05-11 20:18:31 -07:00
Daniel Sweet
7546322e99 Ensure all integration daemon logging happens before test exit
As of Go 1.12, the `testing` package panics if a goroutine logs to a
`testing.T` after the relevant test has completed. This was not
documented as a change at all; see the commit
95d06ab6c982f58b127b14a52c3325acf0bd3926 in the Go repository for the
relevant change.

At any point in the integration tests, tests could panic with the
message "Log in goroutine after TEST_FUNCTION has completed". This was
exacerbated by less direct logging I/O, e.g. running `make test` with
its output piped instead of attached to a TTY.

The most common cause of panics was that there was a race condition
between an exit logging goroutine and the `StopWithError` method:
`StopWithError` could return, causing the calling test method to return,
causing the `testing.T` to be marked as finished, before the goroutine
could log that the test daemon had exited. The fix is simple: capture
the result of `cmd.Wait()`, _then_ log, _then_ send the captured
result over the `Wait` channel. This ensures that the message is
logged before `StopWithError` can return, blocking the test method
so that the target `testing.T` is not marked as finished.

Signed-off-by: Daniel Sweet <danieljsweet@icloud.com>
2019-05-09 13:23:43 -04:00
Rob Gulewich
072400fc4b Make cgroup namespaces configurable
This adds both a daemon-wide flag and a container creation property:
- Set the `CgroupnsMode: "host|private"` HostConfig property at
  container creation time to control what cgroup namespace the container
  is created in
- Set the `--default-cgroupns-mode=host|private` daemon flag to control
  what cgroup namespace containers are created in by default
- Set the default if the daemon flag is unset to "host", for backward
  compatibility
- Default to CgroupnsMode: "host" for client versions < 1.40

Signed-off-by: Rob Gulewich <rgulewich@netflix.com>
2019-05-07 10:22:16 -07:00
Brian Goff
595987fd08 Add log entries for daemon startup/shutdown
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-05-06 10:36:05 -07:00
Brian Goff
20ea8942b8 Optimize test daemon startup
This adds some logs, handles timers better, and sets a request timeout
for the ping request.

I'm not sure the ticker in that loop is what we really want since the
ticker keeps ticking while we are (attempting) to make a request... but
I opted to not change that for now.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-05-06 10:36:05 -07:00
Sebastiaan van Stijn
e6c0d19c3a
client: define "Opt" type
Minor improvement, but makes defining a list of options
a bit cleaner, and more descriptive;

Before:

    opts := make([]func(*client.Client) error, 0)

After:

    opts := make([]client.Opt, 0)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-10 01:23:45 +02:00
Sune Keller
fca5ee3bd5 Support environment vars in Swarm plugins services
Allow specifying environment variables when installing an engine plugin
as a Swarm service. Invalid environment variable entries (without an
equals (`=`) char) will be ignored.

Signed-off-by: Sune Keller <absukl@almbrand.dk>
2019-04-07 09:48:19 +02:00
Aleksa Sarai
ba0afa6ba8
internal: test/env: switch to assert.TestingT
Signed-off-by: Aleksa Sarai <asarai@suse.de>
2019-03-13 11:48:40 +11:00
Aleksa Sarai
175b1d7830
integration-cli: don't build -test images if they already exist
There's no need to try to re-build the test images if they already
exist. This change makes basically no difference to the upstream
integration test-suite running, but for users who want to run the
integration-cli suite on a host machine (such as distributions doing
tests) this change allows images to be pre-loaded such that compilers
aren't needed on the test machine.

However, this does remove the accidental re-compilation of nnp-test, as
well as handling errors far more cleanly (previously if an error
occurred during a test build, further tests won't attempt to rebuild
it).

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2019-03-13 11:48:40 +11:00
Justin Cormack
98fc09128b Remove the rest of v1 manifest support
As people are using the UUID in `docker info` that was based on the v1 manifest signing key, replace
with a UUID instead.

Remove deprecated `--disable-legacy-registry` option that was scheduled to be removed in 18.03.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2019-03-02 10:46:37 -08:00
Sebastiaan van Stijn
b3407d2029
Testing: create new daemon (only) if needed
Some tests were skipped if the local daemon did not have
experimental features enabled; at the same time, some tests
unconditionally created a new (experimental) daemon, even if
the local daemon already had experimental enabled.

This patch;

- Checks if the "testEnv" is an experimental Linux daemon
- If not, and the daemon is running locally; spin up a new
  experimental daemon to be used during the test.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-02-23 13:32:59 +01:00
Brian Goff
e063099f91 Completely remove d.NewClient from testing tools
Favor `d.NewClientT` instead.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-02-07 16:07:02 -08:00
Sebastiaan van Stijn
7e7e100be0
Add HEAD support for /_ping endpoint
Monitoring systems and load balancers are usually configured to use HEAD
requests for health monitoring. The /_ping endpoint currently does not
support this type of request, which means that those systems have fallback
to GET requests.

This patch adds support for HEAD requests on the /_ping endpoint.

Although optional, this patch also returns `Content-Type` and `Content-Length`
headers in case of a HEAD request; Refering to RFC 7231, section 4.3.2:

    The HEAD method is identical to GET except that the server MUST NOT
    send a message body in the response (i.e., the response terminates at
    the end of the header section).  The server SHOULD send the same
    header fields in response to a HEAD request as it would have sent if
    the request had been a GET, except that the payload header fields
    (Section 3.3) MAY be omitted.  This method can be used for obtaining
    metadata about the selected representation without transferring the
    representation data and is often used for testing hypertext links for
    validity, accessibility, and recent modification.

    A payload within a HEAD request message has no defined semantics;
    sending a payload body on a HEAD request might cause some existing
    implementations to reject the request.

    The response to a HEAD request is cacheable; a cache MAY use it to
    satisfy subsequent HEAD requests unless otherwise indicated by the
    Cache-Control header field (Section 5.2 of [RFC7234]).  A HEAD
    response might also have an effect on previously cached responses to
    GET; see Section 4.3.5 of [RFC7234].

With this patch applied, either `GET` or `HEAD` requests work; the only
difference is that the body is empty in case of a `HEAD` request;

    curl -i --unix-socket /var/run/docker.sock http://localhost/_ping
    HTTP/1.1 200 OK
    Api-Version: 1.40
    Cache-Control: no-cache, no-store, must-revalidate
    Docker-Experimental: false
    Ostype: linux
    Pragma: no-cache
    Server: Docker/dev (linux)
    Date: Mon, 14 Jan 2019 12:35:16 GMT
    Content-Length: 2
    Content-Type: text/plain; charset=utf-8

    OK

    curl --head -i --unix-socket /var/run/docker.sock http://localhost/_ping
    HTTP/1.1 200 OK
    Api-Version: 1.40
    Cache-Control: no-cache, no-store, must-revalidate
    Content-Length: 0
    Content-Type: text/plain; charset=utf-8
    Docker-Experimental: false
    Ostype: linux
    Pragma: no-cache
    Server: Docker/dev (linux)
    Date: Mon, 14 Jan 2019 12:34:15 GMT

The client is also updated to use `HEAD` by default, but fallback to `GET`
if the daemon does not support this method.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-31 18:18:24 +01:00
Sebastiaan van Stijn
b84bff7f8a
Fix: plugin-tests discarding current environment
By default, exec uses the environment of the current process, however,
if `exec.Env` is not `nil`, the environment is discarded:

e73f489494/src/os/exec/exec.go (L57-L60)

> If Env is nil, the new process uses the current process's environment.

When adding a new environment variable, prepend the current environment,
to make sure it is not discarded.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-12 13:06:25 +01:00
Sebastiaan van Stijn
263e28a830
Fix some minor wording / issues
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-07 13:57:01 +01:00
Sebastiaan van Stijn
55b5b8de79
Merge pull request #38486 from thaJeztah/dont_use_deprecated_envclient
Remove use of deprecated client.NewEnvClient()
2019-01-05 11:23:19 +01:00
Sebastiaan van Stijn
c8ff5ecc09
Remove use of deprecated client.NewEnvClient()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-03 22:49:00 +01:00
Sebastiaan van Stijn
2cb7b73a1b
Test: Replace NewClient() with NewClientT()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-22 15:53:02 +01:00
selansen
32180ac0c7 VXLAN UDP Port configuration support
This commit contains changes to configure DataPathPort
option. By default we use 4789 port number. But this commit
will allow user to configure port number during swarm init.
DataPathPort can't be modified after swarm init.
Signed-off-by: selansen <elango.siva@docker.com>
2018-11-22 17:35:02 -05:00