Starting with this commit, integration tests should no longer rely on
the docker cli, they should be API tests instead. For the existing tests
the scripts will use a frozen version of the docker cli with a
DOCKER_API_VERSION frozen to 1.30, which should ensure that the CI remains
green at all times.
To help contributors develop and test manually with a modified docker
cli, this commit also adds a DOCKER_CLI_PATH environment variable to the
Makefile. This allows to set the path of a custom cli that will be
available inside the development container and used to run the
integration tests.
Signed-off-by: Arnaud Porterie (icecrime) <arnaud.porterie@docker.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
This allows for the default to come from the image/`Dockerfile` rather
than being unconditionally overwritten by the environment.
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
This commit adds contrib/integration-cli-on-swarm/integration-cli-on-swarm.sh,
which enables IT to be running in parallel, using Swarm-mode and Funker.
Please refer to contrib/integration-cli-on-swarm/README.md
The test takes almost 5 to 6 minutes, with 10 n1-standard-4 GCE instances.
$ ./contrib/integration-cli-on-swarm/integration-cli-on-swarm.sh --push-worker-image example.gcr.io/foo/bar --replicas 30 --shuffle
2016/12/29 08:32:15 Loaded 1618 tests (30 chunks)
2016/12/29 08:32:15 Executing 30 chunks in parallel, against "integration-cli-worker"
2016/12/29 08:32:15 Executing chunk 0 (contains 54 test filters)
..
2016/12/29 08:34:34 Finished chunk 28 [1/30] with 54 test filters in 2m19.098068003s, code=0.
2016/12/29 08:34:38 Finished chunk 12 [2/30] with 54 test filters in 2m23.088569511s, code=0.
2016/12/29 08:34:48 Finished chunk 10 [3/30] with 54 test filters in 2m33.880679079s, code=0.
2016/12/29 08:34:54 Finished chunk 20 [4/30] with 54 test filters in 2m39.973747028s, code=0.
2016/12/29 08:35:11 Finished chunk 18 [5/30] with 54 test filters in 2m56.28384361s, code=0.
2016/12/29 08:35:11 Finished chunk 29 [6/30] with 52 test filters in 2m56.54047088s, code=0.
2016/12/29 08:35:15 Finished chunk 1 [7/30] with 54 test filters in 3m0.285044426s, code=0.
2016/12/29 08:35:22 Finished chunk 6 [8/30] with 54 test filters in 3m7.211775338s, code=0.
2016/12/29 08:35:24 Finished chunk 25 [9/30] with 54 test filters in 3m9.938413009s, code=0.
2016/12/29 08:35:30 Finished chunk 27 [10/30] with 54 test filters in 3m15.219834368s, code=0.
2016/12/29 08:35:36 Finished chunk 9 [11/30] with 54 test filters in 3m21.615434162s, code=0.
2016/12/29 08:35:41 Finished chunk 13 [12/30] with 54 test filters in 3m26.576907401s, code=0.
2016/12/29 08:35:45 Finished chunk 17 [13/30] with 54 test filters in 3m30.290752537s, code=0.
2016/12/29 08:35:53 Finished chunk 2 [14/30] with 54 test filters in 3m38.148423321s, code=0.
2016/12/29 08:35:55 Finished chunk 24 [15/30] with 54 test filters in 3m40.09669137s, code=0.
2016/12/29 08:35:57 Finished chunk 8 [16/30] with 54 test filters in 3m42.299945108s, code=0.
2016/12/29 08:35:57 Finished chunk 22 [17/30] with 54 test filters in 3m42.946558809s, code=0.
2016/12/29 08:35:59 Finished chunk 23 [18/30] with 54 test filters in 3m44.232557165s, code=0.
2016/12/29 08:36:02 Finished chunk 3 [19/30] with 54 test filters in 3m47.112051358s, code=0.
2016/12/29 08:36:11 Finished chunk 15 [20/30] with 54 test filters in 3m56.340656645s, code=0.
2016/12/29 08:36:11 Finished chunk 11 [21/30] with 54 test filters in 3m56.882401231s, code=0.
2016/12/29 08:36:22 Finished chunk 19 [22/30] with 54 test filters in 4m7.551093516s, code=0.
2016/12/29 08:36:23 Finished chunk 21 [23/30] with 54 test filters in 4m8.221093446s, code=0.
2016/12/29 08:36:25 Finished chunk 16 [24/30] with 54 test filters in 4m10.450451705s, code=0.
2016/12/29 08:36:27 Finished chunk 5 [25/30] with 54 test filters in 4m12.162272692s, code=0.
2016/12/29 08:36:28 Finished chunk 14 [26/30] with 54 test filters in 4m13.977801031s, code=0.
2016/12/29 08:36:29 Finished chunk 0 [27/30] with 54 test filters in 4m14.34086812s, code=0.
2016/12/29 08:36:49 Finished chunk 26 [28/30] with 54 test filters in 4m34.437085539s, code=0.
2016/12/29 08:37:14 Finished chunk 7 [29/30] with 54 test filters in 4m59.22902721s, code=0.
2016/12/29 08:37:20 Finished chunk 4 [30/30] with 54 test filters in 5m5.103469214s, code=0.
2016/12/29 08:37:20 Executed 30 chunks in 5m5.104379119s. PASS: 30, FAIL: 0.
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
- export DOCKER_GITCOMMIT in the Makefile
- prioritize DOCKER_GITCOMMIT against the `git` command in
`./hack/make.sh`
- Also add `integration-cli/bundles` to gitignore (it's generated when
using integration-cli shell)
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
By using `make CONTAINER_NAME=docker-dev shell` you're able to name
the docker-dev container to what you want. The idea behind it is by
knowing the name of your dev container you can automate easily some
thing using `docker exec` later.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Much easier than the previous method of copying over to the docs
repository and generating the docs.
And, as @cpuguy83 pointed out, that actually didn't work
because the PR that adds Swagger docs isn't merged yet. Oopsy.
Signed-off-by: Ben Firshman <ben@firshman.co.uk>
Changes the default when incremental build is enabled to use named
volumes instead of bindmounts.
The reason for this is, on Mac/Windows the bind mounts will go over the
shared fs, which is incredibly slow and itself uses lots of CPU.
Makes an incremental build on OSX go from ~40s to 10s.
To get the old behavior, can set `PKGCACHE_DIR=.go-pkg-cache`
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This make it more consistent with the other image builds and allow to
build manpages behind a proxy for example.
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
This pins the version of go-swagger used, because
the results generated by different versions
can differ quite a bit (tested between 0.7.2 - 0.7.4),
and can cause CI / validation to fail.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This speeds up docker build time drastically. It still possible to
disable this by setting `DOCKER_INCREMENTAL_BUILD` to `0` (and this is
what should be done on the CI).
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
Allow each script to run directly without the hack/make.sh wrapper. These
scripts do not produce artifacts and do not benefit from the "bundles"
framework.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
The bundles dir used to be automatically made a volume if BINDDIR was
not set.
The `PKGCACHE_*` stuff was added recently which gets added to
`DOCKER_MOUNT`. This breaks the case where someone wants to use the
package cache (DOCKER_INCREMENTAL_BINARY is enabled) but has not
specified a bind dir but needs the bundles dir made a volume.
Moves the package cache handling to after the bundles dir is made a
volume.
Before this patch, if you have `DOCKER_GRAPHDRIVER` set to anything
other than vfs, the tests in the daemon suite (and swarm suite, and
registry suite) will fail to run since they cannot startup a daemon
(except for speicifc cases where <driver> over <driver> is possible,
like devmapper).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Passses down BUILD_APT_MIRROR to the docker env.
Ensures BUILD_APT_MIRROR is used when building debs, but only when the
consuming `Dockerfile` actually uses it, otherwise it will cause the
build to fail (e.g. on Ubuntu builds we aren't using APT_MIRROR).
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Adds a new bundle `verify-integration-tests` which pre-compiles a test
binary for the integration tests.
This makes sure that the integration tests will actually compile before
doing other tasks which take much longer, such as building dockerd and
loading test fixtures.
When it comes time to actually run the tests, the pre-compiled binary
will be used so it doesn't have to compile the tests a 2nd time.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
DOCKER_BUILD_APT_MIRROR
This allows passthrough of other options like HTTP_PROXY from
DOCKER_BUILD_ARGS to work on all builds (for example,
ensure-syscall-test) when APT_MIRROR is set.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
When running a `make manpages` the image `docker-manpage-dev` will be build and started to create
the man pages. But the container will not be deleted afterwards. So I propose to start it with
`docker run --rm ...`, otherwise we'll collect some trash.
```
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e3d9f721d862 docker-manpage-dev "man/generate.sh" 19 minutes ago Exited (0) 16 minutes ago berserk_jang
5738a23fb682 docker-manpage-dev "man/generate.sh" 9 hours ago Exited (0) 9 hours ago boring_bartik
15490b5e63f7 docker-manpage-dev "man/generate.sh" 10 hours ago Exited (0) 10 hours ago lonely_joliot
```
Signed-off-by: Dieter Reuter <dieter.reuter@me.com>
`make run` allows you to fire up a daemon (in a container) just using
the existing built binaries. This allows for more rapid iteration
instead of dealing with firing up a shell just to start the daemon.
By default the daemon will listen on port 2375 on the default network
interface.
If a port forward is required to access the daemon, the user can set
`make DOCKER_PORT=2375 run` to get a port forward on a random port with
the daemon listening on port 2375, or `make DOCKER_PORT=2375:2375 run`
to get a daemon running with port 2375 forwarded to the daemon running
on 2375.
Note that the daemon is automatically configured to listen on whatever
port is set for the container side of the `DOCKER_PORT` port spec.
When running on docker4mac, the user must do the following:
```
$ make BINDDIR=. DOCKER_PORT=2375 run
```
This makes sure the binaries are loaded in the container and a port is
forwarded, since it is currently impossible to route traffic from the
mac directly to a container IP.
To get a fresh binary:
```
$ make BINDDIR=. DOCKER_PORT=2375 binary run
```
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This installs docker and dockerd to `$DOCKER_MAKE_INSTALL_PREFIX/bin`, which
defaults to `/usr/local/bin`
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Add a proxy to support 'docker daemon'
Fix configFile option, and remove a test that is no longer relevant.
Remove daemon build tag.
Remove DOCKER_CLIENTONLY from build scripts.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Change docker-daemon to dockerd.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
Instead of being split between three files, let's let `hack/make/.detect-daemon-osarch` be our single source of truth for multiarch detection/vars. Not only does it make it slightly easier to make sure we change everything properly when these bits have to change, but it also makes it so that all bits of `hack/make.sh` (especially `hack/make/.ensure-frozen-images`) work properly outside the context of the `Makefile` on all platforms.
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
This allows the test suite to be able to run without worrying about
the underlying fs used by the container running the daemon (e.g.
aufs-on-aufs), so long as the host running the container is running a
supported fs.
The volume will be cleaned up when the container is removed due to
`--rm`.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
A few libnetwork integration tests require that the kernel be configured
with the "dummy" network interface and has the module loaded. However,
the dummy module is not available by default on arm images. This ensures
that it is built and loaded.
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
We always need proxy for build, so we always use "docker build"
command instead of "make build", but now for some reasons, we need
macro defined in Makefile, so it would be helpful if we can use
"make build" without touching Dockerfile.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
Also, fix up some minor whitespace consistency issues, remove a little cruft, and update GOARM for armhf to 7 so that we're explicit.
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
Since --exec-driver flag has been removed, we don't need environment
DOCKER_EXECDRIVER in integration-cli and Makefile any more.
Signed-off-by: Lei Jitang <leijiang@huawei.com>
In cases where we are running non-interactively (e.g. on Jenkins), we
shouldn't try to allocate a TTY, which would fail. This patch makes the
flag `-t` be dependent on whether the shell session is interactive or
not.
When it is interactive, it's important to have `-t` so that the user can
pass signals through to the process.
We also remove the DOCKER_RUN_DOCS and the GITCOMMIT variables, which
are no longer in use in this Makefile (seems like the related targets
that used it were migrated to docs/Makefile).
Signed-off-by: Jonathan Lebon <jlebon@redhat.com>
Add golint to the Dockerfile, and a `validate-lint` task to the
Makefile. Currently, the linter will process a harcoded list of packages
that will expand as we fix more warnings. Eventually, the linter should
process all subpackages of the repo (excluding vendored code).
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
Just to make life easier on devs so they don't need to 'cd' into
the docs dir just to test their docs edits. This doesn't do anything
more than "cd docs && make docs" so that all of the smarts are still
in the docs's Makefile and not in docker's.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Tweaking for Hugo
Updating the Dockerfile with new sed; fix broken link on Kitematic
Fixing image pull for Dockerfile
Removing docs targets
Signed-off-by: Mary Anthony <mary@docker.com>
This helps ensure that `github.com/docker/docker/pkg/...` is actually safe to use in isolation (ie, doesn't import anything from `github.com/docker/docker` except other things from `pkg` or vendored dependencies).
Adding `github.com/docker/docker/utils` to the imports of `pkg/version/version.go`:
```
---> Making bundle: validate-pkg (in bundles/1.7.0-dev/validate-pkg)
These files import internal code: (either directly or indirectly)
- pkg/version/version.go imports github.com/docker/docker/autogen/dockerversion
- pkg/version/version.go imports github.com/docker/docker/utils
```
And then removing it again:
```
---> Making bundle: validate-pkg (in bundles/1.7.0-dev/validate-pkg)
Congratulations! "./pkg/..." is safely isolated from internal code.
```
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
The DOCKER_EXPERIMENTAL environment variable drives the activation of
the 'experimental' build tag.
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
- Updated Dockerfile to satisfy libnetwork GOPATH requirements.
- Reworked daemon to allocate network resources using libnetwork.
- Reworked remove link code to also update network resources in libnetwork.
- Adjusted the exec driver command population to reflect libnetwork design.
- Adjusted the exec driver create command steps.
- Updated a few test cases to reflect the change in design.
- Removed the dns setup code from docker as resolv.conf is entirely managed
in libnetwork.
- Integrated with lxc exec driver.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
See https://github.com/docker-library/docker/issues/2 for some of the context around this; essentially, we're looking to create an official `docker` image that includes the Docker CLI (and the dependencies necessary for easy Docker-in-Docker), but it makes sense to use the `docker` namespace for that image. The `docker-dev` official image is already builds of Docker's own `Dockerfile` (but specifically of releases), so this seemed like a good fit. 👍
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
The validation script from #10681 is too pedantic, and does not handle
well situations like:
```
cat <<EOF # or <<-EOF
Whether the leading whitespace is stripped out or not by bash
it should still be considered as valid.
EOF
```
This reverts commit 4e65c1c319.
Signed-off-by: Tibor Vass <tibor@docker.com>
Created a validation that detects all trailing whitespaces from every
text file that isn't *.go, *.md, vendor/*,
docs/theme/mkdocs/tipuesearch*
Removed trailing whitespaces from every text file except from vendor/*
builder/parser/testfiles*, docs/theme/mkdocs/tipuesearch* and *.md
Signed-off-by: André Martins <martins@noironetworks.com>
Clear the cache for all files. Don't worry about what changed.
Move the docs release script back to the docker/docker repo, so we all can
easily watch the changes.
Docker-DCO-1.1-Signed-off-by: Jessie Frazelle <hugs@docker.com> (github: jfrazelle)
Docker-DCO-1.1-Signed-off-by: Jessie Frazelle <princess@docker.com> (github: jfrazelle)
It turns out the back-to-back `D`s here were tripping people up because visually, it looks like a typo. This renames the variable to `BIND_DIR`, but allows `BINDDIR` to continue working for backwards-compatibility.
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
to docs.master.dockerproject.com.
- Make the invaidation profile the bucket variable, not hard coded.
- Add no cache variable for settings cache to "no-cache"
Docker-DCO-1.1-Signed-off-by: Jessica Frazelle <jess@docker.com> (github: jfrazelle)
Signed-off-by: Sven Dowideit <SvenDowideit@home.org.au>
Docker-DCO-1.1-Signed-off-by: Sven Dowideit <SvenDowideit@home.org.au> (github: SvenDowideit)
Signed-off-by: Sven Dowideit <SvenDowideit@docker.com>
- move docker/docker-py clone to the Dockerfile
- put "integration test daemon startup" code in a separate file for both scripts to source
- add new test-docker-py Makefile target
- include "python-websocket" package in Dockerfile for running the tests
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
This clones and run the integration tests for docker-py master as part
of the integration tests created on master. docker-py hits the api
directly and should be a good way to identify regressions in the api.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
The gist here is a reemphasizing of the explicitly "user mutable" bits by putting them first (and hopefully improving readability a little bit in the process).
Signed-off-by: Andrew Page <admwiggin@gmail.com>
I was trying to just build the Docker client but DOCKER_CLIENTONLY wasn't
getting passed thru from the shell to the container building docker.
So, this PR passes this var (via the -e option) on the docker run command
so we pick it up from the devs shell when running "make ...".
While in there I pulled all of the "-e" options into a new Makefile variable
so its easy to see just the list of env vars we pass along.
Signed-off-by: Doug Davis <dug@us.ibm.com>
If "DOCKER_HOST" is set, we can usually assume the user is connecting to a remote Docker and thus not bind mount anything by default (meaning the Makefile will more often DWIM for our users).
Docker-DCO-1.1-Signed-off-by: Andrew Page <admwiggin@gmail.com> (github: tianon)
use the beta-warning area to tell the user what VERSION of docker, git
branch, and links to the official release version docs are.
requires / extends PR #5272
Docker-DCO-1.1-Signed-off-by: Sven Dowideit <SvenDowideit@fosiki.com> (github: SvenDowideit)
This was supposed to be part of my previous PR, but somehow got missed.
Docker-DCO-1.1-Signed-off-by: Andrew Page <admwiggin@gmail.com> (github: tianon)
This makes it possible to choose the graphdriver and the execdriver
which is going to be used for the cli integration tests.
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
Especially but not limited to:
- make BINDDIR= ... - for when you don't want a bind mount at all
- make DOCSPORT=9000 docs - for when you want a not-8000 docs port
- when we can't determine a branch name, we don't try to "docker build -t docker: ." anymore - we just "docker build -t docker ." (thus allowing Docker to assume ":latest")
Docker-DCO-1.1-Signed-off-by: Andrew Page <admwiggin@gmail.com> (github: tianon)
