Commit graph

228 commits

Author SHA1 Message Date
Tibor Vass
f6e58ca3c8 builder: pass DOCKER_BUILDKIT to enable buildkit in tests
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-06-10 10:05:28 -07:00
Vincent Demeester
d2af0d96e3
Allow to add any args when doing a make run
`make DOCKERD_ARGS=--init binary run` should start the daemon with
`--init` as flags (with any other "automagically" added ones).

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-01 19:19:20 +02:00
Brian Goff
1b7fe816e8 Remove unused/unmaintained package builder stuff
This is left-over stuff from building Docker pacakges. These aren't
really maintained outside of bumping the golang version, and are never
tested.

These builders can be found at
https://github.com/docker/docker-ce-packaging where they are kept up to
date.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-04 16:33:40 -04:00
Fangming.Fang
83062593d7 Add quotation marks for $CURDIR
This change fixes bug when $CURDIR contains special character e.g. '&&'.

Change-Id: I658eb60d97c2229cf8b4e0a61dbbecf3edc9d573
Signed-off-by: Fangming.Fang <fangming.fang@arm.com>
2018-04-02 05:48:35 +00:00
Akihiro Suda
cd3c0057ac
Merge pull request #34369 from cyphar/build-buildmode-pie
*: switch to -buildmode=pie
2018-01-29 23:54:03 +09:00
Tibor Vass
9152e63290 api: generalize version information to any platform component
This change adds a Platform struct with a Name field and a general
Components field to the Version API type. This will allow API
consumers to show version information for the whole platform and
it will allow API providers to set the versions for the various
components of the platform.

All changes here are backwards compatible.

Signed-off-by: Tibor Vass <tibor@docker.com>
2017-12-06 23:23:55 +01:00
Sebastiaan van Stijn
a17071e88f
Fix "make test" failing on missing "test-unit"
Commit dbf580be57 removed
this helper script because it's no longer used in CI.

However, the "make test" target in the Makefile still
called this helper, resulting it to fail.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-20 19:12:25 +01:00
Kir Kolyshkin
17708e72a7 graphdriver: custom build-time priority list
Add a way to specify a custom graphdriver priority list
during build. This can be done with something like

  go build -ldflags "-X github.com/docker/docker/daemon/graphdriver.priority=overlay2,devicemapper"

As ldflags are already used by the engine build process, and it seems
that only one (last) `-ldflags` argument is taken into account by go,
an envoronment variable `DOCKER_LDFLAGS` is introduced in order to
be able to append some text to `-ldflags`. With this in place,
using the feature becomes

  make DOCKER_LDFLAGS="-X github.com/docker/docker/daemon/graphdriver.priority=overlay2,devicemapper" dynbinary

The idea behind this is, the priority list might be different
for different distros, so vendors are now able to change it
without patching the source code.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-11-16 19:43:34 -08:00
Aleksa Sarai
1f4e37cf4b
*: switch to -buildmode=pie
Go has supported PIC builds for a while now, and given the security
benefits of using PIC binaries we should really enable them. There also
appears to be some indication that non-PIC builds have been interacting
oddly on ppc64le (the linker cannot load some shared libraries), and
using PIC builds appears to solve this problem.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2017-11-11 21:59:49 +11:00
Daniel Nephin
dbf580be57 Add a new entrypoint for CI
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-20 17:26:30 -04:00
Yong Tang
975675e924 Merge pull request #34682 from dnephin/fail-build-on-integration-suite
Fix integration suite and propagate failures
2017-09-05 12:40:00 -07:00
Eli Uriegas
7664302579 Allows VERSION to be overwritten by env variable
VERSION was hardcoded to be used as the `VERSION` file from the root
directory, this makes it so that you have the option to overwrite this.

Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2017-09-01 15:47:15 -07:00
Daniel Nephin
96707bc600 Fix integration suite and propagate failures
Failures from the integration suite were not propagating to the outter shell
for some reason. Handle the failure with an if exit 1.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-01 13:07:47 -04:00
Daniel Nephin
cef786f787 Pin docker-cli version to the 17.06-ce release version
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 14:29:16 -04:00
Sebastiaan van Stijn
c500bb562f
Fix make test-unit printing deprecation warning
Commit 1fb615599a moved the unit tests out
of `hack/make.sh`, however the Makefile still used the old path, resulting
in a warning being printed when the unit tests were run:

    ---> Making bundle: test-unit (in bundles/17.06.0-dev/test-unit)
    DEPRECATED: use hack/test/unit instead of hack/make.sh test-unit

This patch updates the Makefile to use the new command.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-08-16 16:02:04 +02:00
Daniel Nephin
bc82b13995 Remove test-integration-cli and references to it.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-09 11:02:57 -04:00
Vincent Demeester
6b025a8b66 Introduce test-integration target (and deprecate/freeze test-integration-cli)
This adds a new package `integration` where `engine` integration tests
should live. Those integration tests should not depends on any `cli`
components (except from the `dockerd` daemon for now — to actually
start a daemon).

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-08-09 11:02:57 -04:00
Aleksa Sarai
c81255c9c0
make: add dynbinary target
Recently, it's become necessary to build dynbinaries on certain
distributions, so make it easier for users to build dynbinaries. It
looks like this was just an oversight when we added dynbinary builds.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2017-07-01 04:54:57 +10:00
Harald Albers
0e2c424a7f Add bash completion to the development container
Adds an environment variable `DOCKER_BASH_COMPLETION_PATH` that allows to
include a bash completion script into the development container.

This is needid for development of bash completion.

Signed-off-by: Harald Albers <github@albersweb.de>
2017-06-23 18:11:55 +02:00
Daniel Nephin
2b45ec7f78 Remove yaml docs generator and man pages
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-21 11:20:05 -04:00
Daniel Nephin
b877fc31c5 Remove pkcs11, libltdl-dev, and clang for osx.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-21 11:18:44 -04:00
Akihiro Suda
63d4000e97 integration-cli-on-swarm/README.md: add a note about Docker->Moby
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-05-15 14:37:05 +00:00
Arnaud Porterie (icecrime)
32915b1d0a Remove cmd/docker and other directories in cli/ in accordance with the new Moby project scope
Starting with this commit, integration tests should no longer rely on
the docker cli, they should be API tests instead. For the existing tests
the scripts will use a frozen version of the docker cli with a
DOCKER_API_VERSION frozen to 1.30, which should ensure that the CI remains
green at all times.

To help contributors develop and test manually with a modified docker
cli, this commit also adds a DOCKER_CLI_PATH environment variable to the
Makefile. This allows to set the path of a custom cli that will be
available inside the development container and used to run the
integration tests.

Signed-off-by: Arnaud Porterie (icecrime) <arnaud.porterie@docker.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
2017-05-05 12:14:29 -07:00
Tianon Gravi
9c9cc5ec43 Only set DOCKER_CROSSPLATFORMS from env if it's set
This allows for the default to come from the image/`Dockerfile` rather
than being unconditionally overwritten by the environment.

Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
2017-05-04 07:13:09 -07:00
Tibor Vass
36bd288162 docs: fix Makefile for yaml docs generation
Signed-off-by: Tibor Vass <tibor@docker.com>
2017-03-29 15:07:16 -07:00
French Ben
5443f0152f docs: added support for CLI yaml file generation
Signed-off-by: French Ben <frenchben@docker.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
2017-03-14 16:04:08 -07:00
Sebastiaan van Stijn
01ed606439 Merge pull request #29775 from AkihiroSuda/integration-cli-on-swarm
[EXPERIMENTAL] Integration Test on Swarm (completes in about 5 minutes, with 10 Swarm nodes)
2017-03-01 21:16:05 +01:00
Victor Vieux
8aa1158164 pass DOCKER_CROSSPLATFORMS to the container in the Makefile
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2017-02-28 10:14:19 -08:00
Akihiro Suda
28797019b3 [EXPERIMENTAL] Integration Test on Swarm
This commit adds contrib/integration-cli-on-swarm/integration-cli-on-swarm.sh,
which enables IT to be running in parallel, using Swarm-mode and Funker.

Please refer to contrib/integration-cli-on-swarm/README.md

The test takes almost 5 to 6 minutes, with 10 n1-standard-4 GCE instances.

  $ ./contrib/integration-cli-on-swarm/integration-cli-on-swarm.sh --push-worker-image example.gcr.io/foo/bar --replicas 30 --shuffle
  2016/12/29 08:32:15 Loaded 1618 tests (30 chunks)
  2016/12/29 08:32:15 Executing 30 chunks in parallel, against "integration-cli-worker"
  2016/12/29 08:32:15 Executing chunk 0 (contains 54 test filters)
  ..
  2016/12/29 08:34:34 Finished chunk 28 [1/30] with 54 test filters in 2m19.098068003s, code=0.
  2016/12/29 08:34:38 Finished chunk 12 [2/30] with 54 test filters in 2m23.088569511s, code=0.
  2016/12/29 08:34:48 Finished chunk 10 [3/30] with 54 test filters in 2m33.880679079s, code=0.
  2016/12/29 08:34:54 Finished chunk 20 [4/30] with 54 test filters in 2m39.973747028s, code=0.
  2016/12/29 08:35:11 Finished chunk 18 [5/30] with 54 test filters in 2m56.28384361s, code=0.
  2016/12/29 08:35:11 Finished chunk 29 [6/30] with 52 test filters in 2m56.54047088s, code=0.
  2016/12/29 08:35:15 Finished chunk 1 [7/30] with 54 test filters in 3m0.285044426s, code=0.
  2016/12/29 08:35:22 Finished chunk 6 [8/30] with 54 test filters in 3m7.211775338s, code=0.
  2016/12/29 08:35:24 Finished chunk 25 [9/30] with 54 test filters in 3m9.938413009s, code=0.
  2016/12/29 08:35:30 Finished chunk 27 [10/30] with 54 test filters in 3m15.219834368s, code=0.
  2016/12/29 08:35:36 Finished chunk 9 [11/30] with 54 test filters in 3m21.615434162s, code=0.
  2016/12/29 08:35:41 Finished chunk 13 [12/30] with 54 test filters in 3m26.576907401s, code=0.
  2016/12/29 08:35:45 Finished chunk 17 [13/30] with 54 test filters in 3m30.290752537s, code=0.
  2016/12/29 08:35:53 Finished chunk 2 [14/30] with 54 test filters in 3m38.148423321s, code=0.
  2016/12/29 08:35:55 Finished chunk 24 [15/30] with 54 test filters in 3m40.09669137s, code=0.
  2016/12/29 08:35:57 Finished chunk 8 [16/30] with 54 test filters in 3m42.299945108s, code=0.
  2016/12/29 08:35:57 Finished chunk 22 [17/30] with 54 test filters in 3m42.946558809s, code=0.
  2016/12/29 08:35:59 Finished chunk 23 [18/30] with 54 test filters in 3m44.232557165s, code=0.
  2016/12/29 08:36:02 Finished chunk 3 [19/30] with 54 test filters in 3m47.112051358s, code=0.
  2016/12/29 08:36:11 Finished chunk 15 [20/30] with 54 test filters in 3m56.340656645s, code=0.
  2016/12/29 08:36:11 Finished chunk 11 [21/30] with 54 test filters in 3m56.882401231s, code=0.
  2016/12/29 08:36:22 Finished chunk 19 [22/30] with 54 test filters in 4m7.551093516s, code=0.
  2016/12/29 08:36:23 Finished chunk 21 [23/30] with 54 test filters in 4m8.221093446s, code=0.
  2016/12/29 08:36:25 Finished chunk 16 [24/30] with 54 test filters in 4m10.450451705s, code=0.
  2016/12/29 08:36:27 Finished chunk 5 [25/30] with 54 test filters in 4m12.162272692s, code=0.
  2016/12/29 08:36:28 Finished chunk 14 [26/30] with 54 test filters in 4m13.977801031s, code=0.
  2016/12/29 08:36:29 Finished chunk 0 [27/30] with 54 test filters in 4m14.34086812s, code=0.
  2016/12/29 08:36:49 Finished chunk 26 [28/30] with 54 test filters in 4m34.437085539s, code=0.
  2016/12/29 08:37:14 Finished chunk 7 [29/30] with 54 test filters in 4m59.22902721s, code=0.
  2016/12/29 08:37:20 Finished chunk 4 [30/30] with 54 test filters in 5m5.103469214s, code=0.
  2016/12/29 08:37:20 Executed 30 chunks in 5m5.104379119s. PASS: 30, FAIL: 0.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-02-28 02:10:09 +00:00
Vincent Demeester
617be0ef2b
Bind mount .git 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-02-03 21:00:32 +01:00
Vincent Demeester
fae6ca2bb4
Do not required .git in the build context
- export DOCKER_GITCOMMIT in the Makefile
- prioritize DOCKER_GITCOMMIT against the `git` command in
  `./hack/make.sh`
- Also add `integration-cli/bundles` to gitignore (it's generated when
  using integration-cli shell)

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-01-24 14:42:57 +01:00
Vincent Demeester
4bda9bd6e9
Add the possibility to set the dev container name
By using `make CONTAINER_NAME=docker-dev shell` you're able to name
the docker-dev container to what you want. The idea behind it is by
knowing the name of your dev container you can automate easily some
thing using `docker exec` later.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-12-16 10:24:16 +01:00
Ben Firshman
27ca33e965
Add make command for previewing API docs
Much easier than the previous method of copying over to the docs
repository and generating the docs.

And, as @cpuguy83 pointed out, that actually didn't work
because the PR that adds Swagger docs isn't merged yet. Oopsy.

Signed-off-by: Ben Firshman <ben@firshman.co.uk>
2016-12-14 12:27:32 +00:00
Brian Goff
c9623d1b76 Use named volumes for docker pkg cache
Changes the default when incremental build is enabled to use named
volumes instead of bindmounts.
The reason for this is, on Mac/Windows the bind mounts will go over the
shared fs, which is incredibly slow and itself uses lots of CPU.

Makes an incremental build on OSX go from ~40s to 10s.

To get the old behavior, can set `PKGCACHE_DIR=.go-pkg-cache`

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-12-05 20:42:08 -05:00
Brian Goff
fddb5a7f2a Merge pull request #29069 from vdemeester/29031-manpages-buildarg
Use DOCKER_BUILD_ARGS on manpages make target
2016-12-02 12:30:36 -05:00
Vincent Demeester
58505ffdee
Use DOCKER_BUILD_ARGS on manpages make target
This make it more consistent with the other image builds and allow to
build manpages behind a proxy for example.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-12-02 14:20:30 +01:00
Sebastiaan van Stijn
ba20c3e65e
pin go-swagger 0.7.4
This pins the version of go-swagger used, because
the results generated by different versions
can differ quite a bit (tested between 0.7.2 - 0.7.4),
and can cause CI / validation to fail.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2016-12-02 01:36:46 +01:00
Vincent Demeester
2c342cff79
Build docker incrementally by default
This speeds up docker build time drastically. It still possible to
disable this by setting `DOCKER_INCREMENTAL_BUILD` to `0` (and this is
what should be done on the CI).

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-11-15 16:23:48 +01:00
Alexander Morozov
f2614f2107 project: use vndr for vendoring
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2016-11-03 15:31:46 -07:00
Daniel Nephin
22033e1003 Move validation out of hack/make
Allow each script to run directly without the hack/make.sh wrapper. These
scripts do not produce artifacts and do not benefit from the "bundles"
framework.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2016-11-03 14:24:22 -06:00
Daniel Nephin
252963bd3f Fix swagger-gen.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2016-11-02 10:48:34 -04:00
Daniel Nephin
02e1ffd76c Add whitespace and create a script to run generation
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2016-10-20 13:24:23 -07:00
Daniel Nephin
b06c352783 Generate api/types:Image from the swagger spec
and rename it to a more appropriate name ImageSummary.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2016-10-20 13:24:23 -07:00
Daniel Nephin
f06d8d6db9 Generate api/types:Port from swagger spec.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2016-10-20 13:24:23 -07:00
Daniel Nephin
0243936d92 Add swagger.yaml
Generate Volume type from the swagger.yaml
Add makefile target for generating the models

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2016-10-20 13:24:23 -07:00
Misty Stanley-Jones
3aeaa9f8fd Add a link to the new build instructions
Signed-off-by: Misty Stanley-Jones <misty@docker.com>
2016-10-14 14:05:48 -07:00
Brian Goff
a95ad53ca4 Fix issue with bundles dir volume
The bundles dir used to be automatically made a volume if BINDDIR was
not set.

The `PKGCACHE_*` stuff was added recently which gets added to
`DOCKER_MOUNT`. This breaks the case where someone wants to use the
package cache (DOCKER_INCREMENTAL_BINARY is enabled) but has not
specified a bind dir but needs the bundles dir made a volume.

Moves the package cache handling to after the bundles dir is made a
volume.

Before this patch, if you have `DOCKER_GRAPHDRIVER` set to anything
other than vfs, the tests in the daemon suite (and swarm suite, and
registry suite) will fail to run since they cannot startup a daemon
(except for speicifc cases where <driver> over <driver> is possible,
like devmapper).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-09-22 16:49:28 -04:00
Justin Cormack
da7a7f9b76 Merge pull request #26375 from cpuguy83/fix_apt_mirror_propagation
Propagate BUILD_APT_MIRROR and consume in buil-deb
2016-09-09 00:02:42 +01:00
Brian Goff
f672a963c9 Propagate BUILD_APT_MIRROR and consume in buil-deb
Passses down BUILD_APT_MIRROR to the docker env.
Ensures BUILD_APT_MIRROR is used when building debs, but only when the
consuming `Dockerfile` actually uses it, otherwise it will cause the
build to fail (e.g. on Ubuntu builds we aren't using APT_MIRROR).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-09-07 10:57:00 -04:00
Brian Goff
4d0275c819 Verify integration tests before building bundles
Adds a new bundle `verify-integration-tests` which pre-compiles a test
binary for the integration tests.

This makes sure that the integration tests will actually compile before
doing other tasks which take much longer, such as building dockerd and
loading test fixtures.
When it comes time to actually run the tests, the pre-compiled binary
will be used so it doesn't have to compile the tests a 2nd time.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-09-06 20:27:28 -04:00