Commit graph

375 commits

Author SHA1 Message Date
Brett Randall
ba49e8c498 Updated docker-on-docker build-notes.
These are now more in-line with wiki instructions. Also removes
broken/deprecated make target test-unit.

Signed-off-by: Brett Randall <javabrett@gmail.com>
2018-02-14 22:08:37 +11:00
Vincent Demeester
0f6dc962a5
Merge pull request #35749 from thaJeztah/change-swagger-install
Update go-swagger installation steps in Dockerfile
2018-02-14 08:20:06 +01:00
Sebastiaan van Stijn
caeab26843
Bump Golang to 1.9.4
This fixes a vulnerability in `go get` (CVE-2018-6574, http://golang.org/issue/23672),
but shouldn't really affect our code, but it's good to keep in sync.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-02-07 14:49:51 -08:00
Akihiro Suda
cd3c0057ac
Merge pull request #34369 from cyphar/build-buildmode-pie
*: switch to -buildmode=pie
2018-01-29 23:54:03 +09:00
Sebastiaan van Stijn
3cc13511f0
Bump Go to 1.9.3
release notes: https://golang.org/doc/devel/release.html#go1.9.minor

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-22 13:40:19 -08:00
unclejack
871afbb304
Merge pull request #35697 from sargun/use-pgzip
Make image (layer) downloads faster by using pigz
2018-01-17 11:18:20 -08:00
Kir Kolyshkin
c11508a5f4 gometalinter: fix --deadline option
1. Add = between the option and the argument, otherwise the argument
   appears to be passed on to the linters directly, as in:

> DEBUG: [golint.8]: executing /home/kir/go/bin/golint
> -min_confidence 0.800000 ./10m ./api ./api/errdefs <...>

2. Fix setting the default for GOMETALINTER_OPTS -- the default
   was -deadline (rather than --deadline).

Fixes: b96093fa56 ("gometalinter: add per-platform configurable options")

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-01-16 14:37:11 -08:00
Sargun Dhillon
fd35494a25 Make image (layer) downloads faster by using pigz
The Golang built-in gzip library is serialized, and fairly slow
at decompressing. It also only decompresses on demand, versus
pipelining decompression.

This change switches to using the pigz external command
for gzip decompression, as opposed to using the built-in
golang one. This code is not vendored, but will be used
if it autodetected as part of the OS.

This also switches to using context, versus a manually
managed channel to manage cancellations, and synchronization.
There is a little bit of weirdness around manually having
to cancel in the error cases.

Signed-off-by: Sargun Dhillon <sargun@sargun.me>
2018-01-16 10:49:18 -08:00
Dennis Chen
eaae7750ef Upgrade the frozen images to multi-arch
Upgrade the frozen images to the multi-arch ones.

Since issue #35963 is not fixed yet on linux/amd64, so we keep the busybox
image on amd64 untouched.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-01-11 05:14:09 +00:00
Sebastiaan van Stijn
29d77acaf8
Update go-swagger installation steps in Dockerfile
The installation steps for go-swagger was a bit noisy, and not consistent with
other installation steps.

This patch makes it similar to other steps, which makes it less noisy, and
makes the image slightly smaller.

Before:

    b53d7aac3200        14 minutes ago      |1 APT_MIRROR=deb.debian.org /bin/sh -c git …   107MB
    fa74acf32f99        2 hours ago         /bin/sh -c #(nop)  ENV GO_SWAGGER_COMMIT=c28…   0B

After:

    6b2454f1a9a5        10 minutes ago      |1 APT_MIRROR=deb.debian.org /bin/sh -c set …   35.2MB
    fa74acf32f99        2 hours ago         /bin/sh -c #(nop)  ENV GO_SWAGGER_COMMIT=c28…   0B

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-12-08 14:02:32 -08:00
Ross Boucher
f01d2029bb Update CRIU github repository
Signed-off-by: Ross Boucher <rboucher@gmail.com>
2017-12-04 18:51:08 -05:00
Ross Boucher
7ffae3d51a CRIU needs a newer version to work with the updates pulled in by containerd 1.0
Signed-off-by: Ross Boucher <rboucher@gmail.com>
2017-12-04 17:45:04 -05:00
Sebastiaan van Stijn
d6e1cc32d3
Bump Go to 1.9.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 20:57:02 +01:00
Sebastiaan van Stijn
d1a0773d3a
Bump docker-py to 1d6b5b203222ba5df7dedfcd1ee061a452f99c8a
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-14 14:17:44 +01:00
Aleksa Sarai
1f4e37cf4b
*: switch to -buildmode=pie
Go has supported PIC builds for a while now, and given the security
benefits of using PIC binaries we should really enable them. There also
appears to be some indication that non-PIC builds have been interacting
oddly on ppc64le (the linker cannot load some shared libraries), and
using PIC builds appears to solve this problem.

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2017-11-11 21:59:49 +11:00
Brian Goff
503b03a3f0 COPY should be the last op in the Dockerfile
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-11-09 16:00:28 -05:00
Brian Goff
16d83571f5 Update docker-py commit
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-11-02 18:02:52 -04:00
Sebastiaan van Stijn
503fe408da
Bump Golang to 1.8.5
go1.8.5 (released 2017/10/25) includes fixes to the compiler, linker, runtime,
documentation, go command, and the crypto/x509 and net/smtp packages. It
includes a fix to a bug introduced in Go 1.8.4 that broke go get of non-Git
repositories under certain conditions. See the Go 1.8.5 milestone on our issue
tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.8.5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-10-27 01:18:00 +02:00
Sebastiaan van Stijn
33e8141f63
Bump Go to 1.8.4
Bumps the Go version used to 1.8.4, which contains
security fixes; https://groups.google.com/forum/#!topic/golang-announce/1hZYiemnkdE

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-10-04 23:22:23 +02:00
Akihiro Suda
1b51a05375 Dockerfile: use seccomp provided by stretch
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2017-09-25 10:03:37 +00:00
Yong Tang
cfdac1245a Merge pull request #34550 from kolyshkin/libeudev
Update Dockerfiles to use Debian Stretch
2017-09-18 21:03:31 -07:00
Yong Tang
65e88d996a Merge pull request #34759 from kolyshkin/gometalinter
Gometalinter fixups for non-x86
2017-09-18 13:44:15 -07:00
Kir Kolyshkin
7439d360fd Fix test-docker-py on some arches
When running 'make all' on armhf, I got this:

> ---> Making bundle: .integration-daemon-start (in bundles/17.06.0-dev/test-docker-py)
> Using test binary docker
> INFO: Waiting for daemon to start...
> Starting dockerd
> .
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line
> 320, in _importconftest
>     mod = conftestpath.pyimport()
>   File "/usr/local/lib/python2.7/dist-packages/py/_path/local.py", line
> 662, in pyimport
>     __import__(modname)
>   File "/docker-py/tests/integration/conftest.py", line 6, in <module>
>     import docker.errors
>   File "/docker-py/docker/__init__.py", line 2, in <module>
>     from .api import APIClient
>   File "/docker-py/docker/api/__init__.py", line 2, in <module>
>     from .client import APIClient
>   File "/docker-py/docker/api/client.py", line 11, in <module>
>     from .build import BuildApiMixin
>   File "/docker-py/docker/api/build.py", line 6, in <module>
>     from .. import auth
>   File "/docker-py/docker/auth.py", line 6, in <module>
>     import dockerpycreds
> ImportError: No module named dockerpycreds
> ERROR: could not load /docker-py/tests/integration/conftest.py

The fix for this was already provided by commit 0ec8f56a3 and
commit c7c923594, but for some reason it did not made its way
to Dockerfiles for all architectures.

While at it, remove excessive comments.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-09-17 22:04:31 -07:00
Kir Kolyshkin
ce2a0120c1 Dockerfiles: fix test-docker-py
Presumably after switch to debian-stretch as a base, the following
errors happens in Jenkins:

10:48:03 ---> Making bundle: test-docker-py (in
bundles/17.06.0-dev/test-docker-py)
10:48:03 ---> Making bundle: .integration-daemon-start (in
bundles/17.06.0-dev/test-docker-py)
10:48:03 Using test binary docker
10:48:03 # DOCKER_EXPERIMENTAL is set: starting daemon with experimental
features enabled!
10:48:03 /etc/init.d/apparmor: 130: /etc/init.d/apparmor:
systemd-detect-virt: not found
10:48:03 Starting AppArmor profiles:Warning from stdin (line 1):
/sbin/apparmor_parser: cannot use or update cache, disable, or
force-complain via stdin
10:48:03 Warning failed to create cache: (null)
10:48:03 .
10:48:03 INFO: Waiting for daemon to start...
10:48:03 Starting dockerd
10:48:05 .
10:48:06 Traceback (most recent call last):
10:48:06   File
"/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 320, in
_importconftest
10:48:06     mod = conftestpath.pyimport()
10:48:06   File
"/usr/local/lib/python2.7/dist-packages/py/_path/local.py", line 662, in
pyimport
10:48:06     __import__(modname)
10:48:06   File "/docker-py/tests/integration/conftest.py", line 6, in
<module>
10:48:06     import docker.errors
10:48:06   File "/docker-py/docker/__init__.py", line 2, in <module>
10:48:06     from .api import APIClient
10:48:06   File "/docker-py/docker/api/__init__.py", line 2, in <module>
10:48:06     from .client import APIClient
10:48:06   File "/docker-py/docker/api/client.py", line 6, in <module>
10:48:06     import requests
10:48:06 ImportError: No module named requests
10:48:06 ERROR: could not load /docker-py/tests/integration/conftest.py
10:48:06

and

00:38:55   File "/docker-py/docker/transport/ssladapter.py", line 21, in
<module>
00:38:55     from backports.ssl_match_hostname import match_hostname
00:38:55 ImportError: No module named backports.ssl_match_hostname
00:38:55 ERROR: could not load /docker-py/tests/integration/conftest.py

To fix, install the missing python modules.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-09-17 22:04:31 -07:00
Kir Kolyshkin
84f1c054e6 devmapper gd: disable for static build
Static build with devmapper is impossible now since libudev is required
and no static version of libudev is available (as static libraries are
not supported by systemd which udev is part of).

This should not hurt anyone as "[t]he primary user of static builds
is the Editions, and docker in docker via the containers, and none
of those use device mapper".

Also, since the need for static libdevmapper is gone, there is no need
to self-compile libdevmapper -- let's use the one from Debian Stretch.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-09-17 22:04:31 -07:00
Justin Cormack
e89a5e5e91 Update Dockerfiles to use Debian stretch
The main gain here is that they all use exactly the same distro; previously
arm64 was using Ubuntu Xenial because Debian jessie was too old.

Does not seem that we can change any of the downloaded dependencies still,
as eg libseccomp is still not the version we are using.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-09-17 22:02:13 -07:00
Kir Kolyshkin
a436d8a634 Fix fetching LVM2 sources
Version 2.02.173 has disappeared, let's revert back to latest stable
one.

https://github.com/moby/moby/issues/34843

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-09-13 14:34:36 -07:00
Kir Kolyshkin
b96093fa56 gometalinter: add per-platform configurable options
I have run into two separate issues while doing 'make all' on armhf
(a Scaleway C1 machine, same as used in CI). This commit fixes both.

1. There were a lot of "not enough memory" errors, and after that
in a few runs gometalinter just stuck forever on FUTEX_WAIT with
no children left.

Looking into docs, I found the --enable-gc option which solved the issue.

[Update: this has already been added]

2. Timeout of 2 minutes is not enough for the abovementioned platform.
The longest running linter is goimports which takes almost 6 minutes to run.

Set the timeout to the observable run time roughly doubled.

In addition, ARM platforms does not have too much RAM (2GB), so
running too many processes in parallel might be problematic. Limit
it by using -j2

[v2: make the timeout arch-dependent, also tested on aarch64 (2m15s)]
[v3: moved timeout setting to Dockerfiles]
[v4: generalized to GOMETALINTER_OPTS, added -j2 for ARM platforms]
[v5: rebase to master]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-09-12 18:45:34 -07:00
Daniel Nephin
d7e2c4ce77 Use gometalinter for linting
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-18 14:23:43 -04:00
Brian Goff
31d8d2253e Merge pull request #34362 from kolyshkin/update-libdevmapper
Dockerfile*: bump devmapper library version
2017-08-14 09:56:43 -04:00
Daniel Nephin
bc82b13995 Remove test-integration-cli and references to it.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-09 11:02:57 -04:00
Kir Kolyshkin
c4fde49a5c Dockerfile*: bump devmapper library version
Let's use latest lvm2 sources to compile the libdevmapper library.

Initial reason for compiling devmapper lib from sources was a need to
have the static version of the library at hand, in order to build
the static dockerd, but note that the same headers/solib are used
for dynamic build (dynbinary) as well.

The reason for this patch is to enable the deferral removal feature.
The supplied devmapper library (and headers) are too old, lacking the
needed functions, so the daemon is built with 'libdm_no_deferred_remove'
build tag (see the check in hack/make.sh). Because of this, even if the
kernel dm driver is perfectly able to support the feature, it can not
be used. For more details and background story, see [1].

Surely, one can't just change the version number. While at it:
 - improve the comments;
 - remove obsoleted URLs;
 - remove s390 and ppc configure updates that are no longer needed;
 - use pkg-config instead of hardcoding the flags (newer lib added
   some more dependencies);

 [1] https://github.com/moby/moby/issues/34298

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-08-01 22:08:24 -07:00
Tonis Tiigi
72df48d1ad vendor: add archive/tar
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-13 19:08:19 -07:00
Harald Albers
cec5ca75e1 Enable bash completion in development container
Signed-off-by: Harald Albers <github@albersweb.de>
2017-06-24 23:51:06 +02:00
Harald Albers
0e2c424a7f Add bash completion to the development container
Adds an environment variable `DOCKER_BASH_COMPLETION_PATH` that allows to
include a bash completion script into the development container.

This is needid for development of bash completion.

Signed-off-by: Harald Albers <github@albersweb.de>
2017-06-23 18:11:55 +02:00
Daniel Nephin
96e61f3173 Remove bindata
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-21 11:20:05 -04:00
Daniel Nephin
b877fc31c5 Remove pkcs11, libltdl-dev, and clang for osx.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-21 11:18:44 -04:00
Vincent Demeester
9823457ed5 Merge pull request #33748 from cpuguy83/remove_osx_cross
Remove OSX cross stuff from main Dockerfile
2017-06-21 14:14:09 +02:00
Tonis Tiigi
d183d6fa7d dockerfile: update docker-py
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-20 22:03:35 -07:00
Brian Goff
57f0e0c619 Remove OSX cross stuff from main Dockerfile
This is no longer needed here. It was required for compiling the CLI
which we no longer do here.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-06-20 13:06:37 -07:00
allencloud
f257f77c6c choose rpc code to determine status code
Signed-off-by: allencloud <allen.sun@daocloud.io>
2017-06-06 10:08:50 +08:00
Brian Goff
0c7c900e9e Bump go to go1.8.3
Note that go1.8.2 contains a security fix (CVE-2017-8932).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-05-30 10:15:59 -04:00
Sebastiaan van Stijn
8022c5fdd1
Remove zfs from static builds
The libzfs-dev package is not needed for the static builds,
and only used for ubuntu packages, so removing from
the Dockerfile.

This resolves an issue where build fails due to the PPA
having changed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-05-15 13:22:52 +02:00
Brian Goff
470dfd69b3 Update golang to 1.8
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-05-10 20:06:27 -04:00
Sebastiaan van Stijn
a762ceace4 Merge pull request #32670 from perweij/fix-Dockerfile-instruction
Correcting test-instruction.
2017-05-09 17:20:42 +02:00
Arnaud Porterie (icecrime)
32915b1d0a Remove cmd/docker and other directories in cli/ in accordance with the new Moby project scope
Starting with this commit, integration tests should no longer rely on
the docker cli, they should be API tests instead. For the existing tests
the scripts will use a frozen version of the docker cli with a
DOCKER_API_VERSION frozen to 1.30, which should ensure that the CI remains
green at all times.

To help contributors develop and test manually with a modified docker
cli, this commit also adds a DOCKER_CLI_PATH environment variable to the
Makefile. This allows to set the path of a custom cli that will be
available inside the development container and used to run the
integration tests.

Signed-off-by: Arnaud Porterie (icecrime) <arnaud.porterie@docker.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
2017-05-05 12:14:29 -07:00
Per Weijnitz
5a076d7589 Correcting test-instruction.
Signed-off-by: Per Weijnitz <per.weijnitz@gmail.com>

Correcting instructions for test suite run.

Signed-off-by: Per Weijnitz <per.weijnitz@gmail.com>
2017-05-05 20:33:48 +02:00
Sebastiaan van Stijn
1ecd8ed518 Update frozen images in Dockerfiles
This updates the versions of the frozen images used to their current
version. The original reason for updating these images was to make sure
they are not affected by [CVE-2016-1252 / DSA-3733-1](https://lwn.net/Articles/709119/),
which is fixed in apt 1.0.9.8.4 and up.

Note that `CVE-2016-1252` won't affect our test-suite, because no packages
are installed during out tests. It is just "good practice" to keep these
images up to date.

The `debian:jessie`, and `buildpack-deps:jessie` in  `Dockerfile.s390x`,
and `Dockerfile.armhf` have not been updated in this patch, because
those images have not yet been updated to contain apt 1.0.9.8.4.

While working on this, the `busybox` and `hello-world` were also updated
to their latest version.

Also removes a reference to `hack/make/.ensure-frozen-images`, which
was removed in ff91276d1f.

The new busybox image has one layer less than the original,
so updated `TestBuildSquashParent` to take that into account.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-04-10 22:05:39 +02:00
Krasi Georgiev
d1a0afacc2 update criu to 2.12.1
Signed-off-by: Krasi Georgiev <krasi@vip-consult.solutions>
2017-04-01 00:54:28 +03:00
unclejack
c7c9235946 Dockerfile: move comments to fix build error
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2017-03-16 14:07:31 +02:00