This was inadvertedly removed in 7bfe48cc00,
because it was documented as a dependency for docker-py, but
actually used to validate the swagger file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The Golang base images switch to buster, which causes some breakage
in networking and packages that are no longer available; (`btrfs-tools`
is now an empty package, and `libprotobuf-c0-dev` is gone).
Some of out tests also start faiilng on stretch, and will have to be
investigated further;
```
15:13:06 --- FAIL: TestRenameAnonymousContainer (3.37s)
15:13:06 rename_test.go:168: assertion failed: 0 (int) != 1 (inspect.State.ExitCode int): container a7fe866d588d65f353f42ffc5ea5288e52700384e1d90850e9c3d4dce8657666 exited with the wrong exitcode:
15:13:38 --- FAIL: TestHostnameDnsResolution (2.23s)
15:13:38 run_linux_test.go:128: assertion failed:
15:13:38 --- ←
15:13:38 +++ →
15:13:38 @@ -1 +1,2 @@
15:13:38 +ping: bad address 'foobar'
15:13:38
15:13:38
15:13:38 run_linux_test.go:129: assertion failed: 0 (int) != 1 (res.ExitCode int)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This allows overriding the version of Go without making modifications in the
source code, which can be useful to test against multiple versions.
For example:
make GO_VERSION=1.13beta1 shell
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This removes all the installation steps for docker-py from the
Dockerfile, and instead builds the upstream Dockerfile, and runs
docker-py tests in a container.
To test;
```
make test-docker-py
...
Removing bundles/
---> Making bundle: dynbinary (in bundles/dynbinary)
Building: bundles/dynbinary-daemon/dockerd-dev
Created binary: bundles/dynbinary-daemon/dockerd-dev
---> Making bundle: test-docker-py (in bundles/test-docker-py)
---> Making bundle: .integration-daemon-start (in bundles/test-docker-py)
Using test binary docker
Starting dockerd
INFO: Waiting for daemon to start...
.
INFO: Building docker-sdk-python3:3.7.0...
sha256:686428ae28479e9b5c8fdad1cadc9b7a39b462e66bd13a7e35bd79c6a152a402
INFO: Starting docker-py tests...
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.1.0, py-1.8.0, pluggy-0.9.0
rootdir: /src, inifile: pytest.ini
plugins: timeout-1.3.3, cov-2.6.1
collected 359 items
tests/integration/api_build_test.py .......s....
....
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The fastly cdn mirror we're using also mirrors the debian security
repository;
```
Welcome to deb.debian.org (fastly instance)!
This is deb.debian.org. This service provides mirrors for the following Debian archive repositories:
/debian/
/debian-debug/
/debian-ports/
/debian-security/
The server deb.debian.org does not have packages itself, but the name has SRV records in DNS that let apt in stretch and later find places.
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This cross-compiling other binaries simpler.
It would be nice if the cross Makefile target built all the required
bins, but at least this is a first step.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Pretty much cross-compile doesn't work because of this:
> profiles/seccomp/seccomp.go:13:2: build constraints exclude all Go files in /go/src/github.com/docker/docker/vendor/github.com/seccomp/libseccomp-golang
This changes adds a new Dockerfile target for cross compilation with the
neccesary arch specific libseccomp packages and CC toolchains.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Also removed the `-stretch` suffix, because Debian Stretch
is the default base-image now, so there should be no need
to keep the suffix
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Please refer to `docs/rootless.md`.
TLDR:
* Make sure `/etc/subuid` and `/etc/subgid` contain the entry for you
* `dockerd-rootless.sh --experimental`
* `docker -H unix://$XDG_RUNTIME_DIR/docker.sock run ...`
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
build the final stage of the Dockerfile (including COPY ...) if no BIND_DIR
is used.
if BIND_DIR is used, build the dev stage, thus skipping the COPY.
Original author: @thaJeztah
Signed-off-by: Mohammad Nasirifar <farnasirim@gmail.com>
I noticed this in the build output;
```
16:05:07 [100%] Built target tini-static
16:05:07 + mkdir -p /build/
16:05:07 + cp tini-static /build//docker-init
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.4 (released 2018/12/14) includes fixes to cgo, the compiler, linker,
runtime, documentation, go command, and the net/http and go/types packages. It
includes a fix to a bug introduced in Go 1.11.3 that broke go get for import
path patterns containing "...".
See the Go 1.11.4 milestone for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.4+label%3ACherryPickApproved
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.13 (released 2018/12/14)
- crypto/x509: CPU denial of service in chain validation golang/go#29233
- cmd/go: directory traversal in "go get" via curly braces in import paths golang/go#29231
- cmd/go: remote command execution during "go get -u" golang/go#29230
See the Go 1.11.3 milestone on the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.3
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Includes fixes to the compiler, linker, documentation, go command, and the
database/sql and go/types packages. See the Go 1.11.2 milestone on the issue
tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.2
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.11.1 (released 2018/10/01) includes fixes to the compiler,
documentation, go command, runtime, and the crypto/x509, encoding/json,
go/types, net, net/http, and reflect packages.
See the Go 1.11.1 milestone on our issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.11.1
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It's that time of year again! Go 1.11 is released, time to use it.
This commit also
* removes our archive/tar fork, since upstream archive/tar
is fixed for static builds, and osusergo build tag is set.
* removes ENV GO_VERSION from Dockerfile as it's not needed
anymore since PR #37592 is merged.
[v2: switch to beta2]
[v3: switch to beta3]
[v4: rc1]
[v5: remove ENV GO_VERSION as PR #37592 is now merged]
[v6: rc2]
[v7: final!]
[v8: use 1.11.0]
[v9: back to 1.11]
[v8: use 1.11.0]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Includes fixes to the go command, linker, and the net/http, mime/multipart,
ld/macho, bytes, and strings packages. See the Go 1.10.4 milestone on the
issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.10.4
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
go1.9.5 (released 2018/03/28) includes fixes to the compiler, go
command, and net/http/pprof package. See the Go 1.9.5 milestone on
the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.9.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Using `dpkg --print-architecture` instead of the `uname -m` to abstract
the architecture value from the container images, which the build process
is running inside, to match exactly the behavior specified by the following
Docker file while not 'passthru' to the host.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Since the recent release of CRIU has already supported other
arches such as AArch64, ppc64le, and s390x, so we can enable
it now.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
This PR consolidates the existing arch-specific Dockerfiles into only
one file `Dockefile` to ease the code maintenance effort.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
This fix update docker-py so that containers from the tests run
could be cleaned up during teardown:
```diff
-ENV DOCKER_PY_COMMIT 5e28dcaace5f7b70cbe44c313b7a3b288fa38916
+ENV DOCKER_PY_COMMIT 8b246db271a85d6541dc458838627e89c683e42f
```
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Originally I worked on this for the multi-stage build Dockerfile
changes. Decided to split this out as we are still waiting for
multi-stage to be available on CI and rebasing these is pretty annoying.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This fix updates docker-py:
```
-ENV DOCKER_PY_COMMIT 1d6b5b203222ba5df7dedfcd1ee061a452f99c8a
+ENV DOCKER_PY_COMMIT 5e28dcaace5f7b70cbe44c313b7a3b288fa38916
```
The updated docker-py includes https://github.com/docker/docker-py/pull/1909
which is required to have #36292 pass the tests.
Full diff is in 1d6b5b2032...5e28dcaace.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
These are now more in-line with wiki instructions. Also removes
broken/deprecated make target test-unit.
Signed-off-by: Brett Randall <javabrett@gmail.com>
Update and unify the `busybox` images on all arches to the `glibc` multi-arch
version and remove the temp workaround on amd64 which uses the old version
busybox (v1.26) before this PR to bypass the failure of those network related
test cases. Also, this PR will fix all the network related issues with `glibc`
version `busybox` image.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
This fixes a vulnerability in `go get` (CVE-2018-6574, http://golang.org/issue/23672),
but shouldn't really affect our code, but it's good to keep in sync.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
1. Add = between the option and the argument, otherwise the argument
appears to be passed on to the linters directly, as in:
> DEBUG: [golint.8]: executing /home/kir/go/bin/golint
> -min_confidence 0.800000 ./10m ./api ./api/errdefs <...>
2. Fix setting the default for GOMETALINTER_OPTS -- the default
was -deadline (rather than --deadline).
Fixes: b96093fa56 ("gometalinter: add per-platform configurable options")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The Golang built-in gzip library is serialized, and fairly slow
at decompressing. It also only decompresses on demand, versus
pipelining decompression.
This change switches to using the pigz external command
for gzip decompression, as opposed to using the built-in
golang one. This code is not vendored, but will be used
if it autodetected as part of the OS.
This also switches to using context, versus a manually
managed channel to manage cancellations, and synchronization.
There is a little bit of weirdness around manually having
to cancel in the error cases.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Upgrade the frozen images to the multi-arch ones.
Since issue #35963 is not fixed yet on linux/amd64, so we keep the busybox
image on amd64 untouched.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
The installation steps for go-swagger was a bit noisy, and not consistent with
other installation steps.
This patch makes it similar to other steps, which makes it less noisy, and
makes the image slightly smaller.
Before:
b53d7aac3200 14 minutes ago |1 APT_MIRROR=deb.debian.org /bin/sh -c git … 107MB
fa74acf32f99 2 hours ago /bin/sh -c #(nop) ENV GO_SWAGGER_COMMIT=c28… 0B
After:
6b2454f1a9a5 10 minutes ago |1 APT_MIRROR=deb.debian.org /bin/sh -c set … 35.2MB
fa74acf32f99 2 hours ago /bin/sh -c #(nop) ENV GO_SWAGGER_COMMIT=c28… 0B
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Go has supported PIC builds for a while now, and given the security
benefits of using PIC binaries we should really enable them. There also
appears to be some indication that non-PIC builds have been interacting
oddly on ppc64le (the linker cannot load some shared libraries), and
using PIC builds appears to solve this problem.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
go1.8.5 (released 2017/10/25) includes fixes to the compiler, linker, runtime,
documentation, go command, and the crypto/x509 and net/smtp packages. It
includes a fix to a bug introduced in Go 1.8.4 that broke go get of non-Git
repositories under certain conditions. See the Go 1.8.5 milestone on our issue
tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.8.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When running 'make all' on armhf, I got this:
> ---> Making bundle: .integration-daemon-start (in bundles/17.06.0-dev/test-docker-py)
> Using test binary docker
> INFO: Waiting for daemon to start...
> Starting dockerd
> .
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line
> 320, in _importconftest
> mod = conftestpath.pyimport()
> File "/usr/local/lib/python2.7/dist-packages/py/_path/local.py", line
> 662, in pyimport
> __import__(modname)
> File "/docker-py/tests/integration/conftest.py", line 6, in <module>
> import docker.errors
> File "/docker-py/docker/__init__.py", line 2, in <module>
> from .api import APIClient
> File "/docker-py/docker/api/__init__.py", line 2, in <module>
> from .client import APIClient
> File "/docker-py/docker/api/client.py", line 11, in <module>
> from .build import BuildApiMixin
> File "/docker-py/docker/api/build.py", line 6, in <module>
> from .. import auth
> File "/docker-py/docker/auth.py", line 6, in <module>
> import dockerpycreds
> ImportError: No module named dockerpycreds
> ERROR: could not load /docker-py/tests/integration/conftest.py
The fix for this was already provided by commit 0ec8f56a3 and
commit c7c923594, but for some reason it did not made its way
to Dockerfiles for all architectures.
While at it, remove excessive comments.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Presumably after switch to debian-stretch as a base, the following
errors happens in Jenkins:
10:48:03 ---> Making bundle: test-docker-py (in
bundles/17.06.0-dev/test-docker-py)
10:48:03 ---> Making bundle: .integration-daemon-start (in
bundles/17.06.0-dev/test-docker-py)
10:48:03 Using test binary docker
10:48:03 # DOCKER_EXPERIMENTAL is set: starting daemon with experimental
features enabled!
10:48:03 /etc/init.d/apparmor: 130: /etc/init.d/apparmor:
systemd-detect-virt: not found
10:48:03 Starting AppArmor profiles:Warning from stdin (line 1):
/sbin/apparmor_parser: cannot use or update cache, disable, or
force-complain via stdin
10:48:03 Warning failed to create cache: (null)
10:48:03 .
10:48:03 INFO: Waiting for daemon to start...
10:48:03 Starting dockerd
10:48:05 .
10:48:06 Traceback (most recent call last):
10:48:06 File
"/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 320, in
_importconftest
10:48:06 mod = conftestpath.pyimport()
10:48:06 File
"/usr/local/lib/python2.7/dist-packages/py/_path/local.py", line 662, in
pyimport
10:48:06 __import__(modname)
10:48:06 File "/docker-py/tests/integration/conftest.py", line 6, in
<module>
10:48:06 import docker.errors
10:48:06 File "/docker-py/docker/__init__.py", line 2, in <module>
10:48:06 from .api import APIClient
10:48:06 File "/docker-py/docker/api/__init__.py", line 2, in <module>
10:48:06 from .client import APIClient
10:48:06 File "/docker-py/docker/api/client.py", line 6, in <module>
10:48:06 import requests
10:48:06 ImportError: No module named requests
10:48:06 ERROR: could not load /docker-py/tests/integration/conftest.py
10:48:06
and
00:38:55 File "/docker-py/docker/transport/ssladapter.py", line 21, in
<module>
00:38:55 from backports.ssl_match_hostname import match_hostname
00:38:55 ImportError: No module named backports.ssl_match_hostname
00:38:55 ERROR: could not load /docker-py/tests/integration/conftest.py
To fix, install the missing python modules.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Static build with devmapper is impossible now since libudev is required
and no static version of libudev is available (as static libraries are
not supported by systemd which udev is part of).
This should not hurt anyone as "[t]he primary user of static builds
is the Editions, and docker in docker via the containers, and none
of those use device mapper".
Also, since the need for static libdevmapper is gone, there is no need
to self-compile libdevmapper -- let's use the one from Debian Stretch.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The main gain here is that they all use exactly the same distro; previously
arm64 was using Ubuntu Xenial because Debian jessie was too old.
Does not seem that we can change any of the downloaded dependencies still,
as eg libseccomp is still not the version we are using.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Version 2.02.173 has disappeared, let's revert back to latest stable
one.
https://github.com/moby/moby/issues/34843
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
I have run into two separate issues while doing 'make all' on armhf
(a Scaleway C1 machine, same as used in CI). This commit fixes both.
1. There were a lot of "not enough memory" errors, and after that
in a few runs gometalinter just stuck forever on FUTEX_WAIT with
no children left.
Looking into docs, I found the --enable-gc option which solved the issue.
[Update: this has already been added]
2. Timeout of 2 minutes is not enough for the abovementioned platform.
The longest running linter is goimports which takes almost 6 minutes to run.
Set the timeout to the observable run time roughly doubled.
In addition, ARM platforms does not have too much RAM (2GB), so
running too many processes in parallel might be problematic. Limit
it by using -j2
[v2: make the timeout arch-dependent, also tested on aarch64 (2m15s)]
[v3: moved timeout setting to Dockerfiles]
[v4: generalized to GOMETALINTER_OPTS, added -j2 for ARM platforms]
[v5: rebase to master]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Let's use latest lvm2 sources to compile the libdevmapper library.
Initial reason for compiling devmapper lib from sources was a need to
have the static version of the library at hand, in order to build
the static dockerd, but note that the same headers/solib are used
for dynamic build (dynbinary) as well.
The reason for this patch is to enable the deferral removal feature.
The supplied devmapper library (and headers) are too old, lacking the
needed functions, so the daemon is built with 'libdm_no_deferred_remove'
build tag (see the check in hack/make.sh). Because of this, even if the
kernel dm driver is perfectly able to support the feature, it can not
be used. For more details and background story, see [1].
Surely, one can't just change the version number. While at it:
- improve the comments;
- remove obsoleted URLs;
- remove s390 and ppc configure updates that are no longer needed;
- use pkg-config instead of hardcoding the flags (newer lib added
some more dependencies);
[1] https://github.com/moby/moby/issues/34298
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Adds an environment variable `DOCKER_BASH_COMPLETION_PATH` that allows to
include a bash completion script into the development container.
This is needid for development of bash completion.
Signed-off-by: Harald Albers <github@albersweb.de>
