go1.9.5 (released 2018/03/28) includes fixes to the compiler, go
command, and net/http/pprof package. See the Go 1.9.5 milestone on
the issue tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.9.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Using `dpkg --print-architecture` instead of the `uname -m` to abstract
the architecture value from the container images, which the build process
is running inside, to match exactly the behavior specified by the following
Docker file while not 'passthru' to the host.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
Since the recent release of CRIU has already supported other
arches such as AArch64, ppc64le, and s390x, so we can enable
it now.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
This PR consolidates the existing arch-specific Dockerfiles into only
one file `Dockefile` to ease the code maintenance effort.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
This fix update docker-py so that containers from the tests run
could be cleaned up during teardown:
```diff
-ENV DOCKER_PY_COMMIT 5e28dcaace5f7b70cbe44c313b7a3b288fa38916
+ENV DOCKER_PY_COMMIT 8b246db271a85d6541dc458838627e89c683e42f
```
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Originally I worked on this for the multi-stage build Dockerfile
changes. Decided to split this out as we are still waiting for
multi-stage to be available on CI and rebasing these is pretty annoying.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This fix updates docker-py:
```
-ENV DOCKER_PY_COMMIT 1d6b5b203222ba5df7dedfcd1ee061a452f99c8a
+ENV DOCKER_PY_COMMIT 5e28dcaace5f7b70cbe44c313b7a3b288fa38916
```
The updated docker-py includes https://github.com/docker/docker-py/pull/1909
which is required to have #36292 pass the tests.
Full diff is in 1d6b5b2032...5e28dcaace.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
These are now more in-line with wiki instructions. Also removes
broken/deprecated make target test-unit.
Signed-off-by: Brett Randall <javabrett@gmail.com>
Update and unify the `busybox` images on all arches to the `glibc` multi-arch
version and remove the temp workaround on amd64 which uses the old version
busybox (v1.26) before this PR to bypass the failure of those network related
test cases. Also, this PR will fix all the network related issues with `glibc`
version `busybox` image.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
This fixes a vulnerability in `go get` (CVE-2018-6574, http://golang.org/issue/23672),
but shouldn't really affect our code, but it's good to keep in sync.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
1. Add = between the option and the argument, otherwise the argument
appears to be passed on to the linters directly, as in:
> DEBUG: [golint.8]: executing /home/kir/go/bin/golint
> -min_confidence 0.800000 ./10m ./api ./api/errdefs <...>
2. Fix setting the default for GOMETALINTER_OPTS -- the default
was -deadline (rather than --deadline).
Fixes: b96093fa56 ("gometalinter: add per-platform configurable options")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The Golang built-in gzip library is serialized, and fairly slow
at decompressing. It also only decompresses on demand, versus
pipelining decompression.
This change switches to using the pigz external command
for gzip decompression, as opposed to using the built-in
golang one. This code is not vendored, but will be used
if it autodetected as part of the OS.
This also switches to using context, versus a manually
managed channel to manage cancellations, and synchronization.
There is a little bit of weirdness around manually having
to cancel in the error cases.
Signed-off-by: Sargun Dhillon <sargun@sargun.me>
Upgrade the frozen images to the multi-arch ones.
Since issue #35963 is not fixed yet on linux/amd64, so we keep the busybox
image on amd64 untouched.
Signed-off-by: Dennis Chen <dennis.chen@arm.com>
The installation steps for go-swagger was a bit noisy, and not consistent with
other installation steps.
This patch makes it similar to other steps, which makes it less noisy, and
makes the image slightly smaller.
Before:
b53d7aac3200 14 minutes ago |1 APT_MIRROR=deb.debian.org /bin/sh -c git … 107MB
fa74acf32f99 2 hours ago /bin/sh -c #(nop) ENV GO_SWAGGER_COMMIT=c28… 0B
After:
6b2454f1a9a5 10 minutes ago |1 APT_MIRROR=deb.debian.org /bin/sh -c set … 35.2MB
fa74acf32f99 2 hours ago /bin/sh -c #(nop) ENV GO_SWAGGER_COMMIT=c28… 0B
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Go has supported PIC builds for a while now, and given the security
benefits of using PIC binaries we should really enable them. There also
appears to be some indication that non-PIC builds have been interacting
oddly on ppc64le (the linker cannot load some shared libraries), and
using PIC builds appears to solve this problem.
Signed-off-by: Aleksa Sarai <asarai@suse.de>
go1.8.5 (released 2017/10/25) includes fixes to the compiler, linker, runtime,
documentation, go command, and the crypto/x509 and net/smtp packages. It
includes a fix to a bug introduced in Go 1.8.4 that broke go get of non-Git
repositories under certain conditions. See the Go 1.8.5 milestone on our issue
tracker for details:
https://github.com/golang/go/issues?q=milestone%3AGo1.8.5
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
When running 'make all' on armhf, I got this:
> ---> Making bundle: .integration-daemon-start (in bundles/17.06.0-dev/test-docker-py)
> Using test binary docker
> INFO: Waiting for daemon to start...
> Starting dockerd
> .
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line
> 320, in _importconftest
> mod = conftestpath.pyimport()
> File "/usr/local/lib/python2.7/dist-packages/py/_path/local.py", line
> 662, in pyimport
> __import__(modname)
> File "/docker-py/tests/integration/conftest.py", line 6, in <module>
> import docker.errors
> File "/docker-py/docker/__init__.py", line 2, in <module>
> from .api import APIClient
> File "/docker-py/docker/api/__init__.py", line 2, in <module>
> from .client import APIClient
> File "/docker-py/docker/api/client.py", line 11, in <module>
> from .build import BuildApiMixin
> File "/docker-py/docker/api/build.py", line 6, in <module>
> from .. import auth
> File "/docker-py/docker/auth.py", line 6, in <module>
> import dockerpycreds
> ImportError: No module named dockerpycreds
> ERROR: could not load /docker-py/tests/integration/conftest.py
The fix for this was already provided by commit 0ec8f56a3 and
commit c7c923594, but for some reason it did not made its way
to Dockerfiles for all architectures.
While at it, remove excessive comments.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Presumably after switch to debian-stretch as a base, the following
errors happens in Jenkins:
10:48:03 ---> Making bundle: test-docker-py (in
bundles/17.06.0-dev/test-docker-py)
10:48:03 ---> Making bundle: .integration-daemon-start (in
bundles/17.06.0-dev/test-docker-py)
10:48:03 Using test binary docker
10:48:03 # DOCKER_EXPERIMENTAL is set: starting daemon with experimental
features enabled!
10:48:03 /etc/init.d/apparmor: 130: /etc/init.d/apparmor:
systemd-detect-virt: not found
10:48:03 Starting AppArmor profiles:Warning from stdin (line 1):
/sbin/apparmor_parser: cannot use or update cache, disable, or
force-complain via stdin
10:48:03 Warning failed to create cache: (null)
10:48:03 .
10:48:03 INFO: Waiting for daemon to start...
10:48:03 Starting dockerd
10:48:05 .
10:48:06 Traceback (most recent call last):
10:48:06 File
"/usr/local/lib/python2.7/dist-packages/_pytest/config.py", line 320, in
_importconftest
10:48:06 mod = conftestpath.pyimport()
10:48:06 File
"/usr/local/lib/python2.7/dist-packages/py/_path/local.py", line 662, in
pyimport
10:48:06 __import__(modname)
10:48:06 File "/docker-py/tests/integration/conftest.py", line 6, in
<module>
10:48:06 import docker.errors
10:48:06 File "/docker-py/docker/__init__.py", line 2, in <module>
10:48:06 from .api import APIClient
10:48:06 File "/docker-py/docker/api/__init__.py", line 2, in <module>
10:48:06 from .client import APIClient
10:48:06 File "/docker-py/docker/api/client.py", line 6, in <module>
10:48:06 import requests
10:48:06 ImportError: No module named requests
10:48:06 ERROR: could not load /docker-py/tests/integration/conftest.py
10:48:06
and
00:38:55 File "/docker-py/docker/transport/ssladapter.py", line 21, in
<module>
00:38:55 from backports.ssl_match_hostname import match_hostname
00:38:55 ImportError: No module named backports.ssl_match_hostname
00:38:55 ERROR: could not load /docker-py/tests/integration/conftest.py
To fix, install the missing python modules.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Static build with devmapper is impossible now since libudev is required
and no static version of libudev is available (as static libraries are
not supported by systemd which udev is part of).
This should not hurt anyone as "[t]he primary user of static builds
is the Editions, and docker in docker via the containers, and none
of those use device mapper".
Also, since the need for static libdevmapper is gone, there is no need
to self-compile libdevmapper -- let's use the one from Debian Stretch.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The main gain here is that they all use exactly the same distro; previously
arm64 was using Ubuntu Xenial because Debian jessie was too old.
Does not seem that we can change any of the downloaded dependencies still,
as eg libseccomp is still not the version we are using.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Version 2.02.173 has disappeared, let's revert back to latest stable
one.
https://github.com/moby/moby/issues/34843
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
I have run into two separate issues while doing 'make all' on armhf
(a Scaleway C1 machine, same as used in CI). This commit fixes both.
1. There were a lot of "not enough memory" errors, and after that
in a few runs gometalinter just stuck forever on FUTEX_WAIT with
no children left.
Looking into docs, I found the --enable-gc option which solved the issue.
[Update: this has already been added]
2. Timeout of 2 minutes is not enough for the abovementioned platform.
The longest running linter is goimports which takes almost 6 minutes to run.
Set the timeout to the observable run time roughly doubled.
In addition, ARM platforms does not have too much RAM (2GB), so
running too many processes in parallel might be problematic. Limit
it by using -j2
[v2: make the timeout arch-dependent, also tested on aarch64 (2m15s)]
[v3: moved timeout setting to Dockerfiles]
[v4: generalized to GOMETALINTER_OPTS, added -j2 for ARM platforms]
[v5: rebase to master]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Let's use latest lvm2 sources to compile the libdevmapper library.
Initial reason for compiling devmapper lib from sources was a need to
have the static version of the library at hand, in order to build
the static dockerd, but note that the same headers/solib are used
for dynamic build (dynbinary) as well.
The reason for this patch is to enable the deferral removal feature.
The supplied devmapper library (and headers) are too old, lacking the
needed functions, so the daemon is built with 'libdm_no_deferred_remove'
build tag (see the check in hack/make.sh). Because of this, even if the
kernel dm driver is perfectly able to support the feature, it can not
be used. For more details and background story, see [1].
Surely, one can't just change the version number. While at it:
- improve the comments;
- remove obsoleted URLs;
- remove s390 and ppc configure updates that are no longer needed;
- use pkg-config instead of hardcoding the flags (newer lib added
some more dependencies);
[1] https://github.com/moby/moby/issues/34298
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Adds an environment variable `DOCKER_BASH_COMPLETION_PATH` that allows to
include a bash completion script into the development container.
This is needid for development of bash completion.
Signed-off-by: Harald Albers <github@albersweb.de>
The libzfs-dev package is not needed for the static builds,
and only used for ubuntu packages, so removing from
the Dockerfile.
This resolves an issue where build fails due to the PPA
having changed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Starting with this commit, integration tests should no longer rely on
the docker cli, they should be API tests instead. For the existing tests
the scripts will use a frozen version of the docker cli with a
DOCKER_API_VERSION frozen to 1.30, which should ensure that the CI remains
green at all times.
To help contributors develop and test manually with a modified docker
cli, this commit also adds a DOCKER_CLI_PATH environment variable to the
Makefile. This allows to set the path of a custom cli that will be
available inside the development container and used to run the
integration tests.
Signed-off-by: Arnaud Porterie (icecrime) <arnaud.porterie@docker.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
This updates the versions of the frozen images used to their current
version. The original reason for updating these images was to make sure
they are not affected by [CVE-2016-1252 / DSA-3733-1](https://lwn.net/Articles/709119/),
which is fixed in apt 1.0.9.8.4 and up.
Note that `CVE-2016-1252` won't affect our test-suite, because no packages
are installed during out tests. It is just "good practice" to keep these
images up to date.
The `debian:jessie`, and `buildpack-deps:jessie` in `Dockerfile.s390x`,
and `Dockerfile.armhf` have not been updated in this patch, because
those images have not yet been updated to contain apt 1.0.9.8.4.
While working on this, the `busybox` and `hello-world` were also updated
to their latest version.
Also removes a reference to `hack/make/.ensure-frozen-images`, which
was removed in ff91276d1f.
The new busybox image has one layer less than the original,
so updated `TestBuildSquashParent` to take that into account.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Running the rm command on a paused/restarting container
will give an error message saying the container is running
which is incorrect.
To fix that, the error message will have the correct
container state and a procedure to remove it accordingly.
Notice: docker-py was bumped to:
4a08d04aef0595322e1b5ac7c52f28a931da85a5
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
This drops support for migrations from pre-1.10 Docker versions, which
should be done via an external tool or an intermediate upgrade.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
The keyserver infrastructure is unreliable, and just adds another point
of failure without adding any security. Instead, commit the key used at
build time for ZFS to the repo, and inline our signing key into the
install script rather than just its fingerprint.
fix#28510fix#13555
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
criu 2.8 and 2.9 contain various fixes,
so updating the version in the Dockerfile
to match the newer version.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Why? Most of the time I end up needing an editor when in `make shell`.
Spent much time doing `apt-get update && apt-get install vim`.
Since we're already installing vim-common anyway...
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
There is no reason to duplicate efforts and tini is well built and
better than grimes. It is a much stronger option for the default init
and @krallin has done a great job maintaining it and helping make
changes so that it will work with Docker.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
- yamllint to ensure it is a valid YAML file
- go-swagger validate to ensure it is a valid swagger file
Signed-off-by: Ben Firshman <ben@firshman.co.uk>
this switches all go download links to use a redirect,
https://golang.org/dl/ instead of the direct source.
Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
This means we can vendor libnetwork without special casing, and
it is built the same way as the other external binaries.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
- What I did
While building docker from source, to get the dependencies installed had done make build, then had got this error.
- How I did it
In the DockerFIle, instead using space a tab was put, which when was done make build the next line was getting combined and was unable to install the package.
image
image
Refer the below Hex View of the earlier file.
image
- How to verify it
After fixing, changing tab to space, built from source to install dependencies and was success
- Description for the changelog
Fixing Issue #27035
Signed-off-by: Rojin George itsmerojin@gmail.com
Signed-off-by: rojingeorge <itsmerojin@gmail.com>
This adds a small C binary for fighting zombies. It is mounted under
`/dev/init` and is prepended to the args specified by the user. You
enable it via a daemon flag, `dockerd --init`, as it is disable by
default for backwards compat.
You can also override the daemon option or specify this on a per
container basis with `docker run --init=true|false`.
You can test this by running a process like this as the pid 1 in a
container and see the extra zombie that appears in the container as it
is running.
```c
int main(int argc, char ** argv) {
pid_t pid = fork();
if (pid == 0) {
pid = fork();
if (pid == 0) {
exit(0);
}
sleep(3);
exit(0);
}
printf("got pid %d and exited\n", pid);
sleep(20);
}
```
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Interactive integration testing is useful when you're developing new tests, or
making changes to cli code.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
following the announcement;
https://groups.google.com/forum/m/#!topic/golang-announce/7JTsd70ZAT0
> [security] Go 1.6.3 and Go 1.7rc2 pre-announcement
>
> Hello gophers,
> We plan to issue Go 1.6.3 and Go 1.7rc2 on Monday July 18 at approximately 2am UTC.
> These are minor release to fix a security issue.
>
> Following our policy at https://golang.org/security, this is the pre-announcement of those releases.
>
> Because we are so late in the release cycle for Go 1.7, we will not issue a minor release of Go 1.5.
> Additionally, we plan to issue Go 1.7rc3 later next week, which will include any changes between 1.7rc1 and tip.
>
> Cheers,
> Chris on behalf of the Go team
**Note:**
the man/Dockerfile is not yet updated, because
the official image for Go 1.6.2 has not yet
been updated.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This version introduces the following:
- uses nanosecond timestamps for event
- ensure events are sent once their effect is "live"
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
This updates containerd to b93a33be39bc4ef0fb00bfcb79147a28c33d9d43
fixing the start sync issues.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Due to the issue of go-md2man, a numbered list in `man docker login` was not rendered correctly.
a8f937e113
Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
This bumps containerd to cf554d59dd96e459544748290eb9167f4bcde509 and
includes various fixes and updates the grpc package and types generated
for use.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
the llvm apt repo was removed and finding the exact rev of the llvm
toolchain that worked perfecting in our dockerfile is a problem.
We looked at the reasons why we were using this version of clang and it
appears to be this compiler warning.
`was built for newer OSX version (10.11) than being linked
(10.6)`
When you look at the dockerfile this makes sense.
```
ENV OSX_SDK MacOSX10.11.sdk
ENV OSX_CROSS_COMMIT 8aa9b71a394905e6c5f4b59e2b97b87a004658a4
RUN set -x \
&& export OSXCROSS_PATH="/osxcross" \
&& git clone https://github.com/tpoechtrager/osxcross.git $OSXCROSS_PATH \
&& ( cd $OSXCROSS_PATH && git checkout -q $OSX_CROSS_COMMIT) \
&& curl -sSL https://s3.dockerproject.org/darwin/v2/${OSX_SDK}.tar.xz -o "${OSXCROSS_PATH}/tarballs/${OSX_SDK}.tar.xz" \
&& UNATTENDED=yes OSX_VERSION_MIN=10.6 ${OSXCROSS_PATH}/build.sh
ENV PATH /osxcross/target/bin:$PATH
```
We are basically using the 10.11 sdk but linking to 10.6 as the
min version so this warning should be expected.
Also the docs on the osxcross project require clan 3.2+, not 3.8.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
This environment variable is no longer
needed in Go 1.6 (as it's not the default).
Removed this environment variable from
all Dockerfiles except the Dockerfile.s390x,
which is still using gcc 5.3 (Go 1.5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This change adds file version information to docker.exe and dockerd.exe by
adding a Windows version resource with the windres tool.
This change adds a dependency to binutils-mingw-w64 on Linux, but removes
a dependency on rsrc. Most Windows build environments should already have
windres if they have gcc (which is necessary to build dockerd).
Signed-off-by: John Starks <jostarks@microsoft.com>
For context: https://github.com/golang/go/issues/15286
This commit downloads go1.5.3 in addition to go1.5.4 in order to
workaround the issue.
It is not expected to do a Docker release without a proper fix, however
this should help unblock Docker development on Windows TP5.
Signed-off-by: Tibor Vass <tibor@docker.com>
This vendors in new spec/runc that supports
setting readonly and masked paths in the
configuration. Using this allows us to make an
exception for `—-privileged`.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This includes fixes for;
- outputing errors for missing seccomp options on seccomp versions < 2.3
- cap set apply EPERM errors on ARM systems
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
runc expects a systemd cgroupsPath to be in slice:scopePrefix:containerName
format and the "--systemd-cgroup" option to be set. Update docker accordingly.
Fixes 21475
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
Add some missing dependencies in the Dockerfile:
- apt-utils for apt-ftparchive.conf
- bsdmainutils for our use of the column command in
hack/make/generate-index-listing
We also ensure that the docker daemon is started before calling
release-deb or release-rpm, since .detect-daemon-osarch, which is sourced
in each of them, requires the daemon to be running.
This commit also gets completely rid of s3cmd and fixes references to
AWS_* environment variables (changing from AWS_ACCESS_KEY to
AWS_ACCESS_KEY_ID and AWS_SECRET_KEY to AWS_SECRET_ACCESS_KEY) in order
to please awscli. Also AWS_DEFAULT_REGION is now important to specify,
the default has been set to the region used by get.docker.com and
test.docker.com.
Signed-off-by: Tibor Vass <tibor@docker.com>
Contains fixes for:
- pid.max fix that is causing hang on network stats test.
- fix for early stdin close containerd-shim
- better logging for `could not synchronise with container process`
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Fixes#20550
This update to libseccomp supports the new versions of socket
system calls that can be called directly rather than via the
socketcall syscall in kernel versions 4.3 or later with new glibc.
Note this library version now supports s390x and ppc64le, so
seccomp can be potentially be enabled for these architectures now.
Signed-off-by: Justin Cormack <justin.cormack@docker.com>
Fixes broken-pipe issue when piping s3cmd to grep -q, by removing the -q
flag and redirecting to /dev/null instead.
Add net-tools for ifconfig, because some tests rely on ifconfig.
Harmonize all Dockerfiles in this direction.
Signed-off-by: Tibor Vass <tibor@docker.com>
dockerinit has been around for a very long time. It was originally used
as a way for us to do configuration for LXC containers once the
container had started. LXC is no longer supported, and /.dockerinit has
been dead code for quite a while. This removes all code and references
in code to dockerinit.
Signed-off-by: Aleksa Sarai <asarai@suse.com>
Fixes#19400
Note that this introduces an incompatibility with Docker 1.10-rc1,
because the media type used for schema1 manifests has been corrected in
the upstream distribution code. Docker 1.10-rc1 won't be able to pull
old manifests from Registry 2.3-rc0 and up, but because of this vendor
update, Docker 1.10-rc2 won't have this problem.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This results in a significant time savings during repeated builds (since we don't have to re-download gcc for every test run).
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
Tracks source repository information for each blob in the blobsum
service, which is then used to attempt to mount blobs from another
repository when pushing instead of having to re-push blobs to the same
registry.
Signed-off-by: Brian Bland <brian.bland@docker.com>
