Commit graph

34788 commits

Author SHA1 Message Date
John Howard
72ba7f593f
Merge pull request #36609 from thaJeztah/fix-stats-loop
Fix stats collector spinning CPU if no stats are collected
2018-03-16 10:35:10 -07:00
Vincent Demeester
823de22db6
Merge pull request #36606 from kolyshkin/t-36561
integration/TestExportContainerAfterDaemonRestart: add
2018-03-16 12:07:46 +01:00
Vincent Demeester
bbf568e41c
Merge pull request #36584 from cpuguy83/volume_store_tests
Add some tests to the volume store
2018-03-16 09:12:55 +01:00
John Stephens
d16c77bc01
Merge pull request #36610 from Microsoft/jjh/unbreakxenon
Windows: Hyper-V containers are broken after 36586 was merged
2018-03-15 21:38:15 -07:00
John Howard
0f5fe3f9cf Windows: Fix Hyper-V containers regression from 36586
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-15 15:36:36 -07:00
Sebastiaan van Stijn
481b8e54b4
Fix stats collector spinning CPU if no stats are collected
Commit fd0e24b718 changed
the stats collection loop to use a `sleep()` instead
of `time.Tick()` in the for-loop.

This change caused a regression in situations where
no stats are being collected, or an error is hit
in the loop (in which case the loop would `continue`,
and the `sleep()` is not hit).

This patch puts the sleep at the start of the loop
to guarantee it's always hit.

This will delay the sampling, which is similar to the
behavior before fd0e24b718.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-03-15 17:56:15 +01:00
Kir Kolyshkin
6e7141c7a2 integration/TestExportContainerAfterDaemonRestart: add
This test case checks that a container created before start
of the currently running dockerd can be exported (as reported
in #36561). To satisfy this condition, either a pre-existing
container is required, or a daemon restart after container
creation.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-15 00:30:11 -07:00
Tibor Vass
3d14173a29
Merge pull request #36603 from thaJeztah/fix-stale-hns-endpoints
Update libnetwork to fix stale HNS endpoints on Windows
2018-03-14 21:05:27 -07:00
Sebastiaan van Stijn
ae7016427f
Merge pull request #36586 from kolyshkin/do-not-panic
ExportContainer: do not panic
2018-03-15 00:18:24 +01:00
Sebastiaan van Stijn
518d028baa
Merge pull request #36517 from jim-minter/missing_closewrite
ensure hijackedConn implements CloseWrite function
2018-03-15 00:05:54 +01:00
Sebastiaan van Stijn
fb364f0746
Update libnetwork to fix stale HNS endpoints on Windows
Update libnetwork to 1b91bc94094ecfdae41daa465cc0c8df37dfb3dd to bring in a fix
for stale HNS endpoints on Windows:

When Windows Server 2016 is restarted with the Docker service running, it is
possible for endpoints to be deleted from the libnetwork store without being
deleted from HNS. This does not occur if the Docker service is stopped cleanly
first, or forcibly terminated (since the endpoints still exist in both). This
change works around the issue by removing any stale HNS endpoints for a network
when creating it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-03-14 23:45:58 +01:00
Tibor Vass
bc0b0f11f3
Merge pull request #36589 from thaJeztah/fix-duplicate-ip-issues
Update libnetwork with fixes for duplicate IP addresses
2018-03-14 15:04:41 -07:00
Brian Goff
834d0e262a Add some tests to the volume store
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-03-14 17:17:44 -04:00
Sebastiaan van Stijn
55e0fe24db
Update libnetwork with fixes for duplicate IP addresses
This updates libnetwork to 8892d7537c67232591f1f3af60587e3e77e61d41 to bring in
IPAM fixes for duplicate IP addresses.

- IPAM tests (libnetwork PR 2104) (no changes in vendored files)
- Fix for Duplicate IP issues  (libnetwork PR 2105)

Also bump golang/x/sync to match libnetwork (no code-changes, other
than the README being updated)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-03-14 16:31:27 +01:00
Jim Minter
37983921c9 Ensure a hijacked connection implements CloseWrite whenever its underlying
connection does.  If this isn't done, then a container listening on stdin won't
receive an EOF when the client closes the stream at their end.

Signed-off-by: Jim Minter <jminter@redhat.com>
2018-03-14 09:07:55 -06:00
Sebastiaan van Stijn
592a15b7a9
Merge pull request #36571 from kolyshkin/t-win-fail
integration-cli/TestSlowStdinClosing: increase timeout
2018-03-14 12:06:32 +01:00
Kir Kolyshkin
d6ea46ceda container.BaseFS: check for nil before deref
Commit 7a7357dae1 ("LCOW: Implemented support for docker cp + build")
changed `container.BaseFS` from being a string (that could be empty but
can't lead to nil pointer dereference) to containerfs.ContainerFS,
which could be be `nil` and so nil dereference is at least theoretically
possible, which leads to panic (i.e. engine crashes).

Such a panic can be avoided by carefully analysing the source code in all
the places that dereference a variable, to make the variable can't be nil.
Practically, this analisys are impossible as code is constantly
evolving.

Still, we need to avoid panics and crashes. A good way to do so is to
explicitly check that a variable is non-nil, returning an error
otherwise. Even in case such a check looks absolutely redundant,
further changes to the code might make it useful, and having an
extra check is not a big price to pay to avoid a panic.

This commit adds such checks for all the places where it is not obvious
that container.BaseFS is not nil (which in this case means we do not
call daemon.Mount() a few lines earlier).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-13 21:24:48 -07:00
Kir Kolyshkin
81f6307eda daemon.ContainerExport(): do not panic
In case ContainerExport() is called for an unmounted container, it leads
to a daemon panic as container.BaseFS, which is dereferenced here, is
nil.

To fix, do not rely on container.BaseFS; use the one returned from
rwlayer.Mount().

Fixes: 7a7357dae1 ("LCOW: Implemented support for docker cp + build")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-13 21:24:43 -07:00
Brian Goff
9e1c4f9906
Merge pull request #36577 from cpuguy83/info_tweaks
Minor optimizations
2018-03-13 16:48:45 -04:00
Sebastiaan van Stijn
514fb6cf85
Merge pull request #36538 from ctelfer/ingress-fix
Fix automatic removal of ingress sandbox when last service leaves
2018-03-13 20:56:03 +01:00
Brian Goff
04a0d6b863 Change containerd monitor ticker to sleep
With the ticker this could end up just doing back-to-back checks, which
isn't really what we want here.
Instead use a sleep to ensure we actually sleep for the desired
interval.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-03-13 13:20:19 -04:00
Brian Goff
f6a7763b6f Add Len() to image store for info endpoint
In info, we only need the number of images, but `CountImages` was
getting the whole map of images and then grabbing the length from that.
This causes a lot of unnecessary CPU usage and memory allocations, which
increases with O(n) on the number of images.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-03-13 11:58:05 -04:00
Sebastiaan van Stijn
fb95dc7eac
Merge pull request #36569 from kolyshkin/t-etc-hosts
TestLinksEtcHostsContentMatch: use container.Exec()
2018-03-13 15:12:59 +01:00
Brian Goff
4a65cd4d6d
Merge pull request #36567 from kolyshkin/t-parallel
integration/TestContainerShmNoLeak: use --iptables=false
2018-03-13 08:36:20 -04:00
Kir Kolyshkin
5043639645 integration-cli/TestSlowStdinClosing: increase timeout
I noticed this test failed on Windows:

> 17:46:24 docker_cli_run_test.go:4361:
> 17:46:24 c.Fatal("running container timed out") // cleanup in teardown

I also noticed that in general tests are running slower on Windows,
for example TestStartAttachSilent (which runs a container with
`busybox echo test` and then starts it again) took 29.763s.
This means a simple container start can easily take 15s, which
explains the above failure.

Double the timeout from 15s to 30s.

Fixes: 4e262f6387 ("Fix race on sending stdin close event")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-12 20:35:28 -07:00
Kir Kolyshkin
ad2f88d8cc TestLinksEtcHostsContentMatch: use container.Exec()
I am not quite sure why but this test is sometimes failing like this:

> 15:21:41 --- FAIL: TestLinksEtcHostsContentMatch (0.53s)
> 15:21:41 	assertions.go:226:
>
> 	Error Trace:	links_linux_test.go:46
> 15:21:41
> 	Error:      	Not equal:
> 15:21:41
> 	            	expected: "127.0.0.1\tlocalhost\n::1\tlocalhost
> ip6-localhost
> ip6-loopback\nfe00::0\tip6-localnet\nff00::0\tip6-mcastprefix\nff02::1\tip6-allnodes\nff02::2\tip6-allrouters\n172.17.0.2\tf53feb6df161\n"
> 15:21:41
> 	            	received: ""

To eliminate some possible failures (like ignoring stderr from `cat` or
its exit code), let's use container.Exec() to read a file from a container.

Fixes: e6bd20edcb ("Migrate some integration-cli test to api tests")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-12 18:15:16 -07:00
Kir Kolyshkin
c125e10a04 integration/TestContainerShmNoLeak: use --iptables=false
As mentioned in commit 9e31938, test cases that use t.Parallel()
and start a docker daemon might step on each other toes as they
try to configure iptables during startup, resulting in flaky tests.

To avoid this, --iptables=false should be used while starting daemon.

Fixes: eaa5192856 ("Make container resource mounts unbindable")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-12 14:49:15 -07:00
Chris Telfer
3da4ebf355 Delete the load balancer endpoint in Ingress nets
Ingress networks will no longer automatically remove their
load-balancing endpoint (and sandbox) automatically when the network is
otherwise upopulated.   This is to prevent automatic removal of the
ingress networks when all the containers leave them.  Therefore
explicit removal of an ingress network also requires explicit removal
of its load-balancing endpoint.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-03-12 15:19:08 -04:00
Chris Telfer
bebad150c9 Update vendoring for libnetwork PR #2097
This PR prevents automatic removal of the load balancing sandbox
endpoint when the endpoint is the last one in the network but
the network is marked as ingress.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-03-12 15:19:08 -04:00
Chris Telfer
805b6a7f74 Add test for ingress removal on service removal
The commit https://github.com/moby/moby/pull/35422 had the result of
accidentally causing the removal of the ingress network when the
last member of a service left the network.  This did not appear
in swarm instances because the swarm manager would still maintain
and return cluster state about the network even though it had
removed its sandbox and endpoint.  This test verifies that after a
service gets added and removed that the ingress sandbox remains
in a functional state.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-03-12 15:19:02 -04:00
Sebastiaan van Stijn
241c904e6f
Merge pull request #36523 from yolken-stripe/36521-configurable-logfile-perms
Make LogFile perms configurable
2018-03-12 15:07:23 +01:00
Brian Goff
c74cd60473
Merge pull request #36489 from thaJeztah/fix-errdef-is
Change return for errdefs.getImplementer()
2018-03-12 10:04:41 -04:00
Sebastiaan van Stijn
1790ce52e9
Merge pull request #36546 from arm64b/multi-arch-support4Dockerfiles
Unify arch-specific Dockerfiles into multi-arch one
2018-03-12 12:36:58 +01:00
Dennis Chen
8eb7ed673b Remove arch-suffix detection of Dockerfile
Since now we have only one Dockerfile, so the arch-specific suffix
of the Dockerfile is not needed anymore.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-03-12 02:05:37 +00:00
Dennis Chen
162f9aee47 Arch-specific Dockerfile removal
Removing all the existing arch-specific Dockerfiles since we already
have a new multi-arch supported one as the replacement.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-03-12 02:05:37 +00:00
Dennis Chen
f1701a741d Multiarch support for Dockerfile
This PR consolidates the existing arch-specific Dockerfiles into only
one file `Dockefile` to ease the code maintenance effort.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-03-12 02:05:01 +00:00
Akihiro Suda
a575b0b138
Merge pull request #36550 from dnephin/fix-diff-tests
Remove unnecessary diff tests
2018-03-11 11:26:50 +09:00
Yong Tang
cda90892aa
Merge pull request #36526 from kolyshkin/ipc-ro
daemon/setMounts(): do not make /dev/shm ro
2018-03-10 10:30:22 -08:00
Sebastiaan van Stijn
b9cc5cba69
Merge pull request #36504 from dmcgowan/layer-store-remove-metastore-interface
layer: remove metadata store interface
2018-03-10 12:22:08 +01:00
Yong Tang
623b1a5c3c
Merge pull request #36519 from stevvooe/resilient-cpu-sampling
daemon/stats: more resilient cpu sampling
2018-03-09 14:34:45 -08:00
Daniel Nephin
038f3add51 Remove unnecessary diff tests
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-09 12:32:50 -05:00
Vincent Demeester
c8f9e14b50
Merge pull request #36539 from tophj-ibm/skip-oom-tests-ppc64le
[integration] skip ppc64le oom tests temporarily
2018-03-09 09:53:00 +01:00
Vincent Demeester
a21d5bf669
Merge pull request #36506 from kolyshkin/pkg-mount-slice
pkg/mount: use sort.Slice
2018-03-09 09:46:53 +01:00
Christopher Jones
620ddc78a1
[integration] skip ppc64le oom tests for now
These tests were enabled by changing a config option on the ci
machines, instead of from a patch, so let me disable them
for now on ppc64le and open up another patch to enable them, where I can find
out what the issues are with them.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2018-03-08 17:51:37 -06:00
Kir Kolyshkin
cad74056c0 daemon/setMounts(): do not make /dev/shm ro
It has been pointed out that if --read-only flag is given, /dev/shm
also becomes read-only in case of --ipc private.

This happens because in this case the mount comes from OCI spec
(since commit 7120976d74), and is a regression caused by that
commit.

The meaning of --read-only flag is to only have a "main" container
filesystem read-only, not the auxiliary stuff (that includes /dev/shm,
other mounts and volumes, --tmpfs, /proc, /dev and so on).

So, let's make sure /dev/shm that comes from OCI spec is not made
read-only.

Fixes: 7120976d74 ("Implement none, private, and shareable ipc modes")

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-08 14:04:03 -08:00
Kir Kolyshkin
33dd562e3a daemon/oci_linux_test: add TestIpcPrivateVsReadonly
The test case checks that in case of IpcMode: private and
ReadonlyRootfs: true (as in "docker run --ipc private --read-only")
the resulting /dev/shm mount is NOT made read-only.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-08 14:04:03 -08:00
Vincent Demeester
0c01629e17
Merge pull request #36518 from seemethere/fix_static_builds
Build containerd, runc, and proxy statically
2018-03-08 13:59:34 +01:00
Akihiro Suda
4a1d35c546
Merge pull request #36505 from kolyshkin/pkg-mount-tests
pkg/mount unit tests: skip some test under non-root
2018-03-08 17:27:31 +09:00
Benjamin Yolken
d0c1287a8d
Make logfile perms configurable
Signed-off-by: Benjamin Yolken <yolken@stripe.com>
2018-03-07 15:18:51 -08:00
Eli Uriegas
5e4885b9af buildmod => buildmode
There was a typo with the buildmode flag for containerd

Signed-off-by: Eli Uriegas <eli.uriegas@docker.com>
2018-03-07 21:36:23 +00:00