Commit graph

34276 commits

Author SHA1 Message Date
John Howard
afd305c4b5 LCOW: Refactor to multiple layer-stores based on feedback
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-18 08:31:05 -08:00
John Howard
ce8e529e18 LCOW: Re-coalesce stores
Signed-off-by: John Howard <jhoward@microsoft.com>

The re-coalesces the daemon stores which were split as part of the
original LCOW implementation.

This is part of the work discussed in https://github.com/moby/moby/issues/34617,
in particular see the document linked to in that issue.
2018-01-18 08:29:19 -08:00
Brian Goff
6feae06003
Merge pull request #35960 from abhi/service
Disable service on release network
2018-01-18 11:19:47 -05:00
Abhinandan Prativadi
dad093cc34 libnetwork vendor
Signed-off-by: abhi <abhi@docker.com>
2018-01-17 14:20:04 -08:00
Abhinandan Prativadi
a042e5a20a Disable service on release network
This PR contains a fix for moby/moby#30321. There was a moby/moby#31142
PR intending to fix the issue by adding a delay between disabling the
service in the cluster and the shutdown of the tasks. However
disabling the service was not deleting the service info in the cluster.
Added a fix to delete service info from cluster and verified using siege
to ensure there is zero downtime on rolling update of a service.In order
to support it and ensure consitency of enabling and disable service knob
from the daemon, we need to ensure we disable service when we release
the network from the container. This helps in making the enable and
disable service less racy. The corresponding part of libnetwork fix is
part of docker/libnetwork#1824

Signed-off-by: abhi <abhi@docker.com>
2018-01-17 14:19:51 -08:00
Yong Tang
ef3988a81f
Merge pull request #36043 from Microsoft/jjh/fixopenfilecall
LCOW: Fix OpenFile parameters
2018-01-17 12:03:11 -08:00
unclejack
871afbb304
Merge pull request #35697 from sargun/use-pgzip
Make image (layer) downloads faster by using pigz
2018-01-17 11:18:20 -08:00
John Howard
141b9a7471 LCOW: Fix OpenFile parameters
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-17 07:58:18 -08:00
Sebastiaan van Stijn
f0b0f2038d
Merge pull request #35441 from cpuguy83/plugin_timeout
Add timeouts for volume plugin ops
2018-01-17 14:49:41 +01:00
Yong Tang
1338fcf58a
Merge pull request #36033 from cpuguy83/improve_zfs_debug_message
Improve zfs init log message for zfs
2018-01-16 21:03:53 -08:00
Akihiro Suda
991d64eeb3
Merge pull request #36036 from kolyshkin/plugins
daemon.cleanupMetricsPlugins(): fix
2018-01-17 13:54:46 +09:00
Yong Tang
d84b8802b9
Merge pull request #36034 from dnephin/update-gty
update gotestyourself vendor
2018-01-16 19:42:54 -08:00
Akihiro Suda
5488cce5d2
Merge pull request #36035 from kolyshkin/gometalinter
gometalinter: fix --deadline option
2018-01-17 12:04:01 +09:00
Drew Hubl
27b002f4a0 Improve zfs init log message for zfs
Signed-off-by: Drew Hubl <drew.hubl@gmail.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-16 21:42:05 -05:00
Brian Goff
b15f8d2d4f Add timeouts for volume plugin ops
This protects the daemon from volume plugins that are slow or
deadlocked.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-16 20:30:49 -05:00
Kir Kolyshkin
9db2c62488 daemon.cleanupMetricsPlugins(): fix
A linter (vet) found the following bug in the code:

> daemon/metrics.go:124::error: range variable p captured by func literal (vet)

Here a variable p is used in an async fashion by goroutine, and most
probably by the time of use it is set to the last element of a range.

For example, the following code

```go
	for _, c := range []string{"here ", "we ", "go"} {
		go func() {
			fmt.Print(c)
		}()
	}
```

will print `gogogo` rather than `here we go` as one would expect.

Fixes: 0e8e8f0f31 ("Add support for metrics plugins")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-01-16 15:15:11 -08:00
Daniel Nephin
4ac4b690f7 update gotestyourself
pickup changes which use t.Helper()

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-01-16 17:43:47 -05:00
Kir Kolyshkin
c11508a5f4 gometalinter: fix --deadline option
1. Add = between the option and the argument, otherwise the argument
   appears to be passed on to the linters directly, as in:

> DEBUG: [golint.8]: executing /home/kir/go/bin/golint
> -min_confidence 0.800000 ./10m ./api ./api/errdefs <...>

2. Fix setting the default for GOMETALINTER_OPTS -- the default
   was -deadline (rather than --deadline).

Fixes: b96093fa56 ("gometalinter: add per-platform configurable options")

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-01-16 14:37:11 -08:00
Sargun Dhillon
fd35494a25 Make image (layer) downloads faster by using pigz
The Golang built-in gzip library is serialized, and fairly slow
at decompressing. It also only decompresses on demand, versus
pipelining decompression.

This change switches to using the pigz external command
for gzip decompression, as opposed to using the built-in
golang one. This code is not vendored, but will be used
if it autodetected as part of the OS.

This also switches to using context, versus a manually
managed channel to manage cancellations, and synchronization.
There is a little bit of weirdness around manually having
to cancel in the error cases.

Signed-off-by: Sargun Dhillon <sargun@sargun.me>
2018-01-16 10:49:18 -08:00
Vincent Demeester
be14665210
Merge pull request #36021 from yongtang/30897-follow-up
Rename FindUniqueNetwork to FindNetwork
2018-01-16 09:38:16 +01:00
Sebastiaan van Stijn
3c7990fb63
Merge pull request #35983 from yongtang/35980-Dockerfile.simple
Add required pkg-config for Dockerfile.simple
2018-01-16 02:41:36 +01:00
Sebastiaan van Stijn
410e95d368
Merge pull request #36025 from yongtang/01152018-golint
Fix golint error
2018-01-16 02:05:42 +01:00
Yong Tang
e02a3d9f5b Fix golint error
PR #36011 fixed almost all of the golint issues though
there is still one golint error:
https://goreportcard.com/report/github.com/docker/docker#golint
```
Golint is a linter for Go source code.
docker/daemon/reload.go
Line 64: warning: redundant if ...; err != nil check, just return error instead. (golint)
```

This fix fixes the last one.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-01-15 21:35:30 +00:00
Yong Tang
8c31aacebf
Merge pull request #36018 from thaJeztah/remove-deprecated-testenv-functions
Remove deprecated integration-cli/environment functions
2018-01-15 13:24:52 -08:00
Yong Tang
c36274da83
Merge pull request #35638 from cpuguy83/error_helpers2
Add helpers to create errdef errors
2018-01-15 10:56:46 -08:00
Yong Tang
ccc2ed0189 Rename FindUniqueNetwork to FindNetwork
This fix is a follow up to 30397, with `FindUniqueNetwork`
changed to `FindNetwork` based on the review feedback.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-01-15 17:34:40 +00:00
Yong Tang
a44274f1c0
Merge pull request #36011 from thaJeztah/golint-fixes
Golint and Ineffassign  fixes
2018-01-15 08:59:58 -08:00
Sebastiaan van Stijn
18a771a761
Remove deprecated environment.DaemonPlatform()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 15:32:06 +01:00
Sebastiaan van Stijn
dfedc9ef62
Remove deprecated environment.MinimalBaseImage()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 15:31:02 +01:00
Sebastiaan van Stijn
da8032d7d3
Remove deprecated environment.ExperimentalDaemon()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 15:30:05 +01:00
Sebastiaan van Stijn
142b1f8bfb
Remove deprecated environment.DockerBasePath()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 15:29:26 +01:00
Sebastiaan van Stijn
dfac74a9e4
Golint: don't use basic untyped string for context key
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 12:49:14 +01:00
Sebastiaan van Stijn
947eb283dc
Merge pull request #35929 from arm64b/busybox-ls-compatible
Upgrade the frozen images to multi-arch ones
2018-01-15 12:28:13 +01:00
Sebastiaan van Stijn
fb13aeef0f
Merge pull request #36009 from ripcurld0/small_nitpick
Small nitpick: create ErrVolumeTargetIsRoot in the volume package
2018-01-15 12:19:23 +01:00
Sebastiaan van Stijn
b4a6313969
Golint: remove redundant ifs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 00:42:25 +01:00
Boaz Shuster
62143af543 Small nitpick: create ErrVolumeTargetIsRoot in the volume package
Both lcow_parser.go and linux_parser.go are duplicating the error:
"invalid specification: destination can't be '/'"

This commit creates a new error called "ErrVolumeTargetIsRoot"
that is used by both linux_parser and lcow_parser and remove
the duplication in the code.

Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
2018-01-14 11:41:39 +00:00
Sebastiaan van Stijn
63a0e88e8a
ineffassign: fix unused variables
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-13 13:35:10 +01:00
Sebastiaan van Stijn
6415f1dcf5
Merge pull request #36000 from emil2k/fix-container-log-err
Wrap response error for container logs method.
2018-01-13 03:16:38 +01:00
Sebastiaan van Stijn
86cb53e800
Merge pull request #35998 from arm64b/fix-timeout-multi-services
Fix timeout issue of  multi-services creation on AArch64
2018-01-13 03:14:31 +01:00
Sebastiaan van Stijn
bc183a0062
Merge pull request #36001 from tophj-ibm/frozen-z-ci
[ci] use alternate sh comparison
2018-01-13 01:00:21 +01:00
Christopher Jones
24da8a0ed4
[ci] use alternate bash comparison
The pattern `echo str | grep -qE pattern` likes to fail on the z CI here for
an unknown reason. Use `grep -qE pattern <<< str` instead.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2018-01-12 11:05:20 -05:00
Emil Davtyan
2d6fb87ca4 Wrap response error for container logs method.
Signed-off-by: Emil Davtyan <emil2k@gmail.com>
2018-01-12 16:43:51 +01:00
Dennis Chen
4542016cbe Fix timeout issue of multi-services cration on AArch64
Now we only adjust the timeout value for `arm` while not `arm64`,
actually the avarage duration for this test is about 25s to crate
multiple services on arm64, else the integration test will terminate
with below error:

> --- FAIL: TestCreateServiceMultipleTimes (24.11s)
>         daemon.go:285: [ddc3c7c1476c2] waiting for daemon to start
>         daemon.go:317: [ddc3c7c1476c2] daemon started
>         poll.go:121: timeout hit after 10s: task count at 4 waiting for 0
>         daemon.go:275: [ddc3c7c1476c2] exiting daemon
>         clean.go:108: Removing image sha256:e6a8d12d58602a19277ee5632b7ff9fa56a4ea52ba00eedf1d3f6f5a495fe761
>         clean.go:108: Removing image sha256:876244cc2ecb8fe1b0b2e817e3b78709a2a735edb093bc6849f99aa6c18f3a01

This PR adjusts the timeout value for both `arm64` and `arm` to mitigate
this issue on those 2 platforms.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-01-12 11:52:35 +00:00
Brian Goff
d453fe35b9 Move api/errdefs to errdefs
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-11 21:21:43 -05:00
Brian Goff
87a12421a9 Add helpers to create errdef errors
Instead of having to create a bunch of custom error types that are doing
nothing but wrapping another error in sub-packages, use a common helper
to create errors of the requested type.

e.g. instead of re-implementing this over and over:

```go
type notFoundError struct {
  cause error
}

func(e notFoundError) Error() string {
  return e.cause.Error()
}

func(e notFoundError) NotFound() {}

func(e notFoundError) Cause() error {
  return e.cause
}
```

Packages can instead just do:

```
  errdefs.NotFound(err)
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-11 21:21:43 -05:00
Yong Tang
a018046ad0 Add required pkg-config for Dockerfile.simple
This fix tries to address the issue raised in 35980 where
pkg-config was missing and was causing Dockerfile.simple build
to fail.

```
$ docker build -t docker:simple -f Dockerfile.simple .
..........
CGO_ENABLED=1 go build  -tags "seccomp apparmor selinux netgo cgo static_build" -installsuffix netgo -ldflags "-w -extldflags -static -X main.gitCommit="b2567b37d7b75eb4cf325b77297b140ea686ce8f" -X main.version=1.0.0-rc4+dev " -o runc .
pkg-config: exec: "pkg-config": executable file not found in $PATH
make: *** [static] Error 2
Makefile:42: recipe for target 'static' failed
The command '/bin/sh -c /tmp/install-binaries.sh runc containerd tini proxy dockercli' returned a non-zero code: 2
```

This fix fixes 35980.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-01-11 20:12:16 +00:00
Brian Goff
92309e34e4
Merge pull request #35982 from cpuguy83/fix_concurrent_daemon_test_issues
Fix race with concurrent daemon startup in tests
2018-01-11 14:26:35 -05:00
Brian Goff
9e3193810d Fix race with concurrent daemon startup in tests
Using parallel tests is nice, however it can cause an issue with
multiple daemons trying to make changes to iptables at the same time
which causes flakey tests.

This just disables iptables for the set of tests since it is not
required.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-11 11:47:21 -05:00
Yong Tang
cc79c6c7da
Merge pull request #35976 from thaJeztah/bump-libnetwork2
Bump libnetwork to a1dfea384b39779552a3b4837ea9303194950976
2018-01-10 22:56:00 -08:00
Dennis Chen
ec6659a121 busybox: use '-v' option for 'cat'
Use `cat -v` command instead of `catv` for the latest version of
busybox(V1.28.0) with multi-arch

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-01-11 06:35:17 +00:00