Commit graph

22533 commits

Author SHA1 Message Date
Brian Goff
c2f7777603 Revert "Add finer-grained locking for aufs"
This reverts commit f31014197c.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-27 08:01:19 -05:00
David Calavera
df2b74188e Merge pull request #20699 from calavera/remove_static_error_declarations
Remove static errors from errors package.
2016-02-26 16:30:12 -08:00
David Calavera
f666d918fc Merge pull request #20672 from justincormack/personality
Add some uses of personality syscall to default seccomp filter
2016-02-26 14:27:23 -08:00
Tianon Gravi
5bfaab984c Merge pull request #20636 from anusha-ragunathan/apt-ftparchive
Always create apt-ftparchive.conf.
2016-02-26 14:12:06 -08:00
Alexander Morozov
2f797bb1d9 Merge pull request #20275 from cpuguy83/finer_graph_locks
Finer graph locks
2016-02-26 13:33:34 -08:00
Alexander Morozov
51302c29ed Merge pull request #20729 from estesp/pipework
Add synchronization and closure to IO pipes in userns path
2016-02-26 13:33:02 -08:00
David Calavera
a793564b25 Remove static errors from errors package.
Moving all strings to the errors package wasn't a good idea after all.

Our custom implementation of Go errors predates everything that's nice
and good about working with errors in Go. Take as an example what we
have to do to get an error message:

```go
func GetErrorMessage(err error) string {
	switch err.(type) {
	case errcode.Error:
		e, _ := err.(errcode.Error)
		return e.Message

	case errcode.ErrorCode:
		ec, _ := err.(errcode.ErrorCode)
		return ec.Message()

	default:
		return err.Error()
	}
}
```

This goes against every good practice for Go development. The language already provides a simple, intuitive and standard way to get error messages, that is calling the `Error()` method from an error. Reinventing the error interface is a mistake.

Our custom implementation also makes very hard to reason about errors, another nice thing about Go. I found several (>10) error declarations that we don't use anywhere. This is a clear sign about how little we know about the errors we return. I also found several error usages where the number of arguments was different than the parameters declared in the error, another clear example of how difficult is to reason about errors.

Moreover, our custom implementation didn't really make easier for people to return custom HTTP status code depending on the errors. Again, it's hard to reason about when to set custom codes and how. Take an example what we have to do to extract the message and status code from an error before returning a response from the API:

```go
	switch err.(type) {
	case errcode.ErrorCode:
		daError, _ := err.(errcode.ErrorCode)
		statusCode = daError.Descriptor().HTTPStatusCode
		errMsg = daError.Message()

	case errcode.Error:
		// For reference, if you're looking for a particular error
		// then you can do something like :
		//   import ( derr "github.com/docker/docker/errors" )
		//   if daError.ErrorCode() == derr.ErrorCodeNoSuchContainer { ... }

		daError, _ := err.(errcode.Error)
		statusCode = daError.ErrorCode().Descriptor().HTTPStatusCode
		errMsg = daError.Message

	default:
		// This part of will be removed once we've
		// converted everything over to use the errcode package

		// FIXME: this is brittle and should not be necessary.
		// If we need to differentiate between different possible error types,
		// we should create appropriate error types with clearly defined meaning
		errStr := strings.ToLower(err.Error())
		for keyword, status := range map[string]int{
			"not found":             http.StatusNotFound,
			"no such":               http.StatusNotFound,
			"bad parameter":         http.StatusBadRequest,
			"conflict":              http.StatusConflict,
			"impossible":            http.StatusNotAcceptable,
			"wrong login/password":  http.StatusUnauthorized,
			"hasn't been activated": http.StatusForbidden,
		} {
			if strings.Contains(errStr, keyword) {
				statusCode = status
				break
			}
		}
	}
```

You can notice two things in that code:

1. We have to explain how errors work, because our implementation goes against how easy to use Go errors are.
2. At no moment we arrived to remove that `switch` statement that was the original reason to use our custom implementation.

This change removes all our status errors from the errors package and puts them back in their specific contexts.
IT puts the messages back with their contexts. That way, we know right away when errors used and how to generate their messages.
It uses custom interfaces to reason about errors. Errors that need to response with a custom status code MUST implementent this simple interface:

```go
type errorWithStatus interface {
	HTTPErrorStatusCode() int
}
```

This interface is very straightforward to implement. It also preserves Go errors real behavior, getting the message is as simple as using the `Error()` method.

I included helper functions to generate errors that use custom status code in `errors/errors.go`.

By doing this, we remove the hard dependency we have eeverywhere to our custom errors package. Yes, you can use it as a helper to generate error, but it's still very easy to generate errors without it.

Please, read this fantastic blog post about errors in Go: http://dave.cheney.net/2014/12/24/inspecting-errors

Signed-off-by: David Calavera <david.calavera@gmail.com>
2016-02-26 15:49:09 -05:00
Vincent Demeester
b2a15a2226 Merge pull request #20725 from runcom/fix-subid-files-parsing
pkg: idtools: fix subid files parsing
2016-02-26 21:28:08 +01:00
Phil Estes
995386735c Add synchronization and closure to IO pipes in userns path
The execdriver pipes setup uses OS pipes with fds so that they can be
chown'ed to the remapped root user for proper access. Recent flakiness
in certain short-lived tests (usually via the "exec" path) reveals that
the copy routines are not completing before exit/tear-down.

This fix adds synchronization and proper closure such that these
routines exit successfully.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2016-02-26 13:47:34 -05:00
Justin Cormack
39b799ac53 Add some uses of personality syscall to default seccomp filter
We generally want to filter the personality(2) syscall, as it
allows disabling ASLR, and turning on some poorly supported
emulations that have been the target of CVEs. However the use
cases for reading the current value, setting the default
PER_LINUX personality, and setting PER_LINUX32 for 32 bit
emulation are fine.

See issue #20634

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2016-02-26 18:43:08 +01:00
David Calavera
e330d0749c Merge pull request #20703 from riyazdf/notary-v0.2.0-vendor
Vendor in notary v0.2.0
2016-02-26 08:53:07 -08:00
Brian Goff
c47674efda Merge pull request #20428 from jfrazelle/generate-conversion
generate seccomp profile convert type
2016-02-26 10:28:23 -05:00
Antonio Murdaca
bf04d68db2 pkg: idtools: fix subid files parsing
Since Docker is already skipping newlines in /etc/sub{uid,gid},
this patch skips commented out lines - otherwise Docker fails to start.
Add unit test also.

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-02-26 15:42:05 +01:00
Sebastiaan van Stijn
9792308b84 Merge pull request #20515 from raesene/patch-1
Update security.md with basic User Namespace info.
2016-02-26 14:02:46 +01:00
Sebastiaan van Stijn
d622494c3b Merge pull request #20722 from thaJeztah/remove-vivid-reference
remove leftover Ubuntu 15.04 from install docs
2016-02-26 13:59:36 +01:00
Sebastiaan van Stijn
1ca064cb62 remove leftover Ubuntu 15.04 from install docs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2016-02-26 13:00:27 +01:00
Riyaz Faizullabhoy
0bb1acee37 bumping miekg/pkcs11 dependency for go1.6
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-02-25 21:29:37 -08:00
Brian Goff
6748cc10ac Merge pull request #20655 from hqhq/hq_fix_update_memoryswap
Fix problems when update swap memory
2016-02-25 22:28:53 -05:00
David Calavera
6c35350bab Merge pull request #20263 from Microsoft/jjh/testunit-fileutils
Windows CI: Fixes panic in test-unit for FileUtils
2016-02-25 17:35:32 -08:00
David Calavera
dd53ab14e4 Merge pull request #20481 from HackToday/addcheckfd
Add check for non-systemd fd use case
2016-02-25 16:38:53 -08:00
David Calavera
d8b6e62f50 Merge pull request #20580 from BrianBland/crossRepoPushRetry
Improve auth fallback behavior for cross-repository push
2016-02-25 16:37:04 -08:00
Sebastiaan van Stijn
5cb4693300 Merge pull request #20673 from Microsoft/jjh/testkill
Windows CI: Port TestKill*
2016-02-26 01:11:51 +01:00
Brian Goff
f31014197c Add finer-grained locking for aufs
```
benchmark                       old ns/op       new ns/op     delta
BenchmarkConcurrentAccess-8     10269529748     26834747      -99.74%

benchmark                       old allocs     new allocs     delta
BenchmarkConcurrentAccess-8     309948         7232           -97.67%

benchmark                       old bytes     new bytes     delta
BenchmarkConcurrentAccess-8     23943576      1578441       -93.41%
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-25 18:06:41 -05:00
Riyaz Faizullabhoy
84dc2d9e70 Vendor in notary v0.2.0
Signed-off-by: Riyaz Faizullabhoy <riyaz.faizullabhoy@docker.com>
2016-02-25 13:40:00 -08:00
Tibor Vass
6fa5576e30 Merge pull request #20697 from tiborvass/tls-remote-daemon-tests
Support TLS remote test daemon
2016-02-25 16:16:40 -05:00
Tibor Vass
2b819b76df Merge pull request #20663 from calavera/standalone_middlewares
Make server middleware standalone functions.
2016-02-25 15:01:07 -05:00
Brian Goff
55c91f2ab9 Fix some issues with concurrency in aufs.
Adds a benchmark to measure performance under concurrent actions.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-25 14:32:13 -05:00
Tibor Vass
f4a1e3db99 Support TLS remote test daemon
This will allow us to have a windows-to-linux CI, where the linux host
can be anywhere, connecting with TLS.

Signed-off-by: Tibor Vass <tibor@docker.com>
2016-02-25 14:12:17 -05:00
Michael Crosby
6582013207 Merge pull request #20633 from crosbymichael/unit-file
Add "Delegate=yes" to docker's service file
2016-02-25 10:47:46 -08:00
Michael Crosby
d16737f971 Add "Delegate=yes" to docker's service file
We need to add delegate yes to docker's service file so that it can
manage the cgroups of the processes that it launches without systemd
interfering with them and moving the processes after it is reloaded.

```
       Delegate=
           Turns on delegation of further resource control partitioning to
           processes of the unit. For unprivileged services (i.e. those
           using the User= setting), this allows processes to create a
           subhierarchy beneath its control group path. For privileged
           services and scopes, this ensures the processes will have all
           control group controllers enabled.
```

This is the proper fix for issue #20152

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-02-25 10:32:09 -08:00
Alexander Morozov
60e2d5e0b0 Merge pull request #20681 from icecrime/aaronl_maintainer
Add @aaronlehmann to maintainers
2016-02-25 09:18:28 -08:00
Vincent Demeester
61d24e769d Merge pull request #20572 from runcom/sudo-user
resolve the config file from the sudo user
2016-02-25 16:05:25 +01:00
Antonio Murdaca
e4a6a889be Merge pull request #20695 from vdemeester/fix-windowsTP4-check
Fix TestExecApiStartWithDetach on WindowsTP4
2016-02-25 16:04:23 +01:00
Vincent Demeester
21c8511123 Fix TestExecApiStartWithDetach on WindowsTP4
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2016-02-25 14:27:22 +01:00
Vincent Demeester
41f2a674b8 Merge pull request #20691 from vincentbernat/doc/simplify-overlay
docs: simplify some steps of the overlay network guide
2016-02-25 13:31:13 +01:00
Vincent Bernat
db5ded0dfc docs: simplify some steps of the overlay network guide
Instead of using a process expansion to feed the right arguments to
docker to run on "mh-keystore", just moves up the next step which makes
"mh-keystore" the default target. This makes the guide a bit shorter and
easier to understand.

Signed-off-by: Vincent Bernat <vincent@bernat.im>
2016-02-25 13:17:26 +01:00
Sebastiaan van Stijn
7cf03700f8 Merge pull request #20679 from Microsoft/jjh/testrestart
Windows CI: Port docker_cli_restart_test.go
2016-02-25 10:17:25 +01:00
Vincent Demeester
13b6733ee8 Merge pull request #20685 from estesp/userns-dev-fuse-fix
Filter auto-created device list if user namespaces enabled
2016-02-25 10:01:13 +01:00
Vincent Demeester
4c014296a1 Merge pull request #20687 from CWSpear/patch-1
Add the Local Persist plugin to docs
2016-02-25 09:39:07 +01:00
Sebastiaan van Stijn
ca7ce05e81 Merge pull request #20677 from Microsoft/jjh/testclilogs
Windows CI: Port docker_cli_logs_test.go
2016-02-25 09:38:23 +01:00
Vincent Demeester
0ad04242b4 Merge pull request #20682 from cpuguy83/fix_volplugin_panics
Fix panic when plugin responds with null volume
2016-02-25 09:32:54 +01:00
Vincent Demeester
9f1ad7850c Merge pull request #20674 from Microsoft/jjh/testlogsapi
Windows CI: Port TestLogsAPI*
2016-02-25 09:11:15 +01:00
Vincent Demeester
a13945dd16 Merge pull request #20680 from cpuguy83/close_plugin_req_body_on_error
Close resp body on plugin call error
2016-02-25 09:10:17 +01:00
Vincent Demeester
abfc480f4b Merge pull request #20664 from Microsoft/jjh/busyboxtop
Windows CI: Integrity check for busybox top
2016-02-25 09:09:37 +01:00
Vincent Demeester
3f7c078aab Merge pull request #20683 from zhangjianfnst/fix-flaws-in-man
Fix some flaws in man.
2016-02-25 09:01:32 +01:00
Vincent Demeester
46c0c5e86b Merge pull request #20671 from nakulpathak3/readme-extension
Add .md extension to readme for markdown rendering
2016-02-25 08:40:00 +01:00
Cameron Spear
cc085be7cc Add the Local Persist plugin
I wasn't 100% sure if it was appropriate to add plugin here, but @thaJeztah invited me to do so, so here it is!

(see https://github.com/CWSpear/local-persist/issues/17#issuecomment-188523784)

Signed-off-by: Cameron Spear <cameronspear@gmail.com>
2016-02-24 23:29:58 -08:00
Phil Estes
9a554e8c37 Filter auto-created device list if user namespaces enabled
Because devices will be bind-mounted instead of using `mknod`, we need
to make sure the source exists and filter the list by only those whose
source is a valid path/current device entry.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2016-02-25 00:11:36 -05:00
John Howard
10bd587d77 Windows CI: Port docker_cli_logs_test.go
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-02-24 20:19:48 -08:00
David Calavera
cee4ff1c4a Merge pull request #20647 from coolljt0725/fix_20638
Fix exec start api with detach and AttachStdin at same time. fixes #2…
2016-02-24 20:17:15 -08:00