Commit graph

5904 commits

Author SHA1 Message Date
Derek McGowan
ac5d363e67
Update overlay2 to use naive diff for changes
The archive changes function is not implemented correctly
to handle opaque directories.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-06-20 11:07:36 -07:00
Vincent Demeester
3845728524
Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-13 09:04:30 +02:00
Tonis Tiigi
157b0b30db builder: lint fixes
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-10 10:05:29 -07:00
Tonis Tiigi
ea36c3cbaf daemon: access to distribution internals
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-10 10:05:26 -07:00
Tonis Tiigi
44f3dd7653 vendor: update containerd to 63522d9
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-08 19:19:06 -07:00
Sebastiaan van Stijn
5e11f66cb6
Merge pull request #37172 from zq-david-wang/resizefix2
Fix race condition between exec start and resize.
2018-06-08 15:43:25 -07:00
Sebastiaan van Stijn
5cb95f693d
Merge pull request #37184 from cpuguy83/attach_leak
Fix fd leak on attach
2018-06-08 13:08:06 -07:00
Sebastiaan van Stijn
a7b6643c5f
Merge pull request #36161 from fanjiyun/edit
When link id is empty for overlay2, do not remove this link.
2018-06-08 09:12:22 -07:00
Tõnis Tiigi
4703b284e6
Merge pull request #37233 from dnephin/add-metrics-for-pull
Add image metrics for push and pull
2018-06-08 00:26:12 -07:00
David Wang
e6783656f9 Fix race condition between exec start and resize
Signed-off-by: David Wang <00107082@163.com>
2018-06-08 11:07:48 +08:00
fanjiyun
0e8f96e317
When id is empty for overlay2/overlay, do not remove the directories.
Signed-off-by: fanjiyun <fan.jiyun@zte.com.cn>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-06-07 17:35:54 -07:00
Daniel Nephin
6910019bbe Add image metrics for push and pull
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2018-06-07 18:26:12 -04:00
Vincent Demeester
e401b88e59
Add support for init on services
It's already supported by `swarmkit`, and act the same as
`HostConfig.Init` on container creation.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-07 14:17:55 +02:00
Jess Frazelle
3694c1e34e
api: add configurable MaskedPaths and ReadOnlyPaths to the API
This adds MaskedPaths and ReadOnlyPaths options to HostConfig for containers so
that a user can override the default values.

When the value sent through the API is nil the default is used.
Otherwise the default is overridden.

Adds integration tests for MaskedPaths and ReadonlyPaths.

Signed-off-by: Jess Frazelle <acidburn@microsoft.com>
2018-06-05 12:33:14 -04:00
Sebastiaan van Stijn
5037c5a8ce
Merge pull request #36688 from cpuguy83/volumes_service
Extract volume interaction to a volumes service
2018-06-05 02:16:20 +02:00
Brian Goff
0f51477017 Fix fd leak on attach
With a full attach, each attach was leaking 4 goroutines.
This updates attach to use errgroup instead of the hodge-podge of
waitgroups and channels.

In addition, the detach event was never being sent.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-31 10:11:18 -04:00
Brian Goff
7c77df8acc Move plugin client to separate interface
This makes it a bit simpler to remove this interface for v2 plugins
and not break external projects (libnetwork and swarmkit).

Note that before we remove the `Client()` interface from `CompatPlugin`
libnetwork and swarmkit must be updated to explicitly check for the v1
client interface as is done int his PR.

This is just a minor tweak that I realized is needed after trying to
implement the needed changes on libnetwork.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-30 15:22:10 -04:00
Brian Goff
b85799b63f
Merge pull request #36874 from kolyshkin/stop-timeout
daemon.ContainerStop(): fix for a negative timeout
2018-05-30 13:38:42 -04:00
Anusha Ragunathan
cbbe0ff511
Merge pull request #37017 from cpuguy83/plugins_addr
Move plugin client creation to the extension point
2018-05-30 09:54:46 -07:00
Sebastiaan van Stijn
65bd038fc5
Merge pull request #37131 from kolyshkin/top-opt
Optimize ContainerTop() aka docker top
2018-05-30 03:28:07 +02:00
Brian Goff
f51a96c016 Move plugin client creation to the extension point
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 15:18:53 -04:00
Brian Goff
e4b6adc88e Extract volume interaction to a volumes service
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 14:21:07 -04:00
Kir Kolyshkin
a41328d570 ContainerTop: improve error message
If "ps" fails, in many cases it prints a meaningful error message
which a user can benefit from. Let's use it.

While at it, let's use errdefs.System to classify the error,
as well as errors.Wrap.

Before:

> $ docker top $CT <any bad ps options>
> Error response from daemon: Error running ps: exit status 1

After:

> $ docker top $CT auxm
> Error response from daemon: ps: error: thread display conflicts with forest display

or

> $ docker top $CT saur
> Error response from daemon: ps: error: conflicting format options

or, if there's no meaningful error on stderr, same as before:

> $ docker top $CT 1234
> Error response from daemon: ps: exit status 1

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-24 18:24:36 -07:00
Kir Kolyshkin
a076badb8b ContainerTop: speed up
Current ContainerTop (a.k.a. docker top) implementation uses "ps"
to get the info about *all* running processes, then parses it, then
filters the results to only contain PIDs used by the container.
Collecting data only to throw most of it away is inefficient,
especially on a system running many containers (or processes).
For example, "docker top" on a container with a single process
can take up to 0.5 seconds to execute (on a mostly idle system)
which is noticeably slow.

Since the containers PIDs are known beforehand, let's use ps's
"-q" option to provide it with a list of PIDs we want info about.

The problem with this approach is, some ps options can't be used
with "-q" (the only one I'm aware of is "f" ("forest view") but
there might be more). As the list of such options is not known,
in case ps fails, it is executed again without "q" (retaining
the old behavior).

Next, the data produced by "ps" is filtered in the same way as before.
The difference here is, in case "-q" worked, the list is much shorter.

I ran some benchmarks on my laptop, with about 8000 "sleep" processes
running to amplify the savings.

The improvement in "docker top" execution times is 5x to 10x (roughly
0.05s vs 0.5s).

The improvement in ContainerTop() execution time is up to 100x
(roughly 3ms vs 300ms).

I haven't measured the memory or the CPU time savings, guess those
are not that critical.

NOTE that busybox ps does not implement -q so the fallback is always
used, but AFAIK it is not usable anyway and Docker expects a normal
ps to be on the system (say the list of fields produced by
"busybox ps -ef" differs from normal "ps -ef" etc.).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-24 18:24:27 -07:00
Sebastiaan van Stijn
80bfcc3a7a
Merge pull request #37136 from muesli/sdnotify-api-const
Use go-systemd const instead of magic string in Linux version of dockerd
2018-05-24 14:30:28 +02:00
Sebastiaan van Stijn
8e2f920306
Merge pull request #37108 from thaJeztah/remove_unneeded_aliases
Various code-cleanup
2018-05-24 01:38:56 +02:00
Christian Muehlhaeuser
703c3c5735
Adapt listeners to upstream API changes in go-systemd
Signed-off-by: Christian Muehlhaeuser <muesli@gmail.com>
2018-05-23 21:57:30 +02:00
Sebastiaan van Stijn
f23c00d870
Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-23 17:50:54 +02:00
Kir Kolyshkin
654a7625fc daemon/parsePsOutput: minor optimisation
It does not make sense to keep looking for PID once
we found it, so let's give it a break.

The side effect of this patch is, if there's more than one column
titled "PID", the last (rightmost) column was used before, and now
the first (leftmost) column is used. Should make no practical
difference whatsoever.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-22 16:24:13 -07:00
Drew Erny
5b69ff466e
Output network attachment task information
Adds functionality to parse and return network attachment spec
information. Network attachment tasks are phony tasks created in
swarmkit to deal with unmanaged containers attached to swarmkit. Before
this change, attempting `docker inspect` on the task id of a network
attachment task would result in an empty task object. After this change,
a full task object is returned

Fixes #26548 the correct way.

Signed-off-by: Drew Erny <drew.erny@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-22 23:36:30 +02:00
Kir Kolyshkin
16670ed484 daemon unit tests: skip some if non-root
This prevents the following test case failures "go test" is run
as non-root in the daemon/ directory:

> --- FAIL: TestContainerInitDNS (0.02s)
> 	daemon_test.go:209: chown /tmp/docker-container-test-054812199/volumes: operation not permitted
>
> --- FAIL: TestDaemonReloadNetworkDiagnosticPort (0.00s)
>	reload_test.go:525: mkdir /var/lib/docker/network/files/: permission denied
> --- FAIL: TestRootMountCleanup (0.00s)
> 	daemon_linux_test.go:240: assertion failed: error is not nil: operation not permitted

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-22 13:12:29 -07:00
Yong Tang
9bd5d9912f
Merge pull request #37110 from thaJeztah/remove-testutil-errorcontains
Replace deprecated testutil.ErrorContains()
2018-05-21 11:04:45 -07:00
Sebastiaan van Stijn
55bebbaecf
Replace deprecated testutil.ErrorContains()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-21 00:13:04 +02:00
Kir Kolyshkin
91f85d1c78
aufs: log reason why aufs is not supported.
In case aufs driver is not supported because supportsAufs() said so,
it is not possible to get a real reason from the logs.

To fix, log the error returned.

Note we're not using WithError here as the error message itself is the
sole message we want to print (i.e. there's nothing to add to it).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-20 12:02:41 +02:00
Kir Kolyshkin
c6e2af5425
aufs: use a single logger
Simplify the code by using a single logger instance.

While at it, use WithError in Umount.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-20 12:02:12 +02:00
Sebastiaan van Stijn
f9dd74deee
Merge pull request #37022 from cgxu519/overlayutils-addmsg
Add additional message when backendfs is extfs without d_type support
2018-05-19 22:38:45 +02:00
Sebastiaan van Stijn
3646562a5c
Merge pull request #37088 from ohbarye/fix-typos-duplicated-the
Fix typos: remove duplicated "the"
2018-05-19 20:57:47 +02:00
Tibor Vass
1b75e17584
Merge pull request #37091 from thaJeztah/fix-linting-errors
Fix some linting issues
2018-05-19 08:39:39 -10:00
Chengguang Xu
8a21b128d4 Add additional message when backendfs is extfs without d_type support
ext4 support d_type by default, but filetype feature is a tunable so
there is still a chance to disable it for some reasons. In this case,
print additional message to explicitly tell how to support d_type.

Signed-off-by: Chengguang Xu <cgxu519@gmx.com>
2018-05-18 10:32:47 +08:00
Anusha Ragunathan
59698778fe
Merge pull request #37084 from nwneisen/update-ring-buffer-documents
Update documentation for RingLogger's ring buffer
2018-05-17 12:10:44 -07:00
Sebastiaan van Stijn
4f8c870d62
Fix some linting issues
These showed locally when running `make validate`. CI doesn't seem to have the
same (possibly it's disabled in the configuration)

    builder/fscache/fscache.go:618::error: github.com/docker/docker/vendor/github.com/tonistiigi/fsutil.StatInfo composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:44::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:20::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:113::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:110::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:171::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:413::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:203::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:584::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:109::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:388::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/volumes_windows.go:27::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    integration/service/network_test.go:31::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    api/server/server.go:129:10⚠️ should not use basic type string as key in context.WithValue (golint)
    integration/service/network_test.go:54::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:61::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:74::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    pkg/archive/archive_windows.go:76::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    plugin/manager_linux.go:56::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-17 19:28:27 +02:00
Vincent Demeester
9aac187ce1
Merge pull request #37075 from tuscloud/fix-typos
Fix typos
2018-05-17 16:00:43 +02:00
ohbarye
0f95b23d98 Fix typos: remove duplicated "the"
Signed-off-by: Masato Ohba <over.rye@gmail.com>
2018-05-17 21:49:51 +09:00
Nick Neisen
3521d534e5 Update documentation for RingLogger's ring buffer
Signed-off-by: Nick Neisen <nwneisen@gmail.com>
2018-05-16 18:52:50 -06:00
weipeng
f781ec45b5 Fix typos
Signed-off-by: weipeng <weipeng@tuscloud.io>
2018-05-16 09:15:43 +08:00
Sebastiaan van Stijn
6821ffd1a1
Merge pull request #37028 from cpuguy83/log_plugin_broken_pipe
Fix logging plugin crash unrecoverable
2018-05-15 13:57:01 +02:00
Sebastiaan van Stijn
6f68db8c7a
Merge pull request #37033 from cpuguy83/some_container_network_cleanups
Some container/network cleanups
2018-05-15 13:23:41 +02:00
Sebastiaan van Stijn
07dc3aa685
Merge pull request #37062 from cpuguy83/fix_logs_daemon_race
Fix race conditions in logs API
2018-05-15 11:59:40 +02:00
Vincent Demeester
26da99e326
Merge pull request #37063 from cpuguy83/fix_log_rotation_issue
Fix some issues in logfile reader and rotation
2018-05-15 11:46:29 +02:00
Vincent Demeester
bcd86fc2ac
Merge pull request #37060 from cpuguy83/fix_swagger_gen
Fix swagger volume type generation
2018-05-15 11:46:02 +02:00