Commit graph

2792 commits

Author SHA1 Message Date
Sebastiaan van Stijn
4f8c870d62
Fix some linting issues
These showed locally when running `make validate`. CI doesn't seem to have the
same (possibly it's disabled in the configuration)

    builder/fscache/fscache.go:618::error: github.com/docker/docker/vendor/github.com/tonistiigi/fsutil.StatInfo composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:44::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:20::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:113::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:110::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:171::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:413::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:203::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:584::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:109::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:388::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/volumes_windows.go:27::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    integration/service/network_test.go:31::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    api/server/server.go:129:10⚠️ should not use basic type string as key in context.WithValue (golint)
    integration/service/network_test.go:54::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:61::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:74::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    pkg/archive/archive_windows.go:76::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    plugin/manager_linux.go:56::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-17 19:28:27 +02:00
Vincent Demeester
6a9bcc51df
Merge pull request #37074 from skyc024/fix-typos
Fix typos
2018-05-17 16:00:26 +02:00
ohbarye
0f95b23d98 Fix typos: remove duplicated "the"
Signed-off-by: Masato Ohba <over.rye@gmail.com>
2018-05-17 21:49:51 +09:00
chenyuzhu
5ac0d2f0b8 Fix typos
Signed-off-by: chenyuzhu <chenyuzhi@oschina.cn>
2018-05-16 15:31:12 +08:00
Tonis Tiigi
f784907eae jsonmessage: pass message to aux callback
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 15:48:25 -07:00
Sebastiaan van Stijn
d4e48af48d
Merge pull request #36918 from Sh4d1/master
add verification to check if escapeKeys have elements (pkg/term/proxy.go)
2018-04-30 11:57:21 +02:00
Patrik Cyvoct
d339130f30
Add verification and test to check if escapeKeys is not empty
In pkg/term/proxy.go and pkg/term/proxy_test.go, check if escapeKeys is empty and if it is, return the one key read

Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
2018-04-30 00:32:43 +02:00
Kir Kolyshkin
7d62e40f7e Switch from x/net/context -> context
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".

Many vendored packages still use x/net/context, so vendor entry remains
for now.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-23 13:52:44 -07:00
Yong Tang
5c233cf431
Merge pull request #36913 from vdemeester/test-skip-non-root
Skip some tests requires root uid when run as user…
2018-04-23 11:42:42 -07:00
Sebastiaan van Stijn
20b524bf2e
Merge pull request #36920 from kolyshkin/cancel-func
context.WithTimeout: do call the cancel func
2018-04-23 20:14:25 +02:00
Vincent Demeester
a7999aaa53
Skip some tests requires root uid when run as user
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-04-23 10:14:39 +02:00
Vincent Demeester
5d8b88b114
Fix typo in idtools tests
It should check `os.Geteuid` with `uid` instead of `os.Getegid`.
On the container (where the tests run), the uid and gid seems to be
the same, thus this doesn't fail.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-04-23 10:14:07 +02:00
Vincent Demeester
53982e3fc1
Merge pull request #36091 from kolyshkin/mount
pkg/mount improvements
2018-04-21 11:03:54 +02:00
Kir Kolyshkin
05e2f7e2fa context.WithTimeout: do call the cancel func
govet complains (when using standard "context" package):

> the cancel function returned by context.WithTimeout should be called,
> not discarded, to avoid a context leak (vet)

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-20 12:27:13 -07:00
Kir Kolyshkin
a1d095199d mount.Unmount(): don't look into /proc/self/mountinfo
Now, every Unmount() call takes a burden to parse the whole nine yards
of /proc/self/mountinfo to figure out whether the given mount point is
mounted or not (and returns an error in case parsing fails somehow).

Instead, let's just call umount() and ignore EINVAL, which results
in the same behavior, but much better performance.

Note that EINVAL is returned from umount(2) not only in the case when
`target` is not mounted, but also for invalid flags. As the flags are
hardcoded here, it can't be the case.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:50 -07:00
Kir Kolyshkin
c611f18a7f pkg/mount/mountinfo_linux: parser speed up
The mountinfo parser implemented via `fmt.Sscanf()` is slower than the one
using `strings.Split()` and `strconv.Atoi()`. This rewrite helps to speed it
up to a factor of 8x, here is a result from go bench:

> BenchmarkParsingScanf-4      	     300	  22294112 ns/op
> BenchmarkParsingSplit-4      	    3000	   2780703 ns/op

I tried other approaches, such as using `fmt.Sscanf()` for the first
three (integer) fields and `strings.Split()` for the rest, but it slows
things down considerably:

> BenchmarkParsingMixed-4      	    1000	   8827058 ns/op

Note the old code uses `fmt.Sscanf`, when a linear search for '-' field,
when a split for the last 3 fields. The new code relies on a single
split.

I have also added more comments to aid in future development.

Finally, the test data is fixed to now have white space before the first field.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:42 -07:00
Kir Kolyshkin
871c957242 getSourceMount(): simplify
The flow of getSourceMount was:
 1 get all entries from /proc/self/mountinfo
 2 do a linear search for the `source` directory
 3 if found, return its data
 4 get the parent directory of `source`, goto 2

The repeated linear search through the whole mountinfo (which can have
thousands of records) is inefficient. Instead, let's just

 1 collect all the relevant records (only those mount points
   that can be a parent of `source`)
 2 find the record with the longest mountpath, return its data

This was tested manually with something like

```go
func TestGetSourceMount(t *testing.T) {
	mnt, flags, err := getSourceMount("/sys/devices/msr/")
	assert.NoError(t, err)
	t.Logf("mnt: %v, flags: %v", mnt, flags)
}
```

...but it relies on having a specific mount points on the system
being used for testing.

[v2: add unit tests for ParentsFilter]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:17 -07:00
Kir Kolyshkin
bb934c6aca pkg/mount: implement/use filter for mountinfo parsing
Functions `GetMounts()` and `parseMountTable()` return all the entries
as read and parsed from /proc/self/mountinfo. In many cases the caller
is only interested only one or a few entries, not all of them.

One good example is `Mounted()` function, which looks for a specific
entry only. Another example is `RecursiveUnmount()` which is only
interested in mount under a specific path.

This commit adds `filter` argument to `GetMounts()` to implement
two things:
 1. filter out entries a caller is not interested in
 2. stop processing if a caller is found what it wanted

`nil` can be passed to get a backward-compatible behavior, i.e. return
all the entries.

A few filters are implemented:
 - `PrefixFilter`: filters out all entries not under `prefix`
 - `SingleEntryFilter`: looks for a specific entry

Finally, `Mounted()` is modified to use `SingleEntryFilter()`, and
`RecursiveUnmount()` is using `PrefixFilter()`.

Unit tests are added to check filters are working.

[v2: ditch NoFilter, use nil]
[v3: ditch GetMountsFiltered()]
[v4: add unit test for filters]
[v5: switch to gotestyourself]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:48:09 -07:00
Alexandre Jomin
0d82bc142c pkg/term add build tag for netbsd
Signed-off-by: Alexandre Jomin <alexandrejomin@gmail.com>
2018-04-18 15:16:06 +02:00
Tibor Vass
61138fb5fc
Merge pull request #35739 from thaJeztah/bump-go-1.10
Bump Golang to 1.10.1
2018-04-13 13:56:38 -10:00
Anusha Ragunathan
ff6a103067
Merge pull request #36778 from odg0318/master
Using authorization plugin, I changed Content-Type check routine.
2018-04-12 17:39:38 -07:00
Derek McGowan
fb170206ba
Set format in archiver
Prevent changing the tar output by setting the format to
PAX and keeping the times truncated.
Without this change the archiver will produce different tar
archives with different hashes with go 1.10.
The addition of the access and changetime timestamps would
also cause diff comparisons to fail.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-04-12 13:34:33 -07:00
Derek McGowan
a422774e59
Fix tarsum for go 1.10
Remove invalid flush commands, flush should only occur when file
has been completely written. This is already handle, remove these calls.
Ensure data gets written after EOF in correct order and before close.
Remove gname and uname from sum for hash compatibility.
Update tarsum tests for gname/uname removal.
Return valid length after eof.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-04-12 13:34:14 -07:00
Brian Goff
74f8e47352 Limit authz response buffer
When the authz response buffer limit is hit, perform a flush.
This prevents excessive buffer sizes, especially on large responses
(e.g. `/containers/<id>/archive` or `/containers/<id>/export`).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-04-11 15:36:36 -04:00
Vincent Demeester
5219725890
Merge pull request #35518 from cyphar/libdm-dlsym-deferred_remove
pkg: devmapper: dynamically load dm_task_deferred_remove
2018-04-11 14:11:16 +02:00
odg0318
6ac73d34e4 If Content-Type is application/json;charset=UTF-8, RequestBody is empty.
Signed-off-by: odg0318 <odg0318@gmail.com>
2018-04-05 04:03:45 -04:00
Manuel Rüger
108bbd96cb Fix tests for pkg/archive
Signed-off-by: Manuel Rüger <manuel@rueg.eu>
2018-04-03 01:17:52 +02:00
Brian Goff
9d46c4c138 Support cancellation in directory.Size()
Makes sure that if the user cancels a request that the daemon stops
trying to traverse a directory.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-03-29 15:49:15 -04:00
Vincent Demeester
7270b01e5d
Merge pull request #36617 from Microsoft/jjh/kernel-registry
Windows: Move kernel_windows to use golang registry functions
2018-03-19 11:47:17 +01:00
John Howard
29f93c4bc7 Windows: Move kernel_windows to use golang registry functions
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-16 09:47:45 -07:00
Daniel Nephin
7d8815ea70 Cleanup pkg/jsonmessage progress tests
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:47 -04:00
Daniel Nephin
c9e52bd0da Post migration assertion fixes
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:46 -04:00
Daniel Nephin
6be0f70983 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:43 -04:00
Daniel Nephin
ef01dea893 Cleanup some assertions
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 10:54:54 -04:00
Vincent Demeester
a21d5bf669
Merge pull request #36506 from kolyshkin/pkg-mount-slice
pkg/mount: use sort.Slice
2018-03-09 09:46:53 +01:00
Akihiro Suda
4a1d35c546
Merge pull request #36505 from kolyshkin/pkg-mount-tests
pkg/mount unit tests: skip some test under non-root
2018-03-08 17:27:31 +09:00
John Stephens
8cf8fe9cf8
Merge pull request #36451 from Microsoft/jjh/ubr
Windows: Report Version and UBR
2018-03-07 06:27:01 -08:00
Kir Kolyshkin
a00310b54c pkg/mount: use sort.Slice
Sorting by mount point length can be implemented in a more
straightforward fashion since Go 1.8 introduced sort.Slice()
with an ability to provide a less() function in place.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-06 12:46:58 -08:00
Kir Kolyshkin
4aae77602a pkg/mount unit tests: skip some test under non-root
This makes `go test .` to pass if run as non-root user, skipping
those tests that require superuser privileges (for `mount`).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-03-06 12:37:27 -08:00
Sebastiaan van Stijn
390b74cb63
Merge pull request #36493 from cpuguy83/add_plugin_timeout_test
Adds a unit test for plugin request timeout
2018-03-06 12:45:17 +01:00
John Stephens
3e1505e3e6
Merge pull request #36327 from Microsoft/jjh/block-pulling-uplevel
Windows: Block pulling uplevel images
2018-03-05 15:12:52 -08:00
Brian Goff
7ca971fb49 Adds a unit test for plugin request timeout
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-03-05 15:38:56 -05:00
John Howard
6de9f90417 Windows: Report Version and UBR
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-03-05 08:20:45 -08:00
John Howard
83908836d3 Windows: Block pulling uplevel images
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-02-26 12:33:54 -08:00
Sebastiaan van Stijn
20028325da
Merge pull request #35829 from cpuguy83/no_private_mount_for_plugins
Perform plugin mounts in the runtime
2018-02-21 12:28:13 +01:00
Tonis Tiigi
f6c8266afd builder: fix wrong cache hits building from tars
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-02-15 23:42:42 -08:00
Aleksa Sarai
98fe4bd8f1
pkg: devmapper: dynamically load dm_task_deferred_remove
dm_task_deferred_remove is not supported by all distributions, due to
out-dated versions of devicemapper. However, in the case where the
devicemapper library was updated without rebuilding Docker (which can
happen in some distributions) then we should attempt to dynamically load
the relevant object rather than try to link to it.

This can only be done if Docker was built dynamically, for obvious
reasons.

In order to avoid having issues arise when dlsym(3) was unnecessary,
gate the whole dlsym(3) logic behind a buildflag that we disable by
default (libdm_dlsym_deferred_remove).

Signed-off-by: Aleksa Sarai <asarai@suse.de>
2018-02-16 17:23:23 +11:00
bin liu
7a7a8a33a4 Fix typos in pkg
Signed-off-by: bin liu <liubin0329@gmail.com>
2018-02-10 19:43:13 +08:00
Brian Goff
0e5eaf8ee3 Ensure plugin returns correctly scoped paths
Before this change, volume management was relying on the fact that
everything the plugin mounts is visible on the host within the plugin's
rootfs. In practice this caused some issues with mount leaks, so we
changed the behavior such that mounts are not visible on the plugin's
rootfs, but available outside of it, which breaks volume management.

To fix the issue, allow the plugin to scope the path correctly rather
than assuming that everything is visible in `p.Rootfs`.
In practice this is just scoping the `PropagatedMount` paths to the
correct host path.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-07 15:48:27 -05:00
Daniel Nephin
4f0d95fa6e Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
Anusha Ragunathan
6e5c2d639f
Merge pull request #36119 from cpuguy83/fix_plugin_scanner
Fix issue with plugin scanner going to deep
2018-02-02 20:19:01 -08:00
Brian Goff
b27f70d45a Fix issue with plugin scanner going to deep
The plugin spec says that plugins can live in one of:

- /var/run/docker/plugins/<name>.sock
- /var/run/docker/plugins/<name>/<name>.sock
- /etc/docker/plugins/<name>.[json,spec]
- /etc/docker/plugins/<name>/<name>.<json,spec>
- /usr/lib/docker/plugins/<name>.<json,spec>
- /usr/lib/docker/plugins/<name>/<name>.<json,spec>

However, the plugin scanner which is used by the volume list API was
doing `filepath.Walk`, which will walk the entire tree for each of the
supported paths.
This means that even v2 plugins in
`/var/run/docker/plugins/<id>/<name>.sock` were being detected as a v1
plugin.
When the v1 plugin loader tried to load such a plugin it would log an
error that it couldn't find it because it doesn't match one of the
supported patterns... e.g. when in a subdir, the subdir name must match
the plugin name for the socket.

There is no behavior change as the error is only on the `Scan()` call,
which is passing names to the plugin registry when someone calls the
volume list API.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-02 16:49:14 -05:00
Sebastiaan van Stijn
c41c80026b
Merge pull request #35911 from ASMfreaK/Russian-Scientsists-addition-to-names-generator
Add more russian scientists in names-generator.go
2018-01-26 15:48:58 -08:00
Vincent Demeester
59e86068e6
Merge pull request #34379 from cpuguy83/mount_optimizations
Optimizations for recursive unmount
2018-01-24 14:00:58 -08:00
Sebastiaan van Stijn
db5c006bc8
Merge pull request #35919 from yongtang/35333-carry
Carry #35333: Devicemapper: ignore Nodata errors when delete thin device
2018-01-20 18:47:16 +01:00
John Howard
5b24976ad4 Bump RS3 final build, and remove LCOW_SUPPORTED
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-19 12:22:56 -08:00
John Howard
0cba7740d4 Address feedback from Tonis
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-18 12:30:39 -08:00
John Howard
afd305c4b5 LCOW: Refactor to multiple layer-stores based on feedback
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-01-18 08:31:05 -08:00
John Howard
ce8e529e18 LCOW: Re-coalesce stores
Signed-off-by: John Howard <jhoward@microsoft.com>

The re-coalesces the daemon stores which were split as part of the
original LCOW implementation.

This is part of the work discussed in https://github.com/moby/moby/issues/34617,
in particular see the document linked to in that issue.
2018-01-18 08:29:19 -08:00
unclejack
871afbb304
Merge pull request #35697 from sargun/use-pgzip
Make image (layer) downloads faster by using pigz
2018-01-17 11:18:20 -08:00
Sebastiaan van Stijn
f0b0f2038d
Merge pull request #35441 from cpuguy83/plugin_timeout
Add timeouts for volume plugin ops
2018-01-17 14:49:41 +01:00
Brian Goff
b15f8d2d4f Add timeouts for volume plugin ops
This protects the daemon from volume plugins that are slow or
deadlocked.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-16 20:30:49 -05:00
Pavel Pletenev
e0c3fe7836
Add more russian scientists in names-generator.go
Signed-off-by: ASM <cpp.create@gmail.com>
2018-01-16 21:09:17 +02:00
Sargun Dhillon
fd35494a25 Make image (layer) downloads faster by using pigz
The Golang built-in gzip library is serialized, and fairly slow
at decompressing. It also only decompresses on demand, versus
pipelining decompression.

This change switches to using the pigz external command
for gzip decompression, as opposed to using the built-in
golang one. This code is not vendored, but will be used
if it autodetected as part of the OS.

This also switches to using context, versus a manually
managed channel to manage cancellations, and synchronization.
There is a little bit of weirdness around manually having
to cancel in the error cases.

Signed-off-by: Sargun Dhillon <sargun@sargun.me>
2018-01-16 10:49:18 -08:00
Sebastiaan van Stijn
b4a6313969
Golint: remove redundant ifs
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 00:42:25 +01:00
Brian Goff
dd21087660 Optimizations for recurrsive unmount
When a recursive unmount fails, don't bother parsing the mount table to check
if what we expected to be a mountpoint is still mounted. `EINVAL` is
returned when you try to unmount something that is not a mountpoint, the
other cases of `EINVAL` would not apply here unless everything is just
wrong. Parsing the mount table over and over is relatively expensive,
especially in the code path that it's in.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-01-11 16:13:16 -05:00
Yong Tang
99c22b0034
Merge pull request #35822 from keyolk/35709-fix-docker-build-overlayfs-whiteout
skip container ID remapping, if the file is overlayfs whiteout.
2018-01-03 22:21:00 +08:00
Chanhun Jeong
b013c1541d skip container ID remapping, if the file is overlayfs whiteout.
Signed-off-by: Chanhun Jeong <chanhun.jeong@navercorp.com>
2018-01-03 17:06:59 +09:00
Liu Hua
8451d03d8e Devicemapper: ignore Nodata errors when delete thin device
if thin device is deteled and the metadata exists, you can not
delete related containers. This patch ignore Nodata errors for
thin device deletion

Signed-off-by: Liu Hua <sdu.liu@huawei.com>
2018-01-02 18:04:25 +00:00
Sebastiaan van Stijn
6ed1163c98
Remove redundant build-tags
Files that are suffixed with `_linux.go` or `_windows.go` are
already only built on Linux / Windows, so these build-tags
were redundant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-12-18 17:41:53 +01:00
John Howard
7caa30e893 Windows: Case-insensitive filename matching against builder cache
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-12-14 13:49:40 -08:00
Tõnis Tiigi
cefb33700c
Merge pull request #35589 from keloyang/close-fd
Close pipe in chrootarchive.invokeUnpack when cmd.Start()/json.NewEncoder failed
2017-12-06 17:48:02 -08:00
Michael Crosby
72e45fd54e
Merge pull request #35618 from kolyshkin/mkdir-all
Fix MkdirAll* and its usage
2017-11-30 11:19:29 -05:00
Kir Kolyshkin
516010e92d Simplify/fix MkdirAll usage
This subtle bug keeps lurking in because error checking for `Mkdir()`
and `MkdirAll()` is slightly different wrt to `EEXIST`/`IsExist`:

 - for `Mkdir()`, `IsExist` error should (usually) be ignored
   (unless you want to make sure directory was not there before)
   as it means "the destination directory was already there"

 - for `MkdirAll()`, `IsExist` error should NEVER be ignored.

Mostly, this commit just removes ignoring the IsExist error, as it
should not be ignored.

Also, there are a couple of cases then IsExist is handled as
"directory already exist" which is wrong. As a result, some code
that never worked as intended is now removed.

NOTE that `idtools.MkdirAndChown()` behaves like `os.MkdirAll()`
rather than `os.Mkdir()` -- so its description is amended accordingly,
and its usage is handled as such (i.e. IsExist error is not ignored).

For more details, a quote from my runc commit 6f82d4b (July 2015):

    TL;DR: check for IsExist(err) after a failed MkdirAll() is both
    redundant and wrong -- so two reasons to remove it.

    Quoting MkdirAll documentation:

    > MkdirAll creates a directory named path, along with any necessary
    > parents, and returns nil, or else returns an error. If path
    > is already a directory, MkdirAll does nothing and returns nil.

    This means two things:

    1. If a directory to be created already exists, no error is
    returned.

    2. If the error returned is IsExist (EEXIST), it means there exists
    a non-directory with the same name as MkdirAll need to use for
    directory. Example: we want to MkdirAll("a/b"), but file "a"
    (or "a/b") already exists, so MkdirAll fails.

    The above is a theory, based on quoted documentation and my UNIX
    knowledge.

    3. In practice, though, current MkdirAll implementation [1] returns
    ENOTDIR in most of cases described in #2, with the exception when
    there is a race between MkdirAll and someone else creating the
    last component of MkdirAll argument as a file. In this very case
    MkdirAll() will indeed return EEXIST.

    Because of #1, IsExist check after MkdirAll is not needed.

    Because of #2 and #3, ignoring IsExist error is just plain wrong,
    as directory we require is not created. It's cleaner to report
    the error now.

    Note this error is all over the tree, I guess due to copy-paste,
    or trying to follow the same usage pattern as for Mkdir(),
    or some not quite correct examples on the Internet.

    [1] https://github.com/golang/go/blob/f9ed2f75/src/os/path.go

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-11-27 17:32:12 -08:00
Kir Kolyshkin
2aa13f86f0 idtools.MkdirAs*: error out if dir exists as file
Standard golang's `os.MkdirAll()` function returns "not a directory" error
in case a directory to be created already exists but is not a directory
(e.g. a file). Our own `idtools.MkdirAs*()` functions do not replicate
the behavior.

This is a bug since all `Mkdir()`-like functions are expected to ensure
the required directory exists and is indeed a directory, and return an
error otherwise.

As the code is using our in-house `system.Stat()` call returning a type
which is incompatible with that of golang's `os.Stat()`, I had to amend
the `system` package with `IsDir()`.

A test case is also provided.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-11-27 13:25:44 -08:00
Chetan Birajdar
7c50d39480 added more awesome and notable women to names-generator.go
Signed-off-by: Chetan Birajdar <birajdar.chetan@gmail.com>
2017-11-26 21:40:02 -08:00
y00316549
f5f8f00860 Close pipe in chrootarchive.invokeUnpack when cmd.Start()/json.NewEncoder failed.
Signed-off-by: y00316549 <yangshukui@huawei.com>
2017-11-24 13:41:08 +08:00
Akihiro Suda
84b1f813a4
Merge pull request #35541 from thaJeztah/remove-deprecated-mkdirallas
Remove uses of deprecated MkdirAllAs(), MkdirAs()
2017-11-22 17:49:37 +09:00
Sebastiaan van Stijn
47af9f625d
Remove unused "testutil" package
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 15:09:23 +01:00
Sebastiaan van Stijn
38b3af567f
Remove deprecated MkdirAllAs(), MkdirAs()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 13:53:54 +01:00
Sebastiaan van Stijn
ce66470f54
Minor refactor in idtools
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 13:49:58 +01:00
Sebastiaan van Stijn
9aba019b72
Update idtools tests to test non-deprecated functions
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-11-21 13:47:17 +01:00
Chao Wang
97e406678c Split and remove pkg/stringutils
Signed-off-by: Chao Wang <wangchao.fnst@cn.fujitsu.com>
2017-11-13 10:01:11 +08:00
Chao Wang
5c154cfac8 Copy Inslice() to those parts that use it
Signed-off-by: Chao Wang <wangchao.fnst@cn.fujitsu.com>
2017-11-10 13:42:38 +08:00
Yong Tang
a4bdb304e2
Merge pull request #35250 from joppich/patch-1
Update names-generator.go
2017-11-09 06:21:02 -08:00
Victor Vieux
5745a8531e
Merge pull request #35265 from cpuguy83/32609_defreference_voldriver_on_error
Fixup some issues with plugin refcounting
2017-11-07 09:47:07 -08:00
Yong Tang
4785f1a7ab Remove solaris build tag and `contrib/mkimage/solaris
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2017-11-02 00:01:46 +00:00
Yong Tang
0bcca5ea86
Merge pull request #35341 from tklauser/utsname-x-sys
Simplify Utsname string conversion
2017-11-01 11:36:38 -07:00
joppich
f69f8f3679 Update names-generator.go to include Cédric Villani and Valentina
Tershkova

Signed-off-by: Jorit Kleine-Möllhoff <joppich@bricknet.de>
2017-10-31 15:38:57 +01:00
Tobias Klauser
6d068bc25b Simplify Utsname string conversion
Update golang.org/x/sys to 95c6576299259db960f6c5b9b69ea52422860fce in
order to get the unix.Utsname with byte array instead of int8/uint8
members.

This allows to use simple byte slice to string conversions instead of
using charsToString or its open-coded version.

Also see golang/go#20753 for details.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-10-31 10:59:32 +01:00
Sebastiaan van Stijn
ba3bf8191e
Merge pull request #35313 from charrywanganthony/RandomAlpha
Separate the GenerateRandomAlphaOnlyString function from stringutils
2017-10-29 22:33:57 +01:00
Yong Tang
e8730d052e
Merge pull request #35289 from zuiurs/namesgeneratorCmd
Fix a names-generator binary
2017-10-28 13:16:33 -07:00
chaowang
7c35a24182 Separate the GenerateRandomAlphaOnlyString function from stringutils
Signed-off-by: chaowang <chaowang@localhost.localdomain>
2017-10-28 09:03:02 +08:00
Sebastiaan van Stijn
17bb1d3663 Merge pull request #35285 from crosbymichael/solaris
Remove solaris files
2017-10-25 15:14:04 +02:00
Vincent Demeester
d040d637a9
Rename a few docker to moby
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-10-25 13:56:12 +02:00
Mizuki Urushida
eaab2f7150 Fix a names-generator binary
To ensure that namesgenerator binary outputs random name
by initializing Seed.

Signed-off-by: Mizuki Urushida <z11111001011@gmail.com>

not use init function.

Signed-off-by: Mizuki Urushida <z11111001011@gmail.com>
2017-10-25 18:21:17 +09:00
Sebastiaan van Stijn
d891f2e3ca Merge pull request #35217 from cpuguy83/fix_chroot_mount_race
Use rslave instead of rprivate in chrootarchive
2017-10-24 22:13:28 +02:00
Michael Crosby
5a9b5f10cf Remove solaris files
For obvious reasons that it is not really supported now.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-24 15:39:34 -04:00
Brian Goff
402540708c Merge pull request #34895 from mlaventure/containerd-1.0-client
Containerd 1.0 client
2017-10-23 10:38:03 -04:00
Brian Goff
3816b51438 Fixup some issues with plugin refcounting
In some circumstances we were not properly releasing plugin references,
leading to failures in removing a plugin with no way to recover other
than restarting the daemon.

1. If volume create fails (in the driver)
2. If a driver validation fails (should be rare)
3. If trying to get a plugin that does not match the passed in capability

Ideally the test for 1 and 2 would just be a unit test, however the
plugin interfaces are too complicated as `plugingetter` relies on
github.com/pkg/plugin/Client (a concrete type), which will require
spinning up services from within the unit test... it just wouldn't be a
unit test at this point.
I attempted to refactor this a bit, but since both libnetwork and
swarmkit are reliant on `plugingetter` as well, this would not work.
This really requires a re-write of the lower-level plugin management to
decouple these pieces.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-10-21 15:17:57 -04:00
Yong Tang
113bebe0ee Merge pull request #35059 from cpuguy83/35002_handle_bind_create_errors
idtools don't chown if not needed
2017-10-20 08:39:23 -07:00
Kenfe-Mickael Laventure
ddae20c032
Update libcontainerd to use containerd 1.0
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-20 07:11:37 -07:00
Brian Goff
fa9709a3fc idtools don't chown if not needed
In some cases (e.g. NFS), a chown may technically be a no-op but still
return `EPERM`, so only call `chown` when neccessary.

This is particularly problematic for docker users bind-mounting an NFS
share into a container.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-10-19 16:06:25 -04:00
Tobias Klauser
86f080cff0 Use Mkdev, Major and Minor functions from golang.org/x/sys/unix
Update golang.org/x/sys to 8dbc5d05d6edcc104950cc299a1ce6641235bc86 in
order to get the Major, Minor and Mkdev functions for every unix-like
OS. Use them instead of the locally defined versions which currently use
the Linux specific device major/minor encoding.

This means that the device number should now be properly encoded on e.g.
Darwin, FreeBSD or Solaris.

Also, the SIGUNUSED constant was removed from golang.org/x/sys/unix in
https://go-review.googlesource.com/61771 as it is also removed from the
respective glibc headers.

Remove it from signal.SignalMap as well after the golang.org/x/sys
re-vendoring.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-10-19 08:28:38 +02:00
Kate Heddleston
736fafb2d0 Removing unused code with a TODO that no longer needs to be done
Signed-off-by: Kate Heddleston <kate.heddleston@gmail.com>
2017-10-16 14:57:48 -07:00
Brian Goff
5ede64d63f Use rslave instead of rprivate in chrootarchive
With `rprivate` there exists a race where a reference to a mount has
propagated to the new namespace, when `rprivate` is set the parent
namespace is not able to remove the mount due to that reference.
With `rslave` unmounts will propagate correctly into the namespace and
prevent the sort of transient errors that are possible with `rprivate`.

This is a similar fix to 117c92745b

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-10-16 13:49:31 -04:00
Sebastiaan van Stijn
be4586fffd Merge pull request #33488 from raja-sami-10p/pkg/idtools
Increase Coverage of pkg/idtools
2017-10-10 18:37:49 +02:00
John Howard
d98ecf2d6c LCOW: API change JSON header to string POST parameter
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-10-06 15:26:48 -07:00
John Howard
0380fbff37 LCOW: API: Add platform to /images/create and /build
Signed-off-by: John Howard <jhoward@microsoft.com>

This PR has the API changes described in https://github.com/moby/moby/issues/34617.
Specifically, it adds an HTTP header "X-Requested-Platform" which is a JSON-encoded
OCI Image-spec `Platform` structure.

In addition, it renames (almost all) uses of a string variable platform (and associated)
methods/functions to os. This makes it much clearer to disambiguate with the swarm
"platform" which is really os/arch. This is a stepping stone to getting the daemon towards
fully multi-platform/arch-aware, and makes it clear when "operating system" is being
referred to rather than "platform" which is misleadingly used - sometimes in the swarm
meaning, but more often as just the operating system.
2017-10-06 11:44:18 -07:00
Sebastiaan van Stijn
1c4fad8135 Merge pull request #35056 from tklauser/win-console-mode-consts
Use windows console mode constants from Azure/go-ansiterm
2017-10-05 00:41:22 +02:00
Alexander Morozov
889cfd1b44 pkg/locker: add benchmarks
Signed-off-by: Alexander Morozov <lk4d4math@gmail.com>
2017-10-02 11:20:21 -07:00
Tobias Klauser
9335683fa5 Use all console mode constants from go-ansiterm
The missing console mode constants were added to go-ansiterm in
Azure/go-ansiterm#23. Use these constants instead of defining them
locally.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-10-02 09:49:20 +02:00
Danyal Khaliq
00c0ee885c Increase Coverage of pkg/idtools
Signed-off-by: Danyal Khaliq <danyal.khaliq@tenpearls.com>
2017-09-27 12:55:04 +05:00
Vincent Demeester
7d47823c22 Merge pull request #34946 from dnephin/fix-jsonlog
Move pkg/jsonlog to be a subpackage of the single consumer
2017-09-26 10:49:57 +02:00
Daniel Nephin
035604cca6 Move jsonlog to a subpackage of jsonfilelog
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
a06ad2792a Fix benchmarks and remove more unnecessary code.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
231c5cbd50 Remove unused JSONLog marshaling
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
638d4cc7e4 Remove unused Format
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
7de92de636 Unexport FastTimeMarshalJSON
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
27cfa68af1 Move RFC3339NanoFixed to a more appropriate package.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:24 -04:00
Vincent Demeester
8bebd42df2
Fix TestMount under a selinux system
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-09-25 11:54:30 +02:00
Stephen J Day
0cd4ab3f9a
pkg/package: remove promise package
The promise package represents a simple enough concurrency pattern that
replicating it in place is sufficient. To end the propagation of this
package, it has been removed and the uses have been inlined.

While this code could likely be refactored to be simpler without the
package, the changes have been minimized to reduce the possibility of
defects. Someone else may want to do further refactoring to remove
closures and reduce the number of goroutines in use.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-09-21 17:56:45 -07:00
Yong Tang
777d4a1bf4 Merge pull request #34861 from tklauser/fix-cstring-leaks
Fix CString memory leaks
2017-09-21 09:14:07 -07:00
Yong Tang
149f3aceb2 Merge pull request #31993 from dccurtis/issue-31816
Fixing ‘docker cp’ to allow new target file name in a host symlinked directory
2017-09-19 10:54:53 -07:00
Kir Kolyshkin
46833ee1c3 devmapper: show dmesg if mount fails
If mount fails, the reason might be right there in the kernel log ring buffer.
Let's include it in the error message, it might be of great help.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-09-17 22:04:31 -07:00
Kir Kolyshkin
84f1c054e6 devmapper gd: disable for static build
Static build with devmapper is impossible now since libudev is required
and no static version of libudev is available (as static libraries are
not supported by systemd which udev is part of).

This should not hurt anyone as "[t]he primary user of static builds
is the Editions, and docker in docker via the containers, and none
of those use device mapper".

Also, since the need for static libdevmapper is gone, there is no need
to self-compile libdevmapper -- let's use the one from Debian Stretch.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-09-17 22:04:31 -07:00
Tobias Klauser
593dbfd144 Fix CString memory leaks
Make sure to call C.free on C string allocated using C.CString in every
exit path.

C.CString allocates memory in the C heap using malloc. It is the callers
responsibility to free them. See
https://golang.org/cmd/cgo/#hdr-Go_references_to_C for details.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-09-15 09:57:26 +02:00
Victor Vieux
ff686743c5 Add LCOW behind experimental,
might not be the cleanest way, but it's definitly the way with the
minimum code change.

Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2017-09-14 13:51:16 -07:00
Akash Gupta
7a7357dae1 LCOW: Implemented support for docker cp + build
This enables docker cp and ADD/COPY docker build support for LCOW.
Originally, the graphdriver.Get() interface returned a local path
to the container root filesystem. This does not work for LCOW, so
the Get() method now returns an interface that LCOW implements to
support copying to and from the container.

Signed-off-by: Akash Gupta <akagup@microsoft.com>
2017-09-14 12:07:52 -07:00
Daniel Nephin
f7f101d57e Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-12 12:09:59 -04:00
Marcel Edmund Franke
6d3bc28289 Fix typos in Locker example
Signed-off-by: Marcel Edmund Franke <marcel.edmund.franke@gmail.com>
2017-09-10 11:34:19 +02:00
Daniel Nephin
09652bf878 Add ineffassign linter
Also enable GC in linting to reduce memory usage.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-08 18:23:21 -04:00
Yong Tang
2dcb77b24c Merge pull request #34738 from wgliang/optimization1
Optimize some wrong usage and spelling
2017-09-07 09:45:14 -07:00
wangguoliang
94cefa2145 Optimize some wrong usage and spelling
Signed-off-by: wgliang <liangcszzu@163.com>
2017-09-07 09:44:08 +08:00
Tibor Vass
be97c66708 Merge pull request #34748 from dnephin/cleanup-client-deps
Cleanup client/ package dependencies
2017-09-06 16:14:29 -07:00
Daniel Nephin
6916c215b0 Move tlsconfig to client package.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-06 16:39:55 -04:00
Simon Ferquel
7c9e64a2e1 re-enable some conversion for darwin-amd64 platform
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2017-09-06 12:22:41 +02:00
Brian Goff
a127269803 Merge pull request #34700 from tklauser/pkg-term-winsize
pkg/term: use IoctlGetWinsize/IoctlSetWinsize from golang.org/x/sys/unix
2017-09-01 15:58:25 -04:00
Yong Tang
cb952bf006 Merge pull request #34625 from dnephin/more-linters
Add interfacer and unconvert linters
2017-09-01 08:46:08 -07:00
Tobias Klauser
32dfc0cb64 pkg/term: use IoctlGetWinsize/IoctlSetWinsize from golang.org/x/sys/unix
Use unix.IoctlGetWinsize and unix.IoctlSetWinsize instead of manually
reimplementing them.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-09-01 14:43:09 +02:00
Yong Tang
6ed5db6243 Merge pull request #34615 from dnephin/remove-pkg-testutil-assert
Move ErrorContains to an internal package
2017-08-26 19:09:27 -07:00
Daniel Nephin
a5631c75e1 Merge pull request #34613 from dnephin/remove-pkg-testutil-cmd
Remove pkg testutil cmd
2017-08-25 15:09:52 -04:00
Vincent Demeester
475c9926db Merge pull request #34590 from shouze/add-support-of-ChownOpts-to-TarWithOptions
archive: add ChownOpts support to TarWithOptions
2017-08-25 18:21:31 +02:00
Daniel Nephin
c2c127fa5a Remove pkg/testutil/cmd
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-25 12:07:31 -04:00
Daniel Nephin
e62b2d410c Move ErrorContains to an internal package.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-25 12:04:58 -04:00
Yong Tang
2cea2f5469 Merge pull request #34583 from tklauser/win-event-functions
Use windows event functions from golang.org/x/sys/windows
2017-08-25 07:43:03 -07:00
Daniel Nephin
2f5f0af3fd Add unconvert linter
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-24 15:08:31 -04:00
Daniel Nephin
468154a841 Remove pkg/testutil tempfile and golden
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-23 17:26:27 -04:00
Sébastien HOUZÉ
8a34c67a7e
archive: add ChownOpts support to TarWithOptions
Signed-off-by: Sébastien HOUZÉ <cto@verylastroom.com>
2017-08-23 20:52:29 +02:00
Daniel Nephin
e885af2a6b Remove RunCommandPipelineWithOutput
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
9db68f4dea Remove RandomTmpDirPath
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
e8bd671815 Remove testutil.ParseCgroupPaths
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
1455086c4b Remove ConsumeWithSpeed
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
4f304e72a2 Remove testutil.ReadBody
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
6a0105b452 Remove ChannelBuffer
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
6ffbe3f6a8 Remove ListTar
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
6151d55ee7 Remove testutil.CompareDirectoryEntries and IsKilled
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
d30e51495a Remove testutil.ConvertSliceOfStringsToMap
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
a0c9089971 Remove unused helpers.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
62c1f0ef41 Add deadcode linter
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-21 18:18:50 -04:00
Daniel Nephin
372670b507 Add goimports to linters.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-21 18:15:08 -04:00
Tobias Klauser
e942513ac4 Use event functions from golang.org/x/sys/windows
Use CreateEvent, OpenEvent (which both map to the respective *EventW
function) and PulseEvent from golang.org/x/sys instead of local copies.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-08-21 12:58:09 +02:00
Daniel Nephin
9b47b7b151 Fix golint errors.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-18 14:23:44 -04:00
Douglas Curtis
cd7489f2b7 Replacing os.Lstat with os.Stat to determine directory status in CopyInfoDestinationPath
Signed-off-by: Douglas Curtis <dougcurtis1@gmail.com>

Commenting out tests for now

Signed-off-by: Doug Curtis <dougcurtis1@gmail.com>

Added unit test for CopyInfoDestionationPath.

Signed-off-by: Doug Curtis <dougcurtis1@gmail.com>

Removing integration-cli test case additions

Signed-off-by: Doug Curtis <dougcurtis1@gmail.com>

Removing extra spaces between archive_unix_test.go test cases

Signed-off-by: Doug Curtis <dougcurtis1@gmail.com>

Fixed gofmt issues in archive_unix_test.go

Signed-off-by: Doug Curtis <dougcurtis1@gmail.com>
2017-08-15 23:19:42 +00:00
Brian Goff
ebcb7d6b40 Remove string checking in API error handling
Use strongly typed errors to set HTTP status codes.
Error interfaces are defined in the api/errors package and errors
returned from controllers are checked against these interfaces.

Errors can be wraeped in a pkg/errors.Causer, as long as somewhere in the
line of causes one of the interfaces is implemented. The special error
interfaces take precedence over Causer, meaning if both Causer and one
of the new error interfaces are implemented, the Causer is not
traversed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-08-15 16:01:11 -04:00
Yong Tang
479cc38bb9 Merge pull request #33450 from naveed-jamil-tenpearls/pkg/signal/trap
Add test coverage to signal/trap.go
2017-08-15 00:53:37 -07:00
Brian Goff
31d8d2253e Merge pull request #34362 from kolyshkin/update-libdevmapper
Dockerfile*: bump devmapper library version
2017-08-14 09:56:43 -04:00
Yong Tang
187cd25517 Merge pull request #34276 from keloyang/testcase-docker-cp-hardlink
Add unit  test  for RebaseArchiveEntries
2017-08-09 07:29:09 -07:00
Naveed Jamil
bc756061a3 Add test coverage to signal/trap.go
Signed-off-by: Naveed Jamil <naveed.jamil@tenpearls.com>
2017-08-09 15:13:38 +05:00
yangshukui
1f1e8e9c0e use testify to check error for test unit test of archive package
Signed-off-by: yangshukui <yangshukui@huawei.com>
2017-08-09 10:27:01 +08:00
Tibor Vass
c7911389e6 Merge pull request #34441 from vdemeester/move-pkg-templates-away
Move pkg/templates away
2017-08-08 17:54:17 -07:00
Vincent Demeester
9ef3b53597
Move pkg/templates away
- Remove unused function and variables from the package
- Remove usage of it from `profiles/apparmor` where it wasn't required
- Move the package to `daemon/logger/templates` where it's only used

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-08-08 18:16:41 +02:00
Vincent Demeester
c204fce2ee
Move pkg/listeners away
It is only used in `daemon` and should really live there.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-08-08 18:14:04 +02:00
yangshukui
82eb9002e9 Add unit test for RebaseArchiveEntries
Signed-off-by: yangshukui <yangshukui@huawei.com>
2017-08-07 16:40:54 +08:00
Tobias Klauser
8a3730d251 pkg/term: set termios VMIN and VTIME in MakeRaw on Linux
The BSD and Solaris versions of term.MakeRaw already set VMIN and VTIME
explicitly such that a read returns when one character is available.
cfmakeraw (which was previously used) in glibc also sets these values
explicitly, so it should be done in the Linux version of MakeRaw as well
to be consistent.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-08-03 13:21:42 +02:00
yuexiao-wang
cc4da81128 Remove the logrus from pkg/signal
Signed-off-by: yuexiao-wang <wang.yuexiao@zte.com.cn>
2017-08-02 18:15:25 +08:00
Kir Kolyshkin
c4fde49a5c Dockerfile*: bump devmapper library version
Let's use latest lvm2 sources to compile the libdevmapper library.

Initial reason for compiling devmapper lib from sources was a need to
have the static version of the library at hand, in order to build
the static dockerd, but note that the same headers/solib are used
for dynamic build (dynbinary) as well.

The reason for this patch is to enable the deferral removal feature.
The supplied devmapper library (and headers) are too old, lacking the
needed functions, so the daemon is built with 'libdm_no_deferred_remove'
build tag (see the check in hack/make.sh). Because of this, even if the
kernel dm driver is perfectly able to support the feature, it can not
be used. For more details and background story, see [1].

Surely, one can't just change the version number. While at it:
 - improve the comments;
 - remove obsoleted URLs;
 - remove s390 and ppc configure updates that are no longer needed;
 - use pkg-config instead of hardcoding the flags (newer lib added
   some more dependencies);

 [1] https://github.com/moby/moby/issues/34298

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-08-01 22:08:24 -07:00
John Howard
8af4db6f00 Merge pull request #34272 from dmcgowan/update-logrus
Update logrus to v1.0.1 (Sirupsen -> sirupsen)
2017-08-01 18:53:33 -07:00
Kir Kolyshkin
adce3ca48e pkg/devicemapper: comment nitpicks
1. devmapper_wrapper_{,no_}deferred_remove.go:
Comments about LibraryDeferredRemovalSupport were very totally
misleading to me. This thing has nothing to do with either static
or dynamic linking (but with build tags). Fix the comment accordingly.

2. devmapper.go:
Reveal the source of those magic device* constants.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-07-31 20:05:26 -07:00
Derek McGowan
1009e6a40b
Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-31 13:16:46 -07:00
Tobias Klauser
01f70b028e Switch Stat syscalls to x/sys/unix
Switch some more usage of the Stat function and the Stat_t type from the
syscall package to golang.org/x/sys. Those were missing in PR #33399.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-27 10:09:02 +02:00
Sebastiaan van Stijn
f8c434366e Merge pull request #33987 from dnephin/cleanup-more-hack
Move test-unit out of hack/make
2017-07-27 00:29:13 +02:00
Victor Vieux
fa7db52e26 Merge pull request #34135 from tklauser/more-unix-fns
Replace manually written wrappers by functions from x/sys/unix
2017-07-18 20:43:13 +02:00
Kir Kolyshkin
7da12bcfa9 devmapper_wrapper.go: fix gcc warning
I am getting the following warning from gcc when compiling the daemon:

> # github.com/docker/docker/pkg/devicemapper
> pkg/devicemapper/devmapper_wrapper.go: In function ‘log_cb’:
> pkg/devicemapper/devmapper_wrapper.go:20:2: warning: ignoring return
> value of ‘vasprintf’, declared with attribute warn_unused_result
> [-Wunused-result]
>  vasprintf(&buffer, f, ap);
>  ^

vasprintf(3) man page says if the function returns -1, the buffer is
undefined, so we should not use it. In practice, I assume, this never
happens so we just return.

Introduced by https://github.com/moby/moby/pull/33845 that resulted in
commit 63328c6 ("devicemapper: remove 256 character limit of libdm logs")

Cc: Aleksa Sarai <asarai@suse.de>
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-07-17 17:46:44 -07:00
Evan Hazlett
458f6712d4 Merge pull request #33886 from aaronlehmann/names-in-memdb
Store container names in memdb
2017-07-17 14:16:41 -04:00
Daniel Nephin
1fb615599a Remove test-unit from hack/make
Also remove the test flag from pkg/term and jsut checkuid directly.
Fixed a problem with a pkg/term test that was leaving the terminal in a bad
state.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-07-17 11:39:33 -04:00
Tobias Klauser
bedf09363c loopback: use IoctlGetInt/IoctlSetInt from x/sys/unix
Use IoctlGetInt/IoctlSetInt from golang.org/x/sys/unix (where
applicable) instead of manually reimplementing them.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-17 10:37:42 +02:00
Tobias Klauser
6476504695 [pkg/term] use IoctlGetTermios/IoctlSetTermios from x/sys/unix
Use IoctlGetTermios/IoctlSetTermios from golang.org/x/sys/unix instead
of manually reimplementing them.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-17 10:37:42 +02:00
Tobias Klauser
6c9d715a8c sysinfo: use Prctl() from x/sys/unix
Use unix.Prctl() instead of manually reimplementing it using
unix.RawSyscall. Also use unix.SECCOMP_MODE_FILTER instead of locally
defining it.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-17 10:37:42 +02:00
Vincent Demeester
97e730ae57 Merge pull request #34106 from tonistiigi/fix-tar-prefix-header
Fix tar prefix header
2017-07-17 09:40:39 +02:00
Tonis Tiigi
4a3cfda45e archive: add test for prefix header
With docker-17.06.0 some images pulled do not extract properly. Some files don't appear in correct directories. This may or may not cause the pull to fail. These images can't be pushed or saved. 17.06 is the first version of Docker built with go1.8.

Cause

There are multiple updates to the tar package in go1.8.

https://go-review.googlesource.com/c/32234/ disables using "prefix" field when new tar archives are being written. Prefix field was previously set when a record in the archive used a path longer than 100 bytes.

Another change https://go-review.googlesource.com/c/31444/ makes the reader ignore the "prefix" field value if the record is in GNU format. GNU format defines that same area should be used for access and modified times. If the "prefix" field is not read, a file will only be extracted by the basename.

The problem is that with a previous version of the golang archive package headers could be written, that use the prefix field while at the same time setting the header format to GNU. This happens when numeric fields are big enough that they can not be written as octal strings and need to be written in binary. Usually, this shouldn't happen: uid, gid, devmajor, devminor can use up to 7 bytes, size and timestamp can use 11. If one of the records does overflow it switches the whole writer to GNU mode and all next files will be saved in GNU format.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-07-14 10:20:48 -07:00
Brian Goff
f7ce35f47b Merge pull request #34077 from justincormack/mount_remount_opts
In the case of remounting with changed data, need to call mount
2017-07-14 09:58:30 -04:00
Aaron Lehmann
1128fc1add Store container names in memdb
Currently, names are maintained by a separate system called "registrar".
This means there is no way to atomically snapshot the state of
containers and the names associated with them.

We can add this atomicity and simplify the code by storing name
associations in the memdb. This removes the need for pkg/registrar, and
makes snapshots a lot less expensive because they no longer need to copy
all the names. This change also avoids some problematic behavior from
pkg/registrar where it returns slices which may be modified later on.

Note that while this change makes the *snapshotting* atomic, it doesn't
yet do anything to make sure containers are named at the same time that
they are added to the database. We can do that by adding a transactional
interface, either as a followup, or as part of this PR.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-13 12:35:00 -07:00
Justin Cormack
3a1ab5b479 In the case of remounting with changed data, need to call mount
The case where we are trying to do a remount with changed filesystem specific options was missing,
we need to call `mount` as well here to change those options.

See #33844 for where we need this, as we change `tmpfs` options.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2017-07-13 15:48:58 +01:00
Sebastiaan van Stijn
96bf279c7d Merge pull request #33539 from tklauser/x-sys-unix-symlink-xattrs
Use symlink xattr functions from x/sys/unix
2017-07-12 18:04:33 -07:00
Sebastiaan van Stijn
00b218216d Merge pull request #33845 from cyphar/devicemapper-show-me-your-logs
devicemapper: rework logging and add --storage-opt dm.libdm_log_level
2017-07-12 17:46:14 -07:00
Sebastiaan van Stijn
e04dbe5ac2 Merge pull request #33877 from rhvgoyal/sync-removal
devicemapper: Wait for device removal if deferredRemoval=true and deferredDeletion=…
2017-07-12 17:35:45 -07:00
Tobias Klauser
4d966409bc system: Use symlink xattr functions from x/sys/unix
Use the symlink xattr syscall wrappers Lgetxattr and Lsetxattr from
x/sys/unix (introduced in golang/sys@b90f89a) instead of providing own
wrappers. Leave the functionality of system.Lgetxattr intact with
respect to the retry with a larger buffer, but switch it to use
unix.Lgetxattr. Also leave system.Lsetxattr intact (even though it's
just a wrapper around the corresponding function from unix) in order to
keep moby building for !linux.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-12 14:52:42 +02:00
Tobias Klauser
ef5252fc5d [pkg/term] temporarily use STD_*_HANDLE from syscall again
Due to the CL https://go-review.googlesource.com/c/39608/ in
x/sys/windows which changed the definitions of STD_INPUT_HANDLE,
STD_OUTPUT_HANDLE and STD_ERROR_HANDLE, we get the following failure
after re-vendoring x/sys/windows:

  07:47:01 # github.com/docker/docker/pkg/term
  07:47:01 pkg/term/term_windows.go:82: constant 4294967286 overflows int
  07:47:01 pkg/term/term_windows.go:88: constant 4294967285 overflows int
  07:47:01 pkg/term/term_windows.go:94: constant 4294967284 overflows int
  07:47:12 Build step 'Execute shell' marked build as failure

Temporarily switch back pkg/term to use these constants from the syscall
package and add a comment about it.

To really fix this, go-ansiterm should probably be switched to use
x/sys/windows.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2017-07-12 14:52:42 +02:00
Christopher Jones
069fdc8a08
[project] change syscall to /x/sys/unix|windows
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>

[s390x] switch utsname from unsigned to signed

per 33267e036f
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2017-07-11 08:00:32 -04:00