Commit graph

2666 commits

Author SHA1 Message Date
Tibor Vass
ca8022ec63 builder: return image ID in API when using buildkit
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-07-03 19:11:02 +00:00
Sebastiaan van Stijn
1d7132e3d2
Merge pull request #37356 from Microsoft/jjh/new37316
LCOW: lazycontext: Use correct lstat, fix archive check
2018-06-28 12:32:31 +02:00
Tonis Tiigi
f099771665 system: add back lcow validation function
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-27 15:24:26 -07:00
John Howard
a5aed699cf LCOW: lazycontext: Use correct lstat, fix archive check
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-06-27 13:29:21 -07:00
Tonis Tiigi
48b93419dc builder: buildkit rebase update
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-26 22:59:17 -07:00
Dennis Chen
7f334d3acf Initial support for OCI multi-platform image
Add the OCI spec compatible image support in client side.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-06-26 11:39:29 -07:00
John Howard
35193c0e7d LCOW: Auto-select OS
Signed-off-by: John Howard <jhoward@microsoft.com>

Addresses https://github.com/moby/moby/pull/35089#issuecomment-367802698.
This change enables the daemon to automatically select an image under LCOW
that can be used if the API doesn't specify an explicit platform.

For example:

FROM supertest2014/nyan
ADD Dockerfile /

And docker build . will download the linux image (not a multi-manifest image)

And similarly docker pull ubuntu will match linux/amd64
2018-06-26 11:37:43 -07:00
Sebastiaan van Stijn
cb37987ee1
Merge pull request #37301 from Mischi/fix-define-self
fix build on OpenBSD by defining Self()
2018-06-25 20:25:38 +02:00
Yong Tang
0ed55cd220 Fix a small spacing issue
As a follow up to https://github.com/moby/moby/pull/37331#pullrequestreview-131411625

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-06-24 17:27:50 +00:00
Martin Muzatko
7015fc03b7 Just satisfying my OCD - fixed comment spacing and removed a hidden character
Signed-off-by: Martin Muzatko <martin@happy-css.com>
2018-06-22 17:17:08 +02:00
Derek McGowan
ac5d363e67
Update overlay2 to use naive diff for changes
The archive changes function is not implemented correctly
to handle opaque directories.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-06-20 11:07:36 -07:00
Fabian Raetz
21537b818d fix build on OpenBSD by defining Self()
Signed-off-by: Fabian Raetz <fabian.raetz@gmail.com>
2018-06-16 19:09:04 +02:00
Vincent Demeester
3845728524
Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-13 09:04:30 +02:00
Joe Ferguson
76e99e1a8b Close readclosers returned by DecompressStream
Signed-off-by: Joe Ferguson <joe@infosiftr.com>
2018-06-07 11:16:13 -07:00
Sebastiaan van Stijn
5037c5a8ce
Merge pull request #36688 from cpuguy83/volumes_service
Extract volume interaction to a volumes service
2018-06-05 02:16:20 +02:00
Anusha Ragunathan
71cd53e4a1
Merge pull request #37177 from cpuguy83/minor_plugin_addr_change
Move plugin client to separate interface
2018-05-31 08:22:04 -07:00
Vincent Demeester
2ea4ece674
Merge pull request #37174 from thaJeztah/no_additional_magic_matching
Be explicit about github.com prefix being a legacy feature
2018-05-31 09:45:59 +02:00
Brian Goff
7c77df8acc Move plugin client to separate interface
This makes it a bit simpler to remove this interface for v2 plugins
and not break external projects (libnetwork and swarmkit).

Note that before we remove the `Client()` interface from `CompatPlugin`
libnetwork and swarmkit must be updated to explicitly check for the v1
client interface as is done int his PR.

This is just a minor tweak that I realized is needed after trying to
implement the needed changes on libnetwork.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-30 15:22:10 -04:00
Sebastiaan van Stijn
babb0c14fd
Be explicit about github.com prefix being a legacy feature
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-30 11:09:14 +02:00
Brian Goff
f51a96c016 Move plugin client creation to the extension point
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 15:18:53 -04:00
Brian Goff
e4b6adc88e Extract volume interaction to a volumes service
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 14:21:07 -04:00
Sebastiaan van Stijn
f23c00d870
Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-23 17:50:54 +02:00
Kir Kolyshkin
d78e885326 pkg/mount/TestMount: fix wrt selinux
Sometimes docker-master CI fails on rhel4+selinux configuration,
like this:

--- FAIL: TestMount (0.12s)
    --- FAIL: TestMount/none-remount,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "rw,size=128k"
    --- FAIL: TestMount/none-remount,ro,size=128k (0.01s)
    	mounter_linux_test.go:209: unexpected mount option "seclabel" expected "ro,size=128k"

Earlier, commit 8bebd42df2 (PR #34965) fixed this failure,
but not entirely (i.e. the test is now flaky). It looks like
either selinux detection code is not always working (it won't
work in d-in-d), or the kernel might or might not add 'seclabel'
option).

As the subject of this test case is definitely not selinux,
it can just ignore the option added by it.

While at it, fix error messages:
 - add missing commas;
 - fix a typo;
 - allow for clear distinction between mount
   and vfs (per-superblock) options.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-22 23:30:47 -07:00
Sebastiaan van Stijn
3646562a5c
Merge pull request #37088 from ohbarye/fix-typos-duplicated-the
Fix typos: remove duplicated "the"
2018-05-19 20:57:47 +02:00
Sebastiaan van Stijn
4f8c870d62
Fix some linting issues
These showed locally when running `make validate`. CI doesn't seem to have the
same (possibly it's disabled in the configuration)

    builder/fscache/fscache.go:618::error: github.com/docker/docker/vendor/github.com/tonistiigi/fsutil.StatInfo composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:44::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:20::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:113::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:110::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:171::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:413::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:203::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:584::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:109::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:388::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/volumes_windows.go:27::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    integration/service/network_test.go:31::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    api/server/server.go:129:10⚠️ should not use basic type string as key in context.WithValue (golint)
    integration/service/network_test.go:54::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:61::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:74::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    pkg/archive/archive_windows.go:76::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    plugin/manager_linux.go:56::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-17 19:28:27 +02:00
Vincent Demeester
6a9bcc51df
Merge pull request #37074 from skyc024/fix-typos
Fix typos
2018-05-17 16:00:26 +02:00
ohbarye
0f95b23d98 Fix typos: remove duplicated "the"
Signed-off-by: Masato Ohba <over.rye@gmail.com>
2018-05-17 21:49:51 +09:00
chenyuzhu
5ac0d2f0b8 Fix typos
Signed-off-by: chenyuzhu <chenyuzhi@oschina.cn>
2018-05-16 15:31:12 +08:00
Tonis Tiigi
f784907eae jsonmessage: pass message to aux callback
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-14 15:48:25 -07:00
Sebastiaan van Stijn
d4e48af48d
Merge pull request #36918 from Sh4d1/master
add verification to check if escapeKeys have elements (pkg/term/proxy.go)
2018-04-30 11:57:21 +02:00
Patrik Cyvoct
d339130f30
Add verification and test to check if escapeKeys is not empty
In pkg/term/proxy.go and pkg/term/proxy_test.go, check if escapeKeys is empty and if it is, return the one key read

Signed-off-by: Patrik Cyvoct <patrik@ptrk.io>
2018-04-30 00:32:43 +02:00
Kir Kolyshkin
7d62e40f7e Switch from x/net/context -> context
Since Go 1.7, context is a standard package. Since Go 1.9, everything
that is provided by "x/net/context" is a couple of type aliases to
types in "context".

Many vendored packages still use x/net/context, so vendor entry remains
for now.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-23 13:52:44 -07:00
Yong Tang
5c233cf431
Merge pull request #36913 from vdemeester/test-skip-non-root
Skip some tests requires root uid when run as user…
2018-04-23 11:42:42 -07:00
Sebastiaan van Stijn
20b524bf2e
Merge pull request #36920 from kolyshkin/cancel-func
context.WithTimeout: do call the cancel func
2018-04-23 20:14:25 +02:00
Vincent Demeester
a7999aaa53
Skip some tests requires root uid when run as user
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-04-23 10:14:39 +02:00
Vincent Demeester
5d8b88b114
Fix typo in idtools tests
It should check `os.Geteuid` with `uid` instead of `os.Getegid`.
On the container (where the tests run), the uid and gid seems to be
the same, thus this doesn't fail.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-04-23 10:14:07 +02:00
Vincent Demeester
53982e3fc1
Merge pull request #36091 from kolyshkin/mount
pkg/mount improvements
2018-04-21 11:03:54 +02:00
Kir Kolyshkin
05e2f7e2fa context.WithTimeout: do call the cancel func
govet complains (when using standard "context" package):

> the cancel function returned by context.WithTimeout should be called,
> not discarded, to avoid a context leak (vet)

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-20 12:27:13 -07:00
Kir Kolyshkin
a1d095199d mount.Unmount(): don't look into /proc/self/mountinfo
Now, every Unmount() call takes a burden to parse the whole nine yards
of /proc/self/mountinfo to figure out whether the given mount point is
mounted or not (and returns an error in case parsing fails somehow).

Instead, let's just call umount() and ignore EINVAL, which results
in the same behavior, but much better performance.

Note that EINVAL is returned from umount(2) not only in the case when
`target` is not mounted, but also for invalid flags. As the flags are
hardcoded here, it can't be the case.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:50 -07:00
Kir Kolyshkin
c611f18a7f pkg/mount/mountinfo_linux: parser speed up
The mountinfo parser implemented via `fmt.Sscanf()` is slower than the one
using `strings.Split()` and `strconv.Atoi()`. This rewrite helps to speed it
up to a factor of 8x, here is a result from go bench:

> BenchmarkParsingScanf-4      	     300	  22294112 ns/op
> BenchmarkParsingSplit-4      	    3000	   2780703 ns/op

I tried other approaches, such as using `fmt.Sscanf()` for the first
three (integer) fields and `strings.Split()` for the rest, but it slows
things down considerably:

> BenchmarkParsingMixed-4      	    1000	   8827058 ns/op

Note the old code uses `fmt.Sscanf`, when a linear search for '-' field,
when a split for the last 3 fields. The new code relies on a single
split.

I have also added more comments to aid in future development.

Finally, the test data is fixed to now have white space before the first field.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:42 -07:00
Kir Kolyshkin
871c957242 getSourceMount(): simplify
The flow of getSourceMount was:
 1 get all entries from /proc/self/mountinfo
 2 do a linear search for the `source` directory
 3 if found, return its data
 4 get the parent directory of `source`, goto 2

The repeated linear search through the whole mountinfo (which can have
thousands of records) is inefficient. Instead, let's just

 1 collect all the relevant records (only those mount points
   that can be a parent of `source`)
 2 find the record with the longest mountpath, return its data

This was tested manually with something like

```go
func TestGetSourceMount(t *testing.T) {
	mnt, flags, err := getSourceMount("/sys/devices/msr/")
	assert.NoError(t, err)
	t.Logf("mnt: %v, flags: %v", mnt, flags)
}
```

...but it relies on having a specific mount points on the system
being used for testing.

[v2: add unit tests for ParentsFilter]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:49:17 -07:00
Kir Kolyshkin
bb934c6aca pkg/mount: implement/use filter for mountinfo parsing
Functions `GetMounts()` and `parseMountTable()` return all the entries
as read and parsed from /proc/self/mountinfo. In many cases the caller
is only interested only one or a few entries, not all of them.

One good example is `Mounted()` function, which looks for a specific
entry only. Another example is `RecursiveUnmount()` which is only
interested in mount under a specific path.

This commit adds `filter` argument to `GetMounts()` to implement
two things:
 1. filter out entries a caller is not interested in
 2. stop processing if a caller is found what it wanted

`nil` can be passed to get a backward-compatible behavior, i.e. return
all the entries.

A few filters are implemented:
 - `PrefixFilter`: filters out all entries not under `prefix`
 - `SingleEntryFilter`: looks for a specific entry

Finally, `Mounted()` is modified to use `SingleEntryFilter()`, and
`RecursiveUnmount()` is using `PrefixFilter()`.

Unit tests are added to check filters are working.

[v2: ditch NoFilter, use nil]
[v3: ditch GetMountsFiltered()]
[v4: add unit test for filters]
[v5: switch to gotestyourself]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-04-19 14:48:09 -07:00
Alexandre Jomin
0d82bc142c pkg/term add build tag for netbsd
Signed-off-by: Alexandre Jomin <alexandrejomin@gmail.com>
2018-04-18 15:16:06 +02:00
Tibor Vass
61138fb5fc
Merge pull request #35739 from thaJeztah/bump-go-1.10
Bump Golang to 1.10.1
2018-04-13 13:56:38 -10:00
Anusha Ragunathan
ff6a103067
Merge pull request #36778 from odg0318/master
Using authorization plugin, I changed Content-Type check routine.
2018-04-12 17:39:38 -07:00
Derek McGowan
fb170206ba
Set format in archiver
Prevent changing the tar output by setting the format to
PAX and keeping the times truncated.
Without this change the archiver will produce different tar
archives with different hashes with go 1.10.
The addition of the access and changetime timestamps would
also cause diff comparisons to fail.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-04-12 13:34:33 -07:00
Derek McGowan
a422774e59
Fix tarsum for go 1.10
Remove invalid flush commands, flush should only occur when file
has been completely written. This is already handle, remove these calls.
Ensure data gets written after EOF in correct order and before close.
Remove gname and uname from sum for hash compatibility.
Update tarsum tests for gname/uname removal.
Return valid length after eof.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-04-12 13:34:14 -07:00
Brian Goff
74f8e47352 Limit authz response buffer
When the authz response buffer limit is hit, perform a flush.
This prevents excessive buffer sizes, especially on large responses
(e.g. `/containers/<id>/archive` or `/containers/<id>/export`).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-04-11 15:36:36 -04:00
Vincent Demeester
5219725890
Merge pull request #35518 from cyphar/libdm-dlsym-deferred_remove
pkg: devmapper: dynamically load dm_task_deferred_remove
2018-04-11 14:11:16 +02:00
odg0318
6ac73d34e4 If Content-Type is application/json;charset=UTF-8, RequestBody is empty.
Signed-off-by: odg0318 <odg0318@gmail.com>
2018-04-05 04:03:45 -04:00