Commit graph

44810 commits

Author SHA1 Message Date
Albin Kerouanton
91f2d963c6
Merge iptables.probe() into iptables.detectIptables()
The former was doing some checks and logging warnings, whereas
the latter was doing the same checks but to set some internal variables.
As both are called only once and from the same place, there're now
merged together.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit 205e5278c6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 18:17:14 +01:00
Albin Kerouanton
2bf66f725c
Check ipt options before looking for ip6t
iptables package has a function `detectIptables()` called to initialize
some local variables. Since v20.10.0, it first looks for iptables bin,
then ip6tables and finally it checks what iptables flags are available
(including -C). It early exits when ip6tables isn't available, and
doesn't execute the last check.

To remove port mappings (eg. when a container stops/dies), Docker
first checks if those NAT rules exist and then deletes them. However, in
the particular case where there's no ip6tables bin available, iptables
`-C` flag is considered unavailable and thus it looks for NAT rules by
using some substring matching. This substring matching then fails
because `iptables -t nat -S POSTROUTING` dumps rules in a slighly format
than what's expected.

For instance, here's what `iptables -t nat -S POSTROUTING` dumps:

```
-A POSTROUTING -s 172.18.0.2/32 -d 172.18.0.2/32 -p tcp -m tcp --dport 9999 -j MASQUERADE
```

And here's what Docker looks for:

```
POSTROUTING -p tcp -s 172.18.0.2 -d 172.18.0.2 --dport 9999 -j MASQUERADE
```

Because of that, those rules are considered non-existant by Docker and
thus never deleted. To fix that, this change reorders the code in
`detectIptables()`.

Fixes #42127.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
(cherry picked from commit af7236f85a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 18:07:42 +01:00
Chee Hau Lim
1b27ab4c73
libnetwork/iptables: Fix test panic when execute only one test
- use local variables for chains instead of sharing global variables
- make createNewChain a t.Helper

Signed-off-by: Chee Hau Lim <ch33hau@gmail.com>
(cherry picked from commit a2cea992c2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 18:07:39 +01:00
Sebastiaan van Stijn
316231cced
daemon/graphdriver/devicemapper: simplify Udev log, and update link
Simplify the error message so that we don't have to distinguish between static-
and non-static builds. Also update the link to the storage-driver section to
use a "/go/" redirect in the docs, as the anchor link was no longer correct.
Using a "/go/" redirect  makes sure the link remains functional if docs is  moving
around.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit a5ebd28797)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 17:02:30 +01:00
Akihiro Suda
7bf0572f64
Merge pull request #44724 from thaJeztah/23.0_backport_use_the_osusergo_luke
[23.0 backport] daemon/logger/gcplogs: remove ensureHomeIfIAmStatic workaround
2023-01-01 00:12:14 +09:00
Sebastiaan van Stijn
770ace39d8
daemon/logger/gcplogs: remove ensureHomeIfIAmStatic workaround
This function was added in b86e3bee5a to
work around an issue in os/user.Current(), which SEGFAULTS when compiling
statically with cgo enabled (see golang/go#13470).

We hit similar issues in other parts, and contributed a "osusergo" build-
tag in https://go-review.googlesource.com/c/go/+/330753. The "osusergo"
build tag must be set when compiling static binaries with cgo enabled.
If that build-tag is set, the cgo implementation for user.Current() won't
be used, and a pure-go implementation is used instead;
https://github.com/golang/go/blob/go1.19.4/src/os/user/cgo_lookup_unix.go#L5

With the above in place, we no longer need this workaround, and can remove
the ensureHomeIfIAmStatic() function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 155e39187c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-31 14:50:43 +01:00
Yamazaki Masashi
5e4464798f
api: can marshal and unmarshal when args.fields is empty
Signed-off-by: Yamazaki Masashi <masi19bw@gmail.com>
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 5fed968b1d)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-29 15:40:54 -07:00
Akihiro Suda
2b5ec481e7
Merge pull request #44714 from thaJeztah/23.0_backport_carry_44600_update_delve
[23.0 backport] Update delve version
2022-12-30 07:24:15 +09:00
Bjorn Neergaard
045de829e8
Merge pull request #44708 from zhsj/23.0-btrfs-progs-6.1
[23.0 backport] daemon/graphdriver/btrfs: workaround field rename in btrfs-progs 6.1
2022-12-29 14:47:13 -07:00
Sebastiaan van Stijn
8cb02d8c58
Merge pull request #44703 from thaJeztah/23.0_backport_docker_proxy_version_flag
[23.0 backport] cmd/docker-proxy: add "-v / --version" flag
2022-12-29 21:13:44 +01:00
oscar.chen
515219b716
Update delve version
Signed-off-by: oscar.chen <2972789494@qq.com>
(cherry picked from commit ad8804885c)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-29 19:13:08 +01:00
Shengjing Zhu
0393897695 daemon/graphdriver/btrfs: workaround field rename in btrfs-progs 6.1
Closes: #44698

Signed-off-by: Shengjing Zhu <zhsj@debian.org>
(cherry picked from commit ffbbe3d103)
Signed-off-by: Shengjing Zhu <zhsj@debian.org>
2022-12-29 21:16:47 +08:00
Sebastiaan van Stijn
7199300239
cmd/docker-proxy: add "-v / --version" flag
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 12df6024db)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-28 08:24:47 +01:00
Sebastiaan van Stijn
4ddde6dde5
Merge pull request #44702 from thaJeztah/23.0_backport_fix_buildkit_ci
[23.0 backport] gha: update buildkit to fix integration tests
2022-12-27 21:54:53 +01:00
Sebastiaan van Stijn
e9075f70c7
gha: update buildkit to fix integration tests
full diff: 0bfcd83e6d...d77361423c

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit c42b304f62)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-27 19:23:28 +01:00
Tianon Gravi
cba986b340
Merge pull request #44679 from thaJeztah/23.0_backport_errdefs_improve_debug_logs
[23.0 backport] errdefs: FromStatusCode() don't log "FIXME" debug message
2022-12-21 09:38:50 -08:00
Sebastiaan van Stijn
b2d2f012b4
errdefs: FromStatusCode() don't log "FIXME" debug message
This utility is used by the client, which cannot do anything about errors
received from the API. In situations where no API connection was possible,
for example, if the client has no permissions to connect to the socket,
the request would have a "-1" status-code;
3e39ec60da/client/request.go (L133-L134)

In this case, a client with "debug" enabled, would print _and_ log a confusing
error message:

    DEBU[0000] FIXME: Got an status-code for which error does not match any expected type!!!  error="Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post \"http://%2Fvar%2Frun%2Fdocker.sock/v1.24/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile.repro&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&shmsize=0&t=repro&target=&ulimits=null&version=1\": dial unix /var/run/docker.sock: connect: permission denied" module=api status_code=-1
    Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile.repro&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&shmsize=0&t=repro&target=&ulimits=null&version=1": dial unix /var/run/docker.sock: connect: permission denied

In the above; `DEBU` logs the error (including the "FIXME"), and the second
line is the error message printed.

This was a mistake on my side when I added the `FromStatusCode` utility. I
implemented that to be the counterpart to `FromError`, but in doing so also
copied over the logging (see 1af30c50ca). That
log-message is only intended to be logged on the daemon side, for situations
where we return an error without a proper errdefs (which would result in an
500 "internal server error" to be returned by the API).

This patch removes the debug log, and a minor cleanup to explicitly return
"nil" if we didn't get an error in the first place.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 2e67c827bb)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-20 16:07:07 +01:00
Sebastiaan van Stijn
478f023783
Merge pull request #44675 from thaJeztah/23.0_backport_vendor_containerd_1.6.14
[23.0 backport] vendor: github.com/containerd/containerd v1.6.14
2022-12-20 10:59:36 +01:00
Sebastiaan van Stijn
52c35488a0
Merge pull request #44672 from thaJeztah/23.0__update_containerd_1.6.14
[23.0] update containerd binary to v1.6.14
2022-12-20 10:54:39 +01:00
Sebastiaan van Stijn
9980e64341
vendor: github.com/containerd/containerd v1.6.14
No changes in vendored code, other than the version bump

full diff: https://github.com/containerd/containerd/compare/v1.6.13...v1.6.14

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 83d7ae76e8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-19 22:46:07 +01:00
Sebastiaan van Stijn
5ebd78d5f8
update containerd binary to v1.6.14
Notable Updates

- Fix `memory.memsw.limit_in_bytes: no such file or directory` error in CRI plugin

full diff: https://github.com/containerd/containerd/compare/v1.6.13...v1.6.14

See the changelog for complete list of changes:
https://github.com/containerd/containerd/releases/tag/v1.6.14

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-19 18:38:04 +01:00
Sebastiaan van Stijn
4b78386a3a
Merge pull request #44655 from thaJeztah/23.0_containerd_binary_1.6.13
[23.0] update containerd binary to v1.6.13
2022-12-16 00:19:03 +01:00
Sebastiaan van Stijn
21c1faed45
Merge pull request #44659 from thaJeztah/23.0_backport_dockerfile_targetplatform
[23.0 backport] Dockerfile: use TARGETPLATFORM
2022-12-16 00:18:27 +01:00
CrazyMax
f6c2f20cee
Dockerfile: delve cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 19f3448c21)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:27 +01:00
CrazyMax
3d6908c79b
Dockerfile: swagger cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit ed95e1a9ff)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:27 +01:00
CrazyMax
1dc34e889c
Dockerfile: containerutility cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 879b023706)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:27 +01:00
CrazyMax
effd7661df
Dockerfile: registry cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 8d9a357a67)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:27 +01:00
CrazyMax
f3e3266373
Dockerfile: dockercli cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit a4bc2922c2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:26 +01:00
CrazyMax
fd6316e631
Dockerfile: rootlesskit cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 420591c4a2)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:26 +01:00
CrazyMax
fbe4db5be0
Dockerfile: tini cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit f0307892ea)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:26 +01:00
CrazyMax
f9a356c7e9
Dockerfile: runc cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 5e19d7ec54)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:26 +01:00
Kevin Alvarez
6e7cc3b7e3
Dockerfile: containerd cross build with TARGETPLATFORM
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 81d704d15d)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:59:19 +01:00
CrazyMax
582b29f587
Dockerfile: DOCKER_STATIC arg
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit b19c11621a)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:58:23 +01:00
CrazyMax
daa28fcf2d
Dockerfile: add dummy stage
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 69c9dfd6d5)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:58:23 +01:00
CrazyMax
4cab584a9c
Dockerfile: add cross compilation helper
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 6cba7d22e9)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 22:58:22 +01:00
Sebastiaan van Stijn
1cc21c1b81
Merge pull request #44658 from thaJeztah/23.0_backport_vendor_containerd_1.6.13
[23.0 backport] vendor: github.com/containerd/containerd v1.6.13
2022-12-15 22:36:15 +01:00
Sebastiaan van Stijn
0b12468116
vendor: github.com/containerd/containerd v1.6.13
full diff: https://github.com/containerd/containerd/compare/v1.6.12...v1.6.13

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 35cbdc9661)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 19:36:58 +01:00
Sebastiaan van Stijn
6570b654f4
vendor: github.com/Microsoft/hcsshim v0.9.6
full diff: https://github.com/Microsoft/hcsshim/compare/v0.9.5...v0.9.6

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 9300aca2a6)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 19:36:57 +01:00
Sebastiaan van Stijn
d8d11bca37
update containerd binary to v1.6.13
Notable Updates

- Update overlay snapshotter to check for tmpfs when evaluating usage of userxattr
- Update hcsschim to v0.9.6 to fix resource leak on exec
- Make swapping disabled with memory limit in CRI plugin
- Allow clients to remove created tasks with PID 0
- Fix concurrent map iteration and map write in CRI port forwarding
- Check for nil HugepageLimits to avoid panic in CRI plugin

See the changelog for complete list of changes:
https://github.com/containerd/containerd/releases/tag/v1.6.13

full diff: https://github.com/containerd/containerd/compare/v1.6.12...v1.6.13

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 19:26:40 +01:00
Sebastiaan van Stijn
0593c0600a
Merge pull request #44652 from thaJeztah/23.0_backport_bump_swarmkit
[23.0 backport] vendor: github.com/moby/swarmkit/v2 v2.0.0-20221215132206-0da442b2780f
2022-12-15 18:07:13 +01:00
Sebastiaan van Stijn
a72fb70eb0
vendor: github.com/moby/swarmkit/v2 v2.0.0-20221215132206-0da442b2780f
No changes in vendored code, but removes some indirect dependencies.

full diff: b17f02f0a0...0da442b278

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 0007490b21)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-12-15 14:29:03 +01:00
Sebastiaan van Stijn
1bb2792c76
Merge pull request #44626 from thaJeztah/23.0_backport_update_fluent_deps
[23.0 backport] vendor: update dependencies for fluentd
2022-12-15 10:55:29 +01:00
Sebastiaan van Stijn
ce27df7b3c
Merge pull request #44633 from corhere/backport-23.0/fix-44512
[23.0 Backport] Prevent containers from being included in List API before they are registered
2022-12-14 12:30:11 +01:00
Bjorn Neergaard
98cad005ca
Merge pull request #44631 from neersighted/23.0_vendor_improvements
[23.0 backport] vendor tooling improvements
2022-12-13 18:40:23 -07:00
Cory Snider
dca58c654a daemon: drop side effect from registerLinks()
(*Daemon).registerLinks() calling the WriteHostConfig() method of its
container argument is a vestigial behaviour. In the distant past,
registerLinks() would persist the container links in an SQLite database
and drop the link config from the container's persisted HostConfig. This
changed in Docker v1.10 (#16032) which migrated away from SQLite and
began using the link config in the container's HostConfig as the
persistent source of truth. registerLinks() no longer mutates the
HostConfig at all so persisting the HostConfig to disk falls outside of
its scope of responsibilities.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 388fe4aea8)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-13 18:08:21 -05:00
Cory Snider
6149c333ff daemon: don't checkpoint container until registered
(*Container).CheckpointTo() upserts a snapshot of the container to the
daemon's in-memory ViewDB and also persists the snapshot to disk. It
does not register the live container object with the daemon's container
store, however. The ViewDB and container store are used as the source of
truth for different operations, so having a container registered in one
but not the other can result in inconsistencies. In particular, the List
Containers API uses the ViewDB as its source of truth and the Container
Inspect API uses the container store.

The (*Daemon).setHostConfig() method is called fairly early in the
process of creating a container, long before the container is registered
in the daemon's container store. Due to a rogue CheckpointTo() call
inside setHostConfig(), there is a window of time where a container can
be included in a List Containers API response but "not exist" according
to the Container Inspect API and similar endpoints which operate on a
particular container. Remove the rogue call so that the caller has full
control over when the container is checkpointed and update callers to
checkpoint explicitly. No changes to (*Daemon).create() are needed as it
checkpoints the fully-created container via (*Daemon).Register().

Fixes #44512.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 0141c6db81)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-13 18:08:21 -05:00
Cory Snider
42bffae5ff daemon: fix GetContainer() returning (nil, nil)
GetContainer() would return (nil, nil) when looking up a container
if the container was inserted into the containersReplica ViewDB but not
the containers Store at the time of the lookup. Callers which reasonably
assume that the returned err == nil implies returned container != nil
would dereference a nil pointer and panic. Change GetContainer() so that
it always returns a container or an error.

Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit 00157a42d3)
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-12-13 18:08:21 -05:00
Bjorn Neergaard
a6818fd4cb
hack: introduce validate/no-module
Moby is not a Go module; to prevent anyone from mistakenly trying to
convert it to one before we are ready, introduce a check (usable in CI
and locally) for a go.mod file.

This is preferable to trying to .gitignore the file as we can ensure
that a mistakenly created go.mod is surfaced by Git-based tooling and is
less likely to surprise a contributor.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit 25c3421802)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-13 14:42:07 -07:00
Bjorn Neergaard
45fe08c93c
hack: replace go-mod-prepare.sh with wrapper script
To make the local build environment more correct and consistent, we
should never leave an uncommitted go.mod in the tree; however, we need a
go.mod for certain commands to work properly. Use a wrapper script to
create and destroy the go.mod as needed instead of potentially changing
tooling behavior by leaving it.

If a go.mod already exists, this script will warn and call the wrapped
command with GO111MODULE=on.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit a449f77774)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-13 14:42:05 -07:00
Bjorn Neergaard
c508d13372
hack/go-mod-prepare.sh: find root robustly; make steps transparent
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
(cherry picked from commit af8e955e8f)
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2022-12-13 14:41:28 -07:00