Commit graph

46384 commits

Author SHA1 Message Date
Sebastiaan van Stijn
19bd0730ef
vendor: github.com/moby/swarmkit/v2 v2.0.0-20230627115642-ad0f3ae162fa
no changes to vendored files

full diff: 36334ed187...ad0f3ae162

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-27 14:01:20 +02:00
Sebastiaan van Stijn
b1569b9c18
Merge pull request #45820 from thaJeztah/re_enable_dco
Revert "Temporarily skip DCO check"
2023-06-27 11:00:17 +02:00
Sebastiaan van Stijn
e2dad6c3ff
Revert "Temporarily skip DCO check"
re-enable the DCO check, which was temporarily disabled to migrate
old commits from github.com/docker/libkv

This reverts commit 7d7225fae6.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-27 10:30:34 +02:00
Sebastiaan van Stijn
8d4982ed33
Merge pull request #45817 from thaJeztah/integrate_libkv_2
Integrate github.com/docker/libkv
2023-06-27 10:27:26 +02:00
Sebastiaan van Stijn
7d7225fae6
Temporarily skip DCO check
The migrated history has some commits that missed a DCO:

These commits do not have a proper 'Signed-off-by:' marker:

 - 3fa22634a617e2c52d2c5f061826e5107e27985f
 - 9b11053e9147884c43c9a9d8ebfcd7bb9470e8b5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:52:28 +02:00
Sebastiaan van Stijn
e60cda7051
libnetwork/internal/kvstore/boltdb: fix linting issues
libnetwork/internal/kvstore/boltdb/boltdb.go:452:28: unnecessary conversion (unconvert)
                _ = bucket.Delete([]byte(key))
                                        ^
    libnetwork/internal/kvstore/boltdb/boltdb.go:425:2: S1023: redundant `return` statement (gosimple)
        return
        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:52:04 +02:00
Sebastiaan van Stijn
d18b89ced6
libnetwork/internal/kvstore: remove some unused code
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:51:53 +02:00
Sebastiaan van Stijn
b873d70369
replace libkv with local fork
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:51:42 +02:00
Sebastiaan van Stijn
5d25143ef3
libnetwork/kvstore: rewrite code for new location
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:49:52 +02:00
Sebastiaan van Stijn
3887475971
Integrate github.com/docker/libkv
A reduced set of the dependency, only taking the parts that are used. Taken from
upstream commit: dfacc563de

    # install filter-repo (https://github.com/newren/git-filter-repo/blob/main/INSTALL.md)
    brew install git-filter-repo

    cd ~/projects

    # create a temporary clone of docker
    git clone https://github.com/docker/libkv.git temp_libkv
    cd temp_libkv

    # create branch to work with
    git checkout -b migrate_libkv

    # remove all code, except for the files we need; rename the remaining ones to their new target location
    git filter-repo --force \
        --path libkv.go \
        --path store/store.go \
        --path store/boltdb/boltdb.go \
        --path-rename libkv.go:libnetwork/internal/kvstore/kvstore_manage.go \
        --path-rename store/store.go:libnetwork/internal/kvstore/kvstore.go \
        --path-rename store/boltdb/:libnetwork/internal/kvstore/boltdb/

    # go to the target github.com/moby/moby repository
    cd ~/projects/docker

    # create a branch to work with
    git checkout -b integrate_libkv

    # add the temporary repository as an upstream and make sure it's up-to-date
    git remote add temp_libkv ~/projects/temp_libkv
    git fetch temp_libkv

    # merge the upstream code, rewriting "pkg/symlink" to "symlink"
    git merge --allow-unrelated-histories --signoff -S temp_libkv/migrate_libkv

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 20:47:08 +02:00
Bjorn Neergaard
8805e38398
Merge pull request #45799 from cpuguy83/containerd_logrus
Switch all logging to use containerd log pkg
2023-06-26 11:51:44 -06:00
Bjorn Neergaard
ddae599cab
Merge pull request #45814 from thaJeztah/cleanup_homedir
pkg/homedir: use os.UserHomeDir(), deprecate GetShortcutString(), Key()
2023-06-26 08:11:45 -06:00
Sebastiaan van Stijn
3c1de2e667
pkg/homedir: deprecate Key() utility
This utility was only used in tests, and internally, and no longer
used since we switch to using os.UserHomeDir() from stdlib.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:46:03 +02:00
Sebastiaan van Stijn
ddd9665289
pkg/homedir: deprecate GetShortcutString() utility
This function was last used in the pkg/mflag package, which was removed
in 14712f9ff0, and is no longer used in
libnetwork code since e6de8aec2f

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:45:21 +02:00
Sebastiaan van Stijn
6876e45f9e
pkg/homedir: unify implementations
Unify some of the exported functions instead of maintaining separate functions
per platform.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:43:26 +02:00
Sebastiaan van Stijn
0aca7f2a0d
pkg/homedir: use os.UserHomeDir()
Use os.UserHomeDir() instead of writing our own, but keep the fallback
on Linux.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 14:30:42 +02:00
Sebastiaan van Stijn
db381fb21b
Merge pull request #45812 from thaJeztah/no_homedir
integration-cli: don't use pkg/homedir in test
2023-06-26 14:28:41 +02:00
Sebastiaan van Stijn
0215a62d5b
integration-cli: don't use pkg/homedir in test
I'm considering deprecating the "Key()" utility, as it was only
used in tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 13:39:43 +02:00
Bjorn Neergaard
56261fabcb
Merge pull request #45743 from thaJeztah/testutil_nits
testutil: remove unused TEST_OSTYPE env-var, and remove Execution.OSType field
2023-06-26 05:04:12 -06:00
Sebastiaan van Stijn
76d8bfdff4
testutil/environment: remove Execution.OSType field
This field was added in f0e5b3d7d8 to
account for older versions of the engine (Docker EE LTS versions), which
did not yet provide the OSType field in Docker info, and had to be manually
set using the TEST_OSTYPE env-var.

This patch removes the field in favor of the equivalent in DaemonInfo. It's
more verbose, but also less ambiguous what information we're using (i.e.,
the platform the daemon is running on, not the local platform).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 11:18:09 +02:00
Sebastiaan van Stijn
325786430b
testutil/environment: remove unused TEST_OSTYPE env-var
This env-var was added in f0e5b3d7d8 to
account for older versions of the engine (Docker EE LTS versions), which
did not yet provide the OSType field in Docker info.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 11:17:12 +02:00
Sebastiaan van Stijn
0fe2ac0437
testutil/environment: rename var that collided with import
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-26 11:17:09 +02:00
Brian Goff
7dd547c5ff
Merge pull request #45802 from dperny/fix-missing-csi-topology
Fix missing Topology in NodeCSIInfo
2023-06-24 08:38:40 -07:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Sebastiaan van Stijn
da05a2ef7f
Merge pull request #45800 from corhere/fix-45788-restore-exit-status
daemon: fix restoring container with missing task
2023-06-23 20:21:07 +02:00
Drew Erny
cdb1293eea Fix missing Topology in NodeCSIInfo
Added code to correctly retrieve and convert the Topology from the gRPC
Swarm Node.

Signed-off-by: Drew Erny <derny@mirantis.com>
2023-06-23 11:45:50 -05:00
Cory Snider
165dfd6c3e daemon: fix restoring container with missing task
Before 4bafaa00aa, if the daemon was
killed while a container was running and the container shim is killed
before the daemon is restarted, such as if the host system is
hard-rebooted, the daemon would restore the container to the stopped
state and set the exit code to 255. The aforementioned commit introduced
a regression where the container's exit code would instead be set to 0.
Fix the regression so that the exit code is once against set to 255 on
restore.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-23 11:28:45 -04:00
Bjorn Neergaard
8d070e30f5
Merge pull request #45797 from corhere/fix-health-probe-double-unlock
daemon: fix double-unlock in health check probe
2023-06-22 18:17:09 -06:00
Cory Snider
5dbb5cc3e5
Merge pull request #45792 from corhere/fix-45770-processevent-nil-check
daemon: fix panic on failed exec start
2023-06-22 18:05:10 -04:00
Cory Snider
786c9adaa2 daemon: fix double-unlock in health check probe
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-22 17:48:21 -04:00
Cory Snider
3b28a24e97 daemon: fix panic on failed exec start
If an exec fails to start in such a way that containerd publishes an
exit event for it, daemon.ProcessEvent will race
daemon.ContainerExecStart in handling the failure. This race has been a
long-standing bug, which was mostly harmless until
4bafaa00aa. After that change, the daemon
would dereference a nil pointer and crash if ProcessEvent won the race.
Restore the status quo buggy behaviour by adding a check to skip the
dereference if execConfig.Process is nil.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-06-22 17:04:51 -04:00
Sebastiaan van Stijn
eb76c93098
Merge pull request #45767 from crazy-max/fix-host-gateway
builder: pass host-gateway IP as worker label
2023-06-22 22:56:57 +02:00
Sebastiaan van Stijn
b3843992fc
Merge pull request #45781 from neersighted/c8d_stargz_refcount 2023-06-21 16:46:51 +02:00
Sebastiaan van Stijn
5ff7be32f0
Merge pull request #45784 from vvoland/busybox-5007 2023-06-21 16:29:48 +02:00
Paweł Gronowski
e010223186
contrib/busybox: Update to FRP-5007-g82accfc19
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-21 14:15:05 +02:00
CrazyMax
21e50b89c9
builder: pass host-gateway IP as worker label
We missed a case when parsing extra hosts from the dockerfile
frontend so the build fails.

To handle this case we need to set a dedicated worker label
that contains the host gateway IP so clients like Buildx
can just set the proper host:ip when parsing extra hosts
that contain the special string "host-gateway".

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-06-21 13:21:38 +02:00
Sebastiaan van Stijn
404160ad87
Merge pull request #45579 from tonistiigi/otlp-traces-for-history
builder-next: enable OTLP tracing for history records
2023-06-21 01:05:49 +02:00
Sebastiaan van Stijn
ab60412cb4
Merge pull request #45736 from thaJeztah/reserve_once
daemon: registerName(): don't reserve name twice
2023-06-20 23:47:40 +02:00
Bjorn Neergaard
21c0a54a6b
c8d: mark stargz as requiring reference-counted mounts
The stargz snapshotter cannot be re-mounted, so the reference-counted
path must be used.

Co-authored-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-06-20 12:59:13 -06:00
Sebastiaan van Stijn
66c497c5f1
Merge pull request #45738 from thaJeztah/dont_cancel_stop
don't cancel container stop when cancelling context
2023-06-20 17:01:53 +02:00
Akihiro Suda
3ecceafa32
Merge pull request #45776 from thaJeztah/update_buildx_0.11
Dockerfile: update buildx to v0.11.0
2023-06-20 22:49:33 +09:00
Sebastiaan van Stijn
4d831949a7
Dockerfile: update buildx to v0.11.0
Update the version of buildx we use in the dev-container to v0.11.0;
https://github.com/docker/buildx/releases/tag/v0.11.0

Full diff: https://github.com/docker/buildx/compare/v0.10.5..v0.11.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-20 12:34:21 +02:00
Sebastiaan van Stijn
fc94ed0a86
don't cancel container stop when cancelling context
Commit 90de570cfa passed through the request
context to daemon.ContainerStop(). As a result, cancelling the context would
cancel the "graceful" stop of the container, and would proceed with forcefully
killing the container.

This patch partially reverts the changes from 90de570cfa
and breaks the context to prevent cancelling the context from cancelling the stop.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-20 11:53:23 +02:00
Sebastiaan van Stijn
3fae7ea16f
Merge pull request #45769 from thaJeztah/fix_docker_py
testing: temporarily pin docker-py tests to use "bullseye"
2023-06-20 11:51:23 +02:00
Sebastiaan van Stijn
19d860fa9d
testing: temporarily pin docker-py tests to use "bullseye"
The official Python images on Docker Hub switched to debian bookworm,
which is now the current stable version of Debian.

However, the location of the apt repository config file changed, which
causes the Dockerfile build to fail;

    Loaded image: emptyfs:latest
    Loaded image ID: sha256:0df1207206e5288f4a989a2f13d1f5b3c4e70467702c1d5d21dfc9f002b7bd43
    INFO: Building docker-sdk-python3:5.0.3...
    tests/Dockerfile:6
    --------------------
       5 |     ARG APT_MIRROR
       6 | >>> RUN sed -ri "s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g" /etc/apt/sources.list \
       7 | >>>     && sed -ri "s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g" /etc/apt/sources.list
       8 |
    --------------------
    ERROR: failed to solve: process "/bin/sh -c sed -ri \"s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g\" /etc/apt/sources.list     && sed -ri \"s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g\" /etc/apt/sources.list" did not complete successfully: exit code: 2

This needs to be fixed in docker-py, but in the meantime, we can pin to
the bullseye variant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-20 10:48:42 +02:00
Tonis Tiigi
cfa08f8366
builder-next: enable OTLP tracing for history records
This enables picking up OTLP tracing context for the gRPC
requests.

Also sets up the in-memory recorder that BuildKit History API
can use to store the traces associated with specific build
in a database after build completes.

This doesn't enable Jaeger tracing endpoints from env
but this can be easily enabled by adding another import if
maintainers want it.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2023-06-15 18:40:24 -07:00
Sebastiaan van Stijn
97455cc31f
Merge pull request #45504 from tianon/oci-history-master
Use OCI "History" type instead of inventing our own copy
2023-06-15 15:27:43 +02:00
Sebastiaan van Stijn
d2e7a6e799
Merge pull request #45744 from thaJeztah/update_go_1.20.5
update go to go1.20.5
2023-06-14 22:19:55 +02:00
Sebastiaan van Stijn
98a44bb18e
update go to go1.20.5
go1.20.5 (released 2023-06-06) includes four security fixes to the cmd/go and
runtime packages, as well as bug fixes to the compiler, the go command, the
runtime, and the crypto/rsa, net, and os packages. See the Go 1.20.5 milestone
on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.5+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.4...go1.20.5

These minor releases include 3 security fixes following the security policy:

- cmd/go: cgo code injection
  The go command may generate unexpected code at build time when using cgo. This
  may result in unexpected behavior when running a go program which uses cgo.

  This may occur when running an untrusted module which contains directories with
  newline characters in their names. Modules which are retrieved using the go command,
  i.e. via "go get", are not affected (modules retrieved using GOPATH-mode, i.e.
  GO111MODULE=off, may be affected).

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29402 and Go issue https://go.dev/issue/60167.

- runtime: unexpected behavior of setuid/setgid binaries

  The Go runtime didn't act any differently when a binary had the setuid/setgid
  bit set. On Unix platforms, if a setuid/setgid binary was executed with standard
  I/O file descriptors closed, opening any files could result in unexpected
  content being read/written with elevated prilieges. Similarly if a setuid/setgid
  program was terminated, either via panic or signal, it could leak the contents
  of its registers.

  Thanks to Vincent Dehors from Synacktiv for reporting this issue.

  This is CVE-2023-29403 and Go issue https://go.dev/issue/60272.

- cmd/go: improper sanitization of LDFLAGS

  The go command may execute arbitrary code at build time when using cgo. This may
  occur when running "go get" on a malicious module, or when running any other
  command which builds untrusted code. This is can by triggered by linker flags,
  specified via a "#cgo LDFLAGS" directive.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29404 and CVE-2023-29405 and Go issues https://go.dev/issue/60305 and https://go.dev/issue/60306.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-14 12:47:05 +02:00
Sebastiaan van Stijn
a9788886e6
Merge pull request #45720 from ndeloof/copy_uidgid
fix `docker cp -a` failing to access `/` in container
2023-06-14 01:11:36 +02:00