Commit graph

45436 commits

Author SHA1 Message Date
Jan Garcia
6ab12ec8f4 rootless: move ./rootless to ./pkg/rootless
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
2023-01-09 16:26:06 +01:00
Jan Garcia
c1bd5e9144 Use user paths for plugin discovery in rootless mode
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
2023-01-09 16:26:06 +01:00
Sebastiaan van Stijn
bffe6d5670
vendor: github.com/containerd/containerd v1.6.15
no changes to vendored files

full diff: https://github.com/containerd/containerd/compare/v1.6.14...v1.6.15

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-09 13:26:32 +01:00
Sebastiaan van Stijn
eaa7b49bd6
Merge pull request #44749 from vvoland/fix-test-race-logs-goroutines
integration-cli: Fix hangs in TestLogsFollowGoroutines*
2023-01-09 13:00:14 +01:00
Akihiro Suda
89fb8b32f6
graphdriver/btrfs: needs kernel headers >= 4.12, not >= 4.7
`linux/btrfs_tree.h` was not installed to `/usr/include` until kernel 4.12
fcc8487d47

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2023-01-08 05:04:47 +09:00
Josh Chorlton
c12d7b6d21 upgrade to aws sdk go v2
Co-Authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Josh Chorlton <jchorlton@gmail.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Josh Chorlton <jchorlton@gmail.com>
2023-01-07 17:27:31 +00:00
Bjorn Neergaard
6834304feb
Merge pull request #44761 from neersighted/btrfs_uapi
graphdriver/btrfs: depend on kernel UAPI instead of libbtrfs
2023-01-06 15:20:20 -07:00
Bjorn Neergaard
aa80c33360
Dockerfile.*: drop libbtrfs
This is no longer necessary after the switch to the kernel UAPI.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-06 13:23:09 -07:00
Bjorn Neergaard
c9d632e485
PACKAGERS: update Btrfs graphdriver dependencies
We only need suitable UAPI headers now. They are available on kernel 4.7
and newer; out of the distributions currently in support that users
might be interested in, only Enterprise Linux 7 has too old a kernel
(3.10).

Users of Enterprise Linux 7 distros can compile using a newer platform,
disable the Btrfs graphdriver as documented in this file, or use newer
kernel headers on their older distro.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-06 13:23:06 -07:00
Bjorn Neergaard
d3778d65fa
graphdriver/btrfs: use free wrapper consistently
While the Cgo in this entire file is quite questionable, that is a task
for another day.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-06 13:23:05 -07:00
Bjorn Neergaard
3208dcabdc
graphdriver/btrfs: use kernel UAPI headers
By relying on the kernel UAPI (userspace API), we can drop a dependency
and simplify building Moby, while also ensuring that we are using a
stable/supported source of the C types and defines we need.

btrfs-progs mirrors the kernel headers, but the headers it ships with
are not the canonical source and as [we have seen before][44698], could
be subject to changes.

Depending on the canonical headers from the kernel both is more
idiomatic, and ensures we are protected by the kernel's promise to not
break userspace.

  [44698]: https://github.com/moby/moby/issues/44698

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-06 13:22:58 -07:00
Bjorn Neergaard
1449c82484
graphdriver/btrfs: drop version information
This is actually quite meaningless as we are reporting the libbtrfs
version, but we do not use libbtrfs. We only use the kernel interface to
btrfs instead.

While we could report the version of the kernel headers in play, they're
rather all-or-nothing: they provide the structures and defines we need,
or they don't. As such, drop all version information as the host kernel
version is the only thing that matters.

Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
2023-01-06 11:13:23 -07:00
Sebastiaan van Stijn
937491288e
Merge pull request #43818 from thaJeztah/image_inspect
add support for image inspect with containerd-integration
2023-01-06 17:40:15 +01:00
Brian Goff
73e09ddecf
Merge pull request #43787 from thaJeztah/memdb_nits
container: ViewDB: cleanup error-types
2023-01-06 08:09:06 -08:00
Sebastiaan van Stijn
8fb13036ee
Merge pull request #44754 from tianon/requireRegistryMatch
Remove reference to "requireRegistryMatch"
2023-01-05 22:10:15 +01:00
Tianon Gravi
fdcab58ccf Remove reference to "requireRegistryMatch"
This option appears to never have actually existed in the merged implementation. 😄

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-01-05 11:28:22 -08:00
Sebastiaan van Stijn
57af88a654
Merge pull request #44748 from vvoland/buildflags-nopie-whenrace
hack/make: Don't add -buildmode=pie with -race
2023-01-05 20:24:21 +01:00
Bjorn Neergaard
9353fc6a27
Merge pull request #44744 from KirkEasterson/kirk/fix-broken-contributing-link
Updated outdated docker contributing guidelines link
2023-01-05 10:48:10 -07:00
Bjorn Neergaard
9409b5623d
Merge pull request #44746 from crazy-max/fix-binary-daemon
hack: restore copy_binaries func
2023-01-05 10:45:53 -07:00
Paweł Gronowski
deb4910c5b
integration-cli: Fix hanging TestLogsFollowGoroutines*
cmd.Wait is called twice from different goroutines which can cause the
test to hang completely. Fix by calling Wait only once and sending its
return value over a channel.

In TestLogsFollowGoroutinesWithStdout also added additional closes and
process kills to ensure that we don't leak anything in case test returns
early because of failed test assertion.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-05 18:10:32 +01:00
Bjorn Neergaard
e9275392c1
Merge pull request #44742 from akerouanton/fix-44688
Clear conntrack entries for published UDP ports
2023-01-05 10:04:29 -07:00
Nicolas De Loof
1616a09b61 add support for image inspect with containerd-integration
This is a squashed version of various PRs (or related code-changes)
to implement image inspect with the containerd-integration;

- add support for image inspect
- introduce GetImageOpts to manage image inspect data in backend
- GetImage to return image tags with details
- list images matching digest to discover all tags
- Add ExposedPorts and Volumes to the image returned
- Refactor resolving/getting images
- Return the image ID on inspect
- consider digest and ignore tag when both are set
- docker run --platform

Signed-off-by: Djordje Lukic <djordje.lukic@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-05 16:20:41 +01:00
Paweł Gronowski
bbe6e9e8d1
hack/make: Don't add -buildmode=pie with -race
Make it possible to add `-race` to the BUILDFLAGS without making the
build fail with error:
"-buildmode=pie not supported when -race is enabled"

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-05 15:28:03 +01:00
CrazyMax
7e7b0622a6
hack: restore copy_binaries func
This function is still useful for the dev environment.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-05 13:21:28 +01:00
Albin Kerouanton
b37d34307d
Clear conntrack entries for published UDP ports
Conntrack entries are created for UDP flows even if there's nowhere to
route these packets (ie. no listening socket and no NAT rules to
apply). Moreover, iptables NAT rules are evaluated by netfilter only
when creating a new conntrack entry.

When Docker adds NAT rules, netfilter will ignore them for any packet
matching a pre-existing conntrack entry. In such case, when
dockerd runs with userland proxy enabled, packets got routed to it and
the main symptom will be bad source IP address (as shown by #44688).

If the publishing container is run through Docker Swarm or in
"standalone" Docker but with no userland proxy, affected packets will
be dropped (eg. routed to nowhere).

As such, Docker needs to flush all conntrack entries for published UDP
ports to make sure NAT rules are correctly applied to all packets.

- Fixes #44688
- Fixes #8795
- Fixes #16720
- Fixes #7540
- Fixes moby/libnetwork#2423
- and probably more.

As a precautionary measure, those conntrack entries are also flushed
when revoking external connectivity to avoid those entries to be reused
when a new sandbox is created (although the kernel should already
prevent such case).

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-01-05 12:53:22 +01:00
Kirk Easterson
1fdfd45995 Updated outdated docker contributing guidelines link
Signed-off-by: Kirk Easterson <kirk.easterson@gmail.com>
2023-01-04 09:36:59 +01:00
Sebastiaan van Stijn
fcb52454ac
Merge pull request #44719 from vvoland/fix-volume-createdat
Fix volume CreatedAt being altered on initialization
2023-01-03 18:38:06 +01:00
Sebastiaan van Stijn
01fd23b625
Fix volume CreatedAt being altered on initialization
The CreatedAt date was determined from the volume's `_data`
directory (`/var/lib/docker/volumes/<volumename>/_data`).
However, when initializing a volume, this directory is updated,
causing the date to change.

Instead of using the `_data` directory, use its parent directory,
which is not updated afterwards, and should reflect the time that
the volume was created.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-03 16:57:04 +01:00
Sebastiaan van Stijn
b9fe30dad4
Merge pull request #44546 from crazy-max/dockerfile-cross
Dockerfile: use TARGETPLATFORM to build Docker
2023-01-02 17:56:21 +01:00
Sebastiaan van Stijn
f15d5a0661
Merge pull request #44426 from thaJeztah/deprecate_image_IDFromDigest
image: deprecate IDFromDigest(), and some minor fixes/cleanup
2023-01-02 12:38:29 +01:00
Sebastiaan van Stijn
d131147a5c
use errors.Is() to handle image store errors
The image store's used are an interface, so there's no guarantee
that implementations don't wrap the errors. Make sure to catch
such cases by using errors.Is.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-02 09:41:50 +01:00
CrazyMax
83280d44d0
docs: contributing docs update for cross comp
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:04 +01:00
CrazyMax
e8a82ed24d
Dockerfile: add "all" stage to build binaries and extra tools
We still need a stage that build binaries and extra tools as well for
docker-ce-packaging repo: ff110508ff/static/Makefile (L41-L57)

This could be removed if we create a package for each project
like it's done in docker-packaging repo: https://github.com/docker/packaging/tree/main/pkg

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:04 +01:00
CrazyMax
113e95ec02
ci: add extra steps to check artifacts
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:03 +01:00
CrazyMax
b565b496ba
Dockerfile: rename dev stages
Also remove DEV_IMAGE bake var that is not used

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:03 +01:00
CrazyMax
27ba1766e7
ci: use GITHUB_REF and GITHUB_SHA to set version and commit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:03 +01:00
CrazyMax
0b3af098e1
bake: define additional vars
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:03 +01:00
CrazyMax
9bcf5bed05
bake: define DESTDIR
Keep the same output dir format in the bake definition
as the one used in make scripts.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:03 +01:00
CrazyMax
491467b0b9
hack: update buildmode pie condition
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:02 +01:00
CrazyMax
347a5f68da
hack: do not set buildmode pie for statically linked binary
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:02 +01:00
CrazyMax
e37985f590
hack: use PKG_CONFIG var when checking libdevmapper
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:02 +01:00
CrazyMax
84ea9ee0f9
IAmStatic not used anymore
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:02 +01:00
CrazyMax
04c90b8cf5
hack: remove containerutility script
this script is not used anymore. containerutility is
built in the Dockerfile.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:02 +01:00
CrazyMax
8086f40123
Dockerfile: use TARGETPLATFORM to build Docker
Better support for cross compilation so we can fully rely
on `--platform` flag of buildx for a seamless integration.

This removes unnecessary extra cross logic in the Dockerfile,
DOCKER_CROSSPLATFORMS and CROSS vars and some hack scripts as well.

Non-sandboxed build invocation is still supported and dev stages
in the Dockerfile have been updated accordingly.

Bake definition and GitHub Actions workflows have been updated
accordingly as well.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:01 +01:00
Sebastiaan van Stijn
1de3966b84
image: store: rename vars that collided with types or shadowed vars
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-01 13:05:56 +01:00
Sebastiaan van Stijn
2910163df1
daemon/images: imageKey() accept string
To reduce some type-juggling :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-01 13:05:55 +01:00
Sebastiaan van Stijn
040b5bbc52
image: store.restore(): use more structured logs
This code below is run when restoring all images (which can be "many"),
constructing the "logrus.WithFields" is deliberately not "DRY", as the
logger is only used for error-cases, and we don't want to do allocations
if we don't need it. A "f" type-alias was added to make it ever so slightly
more DRY, but that's just for convenience :)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-01 13:05:49 +01:00
Sebastiaan van Stijn
456ea1bb1d
image: deprecate IDFromDigest()
Having this function hides what it's doing, which is just to type-cast
to an image.ID (which is a digest). Using a cast is more transparent,
so deprecating this function in favor of a regular typecast.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-01 12:33:00 +01:00
Sebastiaan van Stijn
d109e429dd
Merge pull request #44723 from thaJeztah/devmapper_simplify_udev_error
daemon/graphdriver/devicemapper: simplify Udev log, and update link
2022-12-31 17:05:42 +01:00
Sebastiaan van Stijn
237c15888d
Merge pull request #44722 from thaJeztah/use_the_osusergo_luke
daemon/logger/gcplogs: remove ensureHomeIfIAmStatic workaround
2022-12-31 15:45:38 +01:00