Commit graph

46305 commits

Author SHA1 Message Date
Brian Goff
ddd67b2535 Make image save more like the OCI layout for blobs
This moves the blobs around so they follow the OCI spec.
Note that because docker reads paths from the manifest.json inside the
tar this is not a breaking change.

This does, however, remove the old layer "VERSION" file which had a big
"why is this even here" in the code comments. I suspect it does not
matter at all even for really old versions of Docker. In any case it is
a useless file for any even relatively modern version of Docker.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-05-22 22:29:29 +00:00
Sebastiaan van Stijn
6e98a7f2c9
Merge pull request #45548 from tianon/sysvinit-debian
Remove Upstart and cgroups bits from Debian sysvinit script
2023-05-22 16:08:55 +02:00
Bjorn Neergaard
ecbd126d6a
Merge pull request #45586 from corhere/fix-flaky-resolver-test
libnetwork/osl: restore the right thread's netns
2023-05-19 20:45:38 -06:00
Sebastiaan van Stijn
7796891381
Merge pull request #45475 from thaJeztah/remove_old_buildtags 2023-05-20 02:10:19 +02:00
Cory Snider
871cf72363 libnetwork: check for netns leaks from prior tests
TestProxyNXDOMAIN has proven to be susceptible to failing as a
consequence of unlocked threads being set to the wrong network
namespace. As the failure mode looks a lot like a bug in the test
itself, it seems prudent to add a check for mismatched namespaces to the
test so we will know for next time that the root cause lies elsewhere.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-19 19:36:18 -04:00
Sebastiaan van Stijn
49f8e4f1e3
Merge pull request #45574 from thaJeztah/remove_driverwarnings 2023-05-20 01:04:32 +02:00
Cory Snider
6d79864135 libnetwork/osl: restore the right thread's netns
osl.setIPv6 mistakenly captured the calling goroutine's thread's network
namespace instead of the network namespace of the thread getting its
namespace temporarily changed. As this function appears to only be
called from contexts in the process's initial network namespace, this
mistake would be of little consequence at runtime. The libnetwork unit
tests, on the other hand, unshare network namespaces so as not to
interfere with each other or the host's network namespace. But due to
this bug, the isolation backfires and the network namespace of
goroutines used by a test which are expected to be in the initial
network namespace can randomly become the isolated network namespace of
some other test. Symptoms include a loopback network server running in
one goroutine being inexplicably and randomly being unreachable by a
client in another goroutine.

Capture the original network namespace of the thread from the thread to
be tampered with, after locking the goroutine to the thread.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-19 18:35:59 -04:00
Cory Snider
d4f3858a40 libnetwork: leave global logger alone in tests
Swapping out the global logger on the fly is causing tests to flake out
by logging to a test's log output after the test function has returned.
Refactor Resolver to use a dependency-injected logger and the resolver
unit tests to inject a private logger instance into the Resolver under
test.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-19 18:35:58 -04:00
Cory Snider
0cc6e445d7 libnetwork: make resolver tests less confusing
tstwriter mocks the server-side connection between the resolver and the
container, not the resolver and the external DNS server, so returning
the external DNS server's address as w.LocalAddr() is technically
incorrect and misleading. Only the protocols need to match as the
resolver uses the client's choice of protocol to determine which
protocol to use when forwarding the query to the external DNS server.
While this change has no material impact on the tests, it makes the
tests slightly more comprehensible for the next person.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-19 18:35:58 -04:00
Sebastiaan van Stijn
411a9e1b86
daemon: remove devicemapper driver-warnings
commit dc11d2a2d8 removed the devicemapper
storage-driver, so these warnings are no longer relevant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:45:31 +02:00
Sebastiaan van Stijn
424a1c5d21
daemon: remove warning for overlay/overlay2 without d_type
commit 0abb8dec3f removed support for
running overlay/overlay2 on top of a backing filesystem without d_type
support, and  turned it into a fatal error when starting the daemon,
so there's no need to generate warnings for this situation.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:43:51 +02:00
Sebastiaan van Stijn
ab35df454d
remove pre-go1.17 build-tags
Removed pre-go1.17 build-tags with go fix;

    go mod init
    go fix -mod=readonly ./...
    rm go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 20:38:51 +02:00
Bjorn Neergaard
87f8be8110
Merge pull request #45587 from vvoland/c8d-layerless
c8d/list: Show layerless images
2023-05-19 11:17:58 -06:00
Paweł Gronowski
34964c2454
c8d/list: Show layerless images
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-19 18:12:14 +02:00
Bjorn Neergaard
cb14534d78
Merge pull request #45584 from corhere/fix-vfs-xattrs-unit-test
d/graphdriver/vfs: fix unit test
2023-05-19 14:50:58 +01:00
Akihiro Suda
2d3a1418c2
Merge pull request #45577 from thaJeztah/update_cobra
vendor: github.com/spf13/cobra v1.7.0
2023-05-19 21:55:51 +09:00
Cory Snider
61757032da d/graphdriver/vfs: fix unit test
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-19 08:26:15 -04:00
Bjorn Neergaard
1545693255
Merge pull request #45464 from corhere/xattrs-matter-in-images
Fail unpacking images with xattrs to filesystems without xattr support
2023-05-19 13:21:09 +01:00
Sebastiaan van Stijn
6e4bd3bbc2
vendor: github.com/spf13/cobra v1.7.0
release notes: https://github.com/spf13/cobra/releases/tag/v1.7.0

Features

- Allow to preserve ordering of completions in bash, zsh, pwsh, & fish
- Add support for PowerShell 7.2+ in completions
- Allow sourcing zsh completion script

Bug fixes

- Don't remove flag values that match sub-command name
- Fix powershell completions not returning single word
- Remove masked template import variable name
- Correctly detect completions with dash in argument

Testing & CI/CD

- Deprecate Go 1.15 in CI
- Deprecate Go 1.16 in CI
- Add testing for Go 1.20 in CI
- Add tests to illustrate unknown flag bug

Maintenance

- Update main image to better handle dark backgrounds
- Fix stale.yaml mispellings
- Remove stale bot from GitHub actions
- Add makefile target for installing dependencies
- Add Sia to projects using Cobra
- Add Vitess and Arewefastyet to projects using cobra
- Fixup for Kubescape github org
- Fix route for GitHub workflows badge
- Fixup for GoDoc style documentation
- Various bash scripting improvements for completion
- Add Constellation to projects using Cobra

Documentation

- Add documentation about disabling completion descriptions
- Improve MarkFlagsMutuallyExclusive example in user guide
- Update shell_completions.md
- Update copywrite year
- Document suggested layout of subcommands
- Replace deprecated ExactValidArgs with MatchAll in doc

full diff: https://github.com/spf13/cobra/compare/v1.6.1...v1.7.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-19 01:52:30 +02:00
Sebastiaan van Stijn
f8dec8f1c6
Merge pull request #45565 from corhere/fix-dns-servfail
libnetwork: just forward the external DNS response
2023-05-18 23:30:45 +02:00
Cory Snider
6690d2969c pkg/archive: bail if setting xattrs is unsupported
Extended attributes are set on files in container images for a reason.
Fail to unpack if extended attributes are present in a layer and setting
the attributes on the unpacked files fails for any reason.

Add an option to the vfs graph driver to opt into the old behaviour
where ENOTSUPP and EPERM errors encountered when setting extended
attributes are ignored. Make it abundantly clear to users and anyone
triaging their bug reports that they are shooting themselves in the
foot by enabling this option.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-18 17:21:12 -04:00
Cory Snider
0cdfd5f275 pkg/system: return rich errors from L(g|s)etxattr
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-18 16:31:01 -04:00
Bjorn Neergaard
ccd834ea25
Merge pull request #45557 from crazy-max/fix-worker-id
build: use daemon id as worker id for the graph driver controller
2023-05-18 21:17:43 +01:00
Cory Snider
41356227f2 libnetwork: just forward the external DNS response
Our resolver is just a forwarder for external DNS so it should act like
it. Unless it's a server failure or refusal, take the response at face
value and forward it along to the client. RFC 8020 is only applicable to
caching recursive name servers and our resolver is neither caching nor
recursive.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-18 16:04:19 -04:00
Sebastiaan van Stijn
1b3c2743cc
Merge pull request #45550 from corhere/fix-empty-container-decode
Allow empty body in `POST /commit` again
2023-05-18 21:46:00 +02:00
Sebastiaan van Stijn
50d2c94bd6
Merge pull request #43637 from thaJeztah/remove_deprecated_storage_drivers
Remove deprecated devicemapper storage driver
2023-05-18 21:23:41 +02:00
Kevin Alvarez
6d139e5e95
build: use daemon id as worker id for the graph driver controller
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-18 21:17:29 +02:00
Sebastiaan van Stijn
054add40a1
Merge pull request #45552 from miles-to-go/fix-insecure-registries-reload
Fix insecure registries reload
2023-05-18 20:56:55 +02:00
Sebastiaan van Stijn
7a821d4c99
Merge pull request #45567 from crazy-max/ci-bin-image-distribute
ci(bin-image): distribute build across runners
2023-05-18 20:55:47 +02:00
Kevin Alvarez
668af4be82
ci(bin-image): distribute build across runners
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-18 18:31:07 +02:00
Brian Goff
de9f12ea96
Merge pull request #45563 from vvoland/fix-45556
api/inspect: Fix nil RepoTags and RepoDigests
2023-05-18 07:59:56 -07:00
Paweł Gronowski
1be26e9f0c
api/inspect: Fix nil RepoTags and RepoDigests
Make RepoTags and RepoDigests empty arrays instead of nil.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-18 15:57:18 +02:00
Bjorn Neergaard
b42c65d796
Merge pull request #45561 from laurazard/fix-pull-platform
fix: `docker pull` with platform checks wrong image tag
2023-05-18 14:14:12 +01:00
Laura Brehm
f450ea64e6
fix: docker pull with platform checks wrong image tag
This fixes a bug where, if a user pulls an image with a tag != `latest` and
a specific platform, we return an NotFound error for the wrong (`latest`) tag.
see: https://github.com/moby/moby/issues/45558

This bug was introduced in 779a5b3029
in the changes to `daemon/images/image_pull.go`, when we started returning the error from the call to
`GetImage` after the pull. We do this call, if pulling with a specified platform, to check if the platform
of the pulled image matches the requested platform (for cases with single-arch images).
However, when we call `GetImage` we're not passing the image tag, only name, so `GetImage` assumes `latest`
which breaks when the user has requested a different tag, since there might not be such an image in the store.

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2023-05-18 13:14:59 +01:00
Nolan Miles
f3645a2aa3 reorder load funcs to match newServiceConfig()'s order
Signed-off-by: Nolan Miles <nolanpmiles@gmail.com>
2023-05-17 21:34:14 -04:00
Nolan Miles
3b15156e4d add mirror to daemon reload test for insecure registries
Signed-off-by: Nolan Miles <nolanpmiles@gmail.com>
2023-05-17 21:34:14 -04:00
Bjorn Neergaard
2ea5c1af57
Merge pull request #45551 from tianon/rm-upstart
Remove Upstart scripts
2023-05-18 01:21:36 +01:00
Tianon Gravi
0d8087fbbc Remove Upstart scripts
Upstart has been EOL for 8 years and isn't used by any distributions we support any more.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-05-17 14:20:14 -07:00
Tianon Gravi
405f4589a6
Merge pull request #45505 from neersighted/mailmap_update
mailmap update and regenerate AUTHORS
2023-05-17 20:43:48 +00:00
Cory Snider
967c7bc5d3 api/server: allow empty body for POST /commit again
The error returned by DecodeConfig was changed in
b6d58d749c and caused this to regress.
Allow empty request bodies for this endpoint once again.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-17 16:01:11 -04:00
Cory Snider
3ceb3810d7 client: don't marshal typed nils in request body
The internal Client request methods which accept an object as a body use
nil to signal that the request should not have a body. But it is easy to
accidentally pass a typed-nil value as the object, e.g. if the object
comes from a function argument or struct field of a concrete type. The
result is that these requests will, surprisingly, have a JSON body of
`null`. Treat typed-nil pointers the same as untyped nils for the
purposes of determining whether or not the request should include a
body.

Stop assuming that POST requests should always have a body. POST /commit
does not require a body, for example.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-05-17 14:58:12 -04:00
Tianon Gravi
ae737656f9 Remove Upstart and cgroups bits from Debian sysvinit script
Upstart has been EOL for 8 years and isn't used by any distributions we support any more.

Additionally, this removes the "cgroups v1" setup code because it's more reasonable now for us to expect something _else_ to have set up cgroups appropriately (especially cgroups v2).

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2023-05-16 15:39:33 -07:00
Evan Lezar
c2630c93f2 Update vendoring
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-05-16 17:11:09 +02:00
Evan Lezar
dedf5747cc Remove need to set Capabilities for cdi driver
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-05-16 17:07:58 +02:00
Evan Lezar
7ec9561a77 Add support for CDI devices to docker daemon under linux
These changes add basic CDI integration to the docker daemon.

A cdi driver is added to handle cdi device requests. This
is gated by an experimental feature flag and is only supported on linux

This change also adds a CDISpecDirs (cdi-spec-dirs) option to the config.
This allows the default values of `/etc/cdi`, /var/run/cdi` to be overridden
which is useful for testing.

Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-05-16 17:07:57 +02:00
Evan Lezar
5a200ade7c Rename daemon/devices_liunx.go to daemom/devices.go
Signed-off-by: Evan Lezar <elezar@nvidia.com>
2023-05-16 17:07:57 +02:00
Akihiro Suda
1371aee3cc
Merge pull request #45469 from thaJeztah/deprecate_virtualsize_STEP2
API: omit deprecated VirtualSize field for API v1.44 and up
2023-05-16 15:10:07 +09:00
Sebastiaan van Stijn
9548916aa8
Merge pull request #45512 from thaJeztah/libnetwork_fix_readme_example
libnetwork: update example in README.md
2023-05-15 21:48:06 +01:00
Sebastiaan van Stijn
72b2469be0
Merge pull request #45536 from thaJeztah/containerd_binary_1.7.1
update containerd binary to v1.7.1
2023-05-15 15:25:01 +01:00
Sebastiaan van Stijn
484785456c
update containerd binary to v1.7.1
full diff: https://github.com/containerd/containerd/compare/v1.7.0...v1.7.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-15 12:52:50 +01:00