Commit graph

1479 commits

Author SHA1 Message Date
Paweł Gronowski
96c9353e9b
Merge pull request #47723 from vvoland/builder-fix-workdir-slash
builder: Fix `WORKDIR` with a trailing slash causing a cache miss
2024-04-19 13:56:40 +02:00
Paweł Gronowski
7532420f3b
container/SetupWorkingDirectory: Don't mutate config
Don't mutate the container's `Config.WorkingDir` permanently with a
cleaned path when creating a working directory.

Move the `filepath.Clean` to the `translateWorkingDir` instead.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-04-19 12:42:20 +02:00
Paweł Gronowski
e829cca0ee
Merge pull request #47584 from robmry/upstream_dns_windows
Windows DNS resolver forwarding
2024-04-19 11:34:50 +02:00
Rob Murray
6c68be24a2 Windows DNS resolver forwarding
Make the internal DNS resolver for Windows containers forward requests
to upsteam DNS servers when it cannot respond itself, rather than
returning SERVFAIL.

Windows containers are normally configured with the internal resolver
first for service discovery (container name lookup), then external
resolvers from '--dns' or the host's networking configuration.

When a tool like ping gets a SERVFAIL from the internal resolver, it
tries the other nameservers. But, nslookup does not, and with this
change it does not need to.

The internal resolver learns external server addresses from the
container's HNSEndpoint configuration, so it will use the same DNS
servers as processes in the container.

The internal resolver for Windows containers listens on the network's
gateway address, and each container may have a different set of external
DNS servers. So, the resolver uses the source address of the DNS request
to select external resolvers.

On Windows, daemon.json feature option 'windows-no-dns-proxy' can be used
to prevent the internal resolver from forwarding requests (restoring the
old behaviour).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-16 18:57:28 +01:00
Rob Murray
57dd56726a Disable IPv6 for endpoints in '--ipv6=false' networks.
No IPAM IPv6 address is given to an interface in a network with
'--ipv6=false', but the kernel would assign a link-local address and,
in a macvlan/ipvlan network, the interface may get a SLAAC-assigned
address.

So, disable IPv6 on the interface to avoid that.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 17:11:20 +01:00
Rob Murray
9954d7c6bd Run ipvlan tests even if 'modprobe ipvlan' fails
This reverts commit a77e147d32.

The ipvlan integration tests have been skipped in CI because of a check
intended to ensure the kernel has ipvlan support - which failed, but
seems to be unnecessary (probably because kernels have moved on).

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 08:52:08 +01:00
Rob Murray
cd7240f6d9 Stop macvlan with no parent from using ext-dns
We document that an macvlan network with no parent interface is
equivalent to a '--internal' network. But, in this case, an macvlan
network was still configured with a gateway. So, DNS proxying would
be enabled in the internal resolver (and, if the host's resolver
was on a localhost address, requests to external resolvers from the
host's network namespace would succeed).

This change disables configuration of a gateway for a macvlan Endpoint
if no parent interface is specified.

(Note if a parent interface with no external network is supplied as
'-o parent=<dummy>', the gateway will still be set up. Documentation
will need to be updated to note that '--internal' should be used to
prevent DNS request forwarding in this case.)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 08:51:00 +01:00
Rob Murray
17b8631545 Enable DNS proxying for ipvlan-l3
The internal DNS resolver should only forward requests to external
resolvers if the libnetwork.Sandbox served by the resolver has external
network access (so, no forwarding for '--internal' networks).

The test for external network access was whether the Sandbox had an
Endpoint with a gateway configured.

However, an ipvlan-l3 networks with external network access does not
have a gateway, it has a default route bound to an interface.

Also, we document that an ipvlan network with no parent interface is
equivalent to a '--internal' network. But, in this case, an ipvlan-l2
network was configured with a gateway. So, DNS proxying would be enabled
in the internal resolver (and, if the host's resolver was on a localhost
address, requests to external resolvers from the host's network
namespace would succeed).

So, this change adjusts the test for enabling DNS proxying to include
a check for '--internal' (as a shortcut) and, for non-internal networks,
checks for a default route as well as a gateway. It also disables
configuration of a gateway or a default route for an ipvlan Endpoint if
no parent interface is specified.

(Note if a parent interface with no external network is supplied as
'-o parent=<dummy>', the gateway/default route will still be set up
and external DNS proxying will be enabled. The network must be
configured as '--internal' to prevent that from happening.)

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-10 08:50:57 +01:00
Rob Murray
d8b768149b Move dummy DNS server to integration/internal/network
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-04-04 12:02:22 +01:00
Rob Murray
fde80fe2e7 Restore the SetKey prestart hook.
Partially reverts 0046b16 "daemon: set libnetwork sandbox key w/o OCI hook"

Running SetKey to store the OCI Sandbox key after task creation, rather
than from the OCI prestart hook, meant it happened after sysctl settings
were applied by the runtime - which was the intention, we wanted to
complete Sandbox configuration after IPv6 had been disabled by a sysctl
if that was going to happen.

But, it meant '--sysctl' options for a specfic network interface caused
container task creation to fail, because the interface is only moved into
the network namespace during SetKey.

This change restores the SetKey prestart hook, and regenerates config
files that depend on the container's support for IPv6 after the task has
been created. It also adds a regression test that makes sure it's possible
to set an interface-specfic sysctl.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-25 19:35:55 +00:00
Paweł Gronowski
3312b82515
c8d/list: Add a test case for images sharing a top layer
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-20 13:17:56 +01:00
Bjorn Neergaard
641e341eed
Merge pull request #47538 from robmry/libnet-resolver-nxdomain
libnet: Don't forward to upstream resolvers on internal nw
2024-03-18 11:22:59 -06:00
Paweł Gronowski
1c03312378
integration: Remove Parallel from TestDiskUsage
Check if removing the Parallel execution from that test fixes its
flakiness.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-03-18 12:11:34 +01:00
Albin Kerouanton
790c3039d0 libnet: Don't forward to upstream resolvers on internal nw
Commit cbc2a71c2 makes `connect` syscall fail fast when a container is
only attached to an internal network. Thanks to that, if such a
container tries to resolve an "external" domain, the embedded resolver
returns an error immediately instead of waiting for a timeout.

This commit makes sure the embedded resolver doesn't even try to forward
to upstream servers.

Co-authored-by: Albin Kerouanton <albinker@gmail.com>
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-14 17:46:48 +00:00
Sebastiaan van Stijn
0fb845858d
Merge pull request #47505 from akerouanton/fix-TestBridgeICC-ipv6
inte/networking:  ping with -6 specified when needed
2024-03-08 18:33:46 +01:00
Albin Kerouanton
5a009cdd5b inte/networking: add isIPv6 flag
Make sure the `ping` command used by `TestBridgeICC` actually has
the `-6` flag when it runs IPv6 test cases. Without this flag,
IPv6 connectivity isn't tested properly.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-07 17:55:53 +01:00
Paweł Gronowski
66adfc729a
Merge pull request #47521 from robmry/no_ipv6_addr_when_ipv6_disabled
Don't configure IPv6 addr/gw when IPv6 disabled.
2024-03-07 12:57:27 +01:00
Sebastiaan van Stijn
f5a5e3f203
golangci-lint: enable dupword linter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-07 11:44:27 +01:00
Sebastiaan van Stijn
4adc40ac40
fix duplicate words (dupwords)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-03-07 10:57:03 +01:00
Rob Murray
ef5295cda4 Don't configure IPv6 addr/gw when IPv6 disabled.
When IPv6 is disabled in a container by, for example, using the --sysctl
option - an IPv6 address/gateway is still allocated. Don't attempt to
apply that config because doing so enables IPv6 on the interface.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-03-06 18:32:31 +00:00
Albin Kerouanton
7c7e453255
Merge pull request #47474 from robmry/47441_mac_addr_config_migration
Don't create endpoint config for MAC addr config migration
2024-03-06 11:04:17 +01:00
Albin Kerouanton
21835a5696 inte/networking: rename linkLocal flag into isLinkLocal
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2024-03-06 00:16:08 +01:00
Sebastiaan van Stijn
137a9d6a4c
Merge pull request #47395 from robmry/47370_windows_natnw_dns_test
Test DNS on Windows 'nat' networks
2024-03-01 13:02:52 +01:00
Rob Murray
a580544d82 Don't create endpoint config for MAC addr config migration
In a container-create API request, HostConfig.NetworkMode (the identity
of the "main" network) may be a name, id or short-id.

The configuration for that network, including preferred IP address etc,
may be keyed on network name or id - it need not match the NetworkMode.

So, when migrating the old container-wide MAC address to the new
per-endpoint field - it is not safe to create a new EndpointSettings
entry unless there is no possibility that it will duplicate settings
intended for the same network (because one of the duplicates will be
discarded later, dropping the settings it contains).

This change introduces a new API restriction, if the deprecated container
wide field is used in the new API, and EndpointsConfig is provided for
any network, the NetworkMode and key under which the EndpointsConfig is
store must be the same - no mixing of ids and names.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-29 17:02:19 +00:00
Sebastiaan van Stijn
6c3b3523c9
Merge pull request #47041 from robmry/46968_refactor_resolvconf
Refactor 'resolv.conf' generation.
2024-02-29 09:33:55 +01:00
Paweł Gronowski
5fe96e234d
integration: Reset OTEL_EXPORTER_OTLP_ENDPOINT for sub-daemons
When creating a new daemon in the `TestDaemonProxy`, reset the
`OTEL_EXPORTER_OTLP_ENDPOINT` to an empty value to disable OTEL
collection to avoid it hitting the proxy.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:48:07 +01:00
Paweł Gronowski
84eecc4a30
Revert "integration/TestDaemonProxy: Remove OTEL span"
This reverts commit 56aeb548b2.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-28 10:48:03 +01:00
Sebastiaan van Stijn
f6fa6ff9ed
Merge pull request #47391 from vvoland/rro-backwards-compatible
api/pre-1.44: Default `ReadOnlyNonRecursive` to true
2024-02-27 18:04:46 +01:00
Sebastiaan van Stijn
220835106b
Merge pull request #47364 from vvoland/buildkit-v13
vendor: github.com/moby/buildkit v0.13.0-rc2
2024-02-27 16:38:04 +01:00
Rob Murray
9083c2f10d Test DNS on Windows 'nat' networks
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-27 11:40:11 +00:00
Paweł Gronowski
44167988c3
c8d/windows: Temporarily skip two failing tests
They're failing the CI and we have a tracking ticket: #47107

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 12:04:21 +01:00
Paweł Gronowski
e68f71259a
integration/build: Use fsutil.NewFS
StaticDirSource definition changed and can no longer be initialized from
the composite literal.

a80b48544c

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:10 +01:00
Paweł Gronowski
dd6992617e
integration/build: Use new buildkit progressui
Introduced in: 37131781d7

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-27 11:26:09 +01:00
Paweł Gronowski
432390320e
api/pre-1.44: Default ReadOnlyNonRecursive to true
Don't change the behavior for older clients and keep the same behavior.
Otherwise client can't opt-out (because `ReadOnlyNonRecursive` is
unsupported before 1.44).

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-26 11:37:30 +01:00
Paweł Gronowski
56aeb548b2
integration/TestDaemonProxy: Remove OTEL span
Don't use OTEL tracing in this test because we're testing the HTTP proxy
behavior here and we don't want OTEL to interfere.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-20 10:21:53 +01:00
Sebastiaan van Stijn
7bf8d2606e
Merge pull request #47356 from robmry/47329_restore_internal_bridge_addr
Make 'internal' bridge networks accessible from host
2024-02-16 13:23:38 +01:00
Rob Murray
9faf4855d5 Simplify macvlan/ipvlan integration test structure
Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-14 14:09:45 +00:00
Rob Murray
4eb95d01bc Run the macvlan/ipvlan integration tests
The problem was accidentally introduced in:
  e8dc902781

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-14 14:08:10 +00:00
Rob Murray
419f5a6372 Make 'internal' bridge networks accessible from host
Prior to release 25.0.0, the bridge in an internal network was assigned
an IP address - making the internal network accessible from the host,
giving containers on the network access to anything listening on the
bridge's address (or INADDR_ANY on the host).

This change restores that behaviour. It does not restore the default
route that was configured in the container, because packets sent outside
the internal network's subnet have always been dropped. So, a 'connect()'
to an address outside the subnet will still fail fast.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-07 19:12:10 +00:00
Rob Murray
beb97f7fdf Refactor 'resolv.conf' generation.
Replace regex matching/replacement and re-reading of generated files
with a simple parser, and struct to remember and manipulate the file
content.

Annotate the generated file with a header comment saying the file is
generated, but can be modified, and a trailing comment describing how
the file was generated and listing external nameservers.

Always start with the host's resolv.conf file, whether generating config
for host networking, or with/without an internal resolver - rather than
editing a file previously generated for a different use-case.

Resolves an issue where rewrites of the generated file resulted in
default IPv6 nameservers being unnecessarily added to the config.

Signed-off-by: Rob Murray <rob.murray@docker.com>
2024-02-06 22:26:12 +00:00
Sebastiaan van Stijn
8758d08bb4
api: remove handling of HostConfig on POST /containers/{id}/start (api < v1.24)
API v1.20 (Docker Engine v1.11.0) and older allowed a HostConfig to be passed
when starting a container. This feature was deprecated in API v1.21 (Docker
Engine v1.10.0) in 3e7405aea8, and removed in
API v1.23 (Docker Engine v1.12.0) in commit 0a8386c8be.

API v1.23 and older are deprecated, and this patch removes the feature.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
b3a0ff9944
api: remove POST /containers/{id}/copy endpoint (api < v1.23)
This endpoint was deprecated in API v1.20 (Docker Engine v1.8.0) in
commit db9cc91a9e, in favor of the
`PUT /containers/{id}/archive` and `HEAD /containers/{id}/archive`
endpoints, and disabled in API v1.24 (Docker Engine v1.12.0) through
commit 428328908d.

This patch removes the endpoint, and the associated `daemon.ContainerCopy`
method in the backend.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:44 +01:00
Sebastiaan van Stijn
ed93110e11
api: update test to reflect reality on Windows
The TestInspectAPIContainerResponse mentioned that Windows does not
support API versions before v1.25.

While technically, no stable release existed for Windows with API versions
before that (see f811d5b128), API version
v1.24 was enabled in e4af39aeb3, to have
a consistend fallback version for API version negotiation.

This patch updates the test to reflect that change.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
f0dd554e3c
api: remove code for ContainerInspect on api < v1.20
API v1.23 and older are deprecated, so we can remove the code to adjust
responses for API v1.19 and lower.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:43 +01:00
Sebastiaan van Stijn
dfdf2adf0c
api: POST /containers/{id}/kill: remove handling for api < 1.20
API v1.20 and up produces an error when signalling / killing a non-running
container (see c92377e300). Older API versions
allowed this, and an exception was added in 621e3d8587.

API v1.23 and older are deprecated, so we can remove this handling.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2024-02-06 18:44:42 +01:00
Sebastiaan van Stijn
2156635843
Merge pull request #47232 from vvoland/fix-save-manifests
image/save: Fix untagged images not present in index.json
2024-02-05 19:06:54 +01:00
Paweł Gronowski
2ef0b53e51
integration/save: Add tests checking OCI archive output
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2024-02-05 11:17:58 +01:00
Sebastiaan van Stijn
6b83319773
Merge pull request #47299 from laurazard/plugin-install-digest
plugins: Fix panic when fetching by digest
2024-02-05 09:39:05 +01:00
Albin Kerouanton
ca683c1c77
Merge pull request #47233 from robmry/47146-duplicate_mac_addrs2
Only restore a configured MAC addr on restart.
2024-02-02 09:08:17 +01:00
Laura Brehm
82dda18898
tests: add plugin install test w/ digest
Adds a test case for installing a plugin from a remote in the form
of `plugin-content-trust@sha256:d98f2f8061...`, which is currently
causing the daemon to panic, as we found while running the CLI e2e
tests:

```
docker plugin install registry:5000/plugin-content-trust@sha256:d98f2f806144bf4ba62d4ecaf78fec2f2fe350df5a001f6e3b491c393326aedb
```

Signed-off-by: Laura Brehm <laurabrehm@hey.com>
2024-02-01 23:00:38 +00:00