Commit graph

2563 commits

Author SHA1 Message Date
Sebastiaan van Stijn
c4c8a80958
Merge pull request #43813 from thaJeztah/22.06_backport_fix_43781
[22.06 backport] libnetwork: skip firewalld management for rootless
2022-07-18 09:48:12 +02:00
Sebastiaan van Stijn
e34ab5200d
fix formatting of "nolint" tags for go1.19
The correct formatting for machine-readable comments is;

    //<some alphanumeric identifier>:<options>[,<option>...][ // comment]

Which basically means:

- MUST NOT have a space before `<identifier>` (e.g. `nolint`)
- Identified MUST be alphanumeric
- MUST be followed by a colon
- MUST be followed by at least one `<option>`
- Optionally additional `<options>` (comma-separated)
- Optionally followed by a comment

Any other format will not be considered a machine-readable comment by `gofmt`,
and thus formatted as a regular comment. Note that this also means that a
`//nolint` (without anything after it) is considered invalid, same for `//#nosec`
(starts with a `#`).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 4f08346686)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-15 13:45:13 +02:00
Akihiro Suda
863ca3f185
libnetwork: skip firewalld management for rootless
Fix issue 43781

Co-authored-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
(cherry picked from commit 9464898b47)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-15 13:40:07 +02:00
Sebastiaan van Stijn
cdbca4061b
gofmt GoDoc comments with go1.19
Older versions of Go don't format comments, so committing this as
a separate commit, so that we can already make these changes before
we upgrade to Go 1.19.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 52c1a2fae8)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-13 22:42:29 +02:00
Sebastiaan van Stijn
968ff5ab44
fix some minor linting issues
libnetwork/firewall_linux.go:11:21: var-declaration: should drop = nil from declaration of var ctrl; it is the zero value (revive)
        ctrl *controller = nil
                           ^
    distribution/pull_v2_test.go:213:4: S1038: should use t.Fatalf(...) instead of t.Fatal(fmt.Sprintf(...)) (gosimple)
                t.Fatal(fmt.Sprintf("expected formatPlatform to show windows platform with a version, but got '%s'", result))
                ^
    integration-cli/docker_cli_build_test.go:5951:3: S1038: should use c.Skipf(...) instead of c.Skip(fmt.Sprintf(...)) (gosimple)
            c.Skip(fmt.Sprintf("Bug fixed in 18.06 or higher.Skipping it for %s", testEnv.DaemonInfo.ServerVersion))
            ^
    integration-cli/docker_cli_daemon_test.go:240:3: S1038: should use c.Skipf(...) instead of c.Skip(fmt.Sprintf(...)) (gosimple)
            c.Skip(fmt.Sprintf("New base device size (%v) must be greater than (%s)", units.HumanSize(float64(newBasesizeBytes)), units.HumanSize(float64(oldBasesizeBytes))))
            ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-04 10:15:28 +02:00
Youfu Zhang
f70a9788c5 drivers/ipvlan: add ipvlan_flag option, support l3s ipvlan_mode
Signed-off-by: Youfu Zhang <zhangyoufu@gmail.com>
2022-06-28 19:28:49 +08:00
Sebastiaan van Stijn
db977355b0
fix typo (cluser -> cluster)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-27 15:12:14 +02:00
Martin Braun
5edfd6d081 bump netlink library
bump netlink to 1.2.1
change usages of netlink handle .Delete() to Close()
remove superfluous replace in vendor.mod
make requires of github.com/Azure/go-ansiterm direct

Signed-off-by: Martin Braun <braun@neuroforge.de>
2022-06-16 22:25:33 +02:00
Sebastiaan van Stijn
9959eceb9a
Merge pull request #42626 from mfeit-internet2/small-ipv4-networks
Support small ipv4 networks
2022-06-07 22:15:19 +02:00
Sebastiaan van Stijn
6e80c027c5
Merge pull request #43409 from vincentbernat/fix/udp-conntrack
bridge: also flush conntrack entries when setting up endpoints
2022-06-03 11:29:25 +02:00
Sebastiaan van Stijn
b4b2a0323b
libnetwork/portallocator: un-export consts for defaults
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-06-02 22:59:32 +02:00
Albin Kerouanton
414dd017b3
Revert "Added API to set ephemeral port allocator range."
Since commit 2c4a868f64, Docker doesn't
use the value of net.ipv4.ip_local_port_range when choosing an ephemeral
port. This change reverts back to the previous behavior.

Fixes #43054.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2022-06-02 22:56:18 +02:00
Sebastiaan van Stijn
5a8304c9c5
Merge pull request #43670 from evol262/ipvs-module-idempotency
Ensure performance tuning is always applied
2022-05-31 22:00:40 +02:00
Ryan Barry
293cfd6c76 Ensure performance tuning is always applied
Previously, with the patch from #43146, it was possible for a
network configured with a single ingress or load balancer on a
distribution which does not have the `ip_vs` kernel module loaded
by default to try to apply sysctls which did not exist yet, and
subsequently dynamically load the module as part of ipvs/netlink.go.

This module is vendored, and not a great place to try to tie back
into core libnetwork functionality, so also ensure that the sysctls
(which are idempotent) are called after ingress/lb creation once
`ipvs` has been initialized.

Signed-off-by: Ryan Barry <rbarry@mirantis.com>
2022-05-31 11:47:30 -04:00
Ameya Gawde
cba8cf34d2
Set ExternalPortReserved for dummy proxy
Signed-off-by: Ameya Gawde <agawde@mirantis.com>
2022-05-25 07:12:43 -07:00
Sebastiaan van Stijn
1aea4c2bbd
Merge pull request #43146 from evol262/fix/ingress-namespace-performance
Apply peformance tuning to new sandboxes also
2022-05-18 18:28:30 +02:00
Ryan Barry
0dd3a2eade Apply performance tuning to new sandboxes also
relates to #35082, moby/libnetwork#2491

Previously, values for expire_quiescent_template, conn_reuse_mode,
and expire_nodest_conn were set only system-wide. Also apply them
for new lb_* and ingress_sbox sandboxes, so they are appropriately
propagated

Signed-off-by: Ryan Barry <rbarry@mirantis.com>
2022-05-17 15:44:49 -04:00
Eng Zer Jun
7873c27cfb
all: replace strings.Replace with strings.ReplaceAll
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-09 19:45:40 +08:00
Samuel Karp
ccb691a427
Merge pull request #43511 from thaJeztah/no_logrus_fatal 2022-04-21 11:33:43 -07:00
Sebastiaan van Stijn
df650a1aeb
panic() instead of logrus.Fatal() in init funcs
Some packages were using `logrus.Fatal()` in init functions (which logs the error,
and (by default) calls `os.Exit(1)` after logging).

Given that logrus formatting and outputs have not yet been configured during the
initialization stage, it does not provide much benefits over a plain `panic()`.

This patch replaces some instances of `logrus.Fatal()` with `panic()`, which has
the added benefits of not introducing logrus as a dependency in some of these
packages, and also produces a stacktrace, which could help locating the problem
in the unlikely event an `init()` fails.

Before this change, an error would look like:

    $ dockerd
    FATA[0000] something bad happened

After this change, the same error looks like:

    $ dockerd
    panic: something bad happened

    goroutine 1 [running]:
      github.com/docker/docker/daemon/logger/awslogs.init.0()
        /go/src/github.com/docker/docker/daemon/logger/awslogs/cloudwatchlogs.go:128 +0x89

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-21 12:15:20 +02:00
Samuel Karp
c5f18aac9f
Merge pull request #43505 from thaJeztah/libnetwork_no_string_fields
libnetwork: don't use strings.Fields() to improve performance
2022-04-21 00:57:55 -07:00
Sebastiaan van Stijn
301b252b58
libnetwork: don't use strings.Fields() to improve performance
While looking at this code, I noticed that we were wasting quite some resources
by first constructing a string, only to split it again (with `strings.Fields()`)
into a string slice.

Some conversions were also happening multiple times (int to string, IP-address to
string, etc.)

Setting up networking is known to be costing a considerable amount of time when
starting containers, and while this may only be a small part of that, it doesn't
hurt to save some resources (and readability of the code isn't significantly
impacted).

For example, benchmarking the `redirector()` code before/after:

    BenchmarkParseOld-4   	  137646	      8398 ns/op	    4192 B/op	      75 allocs/op
    BenchmarkParseNew-4   	  629395	      1762 ns/op	    2362 B/op	      24 allocs/op

Average over 10 runs:

    benchstat old.txt new.txt

    name     old time/op    new time/op    delta
    Parse-4    8.43µs ± 2%    1.79µs ± 3%  -78.76%  (p=0.000 n=9+8)

    name     old alloc/op   new alloc/op   delta
    Parse-4    4.19kB ± 0%    2.36kB ± 0%  -43.65%  (p=0.000 n=10+10)

    name     old allocs/op  new allocs/op  delta
    Parse-4      75.0 ± 0%      24.0 ± 0%  -68.00%  (p=0.000 n=10+10)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-04-20 14:43:07 +02:00
Trapier Marshall
a46700dbdb Cleanup servicebindings only on Windows
Make the call to cleanupServiceBindings during network deletion
conditional on Windows (where it is required), thereby providing a
performance improvement to network cleanup on Linux.

Signed-off-by: Trapier Marshall <tmarshall@mirantis.com>
2022-04-19 14:22:16 +00:00
Trapier Marshall
6861aade58 Delay network deletion until after lb cleanup
Removal of PolicyLists from Windows VFP must be performed prior to
removing the HNS network. Otherwise PolicyList removal fails with
HNS error "network not found".

Signed-off-by: Trapier Marshall <tmarshall@mirantis.com>
2022-04-19 14:22:08 +00:00
Trapier Marshall
556cb3ae81 Log HNS policylist removal failures
Signed-off-by: Trapier Marshall <tmarshall@mirantis.com>
2022-04-19 14:21:29 +00:00
Vincent Bernat
e5812117a5 bridge: also flush conntrack entries when setting up endpoints
There is a race condition between the local proxy and iptables rule
setting. When we have a lot of UDP traffic, the kernel will create
conntrack entries to the local proxy and will ignore the iptables
rules set after that.

Related to PR #32505. Fix #8795.

Signed-off-by: Vincent Bernat <vincent@bernat.ch>
2022-03-23 08:34:26 +01:00
Martin Dojcak
feab0cca9f libnetwork/overlay:fix join sandbox deadlock
Operations performed on overlay network sandboxes are handled by
dispatching operations send through a channel. This allows for
asynchronous operations to be performed which, since they are
not called from within another function, are able to operate in
an idempotent manner with a known/measurable starting state from
which an identical series of iterative actions can be performed.

However, it was possible in some cases for an operation dispatched
from this channel to write a message back to the channel in the
case of joining a network when a sufficient volume of sandboxes
were operated on.

A goroutine which is simultaneously reading and writing to an
unbuffered channel can deadlock if it sends a message to a channel
then waits for it to be consumed and completed, since the only
available goroutine is more or less "talking to itself". In order
to break this deadlock, in the observed race, a goroutine is now
created to send the message to the channel.

Signed-off-by: Martin Dojcak <martin.dojcak@lablabs.io>
Signed-off-by: Ryan Barry <rbarry@mirantis.com>
2022-03-22 11:15:14 -04:00
Sebastiaan van Stijn
1b3fef5333
Windows: require Windows Server RS5 / ltsc2019 (build 17763) as minimum
Windows Server 2016 (RS1) reached end of support, and Docker Desktop requires
Windows 10 V19H2 (version 1909, build 18363) as a minimum.

This patch makes Windows Server RS5 /  ltsc2019 (build 17763) the minimum version
to run the daemon, and removes some hacks for older versions of Windows.

There is one check remaining that checks for Windows RS3 for a workaround
on older versions, but recent changes in Windows seemed to have regressed
on the same issue, so I kept that code for now to check if we may need that
workaround (again);

085c6a98d5/daemon/graphdriver/windows/windows.go (L319-L341)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 22:58:28 +01:00
Sebastiaan van Stijn
b9c8eca468
libnetwork/networkdb: remove some redundant fmt.Sprintf()'s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-15 12:56:23 +01:00
Sebastiaan van Stijn
85ed9b8746
libnetwork: fix unhandled errors in tests (ineffassign)
libnetwork/libnetwork_test.go:1014:8: ineffectual assignment to err (ineffassign)
        sbx1, err := controller.NewSandbox(containerID,
              ^
    libnetwork/libnetwork_test.go:1024:8: ineffectual assignment to err (ineffassign)
        sbx2, err := controller.NewSandbox("c2")
              ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-08 09:43:27 +01:00
Sebastiaan van Stijn
7b692a421b
libnetwork: remove more config bits related to external k/v stores
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:45:45 +01:00
Sebastiaan van Stijn
745ba3ecbc
libnetwork: remove etcd-related code and tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:45:43 +01:00
Sebastiaan van Stijn
147173b099
libnetwork: remove consul-related code and tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:45:41 +01:00
Sebastiaan van Stijn
25594c33b9
libnetwork: replace consul with boltdb in test
Based on randomLocalStore() in libnetwork/ipam/allocator_test.go

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:45:07 +01:00
Sebastiaan van Stijn
a7d0f3060a
libnetwork: remove zookeeper-related code and tests
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:28:30 +01:00
Anca Iordache
00f9b23c3a
libnetwork: remove external DS-based host discovery
Signed-off-by: Anca Iordache <anca.iordache@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-06 18:28:26 +01:00
Tianon Gravi
8955d8da89
Merge pull request #42981 from frobnicaty/patch-1
Fix grammar for "does not exist"
2021-12-09 13:36:53 -08:00
frobnicaty
d78b883576 Fix grammar for "does not exist"
as opposed to "does not exists"

Signed-off-by: frobnicaty <92033765+frobnicaty@users.noreply.github.com>
2021-12-03 15:50:13 +00:00
Albin Kerouanton
c721bad8cc
Fix flaky TestPortMappingV6Config
Since moby/libnetwork#2635 has been merged, allocatePortsInternal()
checks if IPv6 is enabled by calling IsV6Listenable(). This function
calls `net.Listen("tcp6", "[::1]:0")` and returns false when
net.Listen() fails.

TestPortMappingV6Config() starts by setting up a new net ns to run into
it. The loopback interface is not bring up in this net ns, thus
net.Listen() fails and IsV6Listenable() returns false. This change takes
care of bringing loopback iface up right after moving to the new net ns.

This test has been reported has flaky on s390x in #42468. For some
reason, this test seems to be consistently green on the CI (on amd64
arch) and when running `hack/test/unit` locally. However it consistently
fails when running `TESTFLAGS='-shuffle on' hack/test/unit` locally.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2021-11-16 09:37:34 +01:00
Mark Feit
3a938df4b5
Make the network allocator handle IPv4 blocks too small for network/broadcast addresses.
This was originally in docker/libnetwork#2624, which has been closed since the
code was moved here.

When creating a new network, IPAM's address allocator attempts to reserve the
network and broadcast addresses on IPv4 networks of all sizes. For RFC 3021
point-to-point networks (IPv4 /31s), this consumes both available addresses and
renders any attempt to allocate an address from the block unsuccessful.

This change prevents those reservations from taking place on IPv4 networks having
two or fewer addresses (i.e., /31s and /32s) while retaining the existing behavior
for larger IPv4 blocks and all IPv6 blocks.

In case you're wondering why anyone would allocate /31s:  I work for a network
service provider.  We use a lot of point-to-point networks.  This cuts our
address space utilization for those by 50%, which makes ARIN happy.

This patch modifies the network allocator to recognize when an network is too
small for network and broadcast addresses and skip those reservations.

There are additional unit tests to make sure the functions involved behave as expected.

Try these out:

 * `docker network create --driver bridge --subnet 10.200.1.0/31 --ip-range 10.200.1.0/31 test-31`
 * `docker network create --driver bridge --subnet 10.200.1.0/32 --ip-range 10.200.1.0/32 test-32`

My installation has been running this patch in production with /31s since March.

Signed-off-by: Mark Feit <mfeit@internet2.edu>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-27 13:03:19 +02:00
Sebastiaan van Stijn
eadf839530
fix TestInvalidRemoteDriver() to check underlying error
commit b1a3fe4934 changed how the error was
returned (which is now wrapped), causing the test to fail:

    === RUN   TestInvalidRemoteDriver
        libnetwork_test.go:1289: Did not fail with expected error. Actual error: Plugin does not implement the requested driver: plugin="invalid-network-driver", requested implementation="NetworkDriver"
    --- FAIL: TestInvalidRemoteDriver (0.01s)

Changing the test to use errors.Is()

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-25 14:57:18 +02:00
Sebastiaan van Stijn
79d6e935ad
libnetwork: some minor refactoring / cleanup
- don't pass the query's quetion.name separately, as we're already
  passing the query itself.
- remove a "fallthrough" in favor of combining the cases in the switch

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-15 15:26:15 +02:00
Sebastiaan van Stijn
d86a331fa4
libnetwork: improve consistency in log messages
- Make sure all log messages have the `[resolver]` prefix
- Use `logrus.WithError()` consistently
- Improve information included in some logs

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-15 12:51:02 +02:00
Sebastiaan van Stijn
9a09448540
libnetwork: ServeDNS(): don't panic on unsupported query types
This was added in b3c883bb2f, but resulted
in a panic if the embedded DNS had to handle an unsupported query-type,
such as ANY.

This patch adds a debug log for this case (to better describe how it's
handled.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-14 20:17:39 +02:00
Sebastiaan van Stijn
abd7966165
Merge pull request #42794 from thaJeztah/remove_libnetwork_api
libnetwork: remove API, as it's no longer used
2021-08-28 04:41:21 +02:00
Sebastiaan van Stijn
7bdf98276c
Merge pull request #42787 from thaJeztah/libnetwork_fix_lint
libnetwork: fix some linting issues
2021-08-28 01:18:13 +02:00
Sebastiaan van Stijn
175dc09491
Merge pull request #42795 from thaJeztah/libnetwork_cleanup
libnetwork/types: remove unused functions
2021-08-27 19:43:34 +02:00
Sebastiaan van Stijn
c23eae3114
libnetwork/ipamapi: godoc fixes
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-27 16:52:46 +02:00
Sebastiaan van Stijn
92ea7df3d1
libnetwork: fix some linting issues
- fix incorrectly formatted GoDoc and comments
- rename a variable that collided with the `cap` built-in

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-27 16:52:44 +02:00
Sebastiaan van Stijn
c425188bc0
libnetwork: remove API, as it's no longer used
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-27 16:50:33 +02:00