TestRequestReleaseAddressDuplicate gets flagged by go test -race because
the same err variable inside the test is assigned to from multiple
goroutines without synchronization, which obscures whether or not there
are any data races in the code under test.
Trouble is, the test _depends on_ the data race to exit the loop if an
error occurs inside a spawned goroutine. And the test contains a logical
concurrency bug (not flagged by the Go race detector) which can result
in false-positive test failures. Because a release operation is logged
after the IP is released, the other goroutine could reacquire the
address and log that it was reacquired before the release is logged.
Fix up the test so it is no longer subject to data races or
false-positive test failures, i.e. flakes.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit b62445871e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
"math/rand".Seed
- Migrate to using local RNG instances.
"archive/tar".TypeRegA
- The deprecated constant tar.TypeRegA is the same value as
tar.TypeReg and so is not needed at all.
Signed-off-by: Cory Snider <csnider@mirantis.com>
(cherry picked from commit dea3f2b417)
Signed-off-by: Cory Snider <csnider@mirantis.com>
libnetwork/etchosts/etchosts_test.go:167:54: empty-lines: extra empty line at the end of a block (revive)
libnetwork/osl/route_linux.go:185:74: empty-lines: extra empty line at the start of a block (revive)
libnetwork/osl/sandbox_linux_test.go:323:36: empty-lines: extra empty line at the start of a block (revive)
libnetwork/bitseq/sequence.go:412:48: empty-lines: extra empty line at the start of a block (revive)
libnetwork/datastore/datastore_test.go:67:46: empty-lines: extra empty line at the end of a block (revive)
libnetwork/datastore/mock_store.go:34:60: empty-lines: extra empty line at the end of a block (revive)
libnetwork/iptables/firewalld.go:202:44: empty-lines: extra empty line at the end of a block (revive)
libnetwork/iptables/firewalld_test.go:76:36: empty-lines: extra empty line at the end of a block (revive)
libnetwork/iptables/iptables.go:256:67: empty-lines: extra empty line at the end of a block (revive)
libnetwork/iptables/iptables.go:303:128: empty-lines: extra empty line at the start of a block (revive)
libnetwork/networkdb/cluster.go:183:72: empty-lines: extra empty line at the end of a block (revive)
libnetwork/ipams/null/null_test.go:44:38: empty-lines: extra empty line at the end of a block (revive)
libnetwork/drivers/macvlan/macvlan_store.go:45:52: empty-lines: extra empty line at the end of a block (revive)
libnetwork/ipam/allocator_test.go:1058:39: empty-lines: extra empty line at the start of a block (revive)
libnetwork/drivers/bridge/port_mapping.go:88:111: empty-lines: extra empty line at the end of a block (revive)
libnetwork/drivers/bridge/link.go:26:90: empty-lines: extra empty line at the end of a block (revive)
libnetwork/drivers/bridge/setup_ipv6_test.go:17:34: empty-lines: extra empty line at the end of a block (revive)
libnetwork/drivers/bridge/setup_ip_tables.go:392:4: empty-lines: extra empty line at the start of a block (revive)
libnetwork/drivers/bridge/bridge.go:804:50: empty-lines: extra empty line at the start of a block (revive)
libnetwork/drivers/overlay/ov_serf.go:183:29: empty-lines: extra empty line at the start of a block (revive)
libnetwork/drivers/overlay/ov_utils.go:81:64: empty-lines: extra empty line at the end of a block (revive)
libnetwork/drivers/overlay/peerdb.go:172:67: empty-lines: extra empty line at the start of a block (revive)
libnetwork/drivers/overlay/peerdb.go:209:67: empty-lines: extra empty line at the start of a block (revive)
libnetwork/drivers/overlay/peerdb.go:344:89: empty-lines: extra empty line at the start of a block (revive)
libnetwork/drivers/overlay/peerdb.go:436:63: empty-lines: extra empty line at the start of a block (revive)
libnetwork/drivers/overlay/overlay.go:183:36: empty-lines: extra empty line at the start of a block (revive)
libnetwork/drivers/overlay/encryption.go:69:28: empty-lines: extra empty line at the end of a block (revive)
libnetwork/drivers/overlay/ov_network.go:563:81: empty-lines: extra empty line at the start of a block (revive)
libnetwork/default_gateway.go:32:43: empty-lines: extra empty line at the start of a block (revive)
libnetwork/errors_test.go:9:40: empty-lines: extra empty line at the start of a block (revive)
libnetwork/service_common.go:184:64: empty-lines: extra empty line at the end of a block (revive)
libnetwork/endpoint.go:161:55: empty-lines: extra empty line at the end of a block (revive)
libnetwork/store.go:320:33: empty-lines: extra empty line at the end of a block (revive)
libnetwork/store_linux_test.go:11:38: empty-lines: extra empty line at the end of a block (revive)
libnetwork/sandbox.go:571:36: empty-lines: extra empty line at the start of a block (revive)
libnetwork/service_common.go:317:246: empty-lines: extra empty line at the start of a block (revive)
libnetwork/endpoint.go:550:17: empty-lines: extra empty line at the end of a block (revive)
libnetwork/sandbox_dns_unix.go:213:106: empty-lines: extra empty line at the start of a block (revive)
libnetwork/controller.go:676:85: empty-lines: extra empty line at the end of a block (revive)
libnetwork/agent.go:876:60: empty-lines: extra empty line at the end of a block (revive)
libnetwork/resolver.go:324:69: empty-lines: extra empty line at the end of a block (revive)
libnetwork/network.go:1153:92: empty-lines: extra empty line at the end of a block (revive)
libnetwork/network.go:1955:67: empty-lines: extra empty line at the start of a block (revive)
libnetwork/network.go:2235:9: empty-lines: extra empty line at the start of a block (revive)
libnetwork/libnetwork_internal_test.go:336:26: empty-lines: extra empty line at the start of a block (revive)
libnetwork/resolver_test.go:76:35: empty-lines: extra empty line at the end of a block (revive)
libnetwork/libnetwork_test.go:303:38: empty-lines: extra empty line at the end of a block (revive)
libnetwork/libnetwork_test.go:985:46: empty-lines: extra empty line at the end of a block (revive)
libnetwork/ipam/allocator_test.go:1263:37: empty-lines: extra empty line at the start of a block (revive)
libnetwork/errors_test.go:9:40: empty-lines: extra empty line at the end of a block (revive)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cd381aea56)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This was originally in docker/libnetwork#2624, which has been closed since the
code was moved here.
When creating a new network, IPAM's address allocator attempts to reserve the
network and broadcast addresses on IPv4 networks of all sizes. For RFC 3021
point-to-point networks (IPv4 /31s), this consumes both available addresses and
renders any attempt to allocate an address from the block unsuccessful.
This change prevents those reservations from taking place on IPv4 networks having
two or fewer addresses (i.e., /31s and /32s) while retaining the existing behavior
for larger IPv4 blocks and all IPv6 blocks.
In case you're wondering why anyone would allocate /31s: I work for a network
service provider. We use a lot of point-to-point networks. This cuts our
address space utilization for those by 50%, which makes ARIN happy.
This patch modifies the network allocator to recognize when an network is too
small for network and broadcast addresses and skip those reservations.
There are additional unit tests to make sure the functions involved behave as expected.
Try these out:
* `docker network create --driver bridge --subnet 10.200.1.0/31 --ip-range 10.200.1.0/31 test-31`
* `docker network create --driver bridge --subnet 10.200.1.0/32 --ip-range 10.200.1.0/32 test-32`
My installation has been running this patch in production with /31s since March.
Signed-off-by: Mark Feit <mfeit@internet2.edu>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Perhaps the testutils package in the past had an `init()` function to set up
specific things, but it no longer has. so these imports were doing nothing.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
libnetwork does different stuff depending on if you are running the
tests in a container or not... without telling it we are in a container
a bunch of the tests actually fail.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
After moving libnetwork to this repo, we need to update all the import
paths for libnetwork to point to docker/docker/libnetwork instead of
docker/libnetwork.
This change implements that.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This change brings global default address pool feature into
libnetwork. Idea is to reuse same code flow and functions that were
implemented for local scope default address pool.
Function InitNetworks carries most of the changes. local scope default
address pool init should always happen only once. But Global scope
default address pool can be initialized multiple times.
Signed-off-by: selansen <elango.siva@docker.com>
TestOverlappingRequests checks that pool requests which are supersets or
subsets of existing allocations, and those which overlap with existing
allocations at the beginning or the end.
Multiple allocation is now tested by TestOverlappingRequests, so
TestDoublePoolRelease only needs to test double releasing.
Signed-off-by: Euan Harris <euan.harris@docker.com>
Add a test to confirm that the pool allocator will iterate through all
the pools even if some earlier ones were freed before coming back to
previously allocated pools.
Signed-off-by: Chris Telfer <ctelfer@docker.com>
Go 1.7 added the subtest feature which can make table-driven tests much easier to run and debug. Some tests are not using this feature.
Signed-off-by: Yang Li <idealhack@gmail.com>
Releasing a pool which has already been released should fail; this
change increases coverage by a fraction by exercising this path.
Signed-off-by: Euan Harris <euan.harris@docker.com>
This is new feature that allows user to specify which subnetwork
Docker contrainer should choose from when it creates bridge network.
This libnetwork commit is to address moby PR 36054
Signed-off-by: selansen <elango.siva@docker.com>
Since bit allocation is no longer first available from
the start some verfications are removed/modified to
the change allocation model
Signed-off-by: Abhinandan Prativadi <abhi@docker.com>
Fixing an earlier commit which needlessly registered
boltdb in allocator.go while it is needed only for tests.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Currently ipam/ipamutils has a bunch of dependencies
in osl and netlink which makes the ipam/ipamutils harder
to use independently with other applications. This PR
modularizes ipam/ipamutils into a standalone package
with no OS level dependencies.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
This moves the initialization of the pre-defined networks to where it's
used instead of in package init.
This reason for this change is having this be populated in `init()`
causes it to always consume cpu, and memory (4.3MB of memory), to
populate even if the package is unused (like for instnace, in a re-exec).
Here is a memory profile of docker/docker just after starting the daemon of the
top 10 largest memory consumers:
Before:
```
flat flat% sum% cum cum%
0 0% 0% 11.89MB 95.96% runtime.goexit
0 0% 0% 6.79MB 54.82% runtime.main
0 0% 0% 5.79MB 46.74% main.init
0 0% 0% 4.79MB 38.67% github.com/docker/docker/api/server/router/network.init
0 0% 0% 4.79MB 38.67% github.com/docker/libnetwork.init
0 0% 0% 4.29MB 34.63% github.com/docker/libnetwork/ipam.init
0 0% 0% 4.29MB 34.63% github.com/docker/libnetwork/ipams/builtin.init
0 0% 0% 4.29MB 34.63% github.com/docker/libnetwork/ipamutils.init
0 0% 0% 4.29MB 34.63% github.com/docker/libnetwork/ipamutils.init.1
4.29MB 34.63% 34.63% 4.29MB 34.63% github.com/docker/libnetwork/ipamutils.initGranularPredefinedNetworks
```
After:
```
flat flat% sum% cum cum%
0 0% 0% 4439.37kB 89.66% runtime.goexit
0 0% 0% 4439.37kB 89.66% runtime.main
0 0% 0% 3882.11kB 78.40% github.com/docker/docker/cli.(*Cli).Run
0 0% 0% 3882.11kB 78.40% main.main
3882.11kB 78.40% 78.40% 3882.11kB 78.40% reflect.callMethod
0 0% 78.40% 3882.11kB 78.40% reflect.methodValueCall
0 0% 78.40% 557.26kB 11.25% github.com/docker/docker/api/server.init
557.26kB 11.25% 89.66% 557.26kB 11.25% html.init
0 0% 89.66% 557.26kB 11.25% html/template.init
0 0% 89.66% 557.26kB 11.25% main.init
```
Now, of course the docker daemon will still need to consume this memory, but
at least now re-execs and such won't have to re-init these variables.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
- The pool request code does not behave properly in
case of concurrent requests when client does not
specify a preferred pool. It may dispense the same
predefined pool to different networks.
- The issue is common for local and global
address spaces
Signed-off-by: Alessandro Boch <aboch@docker.com>
- Test random de-allocation of allocated addresses
which is closer to real use case
- Test db reconstruction after read from datastore
Signed-off-by: Alessandro Boch <aboch@docker.com>
Since libnetwork is going to provide createNetwork
notifications only once when the network is created
bridge network needs to save it's network state in
persistent store so that it becomes available even
after restart.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
- Set bridge ipv4 address when bridge is present
- IPv6 changes for bridge
- Convert unit tests to the new model
Signed-off-by: Alessandro Boch <aboch@docker.com>
Remove the need for watching for IPAM data
structures and add multi store support code and
data reorganization to simplify address space
management.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
This way we won't vendor test related functions in docker anymore.
It also moves netns related functions to a new ns package to be able to
call the ns init function in tests. I think this also helps with the
overall package isolation.
Signed-off-by: David Calavera <david.calavera@gmail.com>
- also provided a new utility to compute the
host part ip address which is resilient to
input passed in different representations.
Signed-off-by: Alessandro Boch <aboch@docker.com>
