Currently ipam/ipamutils has a bunch of dependencies
in osl and netlink which makes the ipam/ipamutils harder
to use independently with other applications. This PR
modularizes ipam/ipamutils into a standalone package
with no OS level dependencies.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
This moves the initialization of the pre-defined networks to where it's
used instead of in package init.
This reason for this change is having this be populated in `init()`
causes it to always consume cpu, and memory (4.3MB of memory), to
populate even if the package is unused (like for instnace, in a re-exec).
Here is a memory profile of docker/docker just after starting the daemon of the
top 10 largest memory consumers:
Before:
```
flat flat% sum% cum cum%
0 0% 0% 11.89MB 95.96% runtime.goexit
0 0% 0% 6.79MB 54.82% runtime.main
0 0% 0% 5.79MB 46.74% main.init
0 0% 0% 4.79MB 38.67% github.com/docker/docker/api/server/router/network.init
0 0% 0% 4.79MB 38.67% github.com/docker/libnetwork.init
0 0% 0% 4.29MB 34.63% github.com/docker/libnetwork/ipam.init
0 0% 0% 4.29MB 34.63% github.com/docker/libnetwork/ipams/builtin.init
0 0% 0% 4.29MB 34.63% github.com/docker/libnetwork/ipamutils.init
0 0% 0% 4.29MB 34.63% github.com/docker/libnetwork/ipamutils.init.1
4.29MB 34.63% 34.63% 4.29MB 34.63% github.com/docker/libnetwork/ipamutils.initGranularPredefinedNetworks
```
After:
```
flat flat% sum% cum cum%
0 0% 0% 4439.37kB 89.66% runtime.goexit
0 0% 0% 4439.37kB 89.66% runtime.main
0 0% 0% 3882.11kB 78.40% github.com/docker/docker/cli.(*Cli).Run
0 0% 0% 3882.11kB 78.40% main.main
3882.11kB 78.40% 78.40% 3882.11kB 78.40% reflect.callMethod
0 0% 78.40% 3882.11kB 78.40% reflect.methodValueCall
0 0% 78.40% 557.26kB 11.25% github.com/docker/docker/api/server.init
557.26kB 11.25% 89.66% 557.26kB 11.25% html.init
0 0% 89.66% 557.26kB 11.25% html/template.init
0 0% 89.66% 557.26kB 11.25% main.init
```
Now, of course the docker daemon will still need to consume this memory, but
at least now re-execs and such won't have to re-init these variables.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
- The pool request code does not behave properly in
case of concurrent requests when client does not
specify a preferred pool. It may dispense the same
predefined pool to different networks.
- The issue is common for local and global
address spaces
Signed-off-by: Alessandro Boch <aboch@docker.com>
- Test random de-allocation of allocated addresses
which is closer to real use case
- Test db reconstruction after read from datastore
Signed-off-by: Alessandro Boch <aboch@docker.com>
- Remove from contract predefined errors which are no longer
valid (ex. ErrInvalidIpamService, ErrInvalidIpamConfigService)
- Do not use network driver error for ipam load failure in controller.go
- Bitseq to expose two well-known errors (no more bit available, bit is already set)
- Default ipam to report proper well-known error on RequestAddress()
based on bitseq returned error
- Default ipam errors to comply with types error interface
Signed-off-by: Alessandro Boch <aboch@docker.com>
- Currently allocator pulls all the bitmasks from datastore
before processing each public API. This is not needed as
the APIs already selectively pull the interested bitmask
when needed.
Signed-off-by: Alessandro Boch <aboch@docker.com>
Added a check to see if address space is valid in
addrSpaces map before accessing it. Also fixed some
error strings so that it provides better information
to the user.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Since libnetwork is going to provide createNetwork
notifications only once when the network is created
bridge network needs to save it's network state in
persistent store so that it becomes available even
after restart.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
- Set bridge ipv4 address when bridge is present
- IPv6 changes for bridge
- Convert unit tests to the new model
Signed-off-by: Alessandro Boch <aboch@docker.com>
- libnetwork should reserve only the auxiliary
addresses which belong to the container
addresable pool. And should fail the network
creation if the aux addr does not belong to
the master pool.
Signed-off-by: Alessandro Boch <aboch@docker.com>
Remove the need for watching for IPAM data
structures and add multi store support code and
data reorganization to simplify address space
management.
Signed-off-by: Jana Radhakrishnan <mrjana@docker.com>
Currently the default ipam implementation ignores the prefered ip if the
request is made on an existing sub-pool. The priority should be other
way around.
Signed-off-by: Madhu Venugopal <madhu@docker.com>
This way we won't vendor test related functions in docker anymore.
It also moves netns related functions to a new ns package to be able to
call the ns init function in tests. I think this also helps with the
overall package isolation.
Signed-off-by: David Calavera <david.calavera@gmail.com>
- also provided a new utility to compute the
host part ip address which is resilient to
input passed in different representations.
Signed-off-by: Alessandro Boch <aboch@docker.com>
In that commit, AtomicPutCreate takes previous = nil to Atomically create keys
that don't exist. We need a create operation that is atomic to prevent races
between multiple libnetworks creating the same object.
Previously, we just created new KVs with an index of 0 and wrote them to the
datastore. Consul accepts this behaviour and interprets index of 0 as
non-existing, but other data backends do no.
- Add Exists() to the KV interface. SetIndex() should also modify a KV so
that it exists.
- Call SetIndex() from within the GetObject() method on DataStore interface.
- This ensures objects have the updated values for exists and index.
- Add SetValue() to the KV interface. This allows implementers to define
their own method to marshall and unmarshall (as bitseq and allocator have).
- Update existing users of the DataStore (endpoint, network, bitseq,
allocator, ov_network) to new interfaces.
- Fix UTs.
- At Handle creation, first check if an instance of the
the respective object is already present in the datastore.
- Handle sequence must be saved only if commit
to datastore is succesfull
- Caller (ipam) needs to manage the retry
Signed-off-by: Alessandro Boch <aboch@docker.com>
- Handle contains sequence and identifier.
This way datastore integration can be done
at bitseq level.
Signed-off-by: Alessandro Boch <aboch@docker.com>
- In order to facilitate usage of datastore
- This makes it slower. Efficiency will be
revisited later after datastore integration
is done.
Signed-off-by: Alessandro Boch <aboch@docker.com>
