Commit graph

2642 commits

Author SHA1 Message Date
Flavio Crisciani
7fc1795cdf Allows to set generic knobs on the Sandbox
Refactor the ostweaks file to allows a more easy reuse
Add a method on the osl.Sandbox interface to allow setting
knobs on the sandbox

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-06-28 16:14:08 -07:00
Flavio Crisciani
ef457321a9 Merge pull request #2200 from fcrisciani/networkdb-retry
Adjust corner case for reconnect logic
2018-06-28 16:00:00 -07:00
Flavio Crisciani
a5e7cfe3d7 Merge pull request #2138 from ctelfer/scalable-lb
Improve load balancing scalability in swarm mode
2018-06-28 15:49:55 -07:00
Chris Telfer
ac0aa6485b Adjust warnings for transient LB endpoint conds
Add debug and error logs to notify when a load balancing sandbox
is not found.  This can occur in normal operation during removal.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-06-28 12:08:18 -04:00
Chris Telfer
0b14b45f0c Prevent race between add-binding and net-delete
Lock the network ID in the controller during an addServiceBinding to
prevent racing with network.delete().  This would cause the binding to
be silently ignored in the system.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-06-28 12:08:18 -04:00
Chris Telfer
ea2fa20859 Add endpoint load-balancing mode
This is the heart of the scalability change for services in libnetwork.
The present routing mesh adds load-balancing rules for a network to
every container connected to the network.  This newer approach creates a
load-balancing endpoint per network per node.  For every service on a
network, libnetwork assigns the VIP of the service to the endpoint's
interface as an alias.  This endpoint must have a unique IP address in
order to route return traffic to it.  Traffic destined for a service's
VIP arrives at the load-balancing endpoint on the VIP and from there,
Linux load balances it among backend destinations while SNATing said
traffic to the endpoint's unique IP address.

The net result of this scheme is that each node in a swarm need only
have one set of load balancing state per service instead of one per
container on the node.  This scheme is very similar to how services
currently operate on Windows nodes in libnetwork.  It (as with Windows
nodes) costs the use of extra IP addresses in a network (one per node)
and an extra network hop in the stack, although, always in the stack
local to the container.

In order to prevent existing deployments from suddenly failing if they
failed to allocate sufficient address space to include per-node
load-balancing endpoint IP addresses, this patch preserves the existing
functionality and activates the new functionality on a per-network
basis depending on whether the network has a load-balancing endpoint.
Eventually, moby should always set this option when creating new
networks and should only omit it for networks created as part of a swarm
that are not marked to use endpoint load balancing.

This patch also normalizes the code to treat "load" and "balancer"
as two separate words from the perspectives of variable/function naming.
This means that the 'b' in "balancer" must be capitalized.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-06-28 12:08:18 -04:00
Chris Telfer
85a3483b4b Refactor [add|rm]LBBackend() to use lb struct
This was passing extra information and adding confusion about the
purpose of the load balancing structure.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-06-28 12:08:18 -04:00
Chris Telfer
78b684a24a Add ability to alias any interface in a sanbox
New load balancing code will require ability to add aliases to
load-balncer sandboxes.  So this broadens the OSL interface to allow
adding aliases to any interface, along with the facility to get the
loopback interface's name based on the OS.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-06-28 12:08:18 -04:00
Chris Telfer
f2c6009583 Add SrcName() method to return interface name
This method returns the name of the interface from the perspective
of the host OS pre-container.  This will be required later for
finding matching a sandbox's interface name to an endpoint which
is, in turn, requied for adding an IP alias to a load balancer
endpoint.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-06-28 12:08:18 -04:00
Chris Telfer
31d3de0994 Fix error handling in createLoadBalncerSandbox()
Error unwinding only works if the error variable is used consistently
and isn't hidden in the scope of other if statements.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-06-28 12:08:18 -04:00
Chris Telfer
1449e88f7a Avoid default gateway collisions
Default gateways truncate the endpoint name to 12 characters.  This can
make network endpoints ambiguous especially for load-balancing sandboxes
for networks with lenghty names (such as with our prefixes).  Address
this by detecting an overflow in the sanbox name length and instead
opting to name the gateway endpoint "gateway_<id>" which should never
collide.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-06-28 12:08:18 -04:00
Chris Telfer
04bfc61497 Add option processing to network.Delete()
Change the Delete() method to take optional options and add
NetworkDeleteOptionRemoveLB as one such option.  This option allows
explicit removal of an ingress network along with its load-balancing
endpoint if there are no other endpoints in the network.  Prior to this,
the libnetwork client would have to manually search for and remove the
ingress load balancing endpoint from an ingress network.  This was, of
course, completely hacky.

This commit will require a slight modification in moby to make use of
the option when deleting the ingress network.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-06-28 12:08:12 -04:00
Chris Telfer
1abac50c8d Generate LB sandbox/endpoint names in one place
Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-06-28 00:46:20 -04:00
Flavio Crisciani
9d758090cf Merge pull request #2199 from selansen/new_plugin_intf
Use new plugin interfaces provided by plugin pkg
2018-06-27 17:24:20 -07:00
Chris Telfer
28ebc2617f Merge pull request #2193 from euanh/update-protobuf
Add support for regenerating protocol buffers, and refresh .pb.go files
2018-06-27 17:30:06 -04:00
Brian Goff
5525c22635 Fix unknow driver test error
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 8856c1ec9557e58917421bf1b32724262745a795)
Signed-off-by: selansen <elango.siva@docker.com>
2018-06-27 17:28:52 -04:00
Chris Telfer
9cf5a4ac4f Merge pull request #2205 from CharlieR-o-o-t/fix_createEndpoint
Fix net driver response loss on createEndpoint
2018-06-26 16:16:19 -04:00
Siarhei Rasiukevich
fe9a5a225d Fix net driver response loss on createEndpoint
Fix related to bug: https://github.com/docker/for-linux/issues/348
We should perform updateToStore(ep) after n.addEndpoint or do update twice,
otherwise response from network plugin will not be written to KV storage.
This results in container creation with broken network config.

Signed-off-by: Siarhei Rasiukevich <raskintech@gmail.com>
2018-06-26 13:27:47 +03:00
Flavio Crisciani
d9b5aa4c86 Merge pull request #2197 from fcrisciani/netdb-test
Add retry to cluster-peers
2018-06-25 11:54:52 -07:00
Flavio Crisciani
9140372ec3 Merge pull request #2202 from fcrisciani/ignore
Added back dockerignore
2018-06-25 11:53:47 -07:00
Brian Goff
f04280986e Reduce libcontainer dep
Most of the libcontainer imports was just for a single test to marshal a
simple type, meanwhile this caused all kinds of transient imports that
are not really needed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit a07a1ee9ccdf4c5a3a90eea9fd359f10b5156c84)
Signed-off-by: selansen <elango.siva@docker.com>
2018-06-23 00:15:16 -04:00
Brian Goff
791700aed3 Use new plugin interfaces provided by plugin pkg
The use of `Client()` on v2 plugins is being deprecated so that we can
be more flexible on the protocol used for plugins.

This means checking specifically if the plugin implements the
`Client() *plugins.Client` interface for V1 plugins, and for v2 plugins
building a the client manually.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 45824a226b8a220d6f189c2d25fe16f9efc83db9)
Signed-off-by: selansen <elango.siva@docker.com>
2018-06-22 23:52:02 -04:00
Brian Goff
e07681c8ca Bump docker commit
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit aae1b0e116d0c4ee0e46494864d1540fec22ced3)
Signed-off-by: selansen <elango.siva@docker.com>
2018-06-22 23:51:21 -04:00
Flavio Crisciani
62ebaeac7a Added back dockerignore
Try to reduce the docker context for the build

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-06-22 16:10:22 -07:00
Flavio Crisciani
28ec64d2e5 Merge pull request #2183 from euanh/update-vendoring
Update vendoring to match moby/moby
2018-06-22 08:17:39 -07:00
Euan Harris
96c7cba64c networkdb, drivers: Regenerate protocol buffers
agent.pb.go is unchanged, but the files in networkdb and drivers
are slightly different when regenerated using the current versions
of protoc and gogoproto.    This is probably because agent.pb.go
was last regenerated quite recently, in February 2018, whereas
networkdb.pb.go and overlay/overlay.pb.go were last changed in 2017,
and windows/overlay/overlay.pb.go was last changed in 2016.

Signed-off-by: Euan Harris <euan.harris@docker.com>
2018-06-22 15:03:12 +01:00
Euan Harris
22a24df515 Makefile, Dockerfile: Add support for regenerating protocol buffers
Signed-off-by: Euan Harris <euan.harris@docker.com>
2018-06-22 15:03:12 +01:00
Flavio Crisciani
500d9f4515 Adjust corner case for reconnect logic
Previous logic was not accounting that each node is
in the node list so the bootstrap nodes won't retry
to reconnect because they will always find themselves
in the node map
Added test that validate the gossip island condition

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-06-21 18:04:55 -07:00
Chris Telfer
755ef5bf29 Merge pull request #2196 from fcrisciani/support
Add auto update to support.sh
2018-06-21 17:02:26 -04:00
Flavio Crisciani
4f3fb7ee6a Add retry to cluster-peers
Add retry field to cluster-peers probe

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-06-21 13:08:04 -07:00
Euan Harris
705bf07f05 vendor.conf: Update vendored components to match moby
* Update dependencies to match moby master; add new sub-dependencies
  as necessary.

* Update moby to latest

* Update gocapability

  This moves gocapability beyond the version vendored in moby;
  presumably the code which requires this particular version
  is not used in moby and is removed by vndr.   Moby will need
  to be updated as well.

Signed-off-by: Euan Harris <euan.harris@docker.com>
2018-06-21 15:16:21 +01:00
Euan Harris
9b114971e5 controller: Unwrap error type returned by PluginGetter
moby/moby commit b27f70d45 wraps the ErrNotFound error returned when
a plugin cannot be found, to include a backtrace.   This changes the
type of the error, so contoller.loadDriver no longer converts it to a
libnetwork plugin.NotFoundError.  This causes a couple of tests which
inspect the return type to fail;  most code only checks whether the
error is non-nil and is not affected by the change in type.

Signed-off-by: Euan Harris <euan.harris@docker.com>
2018-06-21 15:08:25 +01:00
Flavio Crisciani
5de876c7ae Add auto update to support.sh
Before running the support script
try to fetch the latest version

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-06-20 15:29:45 -07:00
Flavio Crisciani
6716626d32 Merge pull request #2172 from thaJeztah/update_miekd_dns
Update miekg/dns to v1.0.7
2018-06-20 14:31:49 -07:00
Flavio Crisciani
8282efa3d7 Merge pull request #2194 from fcrisciani/support
Create support as a container
2018-06-20 14:24:24 -07:00
Flavio Crisciani
9dbf55097a Create support as a container
Containerize support.sh

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-06-20 14:05:14 -07:00
Flavio Crisciani
3a5f106ec3 Merge pull request #2192 from fcrisciani/circle-multistage
Multistage for build
2018-06-19 21:58:45 -07:00
Flavio Crisciani
8f1fca3e66 Multistage for build
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-06-19 21:51:11 -07:00
Flavio Crisciani
c548093db6 Merge pull request #2191 from euanh/bump-go-version
Dockerfile.build: Bump Go to 1.10.2
2018-06-19 09:24:11 -07:00
Flavio Crisciani
8dd7572a4c Merge pull request #2168 from selansen/circle2.0
Migration from CircleCI1.0 to CircleCI2.0
2018-06-19 09:23:34 -07:00
Euan Harris
24bdb26483 Dockerfile.build: Bump Go to 1.10.2
This brings libnetwork up to date with moby/moby.

Signed-off-by: Euan Harris <euan.harris@docker.com>
2018-06-19 14:07:58 +01:00
Flavio Crisciani
803f882770 Merge pull request #2189 from abhi/master
Fixing the unit test for overlapping subnet
2018-06-18 12:48:20 -07:00
Abhinandan Prativadi
f5fa40f9aa Fixing the unit test for overlapping subnet
Signed-off-by: Abhinandan Prativadi <abhi@docker.com>
2018-06-18 12:31:48 -07:00
Flavio Crisciani
59a2ef7843 Merge pull request #2128 from fcrisciani/go-sockaddr-vndr
Vendor go-sockaddr
2018-06-18 11:37:12 -07:00
Flavio Crisciani
828a6d788e Merge pull request #2177 from ctelfer/subnet-cursor
Avoid reusing subnets when allocating from pools
2018-06-18 11:23:11 -07:00
Flavio Crisciani
891fb82213 Merge pull request #2148 from abhi/ipam-check
Adding a check for subnet pool overlap
2018-06-18 11:19:41 -07:00
Flavio Crisciani
c7300fec17 Merge pull request #2160 from euanh/portbinding-ipv6
types: Handle IPv6 literals correctly in port bindings
2018-06-18 09:19:48 -07:00
Flavio Crisciani
48196df4a2 Further makefile cleanup
- cleaned the make check
- local build do not require context

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-06-16 11:03:11 -07:00
selansen
fbf02c3794 Migration from CircleCI1.0 to CircleCI2.0
This commit will allow us to use newer version of CircleCI.

Signed-off-by: selansen <elango.siva@docker.com>
2018-06-16 11:03:11 -07:00
Flavio Crisciani
c19cc28bd8 Merge pull request #2186 from trapier/support_sh_add_links
support.sh: add host links
2018-06-15 08:58:38 -07:00