Commit graph

2573 commits

Author SHA1 Message Date
Arko Dasgupta
c7f0b0152e Fix NPE due to null value returned by ep.Iface()
This PR carryforwards https://github.com/moby/libnetwork/pull/2239
and incorporates the suggestions in comments to fix the NPE and
potential NPEs due to a null value returned by ep.Iface()

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-04-02 22:34:55 -07:00
Sebastiaan van Stijn
6b3673877c Merge pull request #2538 from wangyumu/add-error-invalid-default-address-pools
Fixes moby/moby#40388, produce an error with invalid address pool
2020-03-31 19:45:25 +02:00
Arko Dasgupta
2e64325fee Merge pull request #2540 from SamWhited/resolver_panic
Fixes a panic in the DNS resolver
2020-03-18 11:26:00 -07:00
Arko Dasgupta
e49ee8266d Merge pull request #2460 from moby/revert-2450-iptables-policy
Revert "Always configure iptables forward policy"
2020-03-18 11:22:47 -07:00
Sam Whited
1e02aae252 Fixes a panic in the DNS resolver
Under certain conditions it appears that the DNS response and returned
error can be nil. When this happens, checking resp.Truncated results in
a nil panic so we must first check that the response is not nil before
checking if a truncated response was received.

See moby/moby#40715

Signed-off-by: Sam Whited <sam@samwhited.com>
2020-03-18 13:59:59 -04:00
Wang Yumu
4dda50d226 Fixes moby/moby#40388
Signed-off-by: Wang Yumu <37442693@qq.com>
2020-03-18 00:09:10 +08:00
Sebastiaan van Stijn
d846c2b1ab vendor: update vishvananda/netlink v1.1.0
full diff: https://github.com/vishvananda/netlink/compare/v1.0.0...v1.1.0

also updated moby/ipvs, which is compatible with this version of netlink,
and update vishvananda/netns to current master (which added go.mod)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-03-12 18:25:54 +01:00
Brian Goff
a533fe7094 Use vendored ipvs package
The ipvs package was moved to a separate repo.

The ipvs package is a fairly generic set of helpers for managing IPVS.
The ipvs package is used by docker swarm and kubernetes.
Because we want to merge libnetwork back into the moby/moby codebase
while also not creating more dependencies for other projects on
moby/moby itself, it was decided that the best path for ipvs is to live
on it's own since there are no other ties to libnetwork.

Ref: https://github.com/moby/libnetwork/issues/2522

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-03-11 12:13:37 -07:00
elangovan sivanandam
c485bbf754 Merge pull request #2523 from arkodg/seperate-internal-empty-parent
Macvlan: Separate empty parent and internal
2020-03-05 15:26:57 -05:00
elangovan sivanandam
a9596db128 Merge pull request #2524 from trapier/cleanup-vfp-during-network-removal
Cleanup VFP during overlay network removal
2020-03-05 14:55:50 -05:00
Arko Dasgupta
76b5905cbe Macvlan: Separate empty parent and internal
https://github.com/docker/libnetwork/pull/2419 and
https://github.com/docker/libnetwork/pull/2407
attempted to seperate out empty parent and internal for
macvlan and ipvlan networks

However it didnt pass the integration tests in moby
https://github.com/moby/moby/pull/40596 and exposed some
more plumbing that needed to be done to make sure
we separate the two things

If the -o parent is empty we create a dummylink
and if internal is set we dont add a default gateway
and make sure north-south communication cannot take place
(only east-west / container-container can)

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-03-04 13:24:10 -08:00
Arko Dasgupta
402a6b7dbc Merge pull request #2519 from wangyumu/fix-InhibitIPv4-nil-panic
Fix InhibitIPv4 nil panic
2020-03-04 13:20:06 -08:00
Trapier Marshall
c39ac9c05e Cleanup VFP during overlay network removal
Deleting a network sandbox on Linux implicitly clears OS (ipvs) load
balancer state.  Deleting an HNS network on Windows by contrast does not
inherently remove its corresponding VFP load balancers. The method to
remove load balancers belongs to the network and so must be called prior
to or while deleting a network. This commit reverts one line from
ea2fa20859, reintroducing a call to
explicitly remove backend load balancers during network removal.

Signed-off-by: Trapier Marshall <tmarshall@mirantis.com>
2020-03-03 19:48:38 -05:00
Wang Yumu
41a91e9a5d fix InhibitIPv4 nil panic
Signed-off-by: Wang Yumu <37442693@qq.com>
2020-03-02 23:38:13 +08:00
Akihiro Suda
7ca71f2c49 vendor godbus/dbus v5
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2020-02-28 18:47:00 +09:00
Sebastiaan van Stijn
add7253d68 Dockerfile: update to Go 1.13.8
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-27 11:46:18 +01:00
Sebastiaan van Stijn
129e28baa3 Fix gofmt in preparation of Go 1.13 update
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-27 11:46:16 +01:00
elangovan sivanandam
c3cf1da916 Merge pull request #2513 from thaJeztah/bump_golang_1.12.17
Dockerfile: update to Golang 1.12.17
2020-02-26 18:06:17 -05:00
elangovan sivanandam
ff09e6c12a Merge pull request #2512 from thaJeztah/debian_buster
Dockerfile: switch to debian "buster", and regenerate protobuf
2020-02-26 15:53:58 -05:00
Arko Dasgupta
68f261bfff Merge pull request #2407 from lemrouch/2406-fix
Macvlan internal network should not change default gateway
2020-02-26 09:28:30 -08:00
Sebastiaan van Stijn
847f469e76 regenerate protobufs with debian buster
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-26 16:03:42 +01:00
Sebastiaan van Stijn
6dc0d4ff60 Dockerfile: update to Golang 1.12.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-26 15:42:50 +01:00
Sebastiaan van Stijn
c18a3a075e Dockerfile: switch to debian "buster"
Debian Buster is now the current "stable", and will be the default
baseimage for Golang images going forward.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-26 15:40:11 +01:00
Sam Whited
2a480d515e Bump the DNS library and revendor
Signed-off-by: Sam Whited <sam@samwhited.com>
2020-02-25 15:37:30 -05:00
Sam Whited
9cd3fb6796 Re-vendor with newer vndr
Signed-off-by: Sam Whited <sam@samwhited.com>
2020-02-25 15:37:23 -05:00
Arko Dasgupta
812104ceae Merge pull request #2500 from tklauser/cli-pkg-migrate
Migrate from github.com/codegangsta/cli to github.com/urfave/cli
2020-02-21 15:22:28 -08:00
Pavel Matěja
c7f8bfa001 Fix internal ipvlan network to work in swarm
Using dummy interface allows communication beween containers only if
they are running on the same node in swarm.

Signed-off-by: Pavel Matěja <pavel@verotel.cz>
2020-02-20 19:33:48 +00:00
Pavel Matěja
b0bce9159e Fix internal macvlan network to work in swarm
Using dummy interface allows communication beween containers only if
they are running on the same node in swam.

Signed-off-by: Pavel Matěja <pavel@verotel.cz>
2020-02-20 19:33:48 +00:00
Pavel Matěja
76bccc5ad4 Ipvlan internal network should not change gw
Since docker container can be connected to combination of several
internal and external networks change of default gateway of the internal
ones breaks communication via the external ones.

This fixes only ipvlan network type

Signed-off-by: Pavel Matěja <pavel@verotel.cz>
2020-02-20 17:49:42 +00:00
Arko Dasgupta
14fbe41991 Update design.md
The roadmap document was outdated, and removed in #2316, so remove the link from here

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2020-02-18 17:21:39 -08:00
Arko Dasgupta
a6d52f1521 Merge pull request #2491 from ahjumma/master
Improving load balancer performance
2020-02-16 19:31:14 -08:00
Arko Dasgupta
7bb9876f89 Merge pull request #2498 from wawa0210/bump-hcsshim-v0.8.7
Update vendored dependency hcsshim to v0.8.7
2020-02-16 14:30:42 -08:00
Arko Dasgupta
b8b9d67d1a Merge pull request #2454 from arkodg/add-host-ip-snat
Support for com.docker.network.host_ipv4 driver label
2020-02-15 08:32:56 -08:00
Arko Dasgupta
b5b12d8bec Merge pull request #2317 from gsomlo/gls-bridge-inhibit-ipv4
Allow bridge net driver to skip IPv4 configuration of bridge interface
2020-02-14 14:47:24 -08:00
akim01
9ced389e6e Improving load balancer performance
Further improving load balancer performance by expiring
connections to servers with weights set to 0.

Signed-off-by: Andrew Kim <taeyeonkim90@gmail.com>
2020-02-14 13:24:06 -08:00
Madhu Venugopal
cb48244350 Merge pull request #2494 from thaJeztah/add_arko_to_maintainers
Add Arko to maintainers
2020-02-06 05:29:59 -08:00
Tobias Klauser
5cc6ffae0c Migrate from github.com/codegangsta/cli to github.com/urfave/cli
The library was moved quite a while ago, adjust the module path. No code
changes.

Signed-off-by: Tobias Klauser <tklauser@distanz.ch>
2020-01-24 11:12:07 +01:00
wawa0210
cc7847ddd2 Update vendored dependency hcsshim to v0.8.7
Signed-off-by: wawa0210 <xiaozhang0210@hotmail.com>
2020-01-11 08:08:53 +00:00
elangovan sivanandam
e2c0d868ab Merge pull request #2380 from liskin/bridge-atomic-hwaddr
bridge: Fix hwaddr set race between us and udev
2019-12-31 19:30:42 -05:00
Sebastiaan van Stijn
6492936018 Add Arko to maintainers
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-12-31 11:04:32 +01:00
Andrew Kim
8899d916c0 Improving load balancer performance
IPVS module used for swarm load balancer had a performance issue
under a high load situation. conn_reuse_mode=0 sysctl variable can
be set to handle the high load situation by reusing existing
connection entries in the IPVS table.

Under a high load, IPVS module was dropping tcp SYN packets whenever
a port reuse is detected with a connection in TIME_WAIT status forcing
clients to re-initiate tcp connections after request timeout events.
By setting conn_reuse_mode=0, IPVS module avoids special handling of
existing entries in the IPVS connection table.
Along with expire_nodest_conn=1, swarm load balancer can handle
a high load of requests and forward connections to newly joining
backend services.

Signed-off-by: Andrew Kim <taeyeonkim90@gmail.com>
2019-12-16 07:16:40 -08:00
Su Wang
c27bb1189d Added document describing libnetwork traffic flow.
Signed-off-by: Su Wang <su.wang@docker.com>
2019-12-05 16:20:57 +00:00
elangovan sivanandam
f55f6f82ed Merge pull request #2445 from kdomanski/ipv6-addr-in-hosts
etchosts: include the container's IPv6 address if available
2019-12-04 13:19:49 -05:00
elangovan sivanandam
9b62a8a675 Merge pull request #2462 from arkodg/fix-key-spi-panic
Fix panic in drivers/overlay/encryption.go
2019-11-15 17:40:09 -05:00
Su Wang
fcb70a0e86 DOCKER-USER chain not created when IPTableEnable=false.
This fix addresses https://docker.atlassian.net/browse/ENGCORE-1115
Expected behaviors upon docker engine restarts:
1. IPTableEnable=true, DOCKER-USER chain present
   -- no change to DOCKER-USER chain
2. IPTableEnable=true, DOCKER-USER chain not present
   -- DOCKER-USER chain created and inserted top of FORWARD
      chain.
3. IPTableEnable=false, DOCKER-USER chain present
   -- no change to DOCKER-USER chain
      the rational is that DOCKER-USER is populated
      and may be used by end-user for purpose other than
      filtering docker container traffic. Thus even if
      IPTableEnable=false, docker engine does not touch
      pre-existing DOCKER-USER chain.
4. IPTableEnable=false, DOCKER-USER chain not present
   -- DOCKER-USER chain is not created.

Signed-off-by: Su Wang <su.wang@docker.com>
2019-11-12 16:32:42 +00:00
Arko Dasgupta
cd864b50a2 Fix panic in drivers/overlay/encryption.go
Issue - "index out of range" panic in drivers/overlay/encryption.go:539
due to a mismatch in indices between curKeys and spis due to
case where updateKeys might bail out due to an error and
not update the spis

Fix - Reconfigure keys when there is a key update failure

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-10-31 12:59:41 -07:00
Sebastiaan van Stijn
fdf46323f4 Update Golang 1.12.12 (CVE-2019-17596)
Golang 1.12.12
-------------------------------

full diff: https://github.com/golang/go/compare/go1.12.11...go1.12.12

go1.12.12 (released 2019/10/17) includes fixes to the go command, runtime,
syscall and net packages. See the Go 1.12.12 milestone on our issue tracker for
details.

https://github.com/golang/go/issues?q=milestone%3AGo1.12.12

Golang 1.12.11 (CVE-2019-17596)
-------------------------------

full diff: https://github.com/golang/go/compare/go1.12.10...go1.12.11

go1.12.11 (released 2019/10/17) includes security fixes to the crypto/dsa
package. See the Go 1.12.11 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.12.11

    [security] Go 1.13.2 and Go 1.12.11 are released

    Hi gophers,

    We have just released Go 1.13.2 and Go 1.12.11 to address a recently reported
    security issue. We recommend that all affected users update to one of these
    releases (if you're not sure which, choose Go 1.13.2).

    Invalid DSA public keys can cause a panic in dsa.Verify. In particular, using
    crypto/x509.Verify on a crafted X.509 certificate chain can lead to a panic,
    even if the certificates don't chain to a trusted root. The chain can be
    delivered via a crypto/tls connection to a client, or to a server that accepts
    and verifies client certificates. net/http clients can be made to crash by an
    HTTPS server, while net/http servers that accept client certificates will
    recover the panic and are unaffected.

    Moreover, an application might crash invoking
    crypto/x509.(*CertificateRequest).CheckSignature on an X.509 certificate
    request, parsing a golang.org/x/crypto/openpgp Entity, or during a
    golang.org/x/crypto/otr conversation. Finally, a golang.org/x/crypto/ssh client
    can panic due to a malformed host key, while a server could panic if either
    PublicKeyCallback accepts a malformed public key, or if IsUserAuthority accepts
    a certificate with a malformed public key.

    The issue is CVE-2019-17596 and Go issue golang.org/issue/34960.

    Thanks to Daniel Mandragona for discovering and reporting this issue. We'd also
    like to thank regilero for a previous disclosure of CVE-2019-16276.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-10-20 23:38:03 +02:00
elangovan sivanandam
d6689e849b Merge pull request #2461 from suwang48404/master
Allowed libnetwork caller to set ephemeral port
2019-10-14 23:02:04 -04:00
Su Wang
2c4a868f64 Added API to set ephemeral port allocator range.
Also reduce the allowed port range as the total number of containers
per host is typically less than 1K.

This change helps in scenarios where there are other services on
the same host that uses ephemeral ports in iptables manipulation.

The workflow requires changes in docker engine (
https://github.com/moby/moby/pull/40055) and this change. It
works as follows:

1. user can now specified to docker engine an option
   --published-port-range="50000-60000" as cmdline argument or
   in daemon.json.
2. docker engine read and pass this info to libnetwork via
   config.go:OptionDynamicPortRange.
3. libnetwork uses this range to allocate dynamic port henceforth.
4. --published-port-range can be set either via SIGHUP or
   restart docker engine
5. if --published-port-range is not set by user, a OS specific
   default range is used for dynamic port allocation.
   Linux: 49153-60999, Windows: 60000-65000
6 if --published-port-range is invalid, that is, the range
  given is outside of allowed default range, no change takes place.
  libnetwork will continue to use old/existing port range for
  dynamic port allocation.

Signed-off-by: Su Wang <su.wang@docker.com>
2019-10-11 18:48:07 +00:00
Arko Dasgupta
f042605a42 Revert "Merge pull request #2339 from phyber/iptables-check"
This reverts commit 8d76333719, reversing
changes made to bdd0b7bb40.

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-10-11 09:38:19 -07:00
Euan Harris
0ca676db49 Revert "Always configure iptables forward policy"
Reverts 141b53c77a (PR #2450)

Fallout from changing the forwarding default policy to deny was greater than anticipated.

Signed-off-by: Euan Harris <euan.harris@docker.com>
2019-10-07 10:05:07 -07:00
elangovan sivanandam
01f67061b5 Merge pull request #2458 from arkodg/fix-flaky-tests
Fix flaky NetworkDB tests
2019-10-04 16:50:31 -04:00
Arko Dasgupta
34a636bf51 Fix flaky NetworkDB tests
Fixed these tests :

1.TestNetworkDBIslands
Addresses : https://github.com/docker/libnetwork/issues/2402

2.TestNetworkDBCRUDMediumCluster
Addresses : https://github.com/docker/libnetwork/issues/2401

By :

1. Importing gotest.tools/poll to use poll.WaitOn
Above function can be used to check a condition at regular intervals
until a timeout is reached

2. Replacing Sleep with poll.WaitOn

2. Adding closeNetworkDBInstances to close remaining DBs

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-10-04 10:17:19 -07:00
elangovan sivanandam
4540e13414 Merge pull request #2459 from arkodg/fix-error-check
Fix Error Check in NewNetwork
2019-10-03 18:34:21 -04:00
Arko Dasgupta
87b082f365 Fix Error Check in NewNetwork
Use types.MaskableError instead of doing a string comparison

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-10-03 00:54:45 -07:00
elangovan sivanandam
257e0d4d09 Merge pull request #2443 from Rid/shorten-setkey-id
Shorten controller ID in exec-root to not hit UNIX_PATH_MAX
2019-09-28 18:33:13 -04:00
elangovan sivanandam
f7748b11ca Merge pull request #2444 from kdomanski/verbose-ipv6-cannot-add
log the actual error when failing to add IPv6 route
2019-09-28 18:29:39 -04:00
elangovan sivanandam
e481dc9fad Merge pull request #2449 from espensuenson/bugfix_getnetworkfromstore
Fixed getNetworkFromStore, which returned an incorrect struct
2019-09-28 18:27:21 -04:00
elangovan sivanandam
33e9208a19 Merge pull request #2453 from jdrahos/ipvs_weighted_scheduling_constants-2452
weighted scheduling methods constants for ipvs
2019-09-28 18:19:02 -04:00
Su Wang
ff27bb0db4 Resolve "bridge fdb show" hang issue
The output of "bridge fdb show" command invoked under a network
namespace is unpredicable. Sometime it returns empty, and sometime
non-stop rolling output. This perhaps is a bug in kernel
and/or iproute2 implementation. To work around, display fdb  for
 each bridge.

Signed-off-by: Su Wang <su.wang@docker.com>
2019-09-26 21:29:22 +00:00
Jakub Drahos
edd44eede9 trigger new CI run
Signed-off-by: Jakub Drahos <jack.drahos@gmail.com>
2019-09-25 10:39:33 -04:00
Arko Dasgupta
8c8a25d524 Support for com.docker.network.host_ipv4 driver label
This commit allows a user to specify a Host IP via the
com.docker.network.host_ipv4 label which is used as the
Source IP during SNAT for bridge networks .

The use case is for hosts with multiple interfaces and
this label can dictate which IP will be used as Source IP
for North-South traffic

In the absence of this label, MASQUERADE is used which picks the Source IP
based on Next Hop from the Route Table

Addresses: https://github.com/moby/moby/issues/30053

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-09-24 22:15:43 -07:00
Jakub Drahos
0a99a5f152 adding the constants to the test file
Signed-off-by: Jakub Drahos <jack.drahos@gmail.com>
2019-09-24 21:42:32 -04:00
jdrahos
4d1db69bcc weighted scheduling methods constants for ipvs
Signed-off-by: Jakub Drahos <jack.drahos@gmail.com>
2019-09-24 21:01:05 -04:00
Espen Suenson
a1ab732711 return immediately on error
Signed-off-by: Espen Suenson <mail@espensuenson.dk>
2019-09-24 10:58:08 +02:00
Euan Harris
141b53c77a Merge pull request #2450 from TheNodi/iptables-policy
Always configure iptables forward policy
2019-09-17 16:38:09 +01:00
elangovan sivanandam
57fdd1f741 Merge pull request #2429 from pradipd/windows-nosubnet
Updating IPAM config with results from HNS create network call.
2019-09-13 11:19:09 -04:00
Leonardo Nodari
7e584c1d69 Configure iptables forward policy when ip forwarding is enabled
Signed-off-by: Leonardo Nodari <me@leonardonodari.it>
2019-09-12 15:47:27 +02:00
Espen Suenson
43a3151db2 Fixed getNetworkFromStore, which returned incorrect network information - notably, the 'resolver' field was empty.
This fixes https://github.com/moby/moby/issues/38901

Signed-off-by: Espen Suenson <mail@espensuenson.dk>
2019-09-11 21:09:23 +02:00
Kamil Domański
c4fcd7059c etchosts: additionally include the container's IPv6 address if available
Signed-off-by: Kamil Domański <kamil@domanski.co>
2019-09-04 01:58:00 +02:00
Kamil Domański
226fde5cdd etchosts: allow adding multiple container ips
Signed-off-by: Kamil Domański <kamil@domanski.co>
2019-08-30 23:49:33 +02:00
Kamil Domański
27fc6e9bae log the actual error when failing to add IPv6 route
Signed-off-by: Kamil Domański <kamil@domanski.co>
2019-08-30 00:41:24 +02:00
Grant Millar
049966bdc2 Shorten controller ID in exec-root to not hit UNIX_PATH_MAX
Signed-off-by: Grant Millar <rid@cylo.io>
2019-08-28 18:59:49 +01:00
Sebastiaan van Stijn
55adbae783 bump hashicorp/go-sockaddr v1.0.2
full diff: 6d291a969b...v1.0.2

Relevant changes:
  - hashicorp/go-sockaddr#25 Add android os
  - hashicorp/go-sockaddr#28 Add go.mod

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-26 11:59:41 +02:00
Sebastiaan van Stijn
5ee62a9c70 bump hashicorp/go-multierror v1.0.0, add errwrap v1.0.0
full diff: fcdddc395d...v1.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-26 11:52:15 +02:00
Pradip Dhara
f366d37c72 Updating IPAM config with results from HNS create network call.
In windows HNS manages IPAM.  If the user does not specify a subnet, HNS will choose one
for them.  However, in order for the IPAM to show up in the output of "docker inspect",
we need to update the network IPAMv4Config field.

Signed-off-by: Pradip Dhara <pradipd@microsoft.com>
2019-08-23 09:58:29 -07:00
Tom Zhao
7ab62b791f Fix parseIP error when parseIP before get AddressFamily
Signed-off-by: Tom Zhao <zlwangel@gmail.com>
2019-08-20 09:32:57 +08:00
Arko Dasgupta
4818e89751 Bump the GO Version to 1.12.8 which contains security fixes
https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/golang-announce/65QixT3tcmg/DrFiG6vvCwAJ

Also pin to -stretch

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-08-15 15:40:33 -07:00
Arko Dasgupta
f79533d720 Revert "Apply load balancer properly"
This reverts commit 9f58c475940fb0c0d4b69de0af7787b62a40481f.
This commit is causing TestCreateParallel to be flaky

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-08-15 15:39:46 -07:00
Arko Dasgupta
9a4025d79f Revert "Adjust LockOSThread"
This reverts commit 94af1e5af2.

The reason to revert this is, that TestCreateParallel is
continously failing and breaking the CI

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-08-15 14:44:03 -07:00
elangovan sivanandam
5c9a062864 Merge pull request #2243 from fcrisciani/syscalls
Syscalls reduction
2019-07-31 17:57:15 -04:00
elangovan sivanandam
a79bbdb5f9 Merge pull request #2415 from arkodg/ipvlan-docker-restart-issue
Support dockerd and system restarts for ipvlan and macvlan networks
2019-07-26 13:04:34 -04:00
elangovan sivanandam
0acdd724ae Merge pull request #2420 from thaJeztah/golang_arg
Dockerfile: use GO_VERSION build-arg for overriding Go version
2019-07-25 22:47:14 -04:00
Arko Dasgupta
402efabec5 Remove vlanLinkExists and dummyLinkExists since these are redundant
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-07-25 16:35:47 -07:00
elangovan sivanandam
50893512b5 Merge pull request #2351 from daym/fewer-modprobes
Use fewer modprobes
2019-07-25 16:18:00 -04:00
Arko Dasgupta
d22824dc11 Move dummyLinkExists into createDummyLink
Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-07-24 17:06:04 -07:00
Sebastiaan van Stijn
5fa03a4a8e Dockerfile: use GO_VERSION build-arg for overriding Go version
This allows overriding the version of Go without making modifications in the
source code, which can be useful to test against multiple versions.

For example:

    make GO_VERSION=1.13beta1 build

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-07-18 11:01:07 +02:00
Pavel Matěja
66a82d7268 Log when endpoint joins internal macvlan network
Signed-off-by: Pavel Matěja <pavel@verotel.cz>
2019-07-16 14:50:08 +02:00
Arko Dasgupta
ddd22a8198 Support dockerd and system restarts for ipvlan and macvlan networks
This commit carries forward the work done in
https://github.com/docker/libnetwork/pull/2295
and fixes two things
1. Allows macvlan and ipvlan to be restored properly
after dockerd or the system is restarted
2. Makes sure the refcount for the configOnly network
is not incremented for the above case so this network
can be deleted after all the associated ConfigFrom networks
are deleted

Addresses: https://github.com/docker/libnetwork/issues/1743

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-07-15 15:37:21 -07:00
elangovan sivanandam
d07c890b48 Merge pull request #2414 from lemrouch/2413-fix
Allow network with --config-from to be --internal
2019-07-15 16:24:19 -04:00
elangovan sivanandam
43b7bc99dc Merge pull request #2411 from lemrouch/2410-fix
Macvlan network handles netlabel.Internal wrong
2019-07-15 12:37:29 -04:00
Pavel Matěja
12a182e937 Ipvlan network handles netlabel.Internal wrong
check value of netlabel.Internal not just it's existence

Signed-off-by: Pavel Matěja <pavel@verotel.cz>
2019-07-15 10:15:53 +02:00
Tim Wagner
d132acb485 Fix hardcoded AF_INET for IPv6 address handling
Signed-off-by: Tim Wagner <tim.wagner@freenet.ag>
2019-07-11 08:22:10 +02:00
Pavel Matěja
afcb9fb299 Allow network with --config-from to be --internal
The --internal netlabel is discarded now.

Signed-off-by: Pavel Matěja <pavel@verotel.cz>
2019-07-08 15:34:08 +02:00
Tibor Vass
ecc7bd0540 vendor: update netns to 7109fa855
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-27 22:26:46 +00:00
Pavel Matěja
e5353649f6 Macvlan network handles netlabel.Internal wrong
check value of netlabel.Internal not just it's existence

Signed-off-by: Pavel Matěja <pavel@verotel.cz>
2019-06-27 15:53:57 +02:00
Euan Harris
46bed879da Merge pull request #2231 from msabansal/winnetwork
Add support for Internal and Private network types on windows
2019-06-27 11:23:32 +01:00
Euan Harris
587783fb82 Merge pull request #1656 from huikang/remove-unused-testcode
Removed unused integration test code
2019-06-27 09:57:12 +01:00
Euan Harris
3eeda3d91c Dockerfile: Update to Go 1.12.6, matching moby/moby
Signed-off-by: Euan Harris <euan.harris@docker.com>
2019-06-26 16:54:53 +01:00
Euan Harris
84414583f0 gofmt: Fix formatting warnings
Signed-off-by: Euan Harris <euan.harris@docker.com>
2019-06-26 16:54:31 +01:00
Euan Harris
746e680871 vet: Fix format string warnings
Signed-off-by: Euan Harris <euan.harris@docker.com>
2019-06-26 16:51:22 +01:00
Euan Harris
0275b007c6 vet: Fix composite literal uses unkeyed fields warnings
Signed-off-by: Euan Harris <euan.harris@docker.com>
2019-06-26 16:50:56 +01:00
Pavel Matěja
e2b2318e14 Macvlan internal network shouldln't change gw
Since docker container can be connected to combination of several
internal and external networks change of default gateway of the internal
ones breaks communication via the external ones.

This fixes only macvlan network type

Signed-off-by: Pavel Matěja <pavel@verotel.cz>
2019-06-26 14:23:51 +02:00
Tonis Tiigi
75bc82fe5c bridge: add riscv64 build tags
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2019-06-26 11:11:15 +08:00
Euan Harris
5c8048b363 Merge pull request #2404 from smarkm/master
Remove roadmap link from README.md
2019-06-25 15:24:14 +01:00
Euan Harris
1e476e7de6 Merge pull request #2365 from thaJeztah/fix_loadIPAMDriver_error_handling
controller.loadIPAMDriver: Unwrap error type returned by PluginGetter
2019-06-25 15:15:45 +01:00
Euan Harris
c3d402cef3 Merge pull request #2403 from thaJeztah/carry_2370_sctp_update
[carry 2370] Update sctp package
2019-06-25 15:11:50 +01:00
Euan Harris
d297a25283 Merge pull request #1723 from sebradloff/patch-1
Change wording for Endpoint description
2019-06-25 15:09:04 +01:00
Euan Harris
c0c586aacb Merge pull request #2279 from mrbeskin/clarify-design-doc
clarifications and typo fixes for the design documentation
2019-06-25 15:07:57 +01:00
Smark
1f1d9898c9 road map loss tracing, just keep use 'Design' describe the 'Future'
Signed-off-by: Smark <smark@freecoop.net>
2019-06-25 22:06:30 +08:00
Sascha Grunert
c5c8653912 Update sctp package
This commit updates the vendored ishidawataru/sctp and adapts its used
types.

Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-24 17:26:33 +02:00
Sebastiaan van Stijn
1425fd4fcc controller.loadIPAMDriver: Unwrap error type returned by PluginGetter
moby/moby commit b27f70d45 wraps the ErrNotFound error returned when
a plugin cannot be found, to include a backtrace.   This changes the
type of the error, so contoller.loadIPAMDriver no longer converts it
to a libnetwork plugin.NotFoundError.

This is a similar patch as was merged in 9b114971e5

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-24 16:40:51 +02:00
Euan Harris
d9ac962114 Merge pull request #2396 from mavenugo/um
Maintainers update
2019-06-24 15:25:30 +01:00
Sebastiaan van Stijn
e9e18d993c bump github.com/vishvananda/netns 13995c7128ccc8e51e9a6bd2b551020a27180abd
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-24 14:56:52 +02:00
Sebastiaan van Stijn
d152888722 Bump vishvananda/netlink to 1.0.0
Changes included:

- Allow index specification at link creation time
- replace syscall with golang.org/x/sys/unix
  - related: Use IFF_MULTI_QUEUE from x/sys/unix to define TUNTAP_MULTI_QUEUE
  - related: Use IFLA_* constants from x/sys/unix
- Fix index out of range when no metadata for gretap
- added encapsulation attributes for Iptun and Sittun to support SIT tunnels
- Expose xfrm state's statistics
- Support invert in ip rules
- Support LWTUNNEL_ENCAP_SEG6
- Support setting and retrieving route MTU/AdvMSS
- Fix CalcRtable array parameter bug
- added support for Foo-over-UDP netlink calls
- Support num{tx,rx}queues and udp6zerocsum{tx,rx}
- tuntap: Add multiqueue support
- Retrieve VLAN ID when listing neighbour
- Fix LinkAdd for sit tunnel on 3.10 kernel
- Add support for managing source MACVLANs
- Two functions: one for adding bond slave, one for getting veth peer index
- Eliminate cgo from netlink
- Don't overwrite the XDP file descriptor with flags
- Fix reference to BPF instructions (on Kernel 4.13)
- Add Matchall filter
- Send IFA_CACHEINFO when setting up addresses
- Support IPv6 GRE Tun and Tap
- Add List option to RouteSubscribeWithOptions, AddrSubscribeWithOptions, and LinkSubscribeWithOptions
- Add Fq and Fq_Codel Qdisc support

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-06-24 14:56:49 +02:00
Euan Harris
97d292bf26 Merge pull request #2361 from thaJeztah/bump_engine
Reformat vendor.conf, update docker/docker and dependencies
2019-06-24 08:47:51 +01:00
Madhu Venugopal
0fce3b3653 Maintainers update
- Add Euan and Elango
- Remove inactive maintainers

Signed-off-by: Madhu Venugopal <madhu@docker.com>
2019-06-21 14:27:00 -07:00
Euan Harris
8d76333719 Merge pull request #2339 from phyber/iptables-check
controller: Check if IPTables is enabled for arrangeUserFilterRule
2019-06-21 17:31:52 +01:00
Arko Dasgupta
313d2b8a74 Make DNS records and queries case-insensitive
RFC434 states that DNS Servers should be case insensitive
    This commit makes sure that all DNS queries will be translated
    to lower ASCII characters and all svcRecords will be saved in
    lower case to abide by the RFC

    Relates to https://github.com/moby/moby/issues/21169

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-06-19 11:23:31 -07:00
Euan Harris
a926e65406 Merge pull request #2364 from thaJeztah/fix_TestValidRemoteDriver
Fix TestValidRemoteDriver GetCapabilities errors
2019-06-14 17:13:23 +01:00
Madhu Venugopal
fc69adf859 Merge pull request #2372 from xinfengliu/improve-support.sh
add checks on node LB in support.sh
2019-06-04 09:22:10 -07:00
Tibor Vass
631ee43f1b Merge pull request #2373 from arkodg/ungraceful-exit-config-from-net-clean
Clean up --config-only networks after --config-from networks have ungracefully exited
2019-06-04 08:46:31 -07:00
Tibor Vass
599e037a24 remove gosimple - package is gone and it's not important
Also fixes issue reported by ineffassign

Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-04 04:50:37 +00:00
Tibor Vass
c575631528 resolvconf: use /run/systemd/resolve/resolv.conf if systemd-resolved manages DNS
Signed-off-by: Tibor Vass <tibor@docker.com>
2019-06-04 04:50:37 +00:00
Sebastiaan van Stijn
e9bd147bb7 Add Delete endpoint for plugin in test
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-23 14:55:29 +02:00
Sebastiaan van Stijn
cc38e3a35c Fix TestValidRemoteDriver GetCapabilities errors
This test was producing error messages due to missing endpoints
in the plugin API;

```
=== RUN   TestValidRemoteDriver
ERRO[0039] error getting capability for valid-network-driver due to NetworkDriver.GetCapabilities: 404 page not found
 ```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-05-23 14:55:20 +02:00
Tomas Janousek
6038ef390d bridge: Fix hwaddr set race between us and udev
systemd and udev in their default configuration attempt to set a
persistent MAC address for network interfaces that don't have one
already [systemd-def-link]. We set the address only after creating the
interface, so there is a race between us and udev. There are several
outcomes (that actually occur, this race is very much not a theoretical
one):

* We set the address before udev gets to the networking rules, so udev
  sees `/sys/devices/virtual/net/docker0/addr_assign_type = 3`
  (NET_ADDR_SET). This means there's no need to assign a different
  address and everything is fine.

* udev reads `/sys/devices/virtual/net/docker0/addr_assign_type` before
  we set the address, gets `1` (NET_ADDR_RANDOM), and proceeds to
  generate and set a persistent address.

  Old versions of udev (pre-v242, i.e. without [udev-patch]) would then
  fail to generate an address, spit out "Could not generate persistent
  MAC address for docker0: No such file or directory" (see [udev-issue],
  and everything would be probably fine as well.

  Current version of udev (with [udev-patch]) will generate an address
  just fine and then race us setting it. As udev does more work than we,
  the most probable outcome is that udev will overwrite the address we
  set and possibly cause some trouble later on.

On a clean Debian Buster (from Vagrant) VM with systemd/udev 242 from
Debian Experimental, `docker network create net1` up to `net7` resulted
in 3 bridges having a 02:42: address and 4 bridges having a seemingly
random (actually generated from interface name) address. With systemd
241, the result would be all bridges having a 02:42:, but some "Could
not generate persistent MAC address for" messages in the log.

The fix is to revert the MAC address setting fix from 6901ea51dc,
as it is no longer necessary with current netlink [netlink-addr-add],
and set the address atomically when creating the bridge interface, not
after that.

[systemd-def-link]: a166cd3aac/network/99-default.link
[udev-patch]: 6d36464065
[udev-issue]: https://github.com/systemd/systemd/issues/3374
[netlink-addr-add]: 7d9b424492

...

Do note that a similar race happens when creating veth devices as well.
I wasn't able to reproduce getting a wrong (non-02:42:) address,
possibly because the address is set by docker later, maybe only after
the interface is moved to another network namespace (but I'm just
guessing here). Still, different timings result in various error
messages being logged ("link_config: could not get ethtool features for
vethd9c938e" and the like) depending on when the interface disappears
from the primary network namespace. I'm not sure how to fix this and I
don't intend to dig deeper into this.

Signed-off-by: Tomas Janousek <tomi@nomi.cz>
2019-05-19 19:38:35 +02:00
Arko Dasgupta
ee574c1b7d Correctly clean up --config-only networks
The endpoint count for --config-only networks
was being incremented even when the respective --config-from
inherited network failed to create a network

This was due to a variable shadowing problem with err causing
the deferred function to not execute correctly.

Using the same err variable across the entire function fixes
the issue

Fixes: moby/moby#35101

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-05-14 10:20:14 -07:00
Pradip Dhara
a04a30717e Forcing a nil IP specified in PortBindings to IPv4zero (0.0.0.0).
Signed-off-by: Pradip Dhara <pradipd@microsoft.com>
2019-05-13 15:42:32 -07:00
Xinfeng Liu
74013a3899 add check on node LB in support.sh
Starting from 18.09, there's a per node LB for each overlay
network, this change adds the check to node LB.
This change should not break on older docker versions.

Signed-off-by: Xinfeng Liu <xinfeng.liu@gmail.com>
2019-05-07 16:45:01 +08:00
David O'Rourke
301a7724fb Move hasIPTablesEnabled check into firewall_linux.go
Signed-off-by: David O'Rourke <david@scalefactory.com>
2019-04-29 13:51:15 +01:00
David O'Rourke
b4d0319424 controller: Check if IPTables is enabled for arrangeUserFilterRule
This allows the `--iptables=false` argument to the `dockerd` to actually
work.

Signed-off-by: David O'Rourke <david@scalefactory.com>
2019-04-29 13:27:01 +01:00
Pradip Dhara
1909ecb27a Pick a random host port if the user does not specify a host port.
For overlay, l2bridge, and l2tunnel, if the user does not specify a host port, windows driver will select a random port for them.  This matches linux behavior.
For ics and nat networks the windows OS will choose the port.

Signed-off-by: Pradip Dhara <pradipd@microsoft.com>
2019-04-22 17:43:27 +00:00
Danny Milosavljevic
6b5dc55e93 Use fewer modprobes
Signed-off-by: Danny Milosavljevic <dannym@scratchpost.org>
2019-04-10 17:32:39 +02:00
Sebastiaan van Stijn
6ee0b5fcd8 update docker/docker and dependencies
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-08 12:09:49 +02:00
Sebastiaan van Stijn
0b40402976 Reformat vendor.conf for readability, pin to git-sha's
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-08 11:55:15 +02:00
Sebastiaan van Stijn
f126923aeb re-vendor with current version of LK4D4/vndr
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-08 11:50:39 +02:00
Madhu Venugopal
5c6423fb29 Adding error log if peerAdd fails
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2019-04-05 20:31:17 -07:00
Pradip Dhara
94ab7b2df6 Adding synchronization to endpoint addition/deletion to prevent network connectivity issue
Signed-off-by: Pradip Dhara <pradipd@microsoft.com>
2019-04-05 14:19:19 -07:00
Sebastiaan van Stijn
5ae7f4daf7 Use errors.Wrap to preserve original error
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-04-01 20:14:54 +02:00
Flavio Crisciani
2b1e45c682 Merge pull request #2238 from talex5/networkdb-docs
Add NetworkDB docs
2019-03-14 16:05:31 -07:00
Laurent Bernaille
efeae5e7ee [ipvs] Add SetConfig test with timeout=0
Signed-off-by: Laurent Bernaille <laurent.bernaille@datadoghq.com>
2019-03-13 21:43:01 +01:00
Laurent Bernaille
8e650a9685 [ipvs] Add tests for get/set timeout
Signed-off-by: Laurent Bernaille <laurent.bernaille@datadoghq.com>
2019-03-13 21:02:19 +01:00
Laurent Bernaille
7374284841 [ipvs] Add support for timeout configuration (Get/SetConfig)
Signed-off-by: Laurent Bernaille <laurent.bernaille@datadoghq.com>
2019-03-13 21:02:16 +01:00
Arko Dasgupta
068ca7d046 Revert "debian has iptables-legacy and iptables-nft now"
This reverts commit 7adcd856fe.

Libnetwork should only use the iptables binary. Iptables v1.8 and above
uses the nftables backend. The translations for all the rules used by
libnetwork is supported by the new iptables binary.

Signed-off-by: Arko Dasgupta <arko.dasgupta@docker.com>
2019-03-01 11:50:47 -08:00
Flavio Crisciani
8e7aead94f Merge pull request #2230 from mavenugo/exp-ipv
Moving IPVLAN driver out of experimental
2019-02-28 16:28:24 -08:00
Ryoga
e744b62f49 Update setup_ip_forwarding.go
Signed-off-by: Ryoga Saito <proelbtn@users.noreply.github.com>
2019-02-26 13:30:01 +09:00
Kyle Wuolle
7ca0bc66d3 Cleanup the cluster provider when the agent is closed
Signed-off-by: Kyle Wuolle <kyle.wuolle@gmail.com>
2019-01-25 08:36:28 -08:00
Sebastiaan van Stijn
916927d970 Bump Go to 1.10.7
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-23 21:12:12 +01:00
Flavio Crisciani
a52a6ba44e Merge pull request #2303 from thaJeztah/docs_and_error_touchup
Touch-up error-message and godoc for ConfigVXLANUDPPort
2019-01-23 09:42:36 -08:00
Sebastiaan van Stijn
d7f397c236 Touch-up error-message and godoc for ConfigVXLANUDPPort
Minor changes following review of the engine pull request
for this feature;

- Remove the name of the function from the error message
  as it's not a debug message.
- Add the valid range to the error message, so that a
  user has sufficient information to address the problem.
- Update GoDoc for the function to describe the default
  port, and valid port-ranges.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-01-23 10:56:40 +01:00
Flavio Crisciani
151f42aeaa Fix possible nil pointer exception
It is possible that the node is not yet present in
the node list map. In this case just print a warning
and return. The next iteration would be fine

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2019-01-22 17:07:15 -08:00
Flavio Crisciani
d418337502 Fix gosimple
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2019-01-03 15:25:37 -08:00
Flavio Crisciani
25949f1470 Merge pull request #2302 from thaJeztah/vxlan_locking
Use sync.RWMutex for VXLANUDPPort
2019-01-03 11:12:26 -08:00
Flavio Crisciani
9b597f983f Merge pull request #2306 from qudongfang/add_more_details_to_no-available-network_error
Improve error if auto-selecting IP-range failed
2019-01-03 11:11:05 -08:00
Gabriel L. Somlo
10027e8c01 Allow bridge net driver to skip IPv4 configuration of bridge interface
Introduce "com.docker.network.bridge.inhibit_ipv4" option to the bridge
network driver. If set, this option will prevent docker from setting or
modifying Layer-3 (IP) configuration on the bridge interface in any way.

This option should allow connecting containers to pre-existing network
segments (with e.g., pre-existing default gateways) while simultaneously
preserving our ability to communicate with the host and/or configure the
properties of the host-side container virtual network interface (e.g.,
delay/loss/jitter via netem), which can not be done using macvlan.

Signed-off-by: Gabriel Somlo <gsomlo@gmail.com>
2018-12-23 19:05:20 -05:00
Olli Janatuinen
ef2741452c Removed roadmap
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2018-12-19 12:52:52 +02:00
Flavio Crisciani
33d47f95e8 Build with latest docker stable engine
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-12-10 08:36:52 -08:00
Andrew Hsu
5338928eb8 account for removal of configs.HookState
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2018-12-07 01:47:05 +00:00
Andrew Hsu
bb3ae82008 vndr runc 96ec217
Signed-off-by: Andrew Hsu <andrewhsu@docker.com>
2018-12-07 01:46:14 +00:00
Flavio Crisciani
2dc04ddf4f Merge pull request #2299 from olljanat/ip-to-hostname-fix
Do not add IP to name records for aliases
2018-11-29 08:16:29 -08:00
qudongfang
03b2393a80 Improve error if auto-selecting IP-range failed.
Signed-off-by: qudongfang <qudongfang@gmail.com>
2018-11-28 15:57:45 +08:00
Sebastiaan van Stijn
38c8a3f84d Use sync.RWMutex for VXLANUDPPort
Looks like concurrent reads should be possible, so use
a RWMutex instead of Mutex.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-11-22 21:29:20 +01:00
Flavio Crisciani
7667c0a4b2 Merge pull request #2300 from selansen/master
VXLAN port configuration - late review comments update
2018-11-15 08:25:45 -07:00
selansen
56ca280b27 VXLAN port configuration - late review comments update
Some review comments came in very late after merging
	#2282. This PR addresses those review comments.

Signed-off-by: selansen <elango.siva@docker.com>
2018-11-14 13:26:56 -05:00
Olli Janatuinen
d635844ed7 Do not add IP to Name records for aliases
Signed-off-by: Olli Janatuinen <olli.janatuinen@gmail.com>
2018-11-10 20:53:06 +02:00
Flavio Crisciani
ffa1330066 Merge pull request #2282 from selansen/master
VXLAN UDP Port configuration support
2018-11-05 07:41:26 -08:00
selansen
077ccabc45 VXLAN UDP Port configuration support
This PR chnages allow user to configure VxLAN UDP
port number. By default we use 4789 port number. But this commit
will allow user to configure port number during swarm init.
VxLAN port can't be modified after swarm init.

Signed-off-by: selansen <elango.siva@docker.com>
2018-11-01 15:20:30 -04:00
Flavio Crisciani
2d1311fc1b Merge pull request #2290 from fcrisciani/golint
Update golint tool
2018-11-01 10:21:34 -07:00
Flavio Crisciani
387aed91bf Update golint tool
Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-11-01 10:09:47 -07:00
Nathan Herald
7adcd856fe debian has iptables-legacy and iptables-nft now
Signed-off-by: Nathan Herald <me@nathanherald.com>
2018-10-31 09:14:35 -07:00
mrbeskin
1caeac8b64 clarifications and typo fixes for the design documentation
Signed-off-by: mrbeskin <mrbeskin@gmail.com>
2018-10-21 15:43:25 -05:00
Flavio Crisciani
cbf4d5ce89 Merge pull request #2171 from thaJeztah/dont_give_up
Handle NXDOMAIN, REFUSED and log errors
2018-10-12 08:38:25 -07:00
Flavio Crisciani
e0d1cdd3d4 Merge pull request #2270 from ctelfer/lbdsr
Use direct server return in east-west overlay load balancing
2018-10-11 17:44:44 -07:00
Sebastiaan van Stijn
6dd3f45248 Handle NXDOMAIN, REFUSED and log errors
- NXDOMAIN is an authoritive answer, so when receiving an NXDOMAIN, we're done.
  From RFC 1035: Name Error - Meaningful only for responses from an authoritative
  name server, this code signifies that the domain name referenced in the query
  does not exist.
  FROM RFC 8020: When an iterative caching DNS resolver receives an NXDOMAIN
  response, it SHOULD store it in its cache and then all names and resource
  record sets (RRsets) at or below that node SHOULD be considered unreachable.
  Subsequent queries for such names SHOULD elicit an NXDOMAIN response.
- REFUSED can be a transitional status: (https://www.ietf.org/rfc/rfc1035.txt)
  The name server refuses to perform the specified operation for
  policy reasons.  For example, a name server may not wish to provide the
  information to the particular requester, or a name server may not wish to
  perform a particular operation (e.g., zone)

Other errors are now logged as debug-message, which can be useful for
troubleshooting.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-12 00:55:36 +02:00
Sebastiaan van Stijn
a72bff0da3 Remove if/else and redundant brackets in resolver
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-10-12 00:55:28 +02:00
Christopher Adam Telfer
466225b792 Merge pull request #2276 from amoghe/amoghe_dest_stats
Add destination stats extraction to IPVS library
2018-10-11 18:01:20 -04:00
Chris Telfer
013ca3bdf8 Make DSR an overlay-specific driver "option"
Allow DSR to be a configurable option through a generic option to the
overlay driver.  On the one hand this approach makes sense insofar as
only overlay networks can currently perform load balancing.  On the
other hand, this approach has several issues.  First, should we create
another type of swarm scope network, this will prevent it working.
Second, the service core code is separate from the driver code and the
driver code can't influence the core data structures.  So the driver
code can't set this option itself.  Therefore, implementing in this way
requires some hack code to test for this option in
controller.NewNetwork.

A more correct approach would be to make this a generic option for any
network.  Then the driver could ignore, reject or be unaware of the option
depending on the chosen model.  This would require changes to:
  * libnetwork - naturally
  * the docker API - to carry the option
  * swarmkit - to propagate the option
  * the docker CLI - to support the option
  * moby - to translate the API option into a libnetwork option
Given the urgency of requests to address this issue, this approach will
be saved for a future iteration.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-10-11 14:13:19 -04:00
Chris Telfer
9a2464f436 Set east-west load balancing to use direct routing
Modify the loadbalancing for east-west traffic to use direct routing
rather than NAT and update tasks to use direct service return under
linux.  This avoids hiding the source address of the sender and improves
the performance in single-client/single-server tests.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-10-11 14:13:19 -04:00
Akshay
0922883b7e Add a DstStats type to track Destination (real server) stats
Since SvcStats represents the stats for a `Service`, we don't want
to reuse that struct in the `Destination` (for no other reason than
incompatible nomenclature). So this patch adds a `DstStats` struct
to hold the Destination stats.
2018-10-10 13:42:59 -07:00
Akshay
e427c4ee2e Add Stats to the Destination
This patch modifies the `Destination` struct so that the stats for
that destination are also reported.

Signed-off-by: Akshay <akshay.moghe@gmail.com>
2018-10-09 11:53:12 -07:00
Flavio Crisciani
7c3d556f8b Merge pull request #2262 from trilogy-group/increase-dns-max-concurrent
Increase max concurrent requests for DNS from 100 to 1024
2018-10-09 08:02:08 -07:00
John Stephens
4fd54c2aea Do not fail with an empty transparent IPv4 address
When dockerd.exe is not stopped cleanly (such as when Windows is
restarted), the endpoints are not cleaned up. When using a transparent
network, the endpoint IPv4 address is blank. When dockerd.exe starts up
again, libnetwork restores the endpoint, which would not have been
stored on a clean shutdown of dockerd.exe. That fails because the IPv4
address is blank. This change warns instead of failing.

Signed-off-by: John Stephens <johnstep@docker.com>
2018-09-27 12:30:37 -07:00
Yongxin Li
619e3d6706 typo fix about mismatch
Signed-off-by: Yongxin Li <yxli@alauda.io>
2018-09-27 20:43:13 +08:00
Madhu Venugopal
08796e04ea Removing experimental driver interface
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2018-09-25 10:30:56 -07:00
Madhu Venugopal
f7b7e74624 Moving IPVLAN driver out of experimental
IPVLAN driver had been retained in experimental for multiple releases
with the requirement to have a proper L3 control-plane (such as BGP) to
go along with it which will make this driver much more useful. But
based on the community feedback,
https://github.com/moby/moby/issues/21735, am proposing to move this
driver out of experimental.

Signed-off-by: Madhu Venugopal <madhu@docker.com>
2018-09-25 10:30:56 -07:00
Akihiro Suda
ce5bc0079b allow propagating custom exec-root (e.g. "/run/docker") to libnetwork-setkey
The docker daemon needs to be modified as follows:

    diff --git a/daemon/oci_linux.go b/daemon/oci_linux.go
    index 00ace320df..ea7daa72df 100644
    --- a/daemon/oci_linux.go
    +++ b/daemon/oci_linux.go
    @@ -809,7 +809,7 @@ func (daemon *Daemon) createSpec(c *container.Container) (retSpec *specs.Spec, e
                        s.Hooks = &specs.Hooks{
                                Prestart: []specs.Hook{{
                                        Path: target,
    -                                   Args: []string{"libnetwork-setkey", c.ID, daemon.netController.ID()},
    +                                   Args: []string{"libnetwork-setkey", c.ID, daemon.netController.ID(), "-exec-root="+daemon.configStore.GetExecRoot()},
                                }},
                        }
                }

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-09-14 14:09:09 +09:00
Christopher Adam Telfer
e1b464e78f Merge pull request #2268 from jhowardmsft/boltdb
boltdb/bolt==>bbolt, revendor Microsoft/*, Windows compilation.
2018-09-13 16:00:09 -04:00
John Howard
55b80035ed Don't build portallocator on Windows
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-13 09:20:03 -07:00
John Howard
40b6ebfe75 Add init_windows.go for compilation
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-13 09:20:03 -07:00
John Howard
44094fae9f Remove unused syndtr/gocapability from vendor.conf
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-13 09:20:03 -07:00
John Howard
9ca55c7a60 Bump libkv to 458977154600b9f23984d9f4b82e79570b5ae12b
Signed-off-by: John Howard <jhoward@microsoft.com>

As well as bumping, libkv now requires go.etcd.io/bolt rather
than boltdb/bolt. Hence removed bolt from vendor.conf,
vendored go.etcd.io/bbot @ v1.3.1-etcd.8 and rerun vndr.
2018-09-13 09:20:03 -07:00
Rui Cao
65860893bf Fix typo: assigment -> assignment
Signed-off-by: Rui Cao <ruicao@alauda.io>
2018-09-13 09:59:39 +08:00
John Howard
a906968a3f Bump Microsoft/go-winio to v0.4.11
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-12 14:05:23 -07:00
John Howard
c15a478047 Bump Microsoft/hcsshim to v0.7.3
Signed-off-by: John Howard <jhoward@microsoft.com>
2018-09-12 14:05:04 -07:00
Christopher Adam Telfer
33eca9a291 Merge pull request #2069 from fanjiyun/1-Rolling-back-the-port-configs
Roll back the port configurations upon failure within programIngress()
2018-09-11 14:02:59 -04:00
Thiago Alves Silva
d642cfdeb6 Increase max concurrent requests for DNS from 100 to 1000
This addresses/alleviates https://github.com/docker/libnetwork/issues/2214

The new proposed limit should remediate the issue for most users.

Signed-off-by: Thiago Alves Silva <thiago.alves@aurea.com>
2018-09-11 09:08:58 -03:00
fanjiyun
03ba96c5cf Rolling back the port configs if failed to programIngress()
Signed-off-by: fanjiyun <fan.jiyun@zte.com.cn>
2018-09-11 19:10:59 +08:00
Flavio Crisciani
79074c268f Merge pull request #2266 from riverzhang/typo
Fix some typos
2018-09-08 11:16:35 -07:00
Lei Gong
1adcfa9aa1 fix error when make lint
```
make lint
networkdb/networkdb_test.go:88:2: should replace t.Error(fmt.Sprintf(...)) with t.Errorf(...)
networkdb/networkdb_test.go:136:2: should replace t.Error(fmt.Sprintf(...)) with t.Errorf(...)
make: *** [lint] Error 1
```

Signed-off-by: Lei Gong <lgong@alauda.io>
2018-09-08 21:06:07 +08:00
rongzhang
36375881f3 Fix some typos
Signed-off-by: rongzhang <rongzhang@alauda.io>
2018-09-08 09:33:24 +08:00
Yan Zhu
ce46100a27 doc: fix typo
Signed-off-by: Yan Zhu <yanzhu@alauda.io>
2018-09-07 11:48:15 +08:00
Christopher Adam Telfer
e29452841e Merge pull request #2259 from resin-os/handle-invalid-default-gateways
bridge: fix error handling for stale default gateways
2018-08-30 11:14:22 -04:00
Christopher Adam Telfer
10008597e0 Merge pull request #2077 from quadespresso/master
Added support for Swarm Service Driller (ssd)
2018-08-28 13:35:24 -04:00
Petros Angelatos
72eed906b8 bridge: fix error handling for stale default gateways
Signed-off-by: Petros Angelatos <petrosagg@gmail.com>
2018-08-24 17:43:12 -07:00
liangwei
99c4c6d5fc ipvs support rs connection information
Signed-off-by: liangwei <liangwei14@huawei.com>
2018-08-22 21:14:26 +08:00
Chris Telfer
afcf6c526d Merge pull request #2253 from selansen/global_add_pool
Add getter functions for Default Address Pools
2018-08-16 16:31:12 -04:00
selansen
814f6c1f4b Add getter function for Default Address Pools
ipamutils has two default address pool. Instead of allowing them to
be accessed directly, adding get functions so that other packages
can use get APIs.

Signed-off-by: selansen <elango.siva@docker.com>
2018-08-16 15:48:42 -04:00
Jim Carroll
a478edc819 Added support for Swarm Service Driller (ssd)
Signed-off-by: Jim Carroll <jim.carroll@docker.com>
2018-08-16 14:30:23 -05:00
Madhu Venugopal
bc2180c4b1 Merge pull request #2251 from dani-docker/ssd_image_rename
Move SSD image to docker
2018-08-16 11:20:51 -07:00
selansen
52e85b4b9a Global Default Address Pool support
This change brings global default address pool feature into
libnetwork. Idea is to reuse same code flow and functions that were
implemented for local scope default address pool.
Function InitNetworks carries most of the changes. local scope default
address pool init should always happen only once. But Global scope
default address pool can be initialized multiple times.

Signed-off-by: selansen <elango.siva@docker.com>
2018-08-16 11:28:24 -04:00
Dani Louca
17966c940a Move SSD image to docker
Signed-off-by: Dani Louca <dani.louca@docker.com>
2018-08-14 14:41:33 -04:00
Thomas Leonard
05c05ea5e9 Add NetworkDB docs
This is based on reading the code in the `networkdb` directory.

Signed-off-by: Thomas Leonard <thomas.leonard@docker.com>
2018-08-08 13:35:11 +01:00
Flavio Crisciani
3321709a62 Merge pull request #2242 from euanh/pin-gogoprotobuf
Dockerfile: Install a fixed version of gogoprotobuf
2018-07-26 10:51:42 -07:00
Flavio Crisciani
f9442ee314 Apply load balancer properly
The load balancer options have to be applied
inside the load balancer namespace

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-07-25 13:51:33 -07:00
Flavio Crisciani
94af1e5af2 Adjust LockOSThread
Go 1.10 fixed the problem related to thread and namespaces.
Details:
2595fe7fb6
In few words there is no more the possibility to have a go routine
running on a thread that is another namespace.
In this commit some cleanup is done and the method SetNamespace is
being removed. This will save tons of setns syscall, that were happening
way too frequently possibily to make sure that each operation was being
done in the host namespace.
I suspect that also all the drivers not running in a different
namespace would be able to drop also the lock of the OS Thread but
will address it in a different commit

Removed useless LockOSThreads around

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-07-25 13:51:15 -07:00
Flavio Crisciani
92dd7fda05 Merge pull request #2240 from ctelfer/nice-lb-names
Give LB sandboxes predictable names
2018-07-25 10:14:10 -07:00
Chris Telfer
6d27900ffe Merge pull request #2232 from fcrisciani/ubuntu-dns
Proper handling sandbox options
2018-07-25 13:13:05 -04:00
Euan Harris
72905ebb13 Dockerfile: Install a fixed version of gogoprotobuf
This avoids build failures when gogoprotobuf is changed upstream.

Signed-off-by: Euan Harris <euan.harris@docker.com>
2018-07-25 15:42:25 +01:00
Chris Telfer
5041b74451 Give LB sandboxes predictable names
Change the sandbox IDs for the sandboxes of load-balancing endpoints to
be "lb_XXXXXXXXX" where XXXXXXXXX is the network ID that this sandbox
load balances for.  This makes it easier to find these sandboxes in
/var/run/docker/netns and thus makes debugging easier.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-07-24 17:10:41 -04:00
Flavio Crisciani
55ad3ef1a4 Fix handling of the resolv.conf
Leverage what is it passed from the daemon
Fix check about the host networking

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-07-24 10:18:10 -07:00
Flavio Crisciani
204ce3e31d Create internal directory
Internal directory is designed to contain libraries
that are exclusively used by this project

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-07-16 17:34:20 -07:00
Thomas Leonard
e88eed1d65 Add required imports to example code in README.md
Signed-off-by: Thomas Leonard <thomas.leonard@docker.com>
2018-07-16 14:50:53 +01:00
Sandeep Bansal
012fcea177 Add support for Internal and Private network types on windows
Signed-off-by: Sandeep Bansal <sabansal@microsoft.com>
2018-07-12 13:58:10 -07:00
Flavio Crisciani
8698ad1af1 Merge pull request #2229 from fcrisciani/spelling
Spelling
2018-07-12 13:11:28 -07:00
Flavio Crisciani
6defa7c807 Make protobuf check silent
Avoid printing the if condition

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-07-12 12:54:59 -07:00
Josh Soref
a06f1b2c4e Spelling fixes
* addresses
* assigned
* at least
* attachments
* auxiliary
* available
* cleanup
* communicate
* communications
* configuration
* connection
* connectivity
* destination
* encountered
* endpoint
* example
* existing
* expansion
* expected
* external
* forwarded
* gateway
* implementations
* implemented
* initialize
* internally
* loses
* message
* network
* occurred
* operational
* origin
* overlapping
* reaper
* redirector
* release
* representation
* resolver
* retrieve
* returns
* sanbdox
* sequence
* succesful
* synchronizing
* update
* validates

Signed-off-by: Josh Soref <jsoref@gmail.com>
2018-07-12 12:54:44 -07:00
Trapier Marshall
b9223019c2 support.sh: also inspect stopped containers
Signed-off-by: Trapier Marshall <trapier.marshall@docker.com>
2018-07-12 15:22:42 -04:00
Flavio Crisciani
9db90d27ed Merge pull request #2217 from euanh/protobuf-check
Check that generated protocol buffer code is up to date
2018-07-11 15:12:54 -07:00
Flavio Crisciani
0f593ae92b Merge pull request #2146 from ctelfer/fix-overlay-vxlan-races
Fix overlay vxlan races
2018-07-11 10:41:46 -07:00
Chris Telfer
4e6580c4c1 Refactor locking for join/leave to avoid race
Instead of using "sync.Once" to determine whether to initialize a
network sandbox or subnet sandbox, we use a traditional mutex +
initialization boolean.  This is because the initialization state isn't
truly a once-and-done condition.  Rather, libnetwork destroys network
and subnet sandboxes when the last endpoint leaves them.  The use of
sync.Once in this kind of scenario requires, therefore, re-initializing
the Once which is impoissible.  So the approach that libnetwork
currently takes is to use a pointer to a Once and redirect that pointer
to a new Once on reset.  This leads to nasty race conditions.

In addition to refactoring the locking, this patch merges the functions
joinSandbox(), and joinSubnetSandbox(). This makes the code both cleaner
and it also holds the network and subnet locks through the series of
read-modify-writes avoiding further potential races.  This does reduce
the potential parallelism which could be applied should there be many
joins coming in on many different subnets in the same overlay network.
However, this should be an extremely minor performance hit for a very
obscure case.

One important pattern in this commit is that it is crucial to avoid
sending peerDB messages while holding a driver or network lock.  The
changes herein defer such (asynchronous) notifications until after
release of such locks.  This prevents deadlocks where the peerDB
blocks acquiring said locks while the network method blocks trying
to send to the peerDB's channel.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-07-10 12:13:39 -04:00
Euan Harris
56c4a6dd3f Makefile: Add check for out of date protocol buffer code
'make check' will now fail if the files produced by re-running protoc
differ from those which are checked into the repository.

Signed-off-by: Euan Harris <euan.harris@docker.com>
2018-07-10 17:01:48 +01:00
Euan Harris
2b602bf3d0 Makefile: Add protobuf-local target, runnable within build container
Outside the build container, run: make protobuf
Inside the build container, run: make protobuf-local

Signed-off-by: Euan Harris <euan.harris@docker.com>
2018-07-10 17:01:48 +01:00
Euan Harris
3bebfbc34e Makefile: Document and organize into sections
Add documentation and move protobuf target into Build section

Signed-off-by: Euan Harris <euan.harris@docker.com>
2018-07-10 17:01:48 +01:00
Euan Harris
348ed0a1a8 circleci: Rename 'lint' to 'check' to match build target
Signed-off-by: Euan Harris <euan.harris@docker.com>
2018-07-10 17:01:48 +01:00
Flavio Crisciani
e353363bab Merge pull request #2228 from euanh/use-init
Makefile: Run containers with --init for proper signal handling
2018-07-10 08:09:09 -07:00
Santhosh Manohar
5fdfa8c52c Cleanup interfaces properly when vxlan plumbling fails
Signed-off-by: Santhosh Manohar <santhosh@docker.com>
Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-07-10 10:33:46 -04:00
Euan Harris
c699a1215e Makefile: Run containers with --init for proper signal handling
This makes it possible to Ctrl-C tests and builds again.   Zombie
processes will also be reaped correctly.

Signed-off-by: Euan Harris <euan.harris@docker.com>
2018-07-10 10:20:44 +01:00
Flavio Crisciani
38ec34e0e7 Merge pull request #2122 from quadespresso/esc-509
Changed loglevel from error to warning
2018-07-06 16:28:11 -07:00
Flavio Crisciani
968b269ec8 Merge pull request #2220 from vdemeester/to-gotest.tools
Migrate to gotest.tools :)
2018-07-06 16:15:13 -07:00
Jim Carroll
9da8e04c70 Updated handling of disable_ipv6 file
Signed-off-by: Jim Carroll <jim.carroll@docker.com>
2018-07-06 16:42:29 -05:00
Flavio Crisciani
e0c5c1b5d3 Merge pull request #2222 from fcrisciani/e2e-test
Create tag for master code
2018-07-06 13:36:29 -07:00
Vincent Demeester
06d471d186 Migrate to gotest.tools :)
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-07-06 11:01:37 -07:00
Vincent Demeester
6c0f597a82 Remove unused deps from vendor.conf
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-07-06 10:40:22 -07:00
Flavio Crisciani
ffe33f932e Create tag for master code
In order to support different branches
create master tag

Signed-off-by: Flavio Crisciani <flavio.crisciani@docker.com>
2018-07-06 10:15:00 -07:00
Flavio Crisciani
b0a0059237 Merge pull request #2216 from fcrisciani/netdb-qlen-issue
NetworkDB qlen optimization
2018-07-05 15:02:58 -07:00
Flavio Crisciani
85862008e9 Merge pull request #2221 from ctelfer/use-fmt-string-precision
Use fmt precision to limit string length
2018-07-05 14:52:49 -07:00
Chris Telfer
06922d2d81 Use fmt precision to limit string length
The previous code used string slices to limit the length of certain
fields like endpoint or sandbox IDs.  This assumes that these strings
are at least as long as the slice length.  Unfortunately, some sandbox
IDs can be smaller than 7 characters.   This fix addresses this issue
by systematically converting format string calls that were taking
fixed-slice arguments to use a precision specifier in the string format
itself.  From the golang fmt package documentation:

    For strings, byte slices and byte arrays, however, precision limits
    the length of the input to be formatted (not the size of the output),
    truncating if necessary. Normally it is measured in runes, but for
    these types when formatted with the %x or %X format it is measured
    in bytes.

This nicely fits the desired behavior: it will limit the number of
runes considered for string interpolation to the precision value.

Signed-off-by: Chris Telfer <ctelfer@docker.com>
2018-07-05 17:44:04 -04:00
Flavio Crisciani
00611a5255 Merge pull request #2215 from hzue/master
Fix error handling about bridgeSetup
2018-07-05 14:34:41 -07:00
hzue
5ef4c59574 Fix error handling about bridgeSetup
Fix the error from bridgeSetup doesn't handle by the defer function in the createNetwork function.

Signed-off-by: Terry Chu <jubosh.tw@gmail.com>
2018-07-05 16:46:56 +08:00