Commit graph

222 commits

Author SHA1 Message Date
Jiri Appl
33a779e308 Properly type handles to prevent overflows
This change is needed for 32 bit targets as the default type is int32
and the handle constants do not fit into it.

Signed-off-by: Jiri Appl <jiria@microsoft.com>
2018-10-10 16:46:37 -07:00
Tibor Vass
34eede0296 Remove 'docker-' prefix for containerd and runc binaries
This allows to run the daemon in environments that have upstream containerd installed.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-09-24 21:49:03 +00:00
Tibor Vass
4a776d0ca7 builder: use buildkit's GC for build cache
This allows users to configure the buildkit GC.

The following enables the default GC:
```
{
  "builder": {
    "gc": {
      "enabled": true
    }
  }
}
```

The default GC policy has a simple config:
```
{
  "builder": {
    "gc": {
      "enabled": true,
      "defaultKeepStorage": "30GB"
    }
  }
}
```

A custom GC policy can be used instead by specifying a list of cache prune rules:
```
{
  "builder": {
    "gc": {
      "enabled": true,
      "policy": [
        {"keepStorage": "512MB", "filter": ["unused-for=1400h"]]},
        {"keepStorage": "30GB", "all": true}
      ]
    }
  }
}
```

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-09-21 22:06:00 +00:00
Anda Xu
171d51c861 add support of registry-mirrors and insecure-registries to buildkit
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-09-20 11:53:02 -07:00
Anda Xu
66ac92cdc6 create newBuildKit function separately in daemon_unix.go and daemon_windows.go for cross platform build
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-09-11 11:22:48 -07:00
Anda Xu
54b3af4c7d update vendor
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-09-07 17:48:41 -07:00
Anda Xu
d52485c2f9 propagate the dockerd cgroup-parent config to buildkitd
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-09-07 17:48:41 -07:00
Xiaoxi He
5c0d2a0932 Fix some typos
Signed-off-by: Xiaoxi He <xxhe@alauda.io>
2018-09-07 13:13:47 +08:00
Tõnis Tiigi
4842f7a867
Merge pull request #37738 from tiborvass/remove-unused-field-in-builder
builder: remove unused netnsRoot field in builder-next
2018-09-06 13:33:35 -07:00
Anda Xu
58a75cebdd allow features option live reloadable
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-08-31 12:43:04 -07:00
Tibor Vass
8ab9e78ee4 builder: remove unused netnsRoot field in builder-next
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-08-31 19:09:52 +00:00
Sebastiaan van Stijn
bf95fbc54f
Use errors.Wrap() in daemon errors, and cleanup messages
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-08-23 16:12:44 +02:00
Tibor Vass
dc7e472db9 builder: fix bridge networking when using buildkit
Signed-off-by: Tibor Vass <tibor@docker.com>
2018-08-23 05:20:01 +00:00
Tonis Tiigi
d6424a088d builder: setup code for a bridge networking
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-08-20 18:55:01 +00:00
Sebastiaan van Stijn
9916827496
Merge pull request #37593 from AntaresS/add-enable-buildkit
[enhancement] add optional fields in daemon.json to enable buildkit
2018-08-20 19:41:56 +01:00
Anda Xu
2be17666b4 add optional fields in daemon.json to enable buildkit
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-08-19 14:58:23 -07:00
Sebastiaan van Stijn
7d4fa69e33
Merge pull request #37664 from dmcgowan/vendor-containerd-1.2
Update containerd vendor to 1.2 beta
2018-08-18 22:22:41 +01:00
Derek McGowan
98e23f2a8e
Update containerd vendor to 1.2 beta
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-17 13:08:22 -07:00
John Stephens
b3e9f7b13b
Merge pull request #35521 from salah-khan/35507
Add --chown flag support for ADD/COPY commands for Windows
2018-08-17 11:31:16 -07:00
Salahuddin Khan
763d839261 Add ADD/COPY --chown flag support to Windows
This implements chown support on Windows. Built-in accounts as well
as accounts included in the SAM database of the container are supported.

NOTE: IDPair is now named Identity and IDMappings is now named
IdentityMapping.

The following are valid examples:
ADD --chown=Guest . <some directory>
COPY --chown=Administrator . <some directory>
COPY --chown=Guests . <some directory>
COPY --chown=ContainerUser . <some directory>

On Windows an owner is only granted the permission to read the security
descriptor and read/write the discretionary access control list. This
fix also grants read/write and execute permissions to the owner.

Signed-off-by: Salahuddin Khan <salah@docker.com>
2018-08-13 21:59:11 -07:00
Derek McGowan
dd2e19ebd5
libcontainerd: split client and supervisor
Adds a supervisor package for starting and monitoring containerd.
Separates grpc connection allowing access from daemon.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-08-06 10:23:04 -07:00
Derek McGowan
8fb5f4d5c9
Add configuration option for containerd cri
Disable cri plugin by default in containerd and
allows an option to enable the plugin. This only
has an effect on containerd when supervised by
dockerd. When containerd is managed outside of
dockerd, the configuration is not effected.

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2018-07-24 11:34:47 -07:00
Tibor Vass
195919d9d6 builder: set buildkit's exported product variable via PRODUCT
This introduces a PRODUCT environment variable that is used to set a constant
at dockerversion.ProductName.

That is then used to set BuildKit's ExportedProduct variable in order to show
useful error messages to users when a certain version of the product doesn't
support a BuildKit feature.

Signed-off-by: Tibor Vass <tibor@docker.com>
2018-07-16 21:41:54 +00:00
Sebastiaan van Stijn
aaa1392279
Pass log-level to containerd
dockerd allows the `--log-level` to be specified, but this log-level
was not forwarded to the containerd process.

This patch sets containerd's log-level to the same as dockerd if a
custom level is provided.

Now that `--log-level` is also passed to containerd, the default "info"
is removed, so that containerd's default (or the level configured in containerd.toml)
is still used if no log-level is set.

Before this change:

containerd would always be started without a log-level set (only the level that's configured in `containerd.toml`);

```
root      1014  2.5  2.1 496484 43468 pts/0    Sl+  12:23   0:00 dockerd
root      1023  1.2  1.1 681768 23832 ?        Ssl  12:23   0:00  \_ docker-containerd --config /var/run/docker/containerd/containerd.toml
```

After this change:

when running `dockerd` without options (same as current);

```
root      1014  2.5  2.1 496484 43468 pts/0    Sl+  12:23   0:00 dockerd
root      1023  1.2  1.1 681768 23832 ?        Ssl  12:23   0:00  \_ docker-containerd --config /var/run/docker/containerd/containerd.toml
```

when running `dockerd --debug`:

```
root       600  0.8  2.1 512876 43180 pts/0    Sl+  12:20   0:00 dockerd --debug
root       608  0.6  1.1 624428 23672 ?        Ssl  12:20   0:00  \_ docker-containerd --config /var/run/docker/containerd/containerd.toml --log-level debug
```

when running `dockerd --log-level=panic`

```
root       747  0.6  2.1 496548 43996 pts/0    Sl+  12:21   0:00 dockerd --log-level=panic
root       755  0.7  1.1 550696 24100 ?        Ssl  12:21   0:00  \_ docker-containerd --config /var/run/docker/containerd/containerd.toml --log-level panic
```

combining `--debug` and `--log-level` (`--debug` takes precedence):

```
root       880  2.7  2.1 634692 43336 pts/0    Sl+  12:23   0:00 dockerd --debug --log-level=panic
root       888  1.0  1.1 616232 23652 ?        Ssl  12:23   0:00  \_ docker-containerd --config /var/run/docker/containerd/containerd.toml --log-level debug
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-07-09 15:38:08 +02:00
Vincent Demeester
3845728524
Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-13 09:04:30 +02:00
Tonis Tiigi
b225258496 builder: export build cache records
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-10 10:05:27 -07:00
Tonis Tiigi
22f7caee03 builder: experimental buildkit base
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-10 10:05:26 -07:00
Tonis Tiigi
44f3dd7653 vendor: update containerd to 63522d9
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-08 19:19:06 -07:00
Brian Goff
e4b6adc88e Extract volume interaction to a volumes service
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 14:21:07 -04:00
Sebastiaan van Stijn
80bfcc3a7a
Merge pull request #37136 from muesli/sdnotify-api-const
Use go-systemd const instead of magic string in Linux version of dockerd
2018-05-24 14:30:28 +02:00
Christian Muehlhaeuser
d393774a53
Use go-systemd const instead of magic string in Linux version of dockerd
Signed-off-by: Christian Muehlhaeuser <muesli@gmail.com>
2018-05-23 19:06:34 +02:00
Sebastiaan van Stijn
f23c00d870
Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-23 17:50:54 +02:00
Vincent Demeester
032a4b6ef6
Merge pull request #37106 from thaJeztah/unfork-cobra
Bump spf13/cobra to v0.0.3, pflag to v1.0.1
2018-05-22 09:39:52 +02:00
Sebastiaan van Stijn
55bebbaecf
Replace deprecated testutil.ErrorContains()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-21 00:13:04 +02:00
Sebastiaan van Stijn
0c3192da8c
Use Cobra built-in --version feature
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-19 03:15:08 +02:00
Sebastiaan van Stijn
ed75c7727b
Bump spf13/cobra to v0.0.3, pflag to v1.0.1
Use a tagged release of Cobra. All relevant PR's were merged, so the fork is
no longer needed.

Relevant changes:

- spf13/cobra#552 Add a field to disable [flags] in UseLine()
- spf13/cobra#567 Add `CalledAs` method to cobra.Command
- spf13/cobra#580 Update error message for missing required flags
- spf13/cobra#584 Add support for --version flag
- spf13/cobra#614 If user has a project in symlink, just use its destination folder and work there
- spf13/cobra#649 terminates the flags when -- is found in commandline
- spf13/cobra#662 Add support for ignoring parse errors
- spf13/cobra#686 doc: hide hidden parent flags

Also various improvements were added for generating Bash
completion scripts (currently not used by us)

Fixes usage output for dockerd;

Before this update:

    dockerd --help

    Usage:	dockerd COMMAND

    A self-sufficient runtime for containers.

After this update:

    dockerd --help

    Usage:	dockerd [OPTIONS] [flags]

    A self-sufficient runtime for containers.

Bump spf13/pflag to v1.0.1

Relevant changes:

- spf13/pflag#106 allow lookup by shorthand
- spf13/pflag#113 Add SortFlags option
- spf13/pflag#138 Generate flag error output for errors returned from the parseFunc
- spf13/pflag#141 Fixing Count flag usage string
- spf13/pflag#143 add int16 flag
- spf13/pflag#122 DurationSlice: implementation and tests
- spf13/pflag#115 Implement BytesHex type of argument
- spf13/pflag#150 Add uintSlice and boolSlice to name prettifier
- spf13/pflag#155 Add multiline wrapping support
- spf13/pflag#158 doc: clarify difference between string slice vs. array
- spf13/pflag#160 add ability to ignore unknown flags
- spf13/pflag#163 Allow Users To Show Deprecated Flags

Hide [flags] in usage output

Hides the [flags] in the usage output of commands (present in newer
versions of Cobra), using the `.DisableFlagsInUseLine` option.

Before this change:

    dockerd --help

    Usage:	dockerd [OPTIONS] [flags]

    A self-sufficient runtime for containers.

After this change:

    dockerd --help

    Usage:	dockerd [OPTIONS]

    A self-sufficient runtime for containers.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>

Â#	modified:   vendor/github.com/spf13/pflag/string_array.go
§

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-19 03:09:32 +02:00
Sebastiaan van Stijn
4f8c870d62
Fix some linting issues
These showed locally when running `make validate`. CI doesn't seem to have the
same (possibly it's disabled in the configuration)

    builder/fscache/fscache.go:618::error: github.com/docker/docker/vendor/github.com/tonistiigi/fsutil.StatInfo composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:44::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    client/swarm_unlock_test.go:20::error: github.com/docker/docker/api/types/swarm.UnlockRequest composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:113::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    cmd/dockerd/daemon_unix.go:110::error: github.com/docker/docker/cmd/dockerd/hack.MalformedHostHeaderOverride composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:171::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay/overlay.go:413::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:203::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/overlay2/overlay.go:584::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:109::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/graphdriver/zfs/zfs.go:388::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    daemon/volumes_windows.go:27::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    integration/service/network_test.go:31::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    api/server/server.go:129:10⚠️ should not use basic type string as key in context.WithValue (golint)
    integration/service/network_test.go:54::error: github.com/docker/docker/api/types/network.NetworkingConfig composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:61::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    libcontainerd/client_daemon_linux.go:74::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    pkg/archive/archive_windows.go:76::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)
    plugin/manager_linux.go:56::error: github.com/docker/docker/pkg/idtools.IDPair composite literal uses unkeyed fields (vet)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-17 19:28:27 +02:00
Sebastiaan van Stijn
57493cd606
Merge pull request #36921 from cyli/filter-namespaced-labels
Warn when reserved-namespace engine labels are configured
2018-05-07 15:12:52 +02:00
Alessandro Boch
173b3c364e Allow user to control the default address pools
- Via daemon flag --default-address-pools base=<CIDR>,size=<int>

Signed-off-by: Elango Siva  <elango@docker.com>
2018-04-30 11:14:08 -04:00
cyli
d1d7bcd6d7 Filter engine labels to exclude com.docker.*, io.docker.*, and org.dockerproject.*
labels as per https://docs.docker.com/config/labels-custom-metadata/.

Signed-off-by: Ying Li <ying.li@docker.com>
2018-04-24 11:42:09 -07:00
Karthik Karanth
05346355db Refactor code in cmd/dockerd/daemon.go
Signed-off-by: Karthik Karanth <karanth.karthik@gmail.com>
2018-04-13 00:14:20 +05:30
David Chung
275a1ca7c8 Expose swarm raft tuning parameters in engine config
Signed-off-by: David Chung <david.chung@docker.com>
2018-03-28 16:54:43 -07:00
Sebastiaan van Stijn
cd3e84c6b3
Split daemon service code to _windows file
This moves some of the code that was conditionally
executed on Windows to a separate, windows-only file.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-03-21 12:57:53 +01:00
Daniel Nephin
6be0f70983 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:43 -04:00
Daniel Nephin
0dab53ff3c Move all daemon image methods into imageService
imageService provides the backend for the image API and handles the
imageStore, and referenceStore.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-26 16:48:29 -05:00
Brian Goff
b0b9a25e7e Move log validator logic after plugins are loaded
This ensures that all log plugins are registered when the log validator
is run.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-02-15 11:53:11 -05:00
Daniel Nephin
e574c5ae73 Move commit to container backend
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-14 16:06:12 -05:00
Sebastiaan van Stijn
250193387c
Remove workaround for Nano server TP5
This workaround for golang/go#15286 was added for Nano server TP5 in
fa82c0aa10, and should no longer be
needed

Due to a security fix in Go 1.9.4/1.8.7, loading the .dll is no longer
allowed, and produces an error:

   .\docker_windows.go:9:3: //go:cgo_import_dynamic main.dummy CommandLineToArgvW%2 "shell32.dll" only allowed in cgo-generated code

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-02-07 23:38:14 -08:00
Daniel Nephin
4f0d95fa6e Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
Sebastiaan van Stijn
6e7715d65b
Fix "--node-generic-resource" singular/plural
Daemon flags that can be specified multiple times use
singlar names for flags, but plural names for the configuration
file.

To make the daemon configuration know how to correlate
the flag with the corresponding configuration option,
`opt.NewNamedListOptsRef()` should be used instead of
`opt.NewListOptsRef()`.

Commit 6702ac590e attempted
to fix the daemon not corresponding the flag with the configuration
file option, but did so by changing the name of the flag
to plural.

This patch reverts that change, and uses `opt.NewNamedListOptsRef()`
instead.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-26 13:53:13 -08:00