Commit graph

7184 commits

Author SHA1 Message Date
Sebastiaan van Stijn
9bf40d7edd
pkg/system: move IsWindowsClient to pkg/parsers/operatingsystem
This function was only used in a single place, and pkg/parsers/operatingsystem
already copied the `verNTWorkstation` const, so we might as well move this function
there as well to "unclutter" pkg/system.

The function had no external users, so not adding an alias / stub.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 10:26:50 +01:00
Sebastiaan van Stijn
5d10c6ec67
Update handling of deprecated kernel (tcp) memory options
- Omit `KernelMemory` and `KernelMemoryTCP` fields in `/info` response if they're
  not supported, or when using API v1.42 or up.
- Re-enable detection of `KernelMemory` (as it's still needed for older API versions)
- Remove warning about kernel memory TCP in daemon logs (a warning is still returned
  by the `/info` endpoint, but we can consider removing that).
- Prevent incorrect "Minimum kernel memory limit allowed" error if the value was
  reset because it's not supported by the host.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 09:56:39 +01:00
aiordache
af6307fbda
Remove KernelMemory option from /containers/create and /update endpoints
- remove KernelMemory option from `v1.42` api docs
 - remove KernelMemory warning on `/info`
 - update changes for `v1.42`
 - remove `KernelMemory` field from endpoints docs

Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-17 09:55:36 +01:00
Sebastiaan van Stijn
4203a97aad
staticcheck: ignore "SA1019: strings.Title is deprecated"
This function is marked deprecated in Go 1.18; however, the suggested replacement
brings in a large amount of new code, and most strings we generate will be ASCII,
so this would only be in case it's used for some user-provided string. We also
don't have a language to use, so would be using the "default".

Adding a `//nolint` comment to suppress the linting failure instead.

    daemon/logger/templates/templates.go:23:14: SA1019: strings.Title is deprecated: The rule Title uses for word boundaries does not handle Unicode punctuation properly. Use golang.org/x/text/cases instead. (staticcheck)
        "title":    strings.Title,
                    ^
    pkg/plugins/pluginrpc-gen/template.go:67:9: SA1019: strings.Title is deprecated: The rule Title uses for word boundaries does not handle Unicode punctuation properly. Use golang.org/x/text/cases instead. (staticcheck)
        return strings.Title(s)
               ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-16 12:11:54 +01:00
Akihiro Suda
de6732a403
version: add RootlessKit, slirp4netns, and VPNKit version
```console
$ docker --context=rootless version
...
Server:
...
 rootlesskit:
  Version:          0.14.2
  ApiVersion:       1.1.1
  NetworkDriver:    slirp4netns
  PortDriver:       builtin
  StateDir:         /tmp/rootlesskit245426514
 slirp4netns:
  Version:          1.1.9
  GitCommit:        4e37ea557562e0d7a64dc636eff156f64927335e
```

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-03-15 15:44:42 +09:00
Cory Snider
098a44c07f Finish refactor of UID/GID usage to a new struct
Finish the refactor which was partially completed with commit
34536c498d, passing around IdentityMapping structs instead of pairs of
[]IDMap slices.

Existing code which uses []IDMap relies on zero-valued fields to be
valid, empty mappings. So in order to successfully finish the
refactoring without introducing bugs, their replacement therefore also
needs to have a useful zero value which represents an empty mapping.
Change IdentityMapping to be a pass-by-value type so that there are no
nil pointers to worry about.

The functionality provided by the deprecated NewIDMappingsFromMaps
function is required by unit tests to to construct arbitrary
IdentityMapping values. And the daemon will always need to access the
mappings to pass them to the Linux kernel. Accommodate these use cases
by exporting the struct fields instead. BuildKit currently depends on
the UIDs and GIDs methods so we cannot get rid of them yet.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-14 16:28:57 -04:00
Cory Snider
b36fb04e03 vendor: github.com/containerd/containerd v1.6.1
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-10 17:48:10 -05:00
Cory Snider
06c797f517 vendor: github.com/docker/swarmkit 616e8db4c3b0
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-10 17:48:09 -05:00
Sebastiaan van Stijn
2c7c092e27
Merge pull request #41675 from thaJeztah/remove_containerd_plugin_config
daemon: remove v1 shim configuration for containerd
2022-03-08 13:09:00 +01:00
Sebastiaan van Stijn
327699c313
Merge pull request #43136 from jaen/zfs-driver-fix
Add locking to the ZFS driver
2022-03-07 19:52:56 +01:00
Sebastiaan van Stijn
2c97295ad8
daemon: remove v1 shim configuration for containerd
This removes the plugin section from the containerd configuration file
(`/var/run/docker/containerd/containerd.toml`) that is generated when
starting containerd as child process;

```toml
[plugins]
  [plugins.linux]
    shim = "containerd-shim"
    runtime = "runc"
    runtime_root = "/var/lib/docker/runc"
    no_shim = false
    shim_debug = true
```

This configuration doesn't appear to be used since commit:
0b14c2b67a, which switched the default runtime
to to io.containerd.runc.v2.

Note that containerd itself uses `containerd-shim` and `runc` as default
for `shim` and `runtime` v1, so omitting that configuration doesn't seem
to make a difference.

I'm slightly confused if any of the other options in this configuration were
actually used: for example, even though `runtime_root` was configured to be
`/var/lib/docker/runc`, when starting a container with that coniguration set
on docker 19.03, `/var/lib/docker/runc` doesn't appear to exist:

```console
$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES
098baa4cb0e7        nginx:alpine        "/docker-entrypoint.…"   59 minutes ago      Up 59 minutes       80/tcp              foo

$ ls /var/lib/docker/runc
ls: /var/lib/docker/runc: No such file or directory

$ ps auxf
PID   USER     TIME  COMMAND
    1 root      0:00 sh
   16 root      0:11 dockerd --debug
   26 root      0:09 containerd --config /var/run/docker/containerd/containerd.toml --log-level debug
  234 root      0:00 containerd-shim -namespace moby -workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/09
  251 root      0:00 nginx: master process nginx -g daemon off;
  304 101       0:00 nginx: worker process
...

```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-07 19:31:24 +01:00
Sebastiaan van Stijn
011e1c71ff
Merge pull request #43131 from thaJeztah/move_cpu_realtime_checks
daemon: move check for CPU-realtime daemon options
2022-03-07 19:27:12 +01:00
Tomek Mańko
a34fe9b422 Add locking to the ZFS driver
Trying to build Docker images with buildkit using a ZFS-backed storage
was unreliable due to apparent race condition between adding and
removing layers to the storage (see: https://github.com/moby/buildkit/issues/1758).
The issue describes a similar problem with the BTRFS driver that was
resolved by adding additional locking based on the scheme used in the
OverlayFS driver. This commit replicates the scheme to the ZFS driver
which makes the problem as reported in the issue stop happening.

Signed-off-by: Tomasz Mańko <hi@jaen.me>
2022-03-06 09:45:02 +01:00
Sebastiaan van Stijn
85f1bfc6f7
Merge pull request #43255 from thaJeztah/imageservice_nologs
daemon/images: ImageService.Cleanup(): return error instead of logging
2022-03-05 21:23:38 +01:00
Brian Goff
df664877e3
Merge pull request #43323 from thaJeztah/unalias
remove unneeded "digest" alias for "go-digest"
2022-03-04 16:28:05 -08:00
Sebastiaan van Stijn
7025029b98
Merge pull request #43306 from corhere/logfile-data-race
daemon/logger: fix data race in LogFile
2022-03-05 00:05:58 +01:00
Sebastiaan van Stijn
a0230f3d9a
remove unneeded "digest" alias for "go-digest"
I think this was there for historic reasons (may have been goimports expected
this, and we used to have a linter that wanted it), but it's not needed, so
let's remove it (to make my IDE less complaining about unneeded aliases).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-04 14:49:42 +01:00
Sebastiaan van Stijn
3e8bfcc9f2
Merge pull request #43263 from thaJeztah/daemon_config_tweak
daemon/config: DefaultShmSize: minor tweak and improve docs
2022-03-03 21:24:28 +01:00
Cory Snider
90c54320c8 daemon/logger: fix data race in LogFile
The log message's timestamp was being read after it was returned to the
pool. By coincidence the timestamp field happened to not be zeroed on
reset so much of the time things would work as expected. But if the
message value was to be taken back out of the pool before WriteLogEntry
returned, the timestamp recorded in the gzip header of compressed
rotated log files would be incorrect.

Make future use-after-put bugs fail fast by zeroing all fields of the
Message value, including the timestamp, when it is put into the pool.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-03 14:56:25 -05:00
Cory Snider
9080e5a1f7 daemon/logger: add test to detect data races
Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-03 14:56:25 -05:00
Sebastiaan van Stijn
c8cf4517fc
Merge pull request #43309 from thaJeztah/daemon_refactor_statecounter
daemon: SystemInfo() extract collecting data to more helper functions
2022-03-03 20:49:27 +01:00
Sebastiaan van Stijn
5263bea70f
daemon: move check for CPU-realtime daemon options
Perform the validation when the daemon starts instead of performing these
validations for each individual container, so that we can fail early.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-03 19:50:27 +01:00
Sebastiaan van Stijn
36ec581e5c
Merge pull request #43277 from thaJeztah/fix_kernelmem_docs_and_error
api: add missing docs for KernelMemoryTCP, and fix error message
2022-03-03 19:33:49 +01:00
Sebastiaan van Stijn
25ee00c494
pkg/system: move EnsureRemoveAll() to pkg/containerfs
pkg/system historically has been a bit of a kitchen-sink of things that were
somewhat "system" related, but didn't have a good place for. EnsureRemoveAll()
is one of those utilities. EnsureRemoveAll() is used to both unmount and remove
a path, for which it depends on both github.com/moby/sys/mount, which in turn
depends on github.com/moby/sys/mountinfo.

pkg/system is imported in the CLI, but neither EnsureRemoveAll(), nor any of its
moby/sys dependencies are used on the client side, so let's move this function
somewhere else, to remove those dependencies from the CLI.

I looked for plausible locations that were related; it's used in:

- daemon
- daemon/graphdriver/XXX/
- plugin

I considered moving it into a (e.g.) "utils" package within graphdriver (but not
a huge fan of "utils" packages), and given that it felt (mostly) related to
cleaning up container filesystems, I decided to move it there.

Some things to follow-up on after this:

- Verify if this function is still needed (it feels a bit like a big hammer in
  a "YOLO, let's try some things just in case it fails")
- Perhaps it should be integrated in `containerfs.Remove()` (so that it's used
  automatically)
- Look if there's other implementations (and if they should be consolidated),
  although (e.g.) the one in containerd is a copy of ours:
  https://github.com/containerd/containerd/blob/v1.5.9/pkg/cri/server/helpers_linux.go#L200

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-03 00:22:26 +01:00
Sebastiaan van Stijn
d492101172
daemon: SystemInfo() extract collecting debugging information to a helper
This makes it more inline with other data we collect, and can be used to make
some info optional at some point.

fillDebugInfo sets the current debugging state of the daemon, and additional
debugging information, such as the number of Go-routines, and file descriptors.

Note that this currently always collects the information, but the CLI only
prints it if the daemon has debug enabled. We should consider to either make
this information optional (cli to request "with debugging information"), or
only collect it if the daemon has debug enabled. For the CLI code, see
https://github.com/docker/cli/blob/v20.10.12/cli/command/system/info.go#L239-L244

Additional note: the CLI considers info.SystemTime debugging information. This
felt a bit "odd" (daemon time could be useful for standard use), so I left this
out of this function.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-02 22:19:24 +01:00
Sebastiaan van Stijn
ac2cd5a8f2
daemon: unexport Daemon.ID and Daemon.RegistryService
These are used internally only, and set by daemon.NewDaemon(). If they're
used externally, we should add an accessor added (which may be something
we want to do for daemon.registryService (which should be its own backend)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-02 22:19:22 +01:00
Sebastiaan van Stijn
a27f8aecad
daemon: SystemInfo() extract container counts to a helper function
This makes it more inline with other data we collect, and can be used to
make some info optional at some point.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-03-02 22:19:20 +01:00
Sebastiaan van Stijn
3c44ade6d0
daemon: fix error-message for minimum allowed kernel-memory limit
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-22 10:25:48 +01:00
Sebastiaan van Stijn
821b4d4108
daemon/config: DefaultShmSize: minor tweak and improve docs
I had to check what the actual size was, so added it to the const's documentation.

While at it, also made use of it in a test, so that we're testing against the expected
value, and changed one alias to be consistent with other places where we alias this
import.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 23:29:36 +01:00
Sebastiaan van Stijn
705f9b68cc
some cleaning up of isolation checks, and platform information
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 22:58:37 +01:00
Sebastiaan van Stijn
1b3fef5333
Windows: require Windows Server RS5 / ltsc2019 (build 17763) as minimum
Windows Server 2016 (RS1) reached end of support, and Docker Desktop requires
Windows 10 V19H2 (version 1909, build 18363) as a minimum.

This patch makes Windows Server RS5 /  ltsc2019 (build 17763) the minimum version
to run the daemon, and removes some hacks for older versions of Windows.

There is one check remaining that checks for Windows RS3 for a workaround
on older versions, but recent changes in Windows seemed to have regressed
on the same issue, so I kept that code for now to check if we may need that
workaround (again);

085c6a98d5/daemon/graphdriver/windows/windows.go (L319-L341)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-18 22:58:28 +01:00
Akihiro Suda
54d35c071d
Merge pull request #43130 from thaJeztah/daemon_cache_sysinfo
daemon: load and cache sysInfo on initialization
2022-02-18 13:46:15 +09:00
Sebastiaan van Stijn
79cad59d97
daemon/images: ImageService.Cleanup(): return error instead of logging
This makes the function a bit more idiomatic, and leaves it to the caller to
decide wether or not the error can be ignored.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-17 22:04:03 +01:00
Sebastiaan van Stijn
32e5fe5099
Merge pull request #43182 from thaJeztah/layer_remove_unused_error
layer: remove unused error return from .Size() and .DiffSize()
2022-02-17 20:51:45 +01:00
Brian Goff
047d58f007
Merge pull request #43187 from thaJeztah/remove_lcow_checks
Remove various leftover LCOW checks
2022-02-17 11:22:19 -08:00
Brian Goff
81db56f0a9
Merge pull request #43253 from thaJeztah/remove_kernel_version_check
daemon: remove kernel version check and DOCKER_NOWARN_KERNEL_VERSION
2022-02-17 11:17:15 -08:00
Sebastiaan van Stijn
dd4cf4b641
daemon: remove some unused stubs on Windows
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-17 17:57:51 +01:00
Sebastiaan van Stijn
1240f8b41d
daemon: remove kernel version check and DOCKER_NOWARN_KERNEL_VERSION
All regular, non-EOL Linux distros now come with more recent kernels
out of the box. There may still be users trying to run on kernel 3.10
or older (some embedded systems, e.g.), but those should be a rare
exception, which we don't have to take into account.

This patch removes the kernel version check on Linux, and the corresponding
DOCKER_NOWARN_KERNEL_VERSION environment that was there to skip this
check.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-17 17:47:22 +01:00
Sebastiaan van Stijn
699174347c
daemon: use RWMutex for stateCounter
Use an RWMutex to allow concurrent reads of these counters

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-15 18:04:18 +01:00
Tianon Gravi
469749947b
Merge pull request #43181 from thaJeztah/image_service_debuglogs
daemon/images.NewImageService() don't print debug logs
2022-02-14 09:45:40 -08:00
Samuel Karp
0d9a37d0c2
oci: inheritable capability set should be empty
The Linux kernel never sets the Inheritable capability flag to anything
other than empty.  Moby should have the same behavior, and leave it to
userspace code within the container to set a non-empty value if desired.

Reported-by: Andrew G. Morgan <morgan@kernel.org>
Signed-off-by: Samuel Karp <skarp@amazon.com>
2022-02-08 14:33:44 -08:00
Sebastiaan van Stijn
b88f4e2604
daemon/logger/awslogs: suppress false positive on hardcoded creds (gosec)
daemon/logger/awslogs/cloudwatchlogs.go:42:2: G101: Potential hardcoded credentials (gosec)
        credentialsEndpointKey = "awslogs-credentials-endpoint"
        ^
    daemon/logger/awslogs/cloudwatchlogs.go:67:2: G101: Potential hardcoded credentials (gosec)
        credentialsEndpoint = "http://169.254.170.2"
        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-02-08 09:43:22 +01:00
Sebastiaan van Stijn
f9fb5d4f25
daemon/graphdriver/fuse-overlayfs: Init(): fix directory permissions (staticcheck)
daemon/graphdriver/fuse-overlayfs/fuseoverlayfs.go:101:63: SA9002: file mode '700' evaluates to 01274; did you mean '0700'? (staticcheck)
        if err := idtools.MkdirAllAndChown(path.Join(home, linkDir), 700, currentID); err != nil {
                                                                     ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-28 12:50:30 +01:00
Akihiro Suda
65b8bcc321
Merge pull request #43174 from thaJeztah/move_platformcheck
distribution: remove RootFSDownloadManager interface, and remove "os" argument from Download()
2022-01-26 14:08:44 +09:00
Sebastiaan van Stijn
b36d896fce
layer: remove OS from layerstore
This was added in commits fc21bf280b and
0380fbff37 in support of LCOW, but was
now always set to runtime.GOOS.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-25 15:23:23 +01:00
Sebastiaan van Stijn
da277f891a
daemon.cleanupContainer() remove named return variable
It only made the code more difficult to read, adding cognitive overload.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-25 15:23:22 +01:00
Sebastiaan van Stijn
cae1dbee01
ImageService.ReleaseLayer(): remove unused containerOS argument
This looks to be a leftover from LCOW.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-25 15:23:20 +01:00
Sebastiaan van Stijn
e30a4a438b
daemon: remove leftover LCOW platform checks
This removes some of the checks that were added in 0cba7740d4,
but should no longer be needed.

- `Daemon.create()`: fix the error message, which assumed it could only occur on Windows.
- `Daemon.cleanupContainer()`: no need to validate container platform to delete it.
- `Daemon.containerExport`: if a container was created, we should be able to
  export it; no need to validate.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-25 15:23:18 +01:00
Sebastiaan van Stijn
b2ef2e8c83
daemon/images: remove leftover LCOW platform checks
This removes some of the checks that were added in 0cba7740d4,
but should no longer be needed.

- `ImageService.ImageDelete()`: no need to validate image platform to delete it.
- `ImageService.ImageHistory()`: no need to validate image platform to show its
  history; if it made it into the local image cache, it should be valid.
- `ImageService.ImportImage()`: `dockerfile.BuildFromConfig()` is used for
  `docker (container) commmit` and `docker (image) import`. For `docker import`,
   it's more transparent to perform validation early.
- `ImageService.LookupImage()`: no need to validate image platform to inspect it;
  if it made it into the local image cache, it should be valid.
- `ImageService.SquashImage()`: same. This code was actually broken, because it
  wrapped an `err` that was always `nil`, so would never return an error.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-25 12:15:50 +01:00
Sebastiaan van Stijn
f5db4b01c0
daemon/images: ImageService.LookupImage(): minor cleanup
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-24 18:45:49 +01:00