Commit graph

348 commits

Author SHA1 Message Date
Brian Goff
4b03f520d3
Merge pull request #40856 from cpuguy83/reduce_allocs_on_env_repalce
Use strings.Index instead of strings.Split
2020-05-12 15:19:10 -07:00
Sebastiaan van Stijn
49f8a4224c
SELinux: fix ENOTSUP errors not being detected when relabeling
Commit 12c7541f1f updated the
opencontainers/selinux dependency to v1.3.1, which had a breaking
change in the errors that were returned.

Before v1.3.1, the "raw" `syscall.ENOTSUP` was returned if the
underlying filesystem did not support xattrs, but later versions
wrapped the error, which caused our detection to fail.

This patch uses `errors.Is()` to check for the underlying error.
This requires github.com/pkg/errors v0.9.1 or above (older versions
could use `errors.Cause()`, but are not compatible with "native"
wrapping of errors in Go 1.13 and up, and could potentially cause
these errors to not being detected again.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-05-10 17:06:44 +02:00
Sebastiaan van Stijn
07d60bc257
Replace errors.Cause() with errors.Is() / errors.As()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-29 00:28:41 +02:00
Brian Goff
5702a89db6 Use strings.Index instead of strings.Split
Since we don't need the actual split values, instead of calling
`strings.Split`, which allocates new slices on each call, use
`strings.Index`.

This significantly reduces the allocations required when doing env value
replacements.
Additionally, pre-allocate the env var slice, even if we allocate a
little more than we need, it keeps us from having to do multiple
allocations while appending.

```
benchmark                                     old ns/op     new ns/op     delta
BenchmarkReplaceOrAppendEnvValues/0-8         486           313           -35.60%
BenchmarkReplaceOrAppendEnvValues/100-8       10553         1535          -85.45%
BenchmarkReplaceOrAppendEnvValues/1000-8      94275         12758         -86.47%
BenchmarkReplaceOrAppendEnvValues/10000-8     1161268       129269        -88.87%

benchmark                                     old allocs     new allocs     delta
BenchmarkReplaceOrAppendEnvValues/0-8         5              2              -60.00%
BenchmarkReplaceOrAppendEnvValues/100-8       110            0              -100.00%
BenchmarkReplaceOrAppendEnvValues/1000-8      1013           0              -100.00%
BenchmarkReplaceOrAppendEnvValues/10000-8     10022          0              -100.00%

benchmark                                     old bytes     new bytes     delta
BenchmarkReplaceOrAppendEnvValues/0-8         192           24            -87.50%
BenchmarkReplaceOrAppendEnvValues/100-8       7360          0             -100.00%
BenchmarkReplaceOrAppendEnvValues/1000-8      64832         0             -100.00%
BenchmarkReplaceOrAppendEnvValues/10000-8     1146049       0             -100.00%
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-04-24 11:10:13 -07:00
Sebastiaan van Stijn
8312004f41
remove uses of deprecated pkg/term
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-04-21 16:29:27 +02:00
Kir Kolyshkin
39048cf656 Really switch to moby/sys/mount*
Switch to moby/sys/mount and mountinfo. Keep the pkg/mount for potential
outside users.

This commit was generated by the following bash script:

```
set -e -u -o pipefail

for file in $(git grep -l 'docker/docker/pkg/mount"' | grep -v ^pkg/mount); do
	sed -i -e 's#/docker/docker/pkg/mount"#/moby/sys/mount"#' \
		-e 's#mount\.\(GetMounts\|Mounted\|Info\|[A-Za-z]*Filter\)#mountinfo.\1#g' \
		$file
	goimports -w $file
done
```

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2020-03-20 09:46:25 -07:00
Brian Goff
750f0d1648 Support configuration of log cacher.
Configuration over the API per container is intentionally left out for
the time being, but is supported to configure the default from the
daemon config.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit cbecf48bc352e680a5390a7ca9cff53098cd16d7)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2020-02-19 17:02:34 -05:00
Brian Goff
e2ceb83a53 Support reads for all log drivers.
This supplements any log driver which does not support reads with a
custom read implementation that uses a local file cache.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit d675e2bf2b75865915c7a4552e00802feeb0847f)
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2020-02-19 17:01:44 -05:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
John Howard
8988448729 Remove refs to jhowardmsft from .go code
Signed-off-by: John Howard <jhoward@microsoft.com>
2019-09-25 10:51:18 -07:00
Sebastiaan van Stijn
07ff4f1de8
goimports: fix imports
Format the source according to latest goimports.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:56:54 +02:00
Brian Goff
ad718029b7
Merge pull request #39794 from tao12345666333/cleanup-set-dead-func
cleanup: remove SetDead function
2019-09-12 15:36:08 -07:00
Sebastiaan van Stijn
6e5a304675
container.ConfigFilePath: use same signature on Windows
This made my IDE unhappy; `ConfigFilePath` is an exported function, so
it makes sense to use the same signature for both Linux and Windows.

This patch also adds error handling (same as on Linux), even though the
current implementation will never return an error (it's good practice
to handle errors, so I assumed this would be the right approach)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-03 10:51:43 +02:00
Tibor Vass
32688a47f3
Merge pull request #39699 from thaJeztah/mkdirall_dropin
Allow system.MkDirAll() to be used as drop-in for os.MkDirAll()
2019-08-27 16:27:53 -07:00
Jintao Zhang
f8f6f7c2a0 cleanup: remove SetDead function
Signed-off-by: Jintao Zhang <zhangjintao9020@gmail.com>
2019-08-25 20:34:30 +08:00
Odin Ugedal
9c94e8260a
Fix phrasing when referring to the freezer cgroup
Signed-off-by: Odin Ugedal <odin@ugedal.com>
2019-08-16 20:39:30 +02:00
Sebastiaan van Stijn
e554ab5589
Allow system.MkDirAll() to be used as drop-in for os.MkDirAll()
also renamed the non-windows variant of this file to be
consistent with other files in this package

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-08 15:05:49 +02:00
Brian Goff
88d093cbb4
Merge pull request #39653 from thaJeztah/fix_unmount_ipc_ignore_enotexist
Fix "no such file or directory" warning when unmounting IPC mount
2019-08-05 11:51:28 -07:00
Sebastiaan van Stijn
060f387c0b
Fix "no such file or directory" warning when unmounting IPC mount
When cleaning up IPC mounts, the daemon could log a warning if the IPC mount was not found;

```
cleanup: failed to unmount IPC: umount /var/lib/docker/containers/90f408e26e205d30676655a08504dddc0d17f5713c1dd4654cf67ded7d3bbb63/mounts/shm, flags: 0x2: no such file or directory"
```

These warnings are safe to ignore, but can cause some confusion;  `container.UnmountIpcMount()`
already attempted to suppress these warnings, however, `mount.Unmount()` returns a `mountError`,
which nests the original error, therefore detecting failed.

This parch uses `errors.Cause()` to get the _underlying_ error to detect if it's a "is not exist".

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-08-02 12:36:54 +02:00
Ziheng Liu
53e0c50126
Avoid a data race in container/health.go
Signed-off-by: Ziheng Liu <lzhfromustc@gmail.com>
2019-07-31 13:17:32 -07:00
Michael Crosby
b5f28865ef Handle blocked I/O of exec'd processes
This is the second part to
https://github.com/containerd/containerd/pull/3361 and will help process
delete not block forever when the process exists but the I/O was
inherited by a subprocess that lives on.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2019-06-21 12:02:15 -04:00
Sebastiaan van Stijn
e511b3be89
Merge pull request #39336 from justincormack/entropy-cannot-be-saved
Entropy cannot be saved
2019-06-11 18:40:19 +02:00
Sebastiaan van Stijn
c85fe2d224
Merge pull request #38522 from cpuguy83/fix_timers
Make sure timers are stopped after use.
2019-06-07 13:16:46 +02:00
Justin Cormack
2df693e533
Entropy cannot be saved
Remove non cryptographic randomness.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2019-06-07 11:54:45 +01:00
Yash Murty
a31a088665 Remove DiskQouta field.
Signed-off-by: Yash Murty <yashmurty@gmail.com>
2019-04-30 15:13:08 +07:00
Sebastiaan van Stijn
20dde01848
Move EnableServiceDiscoveryOnDefaultNetwork to container-operations
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-03-20 18:45:20 +01:00
Justin Cormack
c435551ccc
Switch to google/uuid
pborman/uuid and google/uuid used to be different versions of
the same package, but now pborman/uuid is a compatibility wrapper
around google/uuid, maintained by the same person.

Clean up some of the usage as the functions differ slightly.

Not yet removed some uses of pborman/uuid in vendored code but
I have PRs in process for these.

Signed-off-by: Justin Cormack <justin.cormack@docker.com>
2019-03-13 14:13:58 +00:00
Sunny Gogoi
74eb258ffb Add pids-limit support in docker update
- Adds updating PidsLimit in UpdateContainer().
- Adds setting PidsLimit in toContainerResources().

Signed-off-by: Sunny Gogoi <indiasuny000@gmail.com>
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-02-21 14:17:38 -08:00
Brian Goff
eaad3ee3cf Make sure timers are stopped after use.
`time.After` keeps a timer running until the specified duration is
completed. It also allocates a new timer on each call. This can wind up
leaving lots of uneccessary timers running in the background that are
not needed and consume resources.

Instead of `time.After`, use `time.NewTimer` so the timer can actually
be stopped.
In some of these cases it's not a big deal since the duraiton is really
short, but in others it is much worse.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2019-01-16 14:32:53 -08:00
JoeWrightss
89e3a4af62 fix code typo
Signed-off-by: zhoulin xie <zhoulin.xie@daocloud.io>
2019-01-11 15:14:17 +08:00
Sebastiaan van Stijn
d69968d6d3
Fix missing import
This import got lost after commit 56cc56b0fa
was merged, likely because the PR was built against an outdated
master.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-12-12 17:48:48 +01:00
Vincent Demeester
f34727f704
Merge pull request #38316 from dmandalidis/xattr-fix
Ignore xattr errors on copy (fixes #38155)
2018-12-12 09:04:35 +01:00
Dimitris Mandalidis
d0192ae154 Ignore xattr ENOTSUP errors on copy (fixes #38155)
Signed-off-by: Dimitris Mandalidis <dimitris.mandalidis@gmail.com>
2018-12-11 06:36:51 +02:00
Kir Kolyshkin
6533136961 pkg/mount: wrap mount/umount errors
The errors returned from Mount and Unmount functions are raw
syscall.Errno errors (like EPERM or EINVAL), which provides
no context about what has happened and why.

Similar to os.PathError type, introduce mount.Error type
with some context. The error messages will now look like this:

> mount /tmp/mount-tests/source:/tmp/mount-tests/target, flags: 0x1001: operation not permitted

or

> mount tmpfs:/tmp/mount-test-source-516297835: operation not permitted

Before this patch, it was just

> operation not permitted

[v2: add Cause()]
[v3: rename MountError to Error, document Cause()]
[v4: fixes; audited all users]
[v5: make Error type private; changes after @cpuguy83 reviews]

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-12-10 20:07:02 -08:00
Kir Kolyshkin
77bc327e24 UnmountIpcMount: simplify
As standard mount.Unmount does what we need, let's use it.

In addition, this adds ignoring "not mounted" condition, which
was previously implemented (see PR#33329, commit cfa2591d3f)
via a very expensive call to mount.Mounted().

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-12-10 20:06:10 -08:00
Omri Shiv
fe1083d462 fix typo
Signed-off-by: Omri Shiv <Omri.Shiv@teradata.com>
2018-11-30 12:58:10 -08:00
Akihiro Suda
596cdffb9f mount: add BindOptions.NonRecursive (API v1.40)
This allows non-recursive bind-mount, i.e. mount(2) with "bind" rather than "rbind".

Swarm-mode will be supported in a separate PR because of mutual vendoring.

Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>
2018-11-06 17:51:58 +09:00
Sebastiaan van Stijn
e0ad6d045c
Merge pull request #37092 from cpuguy83/local_logger
Add "local" log driver
2018-08-20 07:01:41 +01:00
Brian Goff
a351b38e72 Add new local log driver
This driver uses protobuf to store log messages and has better defaults
for log file handling (e.g. compression and file rotation enabled by
default).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-08-17 09:36:56 -07:00
Salahuddin Khan
763d839261 Add ADD/COPY --chown flag support to Windows
This implements chown support on Windows. Built-in accounts as well
as accounts included in the SAM database of the container are supported.

NOTE: IDPair is now named Identity and IDMappings is now named
IdentityMapping.

The following are valid examples:
ADD --chown=Guest . <some directory>
COPY --chown=Administrator . <some directory>
COPY --chown=Guests . <some directory>
COPY --chown=ContainerUser . <some directory>

On Windows an owner is only granted the permission to read the security
descriptor and read/write the discretionary access control list. This
fix also grants read/write and execute permissions to the owner.

Signed-off-by: Salahuddin Khan <salah@docker.com>
2018-08-13 21:59:11 -07:00
Vincent Demeester
3845728524
Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-13 09:04:30 +02:00
Sebastiaan van Stijn
5cb95f693d
Merge pull request #37184 from cpuguy83/attach_leak
Fix fd leak on attach
2018-06-08 13:08:06 -07:00
Sebastiaan van Stijn
5037c5a8ce
Merge pull request #36688 from cpuguy83/volumes_service
Extract volume interaction to a volumes service
2018-06-05 02:16:20 +02:00
Brian Goff
0f51477017 Fix fd leak on attach
With a full attach, each attach was leaking 4 goroutines.
This updates attach to use errgroup instead of the hodge-podge of
waitgroups and channels.

In addition, the detach event was never being sent.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-31 10:11:18 -04:00
Brian Goff
b85799b63f
Merge pull request #36874 from kolyshkin/stop-timeout
daemon.ContainerStop(): fix for a negative timeout
2018-05-30 13:38:42 -04:00
Brian Goff
e4b6adc88e Extract volume interaction to a volumes service
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 14:21:07 -04:00
Sebastiaan van Stijn
f23c00d870
Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-23 17:50:54 +02:00
Brian Goff
d6558ad6a4 Use mount package to unmount container stuff
Really these mounts shouldn't be in the container pacakge.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-10 17:16:02 -04:00
Brian Goff
cc8f358c23 Move network operations out of container package
These network operations really don't have anything to do with the
container but rather are setting up the networking.

Ideally these wouldn't get shoved into the daemon package, but doing
something else (e.g. extract a network service into a new package) but
there's a lot more work to do in that regard.
In reality, this probably simplifies some of that work as it moves all
the network operations to the same place.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-10 17:16:00 -04:00
Kir Kolyshkin
69b4fe4065 daemon.ContainerStop(): fix for a negative timeout
1. As daemon.ContainerStop() documentation says,

> If a negative number of seconds is given, ContainerStop
> will wait for a graceful termination.

but since commit cfdf84d5d0 (PR #32237) this is no longer the case.

This happens because `context.WithTimeout(ctx, timeout)` is implemented
as `WithDeadline(ctx, time.Now().Add(timeout))`, resulting in a deadline
which is in the past.

To fix, don't use WithDeadline() if the timeout is negative.

2. Add a test case to validate the correct behavior and
as a means to prevent a similar regression in the future.

3. Fix/improve daemon.ContainerStop() and client.ContainerStop()
description for clarity and completeness.

4. Fix/improve DefaultStopTimeout description.

Fixes: cfdf84d5d0 ("Update Container Wait")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-03 10:04:33 -07:00