Commit graph

2918 commits

Author SHA1 Message Date
Bjorn Neergaard
7310a7cd0c
ci(buildkit): remove early-return from buildkit-ref
This doesn't really make sense now that this script returns a
$GITHUB_ENV snippet.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 09:30:42 -06:00
Bjorn Neergaard
4ecc01f3ad
ci(buildkit): remove misleading code from buildkit-ref
Post-f8c0d92a22bad004cb9cbb4db704495527521c42, BUILDKIT_REPO doesn't
really do what it claims to. Instead, don't allow overloading since the
import path for BuildKit is always the same, and make clear the
provenance of values when generating the final variable definitions.

We also better document the script, and follow some best practices for
both POSIX sh and Bash.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 09:30:40 -06:00
Bjorn Neergaard
48ff8a95cc
hack/with-go-mod.sh: warn on stderr
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-18 09:30:39 -06:00
Sebastiaan van Stijn
058a6e94a1
Merge pull request #45994 from jedevc/buildkit-ref-replace
ci: extract buildkit version correctly with replace-d modules
2023-07-18 15:22:38 +02:00
Bjorn Neergaard
780e8b2332
hack/d/cli.sh: properly handle errors in curl
Add `-f` to output nothing to tar if the curl fails, and `-S` to report
errors if they happen.

Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-07-17 10:18:19 -06:00
Bjorn Neergaard
37dcdac218
Merge pull request #40751 from cpuguy83/tmpfs_for_main_test_daemon
Put integration daemon socket in /run
2023-07-17 10:17:54 -06:00
Justin Chadwell
f8c0d92a22 ci: extract buildkit version correctly with replace-d modules
Signed-off-by: Justin Chadwell <me@jedevc.com>
2023-07-17 14:17:49 +01:00
Sebastiaan van Stijn
1ead2dd35d
update go to go1.20.6
go1.20.6 (released 2023-07-11) includes a security fix to the net/http package,
as well as bug fixes to the compiler, cgo, the cover tool, the go command,
the runtime, and the crypto/ecdsa, go/build, go/printer, net/mail, and text/template
packages. See the Go 1.20.6 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.20.6+label%3ACherryPickApproved

Full diff: https://github.com/golang/go/compare/go1.20.5...go1.20.6

These minor releases include 1 security fixes following the security policy:

net/http: insufficient sanitization of Host header

The HTTP/1 client did not fully validate the contents of the Host header.
A maliciously crafted Host header could inject additional headers or entire
requests. The HTTP/1 client now refuses to send requests containing an
invalid Request.Host or Request.URL.Host value.

Thanks to Bartek Nowotarski for reporting this issue.

Includes security fixes for [CVE-2023-29406 ][1] and Go issue https://go.dev/issue/60374

[1]: https://github.com/advisories/GHSA-f8f7-69v5-w4vx

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-07-14 22:36:46 +02:00
Albin Kerouanton
6b1b71ced4
Disable tls when launching dockerd through hack/make.sh
The daemon sleeps for 15 seconds at start up when the API binds to a TCP
socket with no TLS certificate set. That's what the hack/make/run script
does, but it doesn't explicitly disable tls, thus we're experiencing
this annoying delay every time we use this script.

Signed-off-by: Albin Kerouanton <albinker@gmail.com>
2023-07-09 22:26:07 +02:00
Brian Goff
2ab2503b6c Put integration daemon socket in /run
This fixes a case where on Docker For Mac if you need to bind mount the
bundles dir (e.g. to get test results back).
The unix socket does not work over oxsfs, so instead we put it in a
tmpfs.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-07-05 23:13:08 +00:00
Sebastiaan van Stijn
19d860fa9d
testing: temporarily pin docker-py tests to use "bullseye"
The official Python images on Docker Hub switched to debian bookworm,
which is now the current stable version of Debian.

However, the location of the apt repository config file changed, which
causes the Dockerfile build to fail;

    Loaded image: emptyfs:latest
    Loaded image ID: sha256:0df1207206e5288f4a989a2f13d1f5b3c4e70467702c1d5d21dfc9f002b7bd43
    INFO: Building docker-sdk-python3:5.0.3...
    tests/Dockerfile:6
    --------------------
       5 |     ARG APT_MIRROR
       6 | >>> RUN sed -ri "s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g" /etc/apt/sources.list \
       7 | >>>     && sed -ri "s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g" /etc/apt/sources.list
       8 |
    --------------------
    ERROR: failed to solve: process "/bin/sh -c sed -ri \"s/(httpredir|deb).debian.org/${APT_MIRROR:-deb.debian.org}/g\" /etc/apt/sources.list     && sed -ri \"s/(security).debian.org/${APT_MIRROR:-security.debian.org}/g\" /etc/apt/sources.list" did not complete successfully: exit code: 2

This needs to be fixed in docker-py, but in the meantime, we can pin to
the bullseye variant.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-20 10:48:42 +02:00
Sebastiaan van Stijn
98a44bb18e
update go to go1.20.5
go1.20.5 (released 2023-06-06) includes four security fixes to the cmd/go and
runtime packages, as well as bug fixes to the compiler, the go command, the
runtime, and the crypto/rsa, net, and os packages. See the Go 1.20.5 milestone
on our issue tracker for details:

https://github.com/golang/go/issues?q=milestone%3AGo1.20.5+label%3ACherryPickApproved

full diff: https://github.com/golang/go/compare/go1.20.4...go1.20.5

These minor releases include 3 security fixes following the security policy:

- cmd/go: cgo code injection
  The go command may generate unexpected code at build time when using cgo. This
  may result in unexpected behavior when running a go program which uses cgo.

  This may occur when running an untrusted module which contains directories with
  newline characters in their names. Modules which are retrieved using the go command,
  i.e. via "go get", are not affected (modules retrieved using GOPATH-mode, i.e.
  GO111MODULE=off, may be affected).

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29402 and Go issue https://go.dev/issue/60167.

- runtime: unexpected behavior of setuid/setgid binaries

  The Go runtime didn't act any differently when a binary had the setuid/setgid
  bit set. On Unix platforms, if a setuid/setgid binary was executed with standard
  I/O file descriptors closed, opening any files could result in unexpected
  content being read/written with elevated prilieges. Similarly if a setuid/setgid
  program was terminated, either via panic or signal, it could leak the contents
  of its registers.

  Thanks to Vincent Dehors from Synacktiv for reporting this issue.

  This is CVE-2023-29403 and Go issue https://go.dev/issue/60272.

- cmd/go: improper sanitization of LDFLAGS

  The go command may execute arbitrary code at build time when using cgo. This may
  occur when running "go get" on a malicious module, or when running any other
  command which builds untrusted code. This is can by triggered by linker flags,
  specified via a "#cgo LDFLAGS" directive.

  Thanks to Juho Nurminen of Mattermost for reporting this issue.

  This is CVE-2023-29404 and CVE-2023-29405 and Go issues https://go.dev/issue/60305 and https://go.dev/issue/60306.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-14 12:47:05 +02:00
Brian Goff
e1c92184f0
Merge pull request #44526 from crazy-max/rm-dockerfile-e2e
remove Dockerfile.e2e
2023-06-10 10:53:51 -07:00
Jan Garcia
0b1c1877c5 update RootlessKit to v1.1.1
Signed-off-by: Jan Garcia <github-public@n-garcia.com>
2023-06-06 09:26:27 +02:00
Paweł Gronowski
5433b88e2d
hack/cli.sh: Quiet origin cleanup
Don't show `error: No such remote: 'origin'` error when building for the
first time and the cached git repository doesn't a remote yet.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-05 17:02:46 +02:00
Paweł Gronowski
17c99f7164
Dockerfile: Use separate cli for shell and integration-cli
Use separate cli for integration-cli to allow use newer CLI for
interactive dev shell usage.

Both versions can be overriden with DOCKERCLI_VERSION or
DOCKERCLI_INTEGRATION_VERSION. Binary is downloaded from
download.docker.com if it's available, otherwise it's built from the
source.

For backwards compatibility DOCKER_CLI_PATH overrides BOTH clis.

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-06-05 15:25:21 +02:00
sebthom
d58df1fc6c Update blogpost URL
Signed-off-by: sebthom <sebthom@users.noreply.github.com>
2023-05-29 22:37:09 +02:00
CrazyMax
5efe72415d
remove Dockerfile.e2e
Dockerfile.e2e is not used anymore. Integration tests run
through the main Dockerfile.

Also removes the daemon OS/Arch detection script that is not
necessary anymore. It was used to select the Dockerfile based
on the arch like Dockerfile.arm64 but we don't have those
anymore. Was also used to check referenced frozen images
in the Dockerfile.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 11:03:48 +02:00
CrazyMax
735537d6b1
replace gogofast with gogofaster extension
gogofaster is identical as gogofast but removes XXX_unrecognized

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
Kevin Alvarez
7daaa00120
hack: generated files update and validation
Adds a Dockerfile and make targets to update and validate
generated files (proto, seccomp default profile)

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-05-29 03:28:35 +02:00
Sebastiaan van Stijn
7e01865fb6
Merge pull request #45592 from vvoland/dangling-image-repotagsdigests-test
integration: Add TestImageInspectEmptyTagsAndDigests
2023-05-24 19:18:56 +02:00
Sebastiaan van Stijn
53d1b12bc0
hack/make/.binary: don't use "netgo" when building Windows binaries
Starting with go1.19, the Go runtime on Windows now supports the `netgo` build-
flag to use a native Go DNS resolver. Prior to that version, the build-flag
only had an effect on non-Windows platforms. When using the `netgo` build-flag,
the Windows's host resolver is not used, and as a result, custom entries in
`etc/hosts` are ignored, which is a change in behavior from binaries compiled
with older versions of the Go runtime.

From the go1.19 release notes: https://go.dev/doc/go1.19#net

> Resolver.PreferGo is now implemented on Windows and Plan 9. It previously
> only worked on Unix platforms. Combined with Dialer.Resolver and Resolver.Dial,
> it's now possible to write portable programs and be in control of all DNS name
> lookups when dialing.
>
> The net package now has initial support for the netgo build tag on Windows.
> When used, the package uses the Go DNS client (as used by Resolver.PreferGo)
> instead of asking Windows for DNS results. The upstream DNS server it discovers
> from Windows may not yet be correct with complex system network configurations,
> however.

Our Windows binaries are compiled with the "static" (`make/binary-daemon`)
script, which has the `netgo` option set by default. This patch unsets the
`netgo` option when cross-compiling for Windows.

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-05-24 09:16:41 -06:00
Paweł Gronowski
a93aadc2e6
hack: Rename .ensure-emptyfs to .build-empty-images
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-23 15:50:35 +02:00
Paweł Gronowski
3a31f81838
hack/ensure-emptyfs: Create dangling image
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-23 15:50:33 +02:00
Sebastiaan van Stijn
484785456c
update containerd binary to v1.7.1
full diff: https://github.com/containerd/containerd/compare/v1.7.0...v1.7.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-15 12:52:50 +01:00
Brian Goff
e58c267d66
Merge pull request #45304 from crazy-max/buildkit-version-buildinfo
Set BuildKit version using buildinfo
2023-05-10 16:58:18 -07:00
Jeyanthinath Muthuram
71d7908656
added alias validation
Signed-off-by: Jeyanthinath Muthuram <jeyanthinath10@gmail.com>
2023-05-08 19:03:44 +05:30
Sebastiaan van Stijn
8a4b095a94
Merge pull request #45353 from thaJeztah/api_container_change_type
api/types/container: create type for changes endpoint
2023-05-04 19:48:40 +02:00
Sebastiaan van Stijn
dbb48e4b29
api/types/container: create type for changes endpoint
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-05-03 21:23:42 +02:00
Sebastiaan van Stijn
2d0e899819
update runc binary to v1.1.7
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.7
full diff: https://github.com/opencontainers/runc/compare/v1.1.6...v1.1.7

This is the seventh patch release in the 1.1.z release of runc, and is
the last planned release of the 1.1.z series. It contains a fix for
cgroup device rules with systemd when handling device rules for devices
that don't exist (though for devices whose drivers don't correctly
register themselves in the kernel -- such as the NVIDIA devices -- the
full fix only works with systemd v240+).

- When used with systemd v240+, systemd cgroup drivers no longer skip
  DeviceAllow rules if the device does not exist (a regression introduced
  in runc 1.1.3). This fix also reverts the workaround added in runc 1.1.5,
  removing an extra warning emitted by runc run/start.
- The source code now has a new file, runc.keyring, which contains the keys
  used to sign runc releases.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-27 12:17:14 +02:00
Kevin Alvarez
7665feeb52
hack: enable Go modules when building dockerd and docker-proxy
This is a workaround to have buildinfo with deps embedded in the
binary. We need to create a go.mod file before building with
-modfile=vendor.mod, otherwise it fails with:
"-modfile cannot be used to set the module root directory."

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-04-25 12:30:52 +02:00
Sebastiaan van Stijn
d0efca893b
update runc binary to v1.1.6
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.6
full diff: https://github.com/opencontainers/runc/compare/v1.1.5...v1.1.6

This is the sixth patch release in the 1.1.z series of runc, which fixes
a series of cgroup-related issues.

Note that this release can no longer be built from sources using Go
1.16. Using a latest maintained Go 1.20.x or Go 1.19.x release is
recommended. Go 1.17 can still be used.

- systemd cgroup v1 and v2 drivers were deliberately ignoring UnitExist error
  from systemd while trying to create a systemd unit, which in some scenarios
  may result in a container not being added to the proper systemd unit and
  cgroup.
- systemd cgroup v2 driver was incorrectly translating cpuset range from spec's
  resources.cpu.cpus to systemd unit property (AllowedCPUs) in case of more
  than 8 CPUs, resulting in the wrong AllowedCPUs setting.
- systemd cgroup v1 driver was prefixing container's cgroup path with the path
  of PID 1 cgroup, resulting in inability to place PID 1 in a non-root cgroup.
- runc run/start may return "permission denied" error when starting a rootless
  container when the file to be executed does not have executable bit set for
  the user, not taking the CAP_DAC_OVERRIDE capability into account. This is
  a regression in runc 1.1.4, as well as in Go 1.20 and 1.20.1
- cgroup v1 drivers are now aware of misc controller.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-13 19:43:05 +02:00
Sebastiaan van Stijn
f66374f6a8
golangci-lint: enable importas linter to force consistent aliases
This enforces the github.com/containerd/containerd/errdefs package to
be aliased as "cerrdefs". Any other alias (or no alias used) results
in a linting failure:

    integration/container/pause_test.go:9:2: import "github.com/containerd/containerd/errdefs" imported as "c8derrdefs" but must be "cerrdefs" according to config (importas)
        c8derrdefs "github.com/containerd/containerd/errdefs"
        ^

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-08 20:03:27 +02:00
Sebastiaan van Stijn
77be7b777c
update runc binary to v1.1.5
release notes: https://github.com/opencontainers/runc/releases/tag/v1.1.5
diff: https://github.com/opencontainers/runc/compare/v1.1.4...v1.1.5

This is the fifth patch release in the 1.1.z series of runc, which fixes
three CVEs found in runc.

* CVE-2023-25809 is a vulnerability involving rootless containers where
  (under specific configurations), the container would have write access
  to the /sys/fs/cgroup/user.slice/... cgroup hierarchy. No other
  hierarchies on the host were affected. This vulnerability was
  discovered by Akihiro Suda.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-m8cg-xc2p-r3fc>

* CVE-2023-27561 was a regression which effectively re-introduced
  CVE-2019-19921. This bug was present from v1.0.0-rc95 to v1.1.4. This
  regression was discovered by @Beuc.
  <https://github.com/advisories/GHSA-vpvm-3wq2-2wvm>

* CVE-2023-28642 is a variant of CVE-2023-27561 and was fixed by the same
  patch. This variant of the above vulnerability was reported by Lei
  Wang.
  <https://github.com/opencontainers/runc/security/advisories/GHSA-g2j6-57v7-gm8c>

In addition, the following other fixes are included in this release:

* Fix the inability to use `/dev/null` when inside a container.
* Fix changing the ownership of host's `/dev/null` caused by fd redirection
  (a regression in 1.1.1).
* Fix rare runc exec/enter unshare error on older kernels, including
  CentOS < 7.7.
* nsexec: Check for errors in `write_log()`.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-30 14:43:55 +02:00
Sebastiaan van Stijn
a2c29e9128
update containerd binary to v1.7.0
- release notes: https://github.com/containerd/containerd/releases/tag/v1.7.0
- full diff: https://github.com/containerd/containerd/compare/v1.7.0-beta.4...v1.7.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-03-12 16:09:33 +01:00
Bjorn Neergaard
d4e7a87b96 Upgrade to Go 1.20
Signed-off-by: Bjorn Neergaard <bneergaard@mirantis.com>
Co-authored-by: Cory Snider <csnider@mirantis.com>
Signed-off-by: Cory Snider <csnider@mirantis.com>
2023-02-22 16:37:59 -05:00
Sebastiaan van Stijn
e40c4c997e
update containerd binary to v1.7.0-beta.4
release notes: https://github.com/containerd/containerd/releases/tag/v1.7.0-beta.4
full diff: https://github.com/containerd/containerd/compare/v1.7.0-beta.3...v1.7.0-beta.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-02-17 01:28:29 +01:00
Sebastiaan van Stijn
31f4ec92f6
update containerd binary to v1.7.0-beta.3
full diff: https://github.com/containerd/containerd/compare/v1.7.0-beta.2...v1.7.0-beta.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 09:13:07 +01:00
Sebastiaan van Stijn
abdf23c6c0
Merge pull request #44853 from crazy-max/rm-install-suffix
hack: remove -installsuffix build flag
2023-01-20 18:25:50 +01:00
CrazyMax
67aa86f1e0
hack: check if cross-compiling before setting ARM target name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-20 15:13:11 +01:00
CrazyMax
170a1c4709
hack: display build cmd when DOCKER_DEBUG set
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-19 21:33:39 +01:00
CrazyMax
877baae03e
hack: remove unnecessary vars in make.sh script
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-19 21:33:38 +01:00
CrazyMax
b80f16157e
hack: remove ORIG_BUILDFLAGS var
This var was used for the cross target but it has been removed
in 8086f40123 so not necessary anymore

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-19 21:33:38 +01:00
CrazyMax
0c008cc4f8
hack: remove -installsuffix build flag
Has been introduced in 232d59baeb to work around a bug with
"go build" but not required anymore since go 1.5: 4dab6d01f1

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-19 21:33:38 +01:00
Kevin Alvarez
945704208a
hack: name for target ARM architecture not specified
Build currently doesn't set the right name for target ARM
architecture through switches in CGO_CFLAGS and CGO_CXXFLAGS
when doing cross-compilation. This was previously fixed in https://github.com/moby/moby/pull/43474

Also removes the toolchain configuration. Following changes for
cross-compilation in https://github.com/moby/moby/pull/44546,
we forgot to remove the toolchain configuration that is
not used anymore as xx already sets correct cc/cxx envs already.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-16 11:40:47 +01:00
CrazyMax
94639eb66f
hack: typo in displayed platform name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-16 11:40:47 +01:00
Sebastiaan van Stijn
57af88a654
Merge pull request #44748 from vvoland/buildflags-nopie-whenrace
hack/make: Don't add -buildmode=pie with -race
2023-01-05 20:24:21 +01:00
Paweł Gronowski
bbe6e9e8d1
hack/make: Don't add -buildmode=pie with -race
Make it possible to add `-race` to the BUILDFLAGS without making the
build fail with error:
"-buildmode=pie not supported when -race is enabled"

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-05 15:28:03 +01:00
CrazyMax
7e7b0622a6
hack: restore copy_binaries func
This function is still useful for the dev environment.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-05 13:21:28 +01:00
CrazyMax
27ba1766e7
ci: use GITHUB_REF and GITHUB_SHA to set version and commit
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:03 +01:00