Commit graph

799 commits

Author SHA1 Message Date
Paweł Gronowski
27df42255c
hack: Rename .ensure-emptyfs to .build-empty-images
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit a93aadc2e6)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-25 09:48:58 +02:00
Paweł Gronowski
9ee7d30aef
hack/ensure-emptyfs: Create dangling image
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit 3a31f81838)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-05-25 09:48:56 +02:00
Sebastiaan van Stijn
a753ca64e2
hack/make/.binary: don't use "netgo" when building Windows binaries
Starting with go1.19, the Go runtime on Windows now supports the `netgo` build-
flag to use a native Go DNS resolver. Prior to that version, the build-flag
only had an effect on non-Windows platforms. When using the `netgo` build-flag,
the Windows's host resolver is not used, and as a result, custom entries in
`etc/hosts` are ignored, which is a change in behavior from binaries compiled
with older versions of the Go runtime.

From the go1.19 release notes: https://go.dev/doc/go1.19#net

> Resolver.PreferGo is now implemented on Windows and Plan 9. It previously
> only worked on Unix platforms. Combined with Dialer.Resolver and Resolver.Dial,
> it's now possible to write portable programs and be in control of all DNS name
> lookups when dialing.
>
> The net package now has initial support for the netgo build tag on Windows.
> When used, the package uses the Go DNS client (as used by Resolver.PreferGo)
> instead of asking Windows for DNS results. The upstream DNS server it discovers
> from Windows may not yet be correct with complex system network configurations,
> however.

Our Windows binaries are compiled with the "static" (`make/binary-daemon`)
script, which has the `netgo` option set by default. This patch unsets the
`netgo` option when cross-compiling for Windows.

Co-authored-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
(cherry picked from commit 53d1b12bc0)
Signed-off-by: Bjorn Neergaard <bjorn.neergaard@docker.com>
2023-05-24 10:47:59 -06:00
Sebastiaan van Stijn
abdf23c6c0
Merge pull request #44853 from crazy-max/rm-install-suffix
hack: remove -installsuffix build flag
2023-01-20 18:25:50 +01:00
CrazyMax
67aa86f1e0
hack: check if cross-compiling before setting ARM target name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-20 15:13:11 +01:00
CrazyMax
170a1c4709
hack: display build cmd when DOCKER_DEBUG set
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-19 21:33:39 +01:00
CrazyMax
877baae03e
hack: remove unnecessary vars in make.sh script
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-19 21:33:38 +01:00
Kevin Alvarez
945704208a
hack: name for target ARM architecture not specified
Build currently doesn't set the right name for target ARM
architecture through switches in CGO_CFLAGS and CGO_CXXFLAGS
when doing cross-compilation. This was previously fixed in https://github.com/moby/moby/pull/43474

Also removes the toolchain configuration. Following changes for
cross-compilation in https://github.com/moby/moby/pull/44546,
we forgot to remove the toolchain configuration that is
not used anymore as xx already sets correct cc/cxx envs already.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-16 11:40:47 +01:00
CrazyMax
94639eb66f
hack: typo in displayed platform name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-16 11:40:47 +01:00
Sebastiaan van Stijn
57af88a654
Merge pull request #44748 from vvoland/buildflags-nopie-whenrace
hack/make: Don't add -buildmode=pie with -race
2023-01-05 20:24:21 +01:00
Paweł Gronowski
bbe6e9e8d1
hack/make: Don't add -buildmode=pie with -race
Make it possible to add `-race` to the BUILDFLAGS without making the
build fail with error:
"-buildmode=pie not supported when -race is enabled"

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
2023-01-05 15:28:03 +01:00
CrazyMax
7e7b0622a6
hack: restore copy_binaries func
This function is still useful for the dev environment.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-05 13:21:28 +01:00
CrazyMax
491467b0b9
hack: update buildmode pie condition
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:02 +01:00
CrazyMax
347a5f68da
hack: do not set buildmode pie for statically linked binary
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:02 +01:00
CrazyMax
84ea9ee0f9
IAmStatic not used anymore
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:02 +01:00
CrazyMax
04c90b8cf5
hack: remove containerutility script
this script is not used anymore. containerutility is
built in the Dockerfile.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:02 +01:00
CrazyMax
8086f40123
Dockerfile: use TARGETPLATFORM to build Docker
Better support for cross compilation so we can fully rely
on `--platform` flag of buildx for a seamless integration.

This removes unnecessary extra cross logic in the Dockerfile,
DOCKER_CROSSPLATFORMS and CROSS vars and some hack scripts as well.

Non-sandboxed build invocation is still supported and dev stages
in the Dockerfile have been updated accordingly.

Bake definition and GitHub Actions workflows have been updated
accordingly as well.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2023-01-01 18:03:01 +01:00
Sebastiaan van Stijn
d6d0e4c942
Merge pull request #44494 from crazy-max/go-autogen
hack: remove obsolete sources for go-autogen
2022-11-23 11:59:32 +01:00
CrazyMax
8a46a2a364
Dockerfile: remove hardcoded platforms for vpnkit stage
Current Dockerfile downloads vpnkit for both linux/amd64
and linux/arm64 platforms even if target platform does not
match. This change will download vpnkit only if target
platform matches, otherwise it will just use a dummy scratch
stage.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-22 17:07:54 +01:00
CrazyMax
40069797ef
hack: remove obsolete sources for go-autogen
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-11-18 15:03:39 +01:00
Sebastiaan van Stijn
7ff0f654fb
daemon: add TEST_INTEGRATION_USE_SNAPSHOTTER for CI
This allows us to run CI with the containerd snapshotter enabled, without
patching the daemon.json, or changing how tests set up daemon flags.

A warning log is added during startup, to inform if this variable is set,
as it should only be used for our integration tests.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-17 15:13:53 +02:00
Brian Goff
618f26ccbc Volume prune: only prune anonymous volumes by default
This adds a new filter argument to the volume prune endpoint "all".
When this is not set, or it is a false-y value, then only anonymous
volumes are considered for pruning.

When `all` is set to a truth-y value, you get the old behavior.

This is an API change, but I think one that is what most people would
want.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-04 20:55:13 +00:00
CrazyMax
807c849431
ci(test): send coverage to codecov
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-09-08 12:12:03 +02:00
CrazyMax
df731c745a
integration: TestNetworkLoopbackNat is broken on GitHub Runner
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-09-07 22:52:34 +02:00
Paweł Gronowski
a4b28e23cf
integration-cli: Correct emptyfs id under c8d
The legacy v1 is not supported by the containerd import

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-08-16 13:01:48 +02:00
Sebastiaan van Stijn
71e2aa78e4
hack: don't generate checksums for individual binaries
The script to build (static)binaries also generated checksums for all binaries
and scripts included. These checksums were not used, and not included when
releasing packages (which are a `tar.gz` for which a separate checksum would
be generated).

Removing these checksums, as they're unused, and complicated using these
artifacts for packagers (who would have to exclude them after building).

Before this:

```bash
rm -rf ./bundles
docker buildx build --build-arg VERSION=22.06.0-beta.1 --output ./bundles --target binary .

tree bundles
bundles
└── binary-daemon
    ├── containerd
    ├── containerd-shim-runc-v2
    ├── containerd-shim-runc-v2.md5
    ├── containerd-shim-runc-v2.sha256
    ├── containerd.md5
    ├── containerd.sha256
    ├── ctr
    ├── ctr.md5
    ├── ctr.sha256
    ├── docker-init
    ├── docker-init.md5
    ├── docker-init.sha256
    ├── docker-proxy
    ├── docker-proxy.md5
    ├── docker-proxy.sha256
    ├── dockerd
    ├── dockerd-rootless-setuptool.sh
    ├── dockerd-rootless-setuptool.sh.md5
    ├── dockerd-rootless-setuptool.sh.sha256
    ├── dockerd-rootless.sh
    ├── dockerd-rootless.sh.md5
    ├── dockerd-rootless.sh.sha256
    ├── dockerd.md5
    ├── dockerd.sha256
    ├── rootlesskit
    ├── rootlesskit-docker-proxy
    ├── rootlesskit-docker-proxy.md5
    ├── rootlesskit-docker-proxy.sha256
    ├── rootlesskit.md5
    ├── rootlesskit.sha256
    ├── runc
    ├── runc.md5
    ├── runc.sha256
    ├── vpnkit
    ├── vpnkit.md5
    └── vpnkit.sha256

1 directory, 36 files
```

After this:

```bash
rm -rf ./bundles
docker buildx build --build-arg VERSION=22.06.0-beta.1 --output ./bundles --target binary .

tree bundles
bundles
└── binary-daemon
    ├── containerd
    ├── containerd-shim-runc-v2
    ├── ctr
    ├── docker-init
    ├── docker-proxy
    ├── dockerd
    ├── dockerd-rootless-setuptool.sh
    ├── dockerd-rootless.sh
    ├── rootlesskit
    ├── rootlesskit-docker-proxy
    ├── runc
    └── vpnkit

1 directory, 12 files
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-04 13:23:00 +02:00
Sebastiaan van Stijn
d6f9b4d73b
hack: remove version from binaries, and remove symlinks
There may have been some historic reason for doing this, but I couldn't find
a practical use for building the (some) binaries with a version (default: "dev")
included, only to use a symlink to refer to the actual binary.

This patch removes the "${VERSION}" from the binary names in bundles, and
removes the code that created symlinks for them.

Before this patch:

```bash
rm -rf ./bundles
docker buildx build --build-arg VERSION=22.06.0-beta.1 --output ./bundles --target binary .

tree bundles
bundles
└── binary-daemon
    ├── containerd
    ├── containerd-shim-runc-v2
    ├── containerd-shim-runc-v2.md5
    ├── containerd-shim-runc-v2.sha256
    ├── containerd.md5
    ├── containerd.sha256
    ├── ctr
    ├── ctr.md5
    ├── ctr.sha256
    ├── docker-init
    ├── docker-init.md5
    ├── docker-init.sha256
    ├── docker-proxy -> docker-proxy-22.06.0-beta.1
    ├── docker-proxy-22.06.0-beta.1
    ├── docker-proxy-22.06.0-beta.1.md5
    ├── docker-proxy-22.06.0-beta.1.sha256
    ├── dockerd -> dockerd-22.06.0-beta.1
    ├── dockerd-22.06.0-beta.1
    ├── dockerd-22.06.0-beta.1.md5
    ├── dockerd-22.06.0-beta.1.sha256
    ├── dockerd-rootless-setuptool.sh
    ├── dockerd-rootless-setuptool.sh.md5
    ├── dockerd-rootless-setuptool.sh.sha256
    ├── dockerd-rootless.sh
    ├── dockerd-rootless.sh.md5
    ├── dockerd-rootless.sh.sha256
    ├── rootlesskit
    ├── rootlesskit-docker-proxy
    ├── rootlesskit-docker-proxy.md5
    ├── rootlesskit-docker-proxy.sha256
    ├── rootlesskit.md5
    ├── rootlesskit.sha256
    ├── runc
    ├── runc.md5
    ├── runc.sha256
    ├── vpnkit
    ├── vpnkit.md5
    └── vpnkit.sha256

1 directory, 38 files
```

After this patch:

```bash
rm -rf ./bundles
docker buildx build --build-arg VERSION=22.06.0-beta.1 --output ./bundles --target binary .

tree bundles
bundles
└── binary-daemon
    ├── containerd
    ├── containerd-shim-runc-v2
    ├── containerd-shim-runc-v2.md5
    ├── containerd-shim-runc-v2.sha256
    ├── containerd.md5
    ├── containerd.sha256
    ├── ctr
    ├── ctr.md5
    ├── ctr.sha256
    ├── docker-init
    ├── docker-init.md5
    ├── docker-init.sha256
    ├── docker-proxy
    ├── docker-proxy.md5
    ├── docker-proxy.sha256
    ├── dockerd
    ├── dockerd-rootless-setuptool.sh
    ├── dockerd-rootless-setuptool.sh.md5
    ├── dockerd-rootless-setuptool.sh.sha256
    ├── dockerd-rootless.sh
    ├── dockerd-rootless.sh.md5
    ├── dockerd-rootless.sh.sha256
    ├── dockerd.md5
    ├── dockerd.sha256
    ├── rootlesskit
    ├── rootlesskit-docker-proxy
    ├── rootlesskit-docker-proxy.md5
    ├── rootlesskit-docker-proxy.sha256
    ├── rootlesskit.md5
    ├── rootlesskit.sha256
    ├── runc
    ├── runc.md5
    ├── runc.sha256
    ├── vpnkit
    ├── vpnkit.md5
    └── vpnkit.sha256

1 directory, 36 files
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-02 16:57:51 +02:00
Sebastiaan van Stijn
58e1f8d0b5
Merge pull request #43682 from crazy-max/split-test-suites
ci(integration-cli): split test suites in a matrix
2022-06-22 23:22:43 +02:00
CrazyMax
1887d85e21
ci(integration-cli): dynamically split tests in matrix
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:59:04 +02:00
Akihiro Suda
c3eefab550
hack: remove error_on_leaked_containerd_shims
The function does not support containerd-shim-runc-v2 (io.containerd.runc.v2)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-06-05 18:41:44 +09:00
Akihiro Suda
1fcd9f73d4
hack: stop installing containerd-shim (io.containerd.runtime.v1.linux)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-06-05 18:41:44 +09:00
CrazyMax
493d3ca0c3
fix docker-proxy not statically linked
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-20 13:54:37 +02:00
Sebastiaan van Stijn
219374e2cd
Merge pull request #43275 from kponichtera/43274-delve-debugger
Added Delve debugger to the development container
2022-05-10 18:34:45 +02:00
Konrad Ponichtera
4573cd9c30
Improved readability of the run hack script
Signed-off-by: Konrad Ponichtera <konpon96@gmail.com>
2022-05-10 11:57:39 +02:00
Konrad Ponichtera
7d328ea1d6
Added Delve debugger to the development container (including instructions in the contribution guide).
Signed-off-by: Konrad Ponichtera <konpon96@gmail.com>
2022-04-26 20:14:06 +02:00
CrazyMax
1efda78f2b
use go-winres for windows build and cleanup autogen and winresources
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-14 19:52:36 +02:00
CrazyMax
fd2143e0b0
use go-winres for cross to create Windows resources
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-14 19:52:35 +02:00
CrazyMax
12558c8d6e
fix cross compilation for arm platforms
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-10 21:03:31 +02:00
Sebastiaan van Stijn
037b4c0e8e
docker-py: skip CreateContainerTest::test_create_with_device_cgroup_rules
This test is verifying that the container has the right options set (through
`docker inspect`), but also checks if the cgroup-rules are set within the container
by reading `/sys/fs/cgroup/devices/devices`

Unlike cgroups v1, on cgroups v2, there is no file interface, and rules are handled
through ebpf, which means that the test will fail because this file is not present.

From the Linux documentation for cgroups v2: https://github.com/torvalds/linux/blob/v5.16/Documentation/admin-guide/cgroup-v2.rst#device-controller

> (...)
> Device controller manages access to device files. It includes both creation of
> new device files (using mknod), and access to the existing device files.
>
> Cgroup v2 device controller has no interface files and is implemented on top of
> cgroup BPF. To control access to device files, a user may create bpf programs
> of type BPF_PROG_TYPE_CGROUP_DEVICE and att>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 11:11:22 +01:00
Sebastiaan van Stijn
3e5e7a6de3
CI: update docker-py to 5.0.3
full diff: https://github.com/docker/docker-py/compare/5.0.0...5.0.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-20 09:57:11 +02:00
Sebastiaan van Stijn
b585c64e2b
info: remove "expected" check for tini version
These checks were added when we required a specific version of containerd
and runc (different versions were known to be incompatible). I don't think
we had a similar requirement for tini, so this check was redundant. Let's
remove the check altogether.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-23 13:25:14 +02:00
Brian Goff
e7cf711c02
Move proxy CLI to main cmd/
Since this command is part of the official distribution and even
required for tests, let's move this up to the main cmd's.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-25 18:01:55 +02:00
Frédéric Dalleau
e93132e273 Add s390x to cross platforms target
Update build script and buid environment for supporting s390x
Signed-off-by: Frédéric Dalleau <frederic.dalleau@docker.com>
2021-06-24 18:52:55 +02:00
Frédéric Dalleau
3a208c0ff9 Add ppc64le to cross platforms target
debian uses ppc64el
go uses ppc64le
the cross compiler is powerpc64le
Signed-off-by: Frédéric Dalleau <frederic.dalleau@docker.com>
2021-06-24 18:51:49 +02:00
Tianon Gravi
1dac53860c Adjust buildmode comment to link to commit
This should make sure the link is still meaningful if that file drastically changes (which should make it easier to trace where the interesting block of code moved to and how it changes over time).

Also, add TODO items for Go 1.15+ and 1.16+ where we can "pie" more builds.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2021-06-11 12:34:50 -07:00
Georgy Yakovlev
2cd68d5268
don't use buildmode=pie on ppc64
It's already omitted for ppc64 in
hack/dockerfile/install/install.sh
not using wildcard, because GOARCH=ppc64le supports pie

Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>
2021-06-10 19:06:51 -07:00
Brian Goff
4b981436fe Fixup libnetwork lint errors
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 23:48:32 +00:00
Brian Goff
72c4a7b496 Fix issues running libnetwork tests.
libnetwork does different stuff depending on if you are running the
tests in a container or not... without telling it we are in a container
a bunch of the tests actually fail.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 22:14:41 +00:00
Brian Goff
4da0dc974c Ensure docker-proxy is installed in unit-test ci
(some) libnetwork tests require docker-proxy so we need to install that
in CI.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 22:14:31 +00:00
Brian Goff
86c8b8f1a0 Move proxy build into hack/make
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 22:14:06 +00:00