Commit graph

792 commits

Author SHA1 Message Date
CrazyMax
699e853be3
hack: display build cmd when DOCKER_DEBUG set
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 170a1c4709)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 15:06:33 +01:00
CrazyMax
b7e0725e98
hack: remove unnecessary vars in make.sh script
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 877baae03e)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-30 15:06:32 +01:00
CrazyMax
8fc49af4ab
hack: check if cross-compiling before setting ARM target name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 67aa86f1e0)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-01-20 16:40:33 +01:00
Kevin Alvarez
8c12a6648b
hack: name for target ARM architecture not specified
Build currently doesn't set the right name for target ARM
architecture through switches in CGO_CFLAGS and CGO_CXXFLAGS
when doing cross-compilation. This was previously fixed in https://github.com/moby/moby/pull/43474

Also removes the toolchain configuration. Following changes for
cross-compilation in https://github.com/moby/moby/pull/44546,
we forgot to remove the toolchain configuration that is
not used anymore as xx already sets correct cc/cxx envs already.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 945704208a)
2023-01-17 20:17:18 +01:00
CrazyMax
10c4ada049
hack: typo in displayed platform name
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 94639eb66f)
2023-01-17 20:17:08 +01:00
CrazyMax
8dfd907478
hack: restore copy_binaries func
This function is still useful for the dev environment.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 7e7b0622a6)
2023-01-06 08:38:33 +01:00
Paweł Gronowski
0d1aba547e
hack/make: Don't add -buildmode=pie with -race
Make it possible to add `-race` to the BUILDFLAGS without making the
build fail with error:
"-buildmode=pie not supported when -race is enabled"

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
(cherry picked from commit bbe6e9e8d1)
2023-01-05 20:27:08 +01:00
CrazyMax
171471b613
hack: update buildmode pie condition
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 491467b0b9)
2023-01-02 18:46:02 +01:00
CrazyMax
7d8c689923
hack: do not set buildmode pie for statically linked binary
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 347a5f68da)
2023-01-02 18:46:02 +01:00
CrazyMax
29758a067f
IAmStatic not used anymore
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 84ea9ee0f9)
2023-01-02 18:46:02 +01:00
CrazyMax
5d4ddce5ed
hack: remove containerutility script
this script is not used anymore. containerutility is
built in the Dockerfile.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 04c90b8cf5)
2023-01-02 18:46:01 +01:00
CrazyMax
a85bdbe1ca
Dockerfile: use TARGETPLATFORM to build Docker
Better support for cross compilation so we can fully rely
on `--platform` flag of buildx for a seamless integration.

This removes unnecessary extra cross logic in the Dockerfile,
DOCKER_CROSSPLATFORMS and CROSS vars and some hack scripts as well.

Non-sandboxed build invocation is still supported and dev stages
in the Dockerfile have been updated accordingly.

Bake definition and GitHub Actions workflows have been updated
accordingly as well.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 8086f40123)
2023-01-02 18:46:01 +01:00
Sebastiaan van Stijn
a8184baf3b
Merge pull request #44523 from crazy-max/22.06_dockerfile-vpnkit-platform
[22.06 backport] Dockerfile: remove hardcoded platforms for vpnkit stage
2022-11-25 00:47:12 +01:00
CrazyMax
acb8204a7f
Dockerfile: remove hardcoded platforms for vpnkit stage
Current Dockerfile downloads vpnkit for both linux/amd64
and linux/arm64 platforms even if target platform does not
match. This change will download vpnkit only if target
platform matches, otherwise it will just use a dummy scratch
stage.

Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 8a46a2a364)
2022-11-24 14:21:28 +01:00
CrazyMax
313f105443
hack: remove obsolete sources for go-autogen
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 40069797ef)
2022-11-24 14:20:48 +01:00
Brian Goff
4669418731
Volume prune: only prune anonymous volumes by default
This adds a new filter argument to the volume prune endpoint "all".
When this is not set, or it is a false-y value, then only anonymous
volumes are considered for pruning.

When `all` is set to a truth-y value, you get the old behavior.

This is an API change, but I think one that is what most people would
want.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit 618f26ccbc)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-05 23:50:01 +02:00
CrazyMax
cc7b8cc980
ci(test): send coverage to codecov
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit 807c849431)
2022-09-08 13:50:19 +02:00
CrazyMax
0bfb1bded3
integration: TestNetworkLoopbackNat is broken on GitHub Runner
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
(cherry picked from commit df731c745a)
2022-09-08 13:50:18 +02:00
Sebastiaan van Stijn
71e2aa78e4
hack: don't generate checksums for individual binaries
The script to build (static)binaries also generated checksums for all binaries
and scripts included. These checksums were not used, and not included when
releasing packages (which are a `tar.gz` for which a separate checksum would
be generated).

Removing these checksums, as they're unused, and complicated using these
artifacts for packagers (who would have to exclude them after building).

Before this:

```bash
rm -rf ./bundles
docker buildx build --build-arg VERSION=22.06.0-beta.1 --output ./bundles --target binary .

tree bundles
bundles
└── binary-daemon
    ├── containerd
    ├── containerd-shim-runc-v2
    ├── containerd-shim-runc-v2.md5
    ├── containerd-shim-runc-v2.sha256
    ├── containerd.md5
    ├── containerd.sha256
    ├── ctr
    ├── ctr.md5
    ├── ctr.sha256
    ├── docker-init
    ├── docker-init.md5
    ├── docker-init.sha256
    ├── docker-proxy
    ├── docker-proxy.md5
    ├── docker-proxy.sha256
    ├── dockerd
    ├── dockerd-rootless-setuptool.sh
    ├── dockerd-rootless-setuptool.sh.md5
    ├── dockerd-rootless-setuptool.sh.sha256
    ├── dockerd-rootless.sh
    ├── dockerd-rootless.sh.md5
    ├── dockerd-rootless.sh.sha256
    ├── dockerd.md5
    ├── dockerd.sha256
    ├── rootlesskit
    ├── rootlesskit-docker-proxy
    ├── rootlesskit-docker-proxy.md5
    ├── rootlesskit-docker-proxy.sha256
    ├── rootlesskit.md5
    ├── rootlesskit.sha256
    ├── runc
    ├── runc.md5
    ├── runc.sha256
    ├── vpnkit
    ├── vpnkit.md5
    └── vpnkit.sha256

1 directory, 36 files
```

After this:

```bash
rm -rf ./bundles
docker buildx build --build-arg VERSION=22.06.0-beta.1 --output ./bundles --target binary .

tree bundles
bundles
└── binary-daemon
    ├── containerd
    ├── containerd-shim-runc-v2
    ├── ctr
    ├── docker-init
    ├── docker-proxy
    ├── dockerd
    ├── dockerd-rootless-setuptool.sh
    ├── dockerd-rootless.sh
    ├── rootlesskit
    ├── rootlesskit-docker-proxy
    ├── runc
    └── vpnkit

1 directory, 12 files
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-04 13:23:00 +02:00
Sebastiaan van Stijn
d6f9b4d73b
hack: remove version from binaries, and remove symlinks
There may have been some historic reason for doing this, but I couldn't find
a practical use for building the (some) binaries with a version (default: "dev")
included, only to use a symlink to refer to the actual binary.

This patch removes the "${VERSION}" from the binary names in bundles, and
removes the code that created symlinks for them.

Before this patch:

```bash
rm -rf ./bundles
docker buildx build --build-arg VERSION=22.06.0-beta.1 --output ./bundles --target binary .

tree bundles
bundles
└── binary-daemon
    ├── containerd
    ├── containerd-shim-runc-v2
    ├── containerd-shim-runc-v2.md5
    ├── containerd-shim-runc-v2.sha256
    ├── containerd.md5
    ├── containerd.sha256
    ├── ctr
    ├── ctr.md5
    ├── ctr.sha256
    ├── docker-init
    ├── docker-init.md5
    ├── docker-init.sha256
    ├── docker-proxy -> docker-proxy-22.06.0-beta.1
    ├── docker-proxy-22.06.0-beta.1
    ├── docker-proxy-22.06.0-beta.1.md5
    ├── docker-proxy-22.06.0-beta.1.sha256
    ├── dockerd -> dockerd-22.06.0-beta.1
    ├── dockerd-22.06.0-beta.1
    ├── dockerd-22.06.0-beta.1.md5
    ├── dockerd-22.06.0-beta.1.sha256
    ├── dockerd-rootless-setuptool.sh
    ├── dockerd-rootless-setuptool.sh.md5
    ├── dockerd-rootless-setuptool.sh.sha256
    ├── dockerd-rootless.sh
    ├── dockerd-rootless.sh.md5
    ├── dockerd-rootless.sh.sha256
    ├── rootlesskit
    ├── rootlesskit-docker-proxy
    ├── rootlesskit-docker-proxy.md5
    ├── rootlesskit-docker-proxy.sha256
    ├── rootlesskit.md5
    ├── rootlesskit.sha256
    ├── runc
    ├── runc.md5
    ├── runc.sha256
    ├── vpnkit
    ├── vpnkit.md5
    └── vpnkit.sha256

1 directory, 38 files
```

After this patch:

```bash
rm -rf ./bundles
docker buildx build --build-arg VERSION=22.06.0-beta.1 --output ./bundles --target binary .

tree bundles
bundles
└── binary-daemon
    ├── containerd
    ├── containerd-shim-runc-v2
    ├── containerd-shim-runc-v2.md5
    ├── containerd-shim-runc-v2.sha256
    ├── containerd.md5
    ├── containerd.sha256
    ├── ctr
    ├── ctr.md5
    ├── ctr.sha256
    ├── docker-init
    ├── docker-init.md5
    ├── docker-init.sha256
    ├── docker-proxy
    ├── docker-proxy.md5
    ├── docker-proxy.sha256
    ├── dockerd
    ├── dockerd-rootless-setuptool.sh
    ├── dockerd-rootless-setuptool.sh.md5
    ├── dockerd-rootless-setuptool.sh.sha256
    ├── dockerd-rootless.sh
    ├── dockerd-rootless.sh.md5
    ├── dockerd-rootless.sh.sha256
    ├── dockerd.md5
    ├── dockerd.sha256
    ├── rootlesskit
    ├── rootlesskit-docker-proxy
    ├── rootlesskit-docker-proxy.md5
    ├── rootlesskit-docker-proxy.sha256
    ├── rootlesskit.md5
    ├── rootlesskit.sha256
    ├── runc
    ├── runc.md5
    ├── runc.sha256
    ├── vpnkit
    ├── vpnkit.md5
    └── vpnkit.sha256

1 directory, 36 files
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-07-02 16:57:51 +02:00
Sebastiaan van Stijn
58e1f8d0b5
Merge pull request #43682 from crazy-max/split-test-suites
ci(integration-cli): split test suites in a matrix
2022-06-22 23:22:43 +02:00
CrazyMax
1887d85e21
ci(integration-cli): dynamically split tests in matrix
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-06-17 10:59:04 +02:00
Akihiro Suda
c3eefab550
hack: remove error_on_leaked_containerd_shims
The function does not support containerd-shim-runc-v2 (io.containerd.runc.v2)

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-06-05 18:41:44 +09:00
Akihiro Suda
1fcd9f73d4
hack: stop installing containerd-shim (io.containerd.runtime.v1.linux)
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2022-06-05 18:41:44 +09:00
CrazyMax
493d3ca0c3
fix docker-proxy not statically linked
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-05-20 13:54:37 +02:00
Sebastiaan van Stijn
219374e2cd
Merge pull request #43275 from kponichtera/43274-delve-debugger
Added Delve debugger to the development container
2022-05-10 18:34:45 +02:00
Konrad Ponichtera
4573cd9c30
Improved readability of the run hack script
Signed-off-by: Konrad Ponichtera <konpon96@gmail.com>
2022-05-10 11:57:39 +02:00
Konrad Ponichtera
7d328ea1d6
Added Delve debugger to the development container (including instructions in the contribution guide).
Signed-off-by: Konrad Ponichtera <konpon96@gmail.com>
2022-04-26 20:14:06 +02:00
CrazyMax
1efda78f2b
use go-winres for windows build and cleanup autogen and winresources
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-14 19:52:36 +02:00
CrazyMax
fd2143e0b0
use go-winres for cross to create Windows resources
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-14 19:52:35 +02:00
CrazyMax
12558c8d6e
fix cross compilation for arm platforms
Signed-off-by: CrazyMax <crazy-max@users.noreply.github.com>
2022-04-10 21:03:31 +02:00
Sebastiaan van Stijn
037b4c0e8e
docker-py: skip CreateContainerTest::test_create_with_device_cgroup_rules
This test is verifying that the container has the right options set (through
`docker inspect`), but also checks if the cgroup-rules are set within the container
by reading `/sys/fs/cgroup/devices/devices`

Unlike cgroups v1, on cgroups v2, there is no file interface, and rules are handled
through ebpf, which means that the test will fail because this file is not present.

From the Linux documentation for cgroups v2: https://github.com/torvalds/linux/blob/v5.16/Documentation/admin-guide/cgroup-v2.rst#device-controller

> (...)
> Device controller manages access to device files. It includes both creation of
> new device files (using mknod), and access to the existing device files.
>
> Cgroup v2 device controller has no interface files and is implemented on top of
> cgroup BPF. To control access to device files, a user may create bpf programs
> of type BPF_PROG_TYPE_CGROUP_DEVICE and att>

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-01-20 11:11:22 +01:00
Sebastiaan van Stijn
3e5e7a6de3
CI: update docker-py to 5.0.3
full diff: https://github.com/docker/docker-py/compare/5.0.0...5.0.3

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-10-20 09:57:11 +02:00
Sebastiaan van Stijn
b585c64e2b
info: remove "expected" check for tini version
These checks were added when we required a specific version of containerd
and runc (different versions were known to be incompatible). I don't think
we had a similar requirement for tini, so this check was redundant. Let's
remove the check altogether.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-23 13:25:14 +02:00
Brian Goff
e7cf711c02
Move proxy CLI to main cmd/
Since this command is part of the official distribution and even
required for tests, let's move this up to the main cmd's.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-25 18:01:55 +02:00
Frédéric Dalleau
e93132e273 Add s390x to cross platforms target
Update build script and buid environment for supporting s390x
Signed-off-by: Frédéric Dalleau <frederic.dalleau@docker.com>
2021-06-24 18:52:55 +02:00
Frédéric Dalleau
3a208c0ff9 Add ppc64le to cross platforms target
debian uses ppc64el
go uses ppc64le
the cross compiler is powerpc64le
Signed-off-by: Frédéric Dalleau <frederic.dalleau@docker.com>
2021-06-24 18:51:49 +02:00
Tianon Gravi
1dac53860c Adjust buildmode comment to link to commit
This should make sure the link is still meaningful if that file drastically changes (which should make it easier to trace where the interesting block of code moved to and how it changes over time).

Also, add TODO items for Go 1.15+ and 1.16+ where we can "pie" more builds.

Signed-off-by: Tianon Gravi <admwiggin@gmail.com>
2021-06-11 12:34:50 -07:00
Georgy Yakovlev
2cd68d5268
don't use buildmode=pie on ppc64
It's already omitted for ppc64 in
hack/dockerfile/install/install.sh
not using wildcard, because GOARCH=ppc64le supports pie

Signed-off-by: Georgy Yakovlev <gyakovlev@gentoo.org>
2021-06-10 19:06:51 -07:00
Brian Goff
4b981436fe Fixup libnetwork lint errors
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 23:48:32 +00:00
Brian Goff
72c4a7b496 Fix issues running libnetwork tests.
libnetwork does different stuff depending on if you are running the
tests in a container or not... without telling it we are in a container
a bunch of the tests actually fail.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 22:14:41 +00:00
Brian Goff
4da0dc974c Ensure docker-proxy is installed in unit-test ci
(some) libnetwork tests require docker-proxy so we need to install that
in CI.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 22:14:31 +00:00
Brian Goff
86c8b8f1a0 Move proxy build into hack/make
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2021-06-01 22:14:06 +00:00
Sebastiaan van Stijn
257b4db186
testing: update docker-py 5.0.0
drops support for python 2.

full diff: https://github.com/docker/docker-py/compare/4.4.1...5.0.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-04-06 19:15:42 +02:00
Akihiro Suda
088e6ee790
Include VPNkit binary for arm64
Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
2021-02-25 18:16:00 +09:00
Sebastiaan van Stijn
14fb165085
testing: update docker-py 4.4.1
run docker-py integration tests of the latest release;

full diff: https://github.com/docker/docker-py/compare/4.3.0...4.4.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-01-26 11:30:58 +01:00
Brian Goff
1a1f5cc37b Pass through GOCACHE to test_env
Otherwise the cache gets written to bundles instead of to the volume
that we setup in the Makefile as expected.

Found this because vscode did not like having these huge objects in
bundles/ (didn't harm anything, just that it noticed and said hey I don't
like that).

This enables cache-reuse when building the test binaries.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2020-10-27 19:50:03 +00:00
Sebastiaan van Stijn
3ee1ab87b7
Update docker-py to 4.3.0
full diff: https://github.com/docker/docker-py/compare/4.2.2...4.3.0

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-08-13 10:38:32 +02:00
Sebastiaan van Stijn
6302dbbf46
Disable manifest v2 schema 1 push
For CI, a temporary `DOCKER_ALLOW_SCHEMA1_PUSH_DONOTUSE` environment
variable was added while we work out a solution for testing schema 1
pulls (which currently require pushing them to a local registry first
for testing).

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-31 20:13:58 +02:00
Sebastiaan van Stijn
bf4a5982f0
testing: bump docker-py 4.2.2
full diff: https://github.com/docker/docker-py/compare/4.2.1...4.2.2

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-07-17 14:45:28 +02:00