The binary targets now use buildkit to build/output binaries instead of
doing it in a DOCKER_RUN_DOCKER container. With that change caused
issues when trying to call multiple make targets such as `make binary
cross` since those targets are updating the variables (with conflicting
data) used by the shared `build` prerequisite.
This change has those binary output targets call `docker build` (or
`buildx build`) directly since that is the action they are preforming
and no longer have any pre-reqs.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This clarifies comments about static linking made in commit a8608b5b67.
1. There are two ways to create a static binary, one is to disable
cgo, the other is to set linker flags. When cgo is disabled,
there is no need to use osusergo build tag.
2. osusergo only needs to be set when linking against glibc.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
TL;DR: there is no way to do this right.
We do know that in some combination of build tags set (or unset),
linker flags, environment variables, and libc implementation,
this package won't work right. In fact, there is one specific
combination:
1. `CGO_ENABLED=1` (or unset)
2. static binary is being built (e.g. `go build` is run with `-extldflags -static`)
3. `go build` links the binary against glibc
4. `osusergo` is not set
This particular combination results in the following legitimate linker warning:
> cgo_lookup_unix.go: warning: Using 'getpwuid_r' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
If this warning is ignored and the resulting binary is used on a system
with files from a different glibc version (or without those files), it
could result in a segfault.
The commit being reverted tried to guard against such possibility,
but the problem is, we can only use build tags to account for items
1 and 4 from the above list, while items 2 and 3 do not result in
any build tags being set or unset, making this guard excessive.
Remove it.
This reverts commit 023b072288.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Now that we marked these utilities as helpers, it should be
possible to find which test-case failed (if any), and we
can skip logging in the "happy path".
This makes these tests less noisy, which makes it easier
to find actually important information in the output:
--- PASS: TestDockerSuite/TestCpFromCaseC (0.96s)
docker_cli_cp_utils_test.go:244: checking that file "/tmp/test-cp-from-case-c450122079/file2" contains "file2\n"
docker_cli_cp_utils_test.go:192: running `docker cp 962b1f3311e742b0842e13b2ad350214cea25883999fd26e87e8c9ddf40d5eb4:/root/file1 /tmp/test-cp-from-case-c450122079/file2`
docker_cli_cp_utils_test.go:244: checking that file "/tmp/test-cp-from-case-c450122079/file2" contains "file1\n"
Some of these tests should probably be rewritten to use subtests,
but that's something for a follow-up.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- Updated TestInfoSecurityOptions to not rely on CLI output. Note that this
test should be migrated to the integration suite, but that suite does not yet
have checks for "Seccomp" and "AppArmor"
- TestInfoAPIWarnings: don't start with busybox because we're not running containers in this test
- Migrate TestInfoDebug to integration suite
- Migrate TestInsecureRegistries to integration suite (renamed to TestInfoInsecureRegistries)
- Migrate TestRegistryMirrors to integration suite (renamed to TestInfoRegistryMirrors)
- Migrate TestInfoDiscoveryBackend to integration suite
- Migrate TestInfoDiscoveryInvalidAdvertise to integration suite
- Migrate TestInfoDiscoveryAdvertiseInterfaceName to integration suite
- Remove TestInfoFormat, which is testing the CLI functionality, and there is an
existing test in docker/cli (TestFormatInfo) covering this
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The build-stage would still be in the local cache (and can
be cleaned up with `docker system prune`) but the tagged
image (which usually wouldn't be removed) will take up
less space now.
Note that
- the binary is not statically linked, so we cannot use
a "from scratch" image
- in cases where the binary is cross-compiled (e.g.
on a non-linux machine), the image itself is not
really useful (we may want to consider not tagging
the image in that situation)
Before:
REPOSITORY TAG IMAGE ID CREATED SIZE
moby-buildx latest c9b2af465baf 7 minutes ago 1.71GB
After:
REPOSITORY TAG IMAGE ID CREATED SIZE
moby-buildx latest 345501e2df0a 2 minutes ago 820MB
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Un-indent the comment, so that it doesn't get printed by
the shell script (moved it above the target, as it looked
slightly less cluttered)
Also fixed the "help" comment, so that it shows up in
`make help`, and removed the un-needed dummy `buildx:` target.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This simplifies the makefile a bit, while preserving
the functionality. Using a non-existing Dockerfile
to demonstrate:
make buildx
Successfully tagged moby-buildx:latest
92059305df7371f8b5b3638d4d405d49ff909031a7bc6d2f515cb0a0df03c2f4
github.com/docker/buildx v0.3.0 c967f1d
make BUILDX_DOCKERFILE=foo buildx
BUILDX_DOCKERFILE=foo buildx
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /Users/sebastiaan/go/src/github.com/docker/docker/foo: no such file or directory
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch removes the `BUILDX_COMMIT` make variable. With the
make variable removed, it no longer "masks" environment variables,
and there is no longer a need to export the variable.
A side effect of this change, is that (by default), the buildx
image is tagged as `moby-buildx:latest`. This likely isn't a
problem, because the build-cache would still be preserved in
intermediate images. Having the image tagged as `:latest` also
makes cleaning up easier (without having to remove the image
for each version tagged.
Otherwise, the behavior remains the same as before:
# default
rm -f bundles/buildx && make buildx
# => => naming to docker.io/library/moby-buildx:latest
github.com/docker/buildx v0.3.0 c967f1d
# using a make variable:
rm -f bundles/buildx && make BUILDX_COMMIT=v0.2.1 buildx
# => => naming to docker.io/library/moby-buildx:v0.2.1
github.com/docker/buildx v0.2.1 0eb2df5
# using an environment variable:
rm -f bundles/buildx && BUILDX_COMMIT=v0.2.2 make buildx
# => => naming to docker.io/library/moby-buildx:v0.2.2
github.com/docker/buildx v0.2.2 ab5fe3d
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `BUILDX_COMMIT` variable was set as a Make variable,
which isn't exported, and thus not available in scripts,
unless referenced through `$(VAR)` (non-curly-brackets).
As a result `--build-arg BUILDX_COMMIT` did not set the
`BUILDX_COMMIT` build-arg, and the default from the Dockerfile
(`master`) was used instead.
This patch exports the default version that's set in the
Makefile, so that it can be used as a regular environment
variable. The script was also slighly modified to no longer
use the `Make` variable.
In addition, the `buildx` target now calls `buildx version`,
which is useful to confirm if the binary was successfully built
(and with the correct version).
Before:
rm -f bundles/buildx && make buildx && ./bundles/buildx version
# => => naming to docker.io/library/moby-buildx:v0.3.0
github.com/docker/buildx v0.3.1 6db68d0
# using a make variable:
rm -f bundles/buildx && make BUILDX_COMMIT=v0.2.1 buildx && ./bundles/buildx version
# => => naming to docker.io/library/moby-buildx:v0.2.1
github.com/docker/buildx v0.3.1 6db68d0
# using an environment variable:
rm -f bundles/buildx && BUILDX_COMMIT=v0.2.2 make buildx && ./bundles/buildx version
# => => naming to docker.io/library/moby-buildx:v0.2.2
github.com/docker/buildx v0.3.1 6db68d0
After:
# default
rm -f bundles/buildx && make buildx
# => => naming to docker.io/library/moby-buildx:v0.3.0
github.com/docker/buildx v0.3.0 c967f1d
# using a make variable:
rm -f bundles/buildx && make BUILDX_COMMIT=v0.2.1 buildx
# => => naming to docker.io/library/moby-buildx:v0.2.1
github.com/docker/buildx v0.2.1 0eb2df5
# using an environment variable:
rm -f bundles/buildx && BUILDX_COMMIT=v0.2.2 make buildx
# => => naming to docker.io/library/moby-buildx:v0.2.2
github.com/docker/buildx v0.2.2 ab5fe3d
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The variables were not substituted, because single-quotes were used.
While at it; change the fixed version/commit to use the actual commit
and version, using git.
before this change:
make buildx && ./bundles/buildx version
github.com/docker/buildx ${BUILDX_COMMIT} ${BUILDX_COMMIT}
after this change:
make buildx && ./bundles/buildx version
buildx v0.3.0 c967f1d
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 1be2cc2568 updated the
Makefile to force the use of BuildKit, if `USE_BUILDX` was
not set.
As a side-effect, Jenkins now started using BuildKit on
s390x and ppc64le as well, because it overwrote the
`DOCKER_BUILDKIT=0` that was set.
This commit forces the use of buildx on s390x and ppc64le.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Commit 7019b60d0d added these
env-vars to other stages, but forgot to update the DCO stage,
which also does a diff to validate commits that are in a PR.
Also adding openssh-client, for situations where the upstream
needs to be accessed through an ssh connection.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The validate step in CI was broken, due to a combination of
086b4541cf, fbdd437d29,
and 85733620eb being merged to master.
```
api/types/filters/parse.go:39:1: exported method `Args.Keys` should have comment or be unexported (golint)
func (args Args) Keys() []string {
^
daemon/config/builder.go:19:6: exported type `BuilderGCFilter` should have comment or be unexported (golint)
type BuilderGCFilter filters.Args
^
daemon/config/builder.go:21:1: exported method `BuilderGCFilter.MarshalJSON` should have comment or be unexported (golint)
func (x *BuilderGCFilter) MarshalJSON() ([]byte, error) {
^
daemon/config/builder.go:35:1: exported method `BuilderGCFilter.UnmarshalJSON` should have comment or be unexported (golint)
func (x *BuilderGCFilter) UnmarshalJSON(data []byte) error {
^
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This makes the test less noisy, and won't print the `failed: Error ...` messages,
which were confusing.
Also, running as a subtest allows tracking failures individually through the
junit.xml files.
Before:
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup (3.00s)
daemon.go:26: Creating a new daemon at: "/go/src/github.com/docker/docker/bundles/test-integration/TestDockerSwarmSuite/TestSwarmNetworkCreateDup"
docker_cli_swarm_test.go:1527: Creating a network named "network-test-0" with "bridge", then "bridge"
docker_cli_swarm_test.go:1534: As expected, the attempt to network "network-test-0" with "bridge" failed: Error response from daemon: network with name network-test-0 already exists
docker_cli_swarm_test.go:1527: Creating a network named "network-test-0" with "bridge", then "overlay"
docker_cli_swarm_test.go:1534: As expected, the attempt to network "network-test-0" with "overlay" failed: Error response from daemon: network with name network-test-0 already exists
docker_cli_swarm_test.go:1527: Creating a network named "network-test-1" with "overlay", then "bridge"
docker_cli_swarm_test.go:1534: As expected, the attempt to network "network-test-1" with "bridge" failed: Error response from daemon: network with name network-test-1 already exists
docker_cli_swarm_test.go:1527: Creating a network named "network-test-1" with "overlay", then "overlay"
docker_cli_swarm_test.go:1534: As expected, the attempt to network "network-test-1" with "overlay" failed: Error response from daemon: network with name network-test-1 already exists
After:
=== RUN TestDockerSwarmSuite
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_bridge_then_bridge
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_bridge_then_overlay
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_overlay_then_bridge
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_overlay_then_overlay
--- PASS: TestDockerSwarmSuite (8.12s)
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup (8.12s)
daemon.go:26: Creating a new daemon at: "/go/src/github.com/docker/docker/bundles/test-integration/TestDockerSwarmSuite/TestSwarmNetworkCreateDup"
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_bridge_then_bridge (0.52s)
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_bridge_then_overlay (0.31s)
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_overlay_then_bridge (0.17s)
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateDup/driver_overlay_then_overlay (0.12s)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The TEST_FILTER variable allows running a single integration or integration-cli
test. However, it failed to work properly for integration-cli tests.
Before:
-----------
# Filtering "integration" tests works:
make TEST_FILTER=TestInspectCpusetInConfigPre120 test-integration
...
DONE 1 tests in 18.331s
# But running a single test in "integration-cli" did not:
make TEST_FILTER=TestSwarmNetworkCreateIssue27866 test-integration
...
DONE 0 tests in 17.314s
Trying to manually add the `/` prefix, didn't work either, because that made the
"grep" fail to find which test-suites to run/skip:
make TEST_FILTER=/TestSwarmNetworkCreateIssue27866 test-integration
---> Making bundle: test-integration (in bundles/test-integration)
make: *** [test-integration] Error 1
After:
-----------
make TEST_FILTER=TestInspectCpusetInConfigPre120 test-integration
...
DONE 1 tests in 18.331s
make TEST_FILTER=TestSwarmNetworkCreateIssue27866 test-integration
...
DONE 12 tests in 26.527s
Note that the `12` tests is still a bit misleading, because every _suite_ is
started (which is counted as a test), but no tests are run. This is still
something that could be improved on.
This patch also makes a small modification to the code that's setting
`integration_api_dirs`, and no longer runs `go list` if not needed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Instead of logging on the "happy path", add more details when
we fail to create a daemon. Now that we base the path of the
daemon on the test-name, it should still be easy to find.
Before:
make TEST_FILTER=TestSwarmNetworkCreateIssue27866 test-integration
...
=== RUN TestDockerSwarmSuite
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateIssue27866
--- PASS: TestDockerSwarmSuite (7.47s)
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateIssue27866 (7.47s)
docker_cli_swarm_test.go:1499: Creating a new daemon at: "/go/src/github.com/docker/docker/bundles/test-integration/TestDockerSwarmSuite/TestSwarmNetworkCreateIssue27866"
After:
make TEST_FILTER=TestSwarmNetworkCreateIssue27866 test-integration
...
=== RUN TestDockerSwarmSuite
=== RUN TestDockerSwarmSuite/TestSwarmNetworkCreateIssue27866
--- PASS: TestDockerSwarmSuite (8.67s)
--- PASS: TestDockerSwarmSuite/TestSwarmNetworkCreateIssue27866 (8.67s)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Before:
daemon.go:26: Creating a new daemon at: "/go/src/github.com/docker/docker/bundles/test-integration/TestDockerSwarmSuite/TestSwarmNetworkCreateDup"
After:
docker_cli_swarm_test.go:1522: Creating a new daemon at: "/go/src/github.com/docker/docker/bundles/test-integration/TestDockerSwarmSuite/TestSwarmNetworkCreateDup"
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
