Commit graph

575 commits

Author SHA1 Message Date
Vincent Demeester
894c213b3b
Clean some docker_cli_build_tests that are cli-only
Remove TestBuildRenamedDockerfile and TestBuildDockerfileOutsideContext
that are cli-only tests (and already tested in the docker/cli
repository).

Also adds some comments on few tests that could be migrate to
docker/cli.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-02-27 13:42:29 +01:00
Dennis Chen
3a97100976 Unify the frozen images to the multi-arch version
Update and unify the `busybox` images on all arches to the `glibc` multi-arch
version and remove the temp workaround on amd64 which uses the old version
busybox (v1.26) before this PR to bypass the failure of those network related
test cases. Also, this PR will fix all the network related issues with `glibc`
version `busybox` image.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-02-14 03:59:04 +00:00
Yong Tang
8c31aacebf
Merge pull request #36018 from thaJeztah/remove-deprecated-testenv-functions
Remove deprecated integration-cli/environment functions
2018-01-15 13:24:52 -08:00
Sebastiaan van Stijn
18a771a761
Remove deprecated environment.DaemonPlatform()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 15:32:06 +01:00
Sebastiaan van Stijn
dfedc9ef62
Remove deprecated environment.MinimalBaseImage()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 15:31:02 +01:00
Sebastiaan van Stijn
142b1f8bfb
Remove deprecated environment.DockerBasePath()
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-01-15 15:29:26 +01:00
Dennis Chen
f88c2c04ef busybox: Fix ls compatible issue
BusyBox v1.26.2 (2017-03-09 00:04:38 UTC) supports `-le` option
to get the full date and time information, while BusyBox v1.27.2
(2017-11-01 23:22:25 UTC, which is used by the official multi-arch
image, uses `--full-time` instead of `-e` to get the same data. As
a result, we will get below error for the `DockerSuite.TestBuildLastModified`
test case in case of multi-arch image used:

> docker_cli_build_test.go:446:
>     out2 = cli.DockerCmd(c, "run", name, "ls", "-le", "/file").Combined()
>   o/src/github.com/docker/docker/vendor/github.com/gotestyourself/gotestyourself/icmd/command.go:61:
>     t.Fatalf("at %s:%d - %s\n", filepath.Base(file), line, err.Error())
> ... Error: at cli.go:33 -
> Command:  /usr/local/bin/docker run testbuildlastmodified ls -le /file
> ExitCode: 1
> Error:    exit status 1
> Stdout:
> Stderr:   ls: invalid option -- e
> BusyBox v1.27.2 (2017-11-01 23:22:25 UTC) multi-call binary.

This PR tries to fix the above compatible issue for busybox image.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-01-11 06:34:54 +00:00
Kir Kolyshkin
3676bd8569 TestBuildNotVerboseFailureRemote: fix false positive
I got the following test failure on power:

10:00:56
10:00:56
----------------------------------------------------------------------
10:00:56 FAIL: docker_cli_build_test.go:3521:
DockerSuite.TestBuildNotVerboseFailureRemote
10:00:56
10:00:56 docker_cli_build_test.go:3536:
10:00:56     c.Fatal(fmt.Errorf("Test[%s] expected that quiet stderr and
verbose stdout are equal; quiet [%v], verbose [%v]", name,
quietResult.Stderr(), result.Combined()))
10:00:56 ... Error: Test[quiet_build_wrong_remote] expected that quiet
stderr and verbose stdout are equal; quiet [
10:00:56 unable to prepare context: unable to download remote context
http://something.invalid: Get http://something.invalid: dial tcp: lookup
something.invalid on 172.29.128.11:53: no such host
10:00:56 ], verbose [unable to prepare context: unable to download
remote context http://something.invalid: Get http://something.invalid:
dial tcp: lookup something.invalid on 8.8.8.8:53: no such host
10:00:56 ]
10:00:56
10:00:56
10:00:56
----------------------------------------------------------------------

The reason is, either more than one name server is configured, or
nameserver was reconfigured in the middle of the test run. In any case,
different nameserver IP in an error messages should not be treated
as a failure, so let's strip those out.

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2017-12-28 13:08:41 -08:00
Tonis Tiigi
e90b7c06b4 integration-cli: clarify multi-stage tests names
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-12-20 11:55:27 -08:00
chaowang
7c35a24182 Separate the GenerateRandomAlphaOnlyString function from stringutils
Signed-off-by: chaowang <chaowang@localhost.localdomain>
2017-10-28 09:03:02 +08:00
Simon Ferquel
669c067798 Introduce a typed command system and 2 phase parse/dispatch build
This is a work base to introduce more features like build time
dockerfile optimisations, dependency analysis and parallel build, as
well as a first step to go from a dispatch-inline process to a
frontend+backend process.

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2017-09-18 09:48:56 +02:00
Daniel Nephin
92427b3a81 Update tests to use icmd
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-25 12:07:30 -04:00
Daniel Nephin
9db68f4dea Remove RandomTmpDirPath
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
e8bd671815 Remove testutil.ParseCgroupPaths
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Daniel Nephin
6151d55ee7 Remove testutil.CompareDirectoryEntries and IsKilled
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-22 17:15:26 -04:00
Josh Soref
39bcaee47b
Spelling fixes
* additional
* ambiguous
* anonymous
* anything
* application
* because
* before
* building
* capabilities
* circumstances
* commit
* committer
* compresses
* concatenated
* config
* container
* container's
* current
* definition
* delimiter
* disassociates
* discovery
* distributed
* doesnotexist
* downloads
* duplicates
* either
* enhancing
* enumerate
* escapable
* exactly
* expect
* expectations
* expected
* explicitly
* false
* filesystem
* following
* forbidden
* git with
* healthcheck
* ignore
* independent
* inheritance
* investigating
* irrelevant
* it
* logging
* looking
* membership
* mimic
* minimum
* modify
* mountpoint
* multiline
* notifier
* outputting
* outside
* overridden
* override
* parsable
* plugins
* precedence
* propagation
* provided
* provides
* registries
* repositories
* returning
* settings
* should
* signals
* someone
* something
* specifically
* successfully
* synchronize
* they've
* thinking
* uninitialized
* unintentionally
* unmarshaling
* unnamed
* unreferenced
* verify

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-03 13:13:09 -07:00
Ian Campbell
9777ec3be0 builder: Emit a BuildResult after squashing.
This ensures that the final BuildResult is the actual tagged image.

Fixes #33822.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-06-26 13:19:25 +01:00
Daniel Nephin
5136096520 Fix copy when used with scratch and images with empty RootFS
Commit the rwLayer to get the correct DiffID
Refacator copy in thebuilder
move more code into exportImage
cleanup some windows tests
Release the newly commited layer.
Set the imageID on the buildStage after exporting a new image.
Move archiver to BuildManager.
Have ReleaseableLayer.Commit return a layer
and store the Image from exportImage in the local imageSources cache
Remove NewChild from image interface.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-08 15:07:16 -04:00
Tonis Tiigi
b192218503 Fix resetting image metadata between stages for scratch case
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-05-12 12:00:55 -07:00
Daniel Nephin
ab3a037a5b Refactor interaction between dispatcher.from and dispatchState
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-10 17:58:57 -04:00
Tibor Vass
f47a61c429 fix expected output in tests
Signed-off-by: Tibor Vass <tibor@docker.com>
2017-05-05 12:14:30 -07:00
Ian Campbell
5894bc1abf Add docker build --iidfile=FILE
This is synonymous with `docker run --cidfile=FILE` and writes the digest of
the newly built image to the named file. This is intended to be used by build
systems which want to avoid tagging (perhaps because they are in CI or
otherwise want to avoid fixed names which can clash) by enabling e.g. Makefile
constructs like:

    image.id: Dockerfile
    	docker build --iidfile=image.id .

    do-some-more-stuff: image.id
    	do-stuff-with <image.id

Currently the only way to achieve this is to use `docker build -q` and capture
the stdout, but at the expense of losing the build output.

In non-silent mode (without `-q`) with API >= v1.29 the caller will now see a
`JSONMessage` with the `Aux` field containing a `types.BuildResult` in the
output stream for each image/layer produced during the build, with the final
one being the end product.  Having all of the intermediate images might be
interesting in some cases.

In silent mode (with `-q`) there is no change, on success the only output will
be the resulting image digest as it was previosuly.

There was no wrapper to just output an Aux section without enclosing it in a
Progress, so add one here.

Added some tests to integration cli tests.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-05-05 16:35:54 +01:00
Daniel Nephin
d9371ee807 Fix run with entrypoint in base image
Update a test to use a base image with entrypoint to that the linux build
has at least one test that behaves like all the windows tests.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-03 10:59:38 -04:00
Daniel Nephin
97f8607164 Remove ContainerUpdateCmdOnBuild, it does nothing.
Set a blank entrypoint to preserve the old behaviour.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-03 10:59:38 -04:00
Daniel Nephin
9f738cc574 Cleanup all the mutate + defer revert of b.runConfig in the builder
Instead of mutating and reverting, just create a copy and pass the copy
around.

Add a unit test for builder dispatcher.run

Fix two test failures

Fix image history by adding a CreatedBy to commit options. Previously the
createdBy field was being created by modifying a reference to the runConfig that
was held from when the container was created.

Fix a test that expected a trailing slash. Previously the runConfig was being
modified by container create. Now that we're creating a copy of runConfig
instead of sharing a reference the runConfig retains the trailing slash.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-01 18:36:53 -04:00
Daniel Nephin
721e1736ae Fix cache miss when builtin build args are used.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-28 12:49:50 -04:00
Dave Tucker
1076ab58ec builder: Make builtin arg pruning work with > 1 arg
The previous implementation would error out with "Unexpected EOF" which
was caused by an underlying "array index out-of-bounds" error.
The root cause was deleting items from the same array that was being
iterated over. The iteration was unaware that the array size had
changed, resulting in an error.

The new implementation builds a new array instead of mutating a copy of
the old one.

Fixes: #32744

Signed-off-by: Dave Tucker <dt@docker.com>
2017-04-26 10:24:58 +01:00
Daniel Nephin
e59327aaac Change builder.dispatchers to use a dispatchRequest object
This change starts the process of splitting up the Builder into logical
components. Remove builder.flags and move it to the new dispatchRequest
object.

Use runConfig from dispatchRequest instead of from the builder.

More progress removing things from the Builder struct.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-24 14:21:19 -04:00
Sebastiaan van Stijn
83ee902ecc Merge pull request #32740 from vdemeester/integration-trusted-cmd-cli
Use `cli` for trusted related command
2017-04-22 14:51:12 -07:00
Vincent Demeester
a582d9dc42
[integration] Move fakegit to its own package in cli
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-04-19 15:52:03 +02:00
Vincent Demeester
b0ba39d431
Use cli for trusted relate command
This also removed some skipped test (that are skipped for a long while).

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-04-19 14:04:39 +02:00
Daniel Nephin
df3c425407 Merge pull request #32328 from duglin/fixEnvVars
Minor tweaks to quotes in env vars
2017-04-13 10:41:28 -04:00
Doug Davis
2fb7c3c4f0 Minor tweaks to quotes in env vars
Addresses part of #32140, in particular:
- this will make it so that double backslashes in double-quoted
strings will result in a single backslash. While in single quotes it remains
a double backslash.
- missing closing " and ' will now generate an error

Signed-off-by: Doug Davis <dug@us.ibm.com>
2017-04-12 12:20:14 -07:00
Tõnis Tiigi
d2ab40e5b9 Merge pull request #32532 from dnephin/refactor-some-builder-parts
[builder] cleanup env dispatcher and Builder.build()
2017-04-12 11:11:06 -07:00
Vincent Demeester
330ddf8eb8 Merge pull request #29807 from thaJeztah/update-frozen-images
Update frozen images in Dockerfiles
2017-04-12 15:37:40 +02:00
Daniel Nephin
c7fad9b750 Cleanup in dispatcher.env
Remove commented code blocks
Remove some duplication in comparing and restructuring env

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-11 14:44:28 -04:00
Vincent Demeester
56fb4653e8
Move FakeContext to integration-cli/cli/build/context package…
… and continue emptying `docker_utils_test.go` from build related function.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-04-11 18:25:49 +02:00
Tonis Tiigi
33e07f41ad builder: add an option for specifying build target
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-10 16:21:43 -07:00
Sebastiaan van Stijn
1ecd8ed518 Update frozen images in Dockerfiles
This updates the versions of the frozen images used to their current
version. The original reason for updating these images was to make sure
they are not affected by [CVE-2016-1252 / DSA-3733-1](https://lwn.net/Articles/709119/),
which is fixed in apt 1.0.9.8.4 and up.

Note that `CVE-2016-1252` won't affect our test-suite, because no packages
are installed during out tests. It is just "good practice" to keep these
images up to date.

The `debian:jessie`, and `buildpack-deps:jessie` in  `Dockerfile.s390x`,
and `Dockerfile.armhf` have not been updated in this patch, because
those images have not yet been updated to contain apt 1.0.9.8.4.

While working on this, the `busybox` and `hello-world` were also updated
to their latest version.

Also removes a reference to `hack/make/.ensure-frozen-images`, which
was removed in ff91276d1f.

The new busybox image has one layer less than the original,
so updated `TestBuildSquashParent` to take that into account.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-04-10 22:05:39 +02:00
Vincent Demeester
778e32a2fa Merge pull request #31236 from tonistiigi/docker-stdin
build: accept -f - to read Dockerfile from stdin
2017-04-10 20:14:54 +02:00
Daniel Nephin
4d9e32a08e Merge pull request #31352 from dnephin/allow-arg-in-fromt
[dockerfile] Allow ARG in FROM
2017-04-10 13:23:40 -04:00
Daniel Nephin
239c53bf83 Refactor BuildArgs
Add MetaArgs for ARG that occur before the first FROM
Integration test for these cases.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-07 17:57:47 -04:00
Simon Ferquel
b0e7588873 Test and fix forbidden path for COPY --from on Windows
Paths resolving to c:\ or c:\windows are forbidden

Replaced the obscure (and non-working) regex with a simple case
insensitive comparison to the black listed paths (we should forbid c:\,
c:\windows but not d:\)

Also, add a test ensuring paths are case insensitive on windows

Also, made sure existing multi-staged build tests pass on windows

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2017-04-07 18:00:36 +02:00
Daniel Nephin
8cd6c30a48 Upadte archive.ReplaceFileTarWrapper() to not expect a sorted archive
Improve test coverage of ReplaceFileTarWrapper()

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-06 13:36:34 -04:00
David Sheets
3f6dc81e10 build: accept -f - to read Dockerfile from stdin
Heavily based on implementation by David Sheets

Signed-off-by: David Sheets <sheets@alum.mit.edu>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-05 19:42:31 -04:00
Tonis Tiigi
79ccb3478f Add a multi stage trusted build testcase
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-04 09:56:00 -07:00
Tonis Tiigi
672ea30a94 Add support for FROM using named block
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-04 09:34:28 -07:00
Tonis Tiigi
73b4b8ed7f Implicit copy-from with reference
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-04 09:34:28 -07:00
Tonis Tiigi
87512bbc84 Add named context support
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-04 09:34:28 -07:00
Tonis Tiigi
a1fa59ea12 builder: Fix setting command with custom shell
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-03-30 13:29:21 -07:00