Commit graph

485 commits

Author SHA1 Message Date
Tonis Tiigi
b192218503 Fix resetting image metadata between stages for scratch case
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-05-12 12:00:55 -07:00
Daniel Nephin
6c28e8edd5 Refactor imageContexts into two different structs.
buildStages now tracks the imageID and runConfig for a build stage

imageMounter tracks image mounts so they can released when the build ends.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-10 17:59:30 -04:00
Daniel Nephin
ab3a037a5b Refactor interaction between dispatcher.from and dispatchState
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-10 17:58:57 -04:00
Daniel Nephin
b3bc7b28d0 Expose a smaller interface for the Builder retrieving images from daemon
Removes 3 methods from the builder.Backend interface
Remove the coupling between imageContexts, imageMounts and the builder.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-10 17:58:57 -04:00
Tõnis Tiigi
a3eebaf509 Merge pull request #33090 from tonistiigi/fix-release-mounts
builder: Fix releasing implicit mounts
2017-05-10 14:46:58 -07:00
Vincent Demeester
22a03192fb Merge pull request #33114 from dnephin/fix-builder-warn-on-unused-builtint
Fix warning for unused build args
2017-05-10 09:38:07 +02:00
Daniel Nephin
543dfde637 Merge pull request #32997 from dnephin/fix-builder-scratch-with-name
Fix a rare case where using FROM scratch as NAME would fail
2017-05-09 18:15:33 -04:00
Sebastiaan van Stijn
1a6f8a92b2 Merge pull request #32792 from rogaha/extra_prometheus_metrics
Add extra prometheus metrics
2017-05-09 21:48:34 +02:00
Daniel Nephin
72cc81ee8d Fix warning for unused build args.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-09 11:25:33 -04:00
Roberto Gandolfo Hashioka
a28b173a78 Add extra prometheus metrics
- buildsTriggered
 - buildsFailed
    - valid options:
        metricsDockerfileSyntaxError,
        metricsDockerfileEmptyError,
        metricsCommandNotSupportedError,
        metricsErrorProcessingCommandsError,
        metricsBuildTargetNotReachableError,
        metricsMissingOnbuildArgumentsError,
        metricsUnknownInstructionError,
        metricsBuildCanceled,
- engineInfo

Signed-off-by: Roberto Gandolfo Hashioka <roberto_hashioka@hotmail.com>
2017-05-09 01:04:40 -07:00
Tonis Tiigi
29efb93a19 builder: Fix releasing implicit mounts
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-05-08 10:32:20 -07:00
Ian Campbell
5894bc1abf Add docker build --iidfile=FILE
This is synonymous with `docker run --cidfile=FILE` and writes the digest of
the newly built image to the named file. This is intended to be used by build
systems which want to avoid tagging (perhaps because they are in CI or
otherwise want to avoid fixed names which can clash) by enabling e.g. Makefile
constructs like:

    image.id: Dockerfile
    	docker build --iidfile=image.id .

    do-some-more-stuff: image.id
    	do-stuff-with <image.id

Currently the only way to achieve this is to use `docker build -q` and capture
the stdout, but at the expense of losing the build output.

In non-silent mode (without `-q`) with API >= v1.29 the caller will now see a
`JSONMessage` with the `Aux` field containing a `types.BuildResult` in the
output stream for each image/layer produced during the build, with the final
one being the end product.  Having all of the intermediate images might be
interesting in some cases.

In silent mode (with `-q`) there is no change, on success the only output will
be the resulting image digest as it was previosuly.

There was no wrapper to just output an Aux section without enclosing it in a
Progress, so add one here.

Added some tests to integration cli tests.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-05-05 16:35:54 +01:00
Daniel Nephin
28379bd29a Fix a rare case where using FROM scracth as NAME would fail
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-05 11:14:03 -04:00
Daniel Nephin
2f0ebba0e7 Some refactoring of dispatch()
Remove runConfig from Builder and dispatchRequest. It is not only on
dispatchState.

Move dispatch state fields from Builder to dispatchState

Move stageName tracking to dispatchRequest.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-04 17:11:08 -04:00
Daniel Nephin
37be263826 Merge pull request #32749 from dnephin/rm-ContainerUpdateCmdOnBuild
[Builder] Remove ContainerUpdateCmdOnBuild
2017-05-03 14:02:08 -04:00
Aaron Lehmann
08d7fad45d Merge pull request #32949 from dnephin/refactor-streamformatter-and-progress
Refacator pkg/streamformatter
2017-05-03 09:05:26 -07:00
Daniel Nephin
d9371ee807 Fix run with entrypoint in base image
Update a test to use a base image with entrypoint to that the linux build
has at least one test that behaves like all the windows tests.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-03 10:59:38 -04:00
Daniel Nephin
97f8607164 Remove ContainerUpdateCmdOnBuild, it does nothing.
Set a blank entrypoint to preserve the old behaviour.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-03 10:59:38 -04:00
Daniel Nephin
3dcab28982 Fix setting b.runConfig.Image at arbitrary places.
Previously this value was set at some point attrbitrarily between when it was updated and when it was going to be used next.

Instead always set it as the last step of dispatch.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-03 10:59:37 -04:00
Daniel Nephin
c87d67b0ad Refacator pkg/streamformatter
StreamFormatter suffered was two distinct structs mixed into a single struct
without any overlap.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-02 17:38:12 -04:00
Daniel Nephin
9f738cc574 Cleanup all the mutate + defer revert of b.runConfig in the builder
Instead of mutating and reverting, just create a copy and pass the copy
around.

Add a unit test for builder dispatcher.run

Fix two test failures

Fix image history by adding a CreatedBy to commit options. Previously the
createdBy field was being created by modifying a reference to the runConfig that
was held from when the container was created.

Fix a test that expected a trailing slash. Previously the runConfig was being
modified by container create. Now that we're creating a copy of runConfig
instead of sharing a reference the runConfig retains the trailing slash.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-01 18:36:53 -04:00
Daniel Nephin
0296797f0f Extract squash and tagging from the Dockerfile builder.
Remove pathCache and replace it with syncmap
Cleanup NewBuilder
Create an api/server/backend/build
Extract BuildTagger

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-01 12:07:32 -04:00
Daniel Nephin
721e1736ae Fix cache miss when builtin build args are used.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-28 12:49:50 -04:00
Brian Goff
ec67cea96e Merge pull request #32772 from dnephin/refactor-builder-split-commit
[Builder] Refactor `Builder.commit()`
2017-04-28 11:18:55 -04:00
Brian Goff
9f0ebea335 Merge pull request #32763 from dave-tucker/fix-32744
builder: Make builtin arg pruning work with > 1 arg
2017-04-28 10:24:13 -04:00
Brian Goff
a7519152d9 Merge pull request #32504 from dongluochen/healthcheck_duration
do not allow duration less than 1 ms in healthcheck parameters
2017-04-27 23:54:00 -04:00
Daniel Nephin
0d9e66b98a Extract commitContainer from commit() to separate the distinct branches
Remove unused arguments to commit.
This will allow us to remove all the runConfig mutate+revert code that
is scattered around builder dispatchers/internals

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-27 20:49:38 -04:00
Daniel Nephin
aee2da3bdf Merge pull request #32858 from dnephin/builder-shell-words-interface
[Builder] Remove b.escapeToken, create ShellLex
2017-04-27 13:16:40 -04:00
Brian Goff
ae0f8c7ba1 Merge pull request #31984 from tonistiigi/remote-context
builder: Refactor remote context
2017-04-27 11:50:18 -04:00
Daniel Nephin
2414166e1e Create a new ShellLex struct which provides the expected interface to Builder.
Remove b.escapeToken from builder

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-26 19:39:16 -04:00
Tõnis Tiigi
5eca7f7c5d Merge pull request #31856 from jim-minter/more_races
Resolve a set of race conditions in logging and attaching code
2017-04-26 16:04:02 -07:00
Dave Tucker
1076ab58ec builder: Make builtin arg pruning work with > 1 arg
The previous implementation would error out with "Unexpected EOF" which
was caused by an underlying "array index out-of-bounds" error.
The root cause was deleting items from the same array that was being
iterated over. The iteration was unaware that the array size had
changed, resulting in an error.

The new implementation builds a new array instead of mutating a copy of
the old one.

Fixes: #32744

Signed-off-by: Dave Tucker <dt@docker.com>
2017-04-26 10:24:58 +01:00
Tonis Tiigi
d1faf3df27 Refactor remote context parsing
Redefine a better interface for remote context dependency.

Separate Dockerfile build instruction from remote context.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-25 14:35:31 -07:00
Daniel Nephin
52626bb919 Reduce the number of calls to b.image
In preparation for removing it from builder.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-24 14:21:19 -04:00
Daniel Nephin
e59327aaac Change builder.dispatchers to use a dispatchRequest object
This change starts the process of splitting up the Builder into logical
components. Remove builder.flags and move it to the new dispatchRequest
object.

Use runConfig from dispatchRequest instead of from the builder.

More progress removing things from the Builder struct.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-24 14:21:19 -04:00
Tõnis Tiigi
3beb8d9c8b Merge pull request #32500 from dnephin/refactor-from-dispatcher
Refactor from dispatcher
2017-04-24 10:10:42 -07:00
Jim Minter
32ca1214fa Fix race on ContainerAttachRaw
Signed-off-by: Jim Minter <jminter@redhat.com>
2017-04-20 12:27:50 +01:00
Jean-Pierre Huynh
2c435ab8be MAINTAINER deprecation follow up. Relates to #25466
Signed-off-by: Jean-Pierre Huynh <jean-pierre.huynh@ounet.fr>
2017-04-18 16:44:33 -05:00
Aaron Lehmann
6052f2b396 Remove pkg/testutil/assert in favor of testify
I noticed that we're using a homegrown package for assertions. The
functions are extremely similar to testify, but with enough slight
differences to be confusing (for example, Equal takes its arguments in a
different order). We already vendor testify, and it's used in a few
places by tests.

I also found some problems with pkg/testutil/assert. For example, the
NotNil function seems to be broken. It checks the argument against
"nil", which only works for an interface. If you pass in a nil map or
slice, the equality check will fail.

In the interest of avoiding NIH, I'm proposing replacing
pkg/testutil/assert with testify. The test code looks almost the same,
but we avoid the confusion of having two similar but slightly different
assertion packages, and having to maintain our own package instead of
using a commonly-used one.

In the process, I found a few places where the tests should halt if an
assertion fails, so I've made those cases (that I noticed) use "require"
instead of "assert", and I've vendored the "require" package from
testify alongside the already-present "assert" package.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-04-14 12:03:21 -07:00
Vincent Demeester
41f4c3cf7e Merge pull request #32601 from dnephin/builder-shell-words-buffer
Use a bytes.Buffer for shell_words string concat
2017-04-13 23:22:18 +02:00
Daniel Nephin
aafd7fa969 remove increment flag on imageContexts.new()
Rename the methods to match my understanding of the behaviour.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-13 14:56:57 -04:00
Daniel Nephin
4d62f67117 Extract two functions from the FROM dispatcher
This helps clarify the difference cases for parsing ctxName, and for
getting an image.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-13 14:56:57 -04:00
Vincent Demeester
700b4807c3 Merge pull request #32580 from dnephin/refactor-builder-parser-directive
Refactor Dockerfile.parser and directive
2017-04-13 20:47:42 +02:00
Daniel Nephin
0055a48277 Use a bytes.Buffer for shell_words string concat
It's much faster

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-13 14:39:34 -04:00
Daniel Nephin
df3c425407 Merge pull request #32328 from duglin/fixEnvVars
Minor tweaks to quotes in env vars
2017-04-13 10:41:28 -04:00
Dong Chen
d8b6a35d02 set 1ms as container duration minimum value
Signed-off-by: Dong Chen <dongluo.chen@docker.com>
2017-04-12 15:45:29 -07:00
Daniel Nephin
4cbc953a5d Refactor dockerfile parser
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-12 18:31:52 -04:00
Daniel Nephin
9c53fa2d0c Cleanup processing of directives.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-12 15:42:35 -04:00
Doug Davis
2fb7c3c4f0 Minor tweaks to quotes in env vars
Addresses part of #32140, in particular:
- this will make it so that double backslashes in double-quoted
strings will result in a single backslash. While in single quotes it remains
a double backslash.
- missing closing " and ' will now generate an error

Signed-off-by: Doug Davis <dug@us.ibm.com>
2017-04-12 12:20:14 -07:00
Daniel Nephin
64c4c1c3d5 Keep parser.Directive internal to parser
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-12 14:48:09 -04:00