Commit graph

130 commits

Author SHA1 Message Date
Roberto Gandolfo Hashioka
a28b173a78 Add extra prometheus metrics
- buildsTriggered
 - buildsFailed
    - valid options:
        metricsDockerfileSyntaxError,
        metricsDockerfileEmptyError,
        metricsCommandNotSupportedError,
        metricsErrorProcessingCommandsError,
        metricsBuildTargetNotReachableError,
        metricsMissingOnbuildArgumentsError,
        metricsUnknownInstructionError,
        metricsBuildCanceled,
- engineInfo

Signed-off-by: Roberto Gandolfo Hashioka <roberto_hashioka@hotmail.com>
2017-05-09 01:04:40 -07:00
Ian Campbell
5894bc1abf Add docker build --iidfile=FILE
This is synonymous with `docker run --cidfile=FILE` and writes the digest of
the newly built image to the named file. This is intended to be used by build
systems which want to avoid tagging (perhaps because they are in CI or
otherwise want to avoid fixed names which can clash) by enabling e.g. Makefile
constructs like:

    image.id: Dockerfile
    	docker build --iidfile=image.id .

    do-some-more-stuff: image.id
    	do-stuff-with <image.id

Currently the only way to achieve this is to use `docker build -q` and capture
the stdout, but at the expense of losing the build output.

In non-silent mode (without `-q`) with API >= v1.29 the caller will now see a
`JSONMessage` with the `Aux` field containing a `types.BuildResult` in the
output stream for each image/layer produced during the build, with the final
one being the end product.  Having all of the intermediate images might be
interesting in some cases.

In silent mode (with `-q`) there is no change, on success the only output will
be the resulting image digest as it was previosuly.

There was no wrapper to just output an Aux section without enclosing it in a
Progress, so add one here.

Added some tests to integration cli tests.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-05-05 16:35:54 +01:00
Daniel Nephin
2f0ebba0e7 Some refactoring of dispatch()
Remove runConfig from Builder and dispatchRequest. It is not only on
dispatchState.

Move dispatch state fields from Builder to dispatchState

Move stageName tracking to dispatchRequest.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-04 17:11:08 -04:00
Daniel Nephin
9f738cc574 Cleanup all the mutate + defer revert of b.runConfig in the builder
Instead of mutating and reverting, just create a copy and pass the copy
around.

Add a unit test for builder dispatcher.run

Fix two test failures

Fix image history by adding a CreatedBy to commit options. Previously the
createdBy field was being created by modifying a reference to the runConfig that
was held from when the container was created.

Fix a test that expected a trailing slash. Previously the runConfig was being
modified by container create. Now that we're creating a copy of runConfig
instead of sharing a reference the runConfig retains the trailing slash.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-01 18:36:53 -04:00
Daniel Nephin
0296797f0f Extract squash and tagging from the Dockerfile builder.
Remove pathCache and replace it with syncmap
Cleanup NewBuilder
Create an api/server/backend/build
Extract BuildTagger

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-01 12:07:32 -04:00
Daniel Nephin
aee2da3bdf Merge pull request #32858 from dnephin/builder-shell-words-interface
[Builder] Remove b.escapeToken, create ShellLex
2017-04-27 13:16:40 -04:00
Daniel Nephin
2414166e1e Create a new ShellLex struct which provides the expected interface to Builder.
Remove b.escapeToken from builder

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-26 19:39:16 -04:00
Tonis Tiigi
d1faf3df27 Refactor remote context parsing
Redefine a better interface for remote context dependency.

Separate Dockerfile build instruction from remote context.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-25 14:35:31 -07:00
Daniel Nephin
52626bb919 Reduce the number of calls to b.image
In preparation for removing it from builder.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-24 14:21:19 -04:00
Daniel Nephin
e59327aaac Change builder.dispatchers to use a dispatchRequest object
This change starts the process of splitting up the Builder into logical
components. Remove builder.flags and move it to the new dispatchRequest
object.

Use runConfig from dispatchRequest instead of from the builder.

More progress removing things from the Builder struct.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-24 14:21:19 -04:00
Daniel Nephin
64c4c1c3d5 Keep parser.Directive internal to parser
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-12 14:48:09 -04:00
Daniel Nephin
bb429da9a9 Hide builder.parser.Directive internals
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-12 14:48:09 -04:00
Daniel Nephin
068f344e03 Remove unused Builder.Cancel()
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-11 20:33:13 -04:00
Daniel Nephin
a6abd57b83 Remove unused id field from Builder.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-11 14:44:32 -04:00
Daniel Nephin
bfcd95817a Factor out functions from builder/dockerfile/builder.go:Builder.build()
Remove the block comment which is stale, and redundant now that the
function is just as readable as the comment.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-11 14:44:32 -04:00
Daniel Nephin
c7fad9b750 Cleanup in dispatcher.env
Remove commented code blocks
Remove some duplication in comparing and restructuring env

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-11 14:44:28 -04:00
Akihiro Suda
b764c471bf Merge pull request #32495 from dnephin/cleanup-builder-dispatcher-env
Remove Builder.dockerfile field
2017-04-11 14:50:55 +09:00
Tonis Tiigi
33e07f41ad builder: add an option for specifying build target
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-10 16:21:43 -07:00
Daniel Nephin
f3e205ddc1 Remove the dockerfile field from Builder.
Return dockerfile from parseDockerfile and pass the dockerfile nodes
as an arg

Strip unused arg from builder.NewBuilder.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-10 18:09:17 -04:00
Daniel Nephin
239c53bf83 Refactor BuildArgs
Add MetaArgs for ARG that occur before the first FROM
Integration test for these cases.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-07 17:57:47 -04:00
Daniel Nephin
f0a9c2e3f4 Allow ARG to come before FROM to support variables in FROM.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-07 17:57:47 -04:00
Tõnis Tiigi
6abbc933ae Merge pull request #31750 from dnephin/some-builder-cleanup
Fix `docker build --label` when the label includes single quotes and a space
2017-04-04 09:37:00 -07:00
Alexandre Garnier
d005219830 Display created tags on successful build
Signed-off-by: Alexandre Garnier <alexandre.garnier@zenika.com>
Signed-off-by: Alexandre Garnier <zigarn@gmail.com>
2017-03-25 11:46:20 +01:00
Daniel Nephin
726fb269cf Fix --label on docker build when a single quote is used in the value
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-03-24 12:29:14 -04:00
Tõnis Tiigi
2fa8fe400d Merge pull request #31257 from tonistiigi/nested-build
build: add multi-stage build support
2017-03-23 19:14:13 -07:00
Tonis Tiigi
f95f58283b Add support for COPY from previous rootfs
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-03-23 15:12:15 -07:00
Dave Tucker
89a2a885c8 Ignore built-in allowed build-args in image history
Removes the build-args from the image history if they are in the
BuiltinAllowedBuildArgs map unless they are explicitly defined in an ARG
instruction.

Signed-off-by: Dave Tucker <dt@docker.com>
2017-03-21 16:36:48 +00:00
Tonis Tiigi
acad599210 Fix cache for dockerfiles with multiple FROM
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-03-20 11:24:50 -07:00
Tonis Tiigi
09f308ce21 Fix ARG scoping for Dockerfiles with multiple FROM
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-03-20 11:24:44 -07:00
Daniel Nephin
9ea5e02e92 extract a function
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-03-08 11:46:09 -05:00
Vincent Demeester
254fc83cba Merge pull request #30457 from dmcgowan/distribution-reference-update-2
reference: use distribution reference and remove fork
2017-02-07 22:01:25 +01:00
Derek McGowan
3a1279393f
Use distribution reference
Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.

Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-07 11:08:37 -08:00
Daniel Nephin
80b642ff88 Ignore the daemon log config when building images.
Logs created by build containers should be handled by the daemon, not by logging drivers.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-01-30 10:59:19 -05:00
Boaz Shuster
8f282cdea5 Specify in which line the Dockerfile parser failed
Signed-off-by: Boaz Shuster <ripcurld.github@gmail.com>
2017-01-12 17:45:45 +02:00
Sebastiaan van Stijn
d6be0e9802 Merge pull request #28631 from likel/master
Don't do format if it's unnecessary
2016-12-30 01:13:12 +01:00
Ke Li
514adcf458 Remove redundant format
Signed-off-by: Ke Li <kel@splunk.com>

Add missing changes

Signed-off-by: Ke Li <kel@splunk.com>

User errors.New to create error

Signed-off-by: Ke Li <kel@splunk.com>
2016-12-27 21:46:52 +08:00
Yong Tang
d32efdbf8b Sort the build labels passed from build --labels
This fix tries to fix the issue in 29619 where
labels passed from `build --labels` are not sorted.
As a result, if multiple labels have been passed,
each `docker build --labels A=A --labels B=B --labels C=C`
will generate different layers.

This fix fixes the issue by sort the Labels before
they are concatenated to `LABEL ...`.

A unit test has been added to cover the changes

This fix fixes 29619.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-12-22 07:32:52 -08:00
Doug Davis
cdb8ea90b0 Fix processing of unset build-args during build
This reverts 26103.  26103 was trying to make it so that if someone did:
  docker build --build-arg FOO .
and FOO wasn't set as an env var then it would pick-up FOO from the
Dockerfile's ARG cmd.  However, it went too far and removed the ability
to specify a build arg w/o any value. Meaning it required the --build-arg
param to always be in the form "name=value", and not just "name".

This PR does the right fix - it allows just "name" and it'll grab the value
from the env vars if set. If "name" isn't set in the env then it still needs
to send "name" to the server so that a warning can be printed about an
unused --build-arg. And this is why buildArgs in the options is now a
*string instead of just a string - 'nil' == mentioned but no value.

Closes #29084

Signed-off-by: Doug Davis <dug@us.ibm.com>
2016-12-07 07:41:55 -08:00
Doug Davis
eaf0b5708f Fix case where \\ at EOF made the builder ignore the command
Came from looking at issue #27545

Signed-off-by: Doug Davis <dug@us.ibm.com>
2016-11-03 05:48:49 -07:00
Michael Crosby
22f3e43e9a Merge pull request #22641 from cpuguy83/build_finalization
Adds ability to flatten image after build
2016-11-01 14:30:18 -07:00
Brian Goff
362369b4bb Adds ability to squash image after build
Allow built images to be squash to scratch.
Squashing does not destroy any images or layers, and preserves the
build cache.

Introduce a new CLI argument --squash to docker build
Introduce a new param to the build API endpoint `squash`

Once the build is complete, docker creates a new image loading the diffs
from each layer into a single new layer and references all the parent's
layers.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-11-01 13:41:59 -04:00
Addam Hardy
f150f42009 Convert Unused ARG error to warning
Signed-off-by: Addam Hardy <addam.hardy@gmail.com>
2016-10-31 21:42:50 -05:00
Doug Davis
28a9a7deb0 Don't env-var evaluate labels from docker build cmd line
Fixes #26027

Signed-off-by: Doug Davis <dug@us.ibm.com>
2016-10-05 18:17:02 -07:00
Tõnis Tiigi
690882c2e7 Implement build cache based on history array
Based on work by KJ Tsanaktsidis

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Signed-off-by: KJ Tsanaktsidis <kjtsanaktsidis@gmail.com>
2016-09-23 11:30:06 -07:00
Yong Tang
c8dc2b156a Check bad syntax on dockerfile before building.
This fix tries to address the issue raised in 26453 where bad syntax
on dockerfile is not checked before building, thus user has to wait
before seeing error in dockerfile.

This fix fixes the issue by evaluating all the instructions and check
syntax before dockerfile is invoked actually.

All existing tests pass.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-09-13 21:43:10 -07:00
Michael Crosby
91e197d614 Add engine-api types to docker
This moves the types for the `engine-api` repo to the existing types
package.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-09-07 11:05:58 -07:00
Doug Davis
282b0aff08 Merge pull request #24978 from yongtang/24912-build-with-progress
Add hint of progress to the output of `docker build`
2016-08-18 16:10:48 -04:00
John Howard
755be795b4 Move directive out of globals
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-07-26 10:35:33 -07:00
Yong Tang
35418c1455 Add hint of progress to the output of docker build
This fix tries to address the issue raised in 24912 where docker
build only consists of the current step without overall total steps.

This fix adds the overall total steps so that end user could follow
the progress of the docker build.

An additonal test has been added to cover the changes.

This fix fixes 24912.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-07-24 08:13:58 -07:00
Thomas Leonard
b6c7becbfe
Add support for user-defined healthchecks
This PR adds support for user-defined health-check probes for Docker
containers. It adds a `HEALTHCHECK` instruction to the Dockerfile syntax plus
some corresponding "docker run" options. It can be used with a restart policy
to automatically restart a container if the check fails.

The `HEALTHCHECK` instruction has two forms:

* `HEALTHCHECK [OPTIONS] CMD command` (check container health by running a command inside the container)
* `HEALTHCHECK NONE` (disable any healthcheck inherited from the base image)

The `HEALTHCHECK` instruction tells Docker how to test a container to check that
it is still working. This can detect cases such as a web server that is stuck in
an infinite loop and unable to handle new connections, even though the server
process is still running.

When a container has a healthcheck specified, it has a _health status_ in
addition to its normal status. This status is initially `starting`. Whenever a
health check passes, it becomes `healthy` (whatever state it was previously in).
After a certain number of consecutive failures, it becomes `unhealthy`.

The options that can appear before `CMD` are:

* `--interval=DURATION` (default: `30s`)
* `--timeout=DURATION` (default: `30s`)
* `--retries=N` (default: `1`)

The health check will first run **interval** seconds after the container is
started, and then again **interval** seconds after each previous check completes.

If a single run of the check takes longer than **timeout** seconds then the check
is considered to have failed.

It takes **retries** consecutive failures of the health check for the container
to be considered `unhealthy`.

There can only be one `HEALTHCHECK` instruction in a Dockerfile. If you list
more than one then only the last `HEALTHCHECK` will take effect.

The command after the `CMD` keyword can be either a shell command (e.g. `HEALTHCHECK
CMD /bin/check-running`) or an _exec_ array (as with other Dockerfile commands;
see e.g. `ENTRYPOINT` for details).

The command's exit status indicates the health status of the container.
The possible values are:

- 0: success - the container is healthy and ready for use
- 1: unhealthy - the container is not working correctly
- 2: starting - the container is not ready for use yet, but is working correctly

If the probe returns 2 ("starting") when the container has already moved out of the
"starting" state then it is treated as "unhealthy" instead.

For example, to check every five minutes or so that a web-server is able to
serve the site's main page within three seconds:

    HEALTHCHECK --interval=5m --timeout=3s \
      CMD curl -f http://localhost/ || exit 1

To help debug failing probes, any output text (UTF-8 encoded) that the command writes
on stdout or stderr will be stored in the health status and can be queried with
`docker inspect`. Such output should be kept short (only the first 4096 bytes
are stored currently).

When the health status of a container changes, a `health_status` event is
generated with the new status. The health status is also displayed in the
`docker ps` output.

Signed-off-by: Thomas Leonard <thomas.leonard@docker.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2016-06-02 23:58:34 +02:00