Commit graph

105 commits

Author SHA1 Message Date
Tonis Tiigi
c6703b722e builder: fix long stream sync
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-11-03 12:53:17 -07:00
Simon Ferquel
172e73a1df Test & Fix build with rm/force-rm matrix
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2017-10-12 10:22:40 +02:00
John Howard
d98ecf2d6c LCOW: API change JSON header to string POST parameter
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-10-06 15:26:48 -07:00
John Howard
0380fbff37 LCOW: API: Add platform to /images/create and /build
Signed-off-by: John Howard <jhoward@microsoft.com>

This PR has the API changes described in https://github.com/moby/moby/issues/34617.
Specifically, it adds an HTTP header "X-Requested-Platform" which is a JSON-encoded
OCI Image-spec `Platform` structure.

In addition, it renames (almost all) uses of a string variable platform (and associated)
methods/functions to os. This makes it much clearer to disambiguate with the swarm
"platform" which is really os/arch. This is a stepping stone to getting the daemon towards
fully multi-platform/arch-aware, and makes it clear when "operating system" is being
referred to rather than "platform" which is misleadingly used - sometimes in the swarm
meaning, but more often as just the operating system.
2017-10-06 11:44:18 -07:00
Simon Ferquel
669c067798 Introduce a typed command system and 2 phase parse/dispatch build
This is a work base to introduce more features like build time
dockerfile optimisations, dependency analysis and parallel build, as
well as a first step to go from a dispatch-inline process to a
frontend+backend process.

Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2017-09-18 09:48:56 +02:00
Akash Gupta
7a7357dae1 LCOW: Implemented support for docker cp + build
This enables docker cp and ADD/COPY docker build support for LCOW.
Originally, the graphdriver.Get() interface returned a local path
to the container root filesystem. This does not work for LCOW, so
the Get() method now returns an interface that LCOW implements to
support copying to and from the container.

Signed-off-by: Akash Gupta <akagup@microsoft.com>
2017-09-14 12:07:52 -07:00
Brian Goff
ebcb7d6b40 Remove string checking in API error handling
Use strongly typed errors to set HTTP status codes.
Error interfaces are defined in the api/errors package and errors
returned from controllers are checked against these interfaces.

Errors can be wraeped in a pkg/errors.Causer, as long as somewhere in the
line of causes one of the interfaces is implemented. The special error
interfaces take precedence over Causer, meaning if both Causer and one
of the new error interfaces are implemented, the Causer is not
traversed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-08-15 16:01:11 -04:00
Derek McGowan
1009e6a40b
Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-31 13:16:46 -07:00
Tibor Vass
41445a4745 Remove client/session package, depend on buildkit's session package
gofmt -w -r '"github.com/docker/docker/client/session" -> "github.com/moby/buildkit/session"'
gofmt -w -r '"github.com/docker/docker/client/session/filesync" -> "github.com/moby/buildkit/session/filesync"'

Signed-off-by: Tibor Vass <tibor@docker.com>
2017-07-28 16:34:23 -07:00
Tonis Tiigi
ad46348d7c builder: enable path filtering for filesync session
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-28 09:12:30 -07:00
Daniel Nephin
b47b375cb8 Add a test for warning on empty continuation lines.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-23 13:15:04 -04:00
Tonis Tiigi
5c3d2d552b Implement incremental file sync using client session
Also exposes shared cache and garbage collection/prune
for the source data.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-22 11:52:35 -07:00
Tonis Tiigi
ec7b6238c3 Add long-running client session endpoint
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-06-22 11:22:41 -07:00
John Howard
ba40132366 LCOW: Fix ImageCache to address right store
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:49:53 -07:00
John Howard
08252bc963 LCOW: Builder plumbing
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:49:53 -07:00
Daniel Nephin
5136096520 Fix copy when used with scratch and images with empty RootFS
Commit the rwLayer to get the correct DiffID
Refacator copy in thebuilder
move more code into exportImage
cleanup some windows tests
Release the newly commited layer.
Set the imageID on the buildStage after exporting a new image.
Move archiver to BuildManager.
Have ReleaseableLayer.Commit return a layer
and store the Image from exportImage in the local imageSources cache
Remove NewChild from image interface.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-08 15:07:16 -04:00
Daniel Nephin
bd5f92d263 Remove CopyOnBuild from the daemon.
Add CreateImage() to the daemon
Refactor daemon.Comit() and expose a Image.NewChild()
Update copy to use IDMappings.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-08 15:06:54 -04:00
Daniel Nephin
3f26041577 Fix ONBUILD COPY
the source was missing from the second dispatch

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-02 17:47:03 -04:00
Daniel Nephin
19f3b0715c Extract imageProber and ContainerBackend from Builder
Extract a common function for builder.createContainer
Extract imageCache for doing cache probes
Removes the cacheBuested field from Builder
Create a new containerManager class which reduces the interface between the
builder and managing containers to 3 functions (from 6)

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-23 15:12:58 -04:00
Daniel Nephin
213ed02e25 Refactor builder.calcCopyInfo and dependencies
Remove pathCache from imageContexts
Extract validateCopySourcePath
Extract copyWithWildcards
Extract copyInfoForFile and walkSource from calcCopyInfo
Move copy internals to copy.go
remove source from Builder

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-17 09:54:39 -04:00
Daniel Nephin
6c28e8edd5 Refactor imageContexts into two different structs.
buildStages now tracks the imageID and runConfig for a build stage

imageMounter tracks image mounts so they can released when the build ends.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-10 17:59:30 -04:00
Daniel Nephin
b3bc7b28d0 Expose a smaller interface for the Builder retrieving images from daemon
Removes 3 methods from the builder.Backend interface
Remove the coupling between imageContexts, imageMounts and the builder.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-10 17:58:57 -04:00
Vincent Demeester
22a03192fb Merge pull request #33114 from dnephin/fix-builder-warn-on-unused-builtint
Fix warning for unused build args
2017-05-10 09:38:07 +02:00
Sebastiaan van Stijn
1a6f8a92b2 Merge pull request #32792 from rogaha/extra_prometheus_metrics
Add extra prometheus metrics
2017-05-09 21:48:34 +02:00
Daniel Nephin
72cc81ee8d Fix warning for unused build args.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-09 11:25:33 -04:00
Roberto Gandolfo Hashioka
a28b173a78 Add extra prometheus metrics
- buildsTriggered
 - buildsFailed
    - valid options:
        metricsDockerfileSyntaxError,
        metricsDockerfileEmptyError,
        metricsCommandNotSupportedError,
        metricsErrorProcessingCommandsError,
        metricsBuildTargetNotReachableError,
        metricsMissingOnbuildArgumentsError,
        metricsUnknownInstructionError,
        metricsBuildCanceled,
- engineInfo

Signed-off-by: Roberto Gandolfo Hashioka <roberto_hashioka@hotmail.com>
2017-05-09 01:04:40 -07:00
Ian Campbell
5894bc1abf Add docker build --iidfile=FILE
This is synonymous with `docker run --cidfile=FILE` and writes the digest of
the newly built image to the named file. This is intended to be used by build
systems which want to avoid tagging (perhaps because they are in CI or
otherwise want to avoid fixed names which can clash) by enabling e.g. Makefile
constructs like:

    image.id: Dockerfile
    	docker build --iidfile=image.id .

    do-some-more-stuff: image.id
    	do-stuff-with <image.id

Currently the only way to achieve this is to use `docker build -q` and capture
the stdout, but at the expense of losing the build output.

In non-silent mode (without `-q`) with API >= v1.29 the caller will now see a
`JSONMessage` with the `Aux` field containing a `types.BuildResult` in the
output stream for each image/layer produced during the build, with the final
one being the end product.  Having all of the intermediate images might be
interesting in some cases.

In silent mode (with `-q`) there is no change, on success the only output will
be the resulting image digest as it was previosuly.

There was no wrapper to just output an Aux section without enclosing it in a
Progress, so add one here.

Added some tests to integration cli tests.

Signed-off-by: Ian Campbell <ian.campbell@docker.com>
2017-05-05 16:35:54 +01:00
Daniel Nephin
2f0ebba0e7 Some refactoring of dispatch()
Remove runConfig from Builder and dispatchRequest. It is not only on
dispatchState.

Move dispatch state fields from Builder to dispatchState

Move stageName tracking to dispatchRequest.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-04 17:11:08 -04:00
Daniel Nephin
9f738cc574 Cleanup all the mutate + defer revert of b.runConfig in the builder
Instead of mutating and reverting, just create a copy and pass the copy
around.

Add a unit test for builder dispatcher.run

Fix two test failures

Fix image history by adding a CreatedBy to commit options. Previously the
createdBy field was being created by modifying a reference to the runConfig that
was held from when the container was created.

Fix a test that expected a trailing slash. Previously the runConfig was being
modified by container create. Now that we're creating a copy of runConfig
instead of sharing a reference the runConfig retains the trailing slash.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-01 18:36:53 -04:00
Daniel Nephin
0296797f0f Extract squash and tagging from the Dockerfile builder.
Remove pathCache and replace it with syncmap
Cleanup NewBuilder
Create an api/server/backend/build
Extract BuildTagger

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-01 12:07:32 -04:00
Daniel Nephin
aee2da3bdf Merge pull request #32858 from dnephin/builder-shell-words-interface
[Builder] Remove b.escapeToken, create ShellLex
2017-04-27 13:16:40 -04:00
Daniel Nephin
2414166e1e Create a new ShellLex struct which provides the expected interface to Builder.
Remove b.escapeToken from builder

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-26 19:39:16 -04:00
Tonis Tiigi
d1faf3df27 Refactor remote context parsing
Redefine a better interface for remote context dependency.

Separate Dockerfile build instruction from remote context.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-25 14:35:31 -07:00
Daniel Nephin
52626bb919 Reduce the number of calls to b.image
In preparation for removing it from builder.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-24 14:21:19 -04:00
Daniel Nephin
e59327aaac Change builder.dispatchers to use a dispatchRequest object
This change starts the process of splitting up the Builder into logical
components. Remove builder.flags and move it to the new dispatchRequest
object.

Use runConfig from dispatchRequest instead of from the builder.

More progress removing things from the Builder struct.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-24 14:21:19 -04:00
Daniel Nephin
64c4c1c3d5 Keep parser.Directive internal to parser
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-12 14:48:09 -04:00
Daniel Nephin
bb429da9a9 Hide builder.parser.Directive internals
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-12 14:48:09 -04:00
Daniel Nephin
068f344e03 Remove unused Builder.Cancel()
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-11 20:33:13 -04:00
Daniel Nephin
a6abd57b83 Remove unused id field from Builder.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-11 14:44:32 -04:00
Daniel Nephin
bfcd95817a Factor out functions from builder/dockerfile/builder.go:Builder.build()
Remove the block comment which is stale, and redundant now that the
function is just as readable as the comment.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-11 14:44:32 -04:00
Daniel Nephin
c7fad9b750 Cleanup in dispatcher.env
Remove commented code blocks
Remove some duplication in comparing and restructuring env

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-11 14:44:28 -04:00
Akihiro Suda
b764c471bf Merge pull request #32495 from dnephin/cleanup-builder-dispatcher-env
Remove Builder.dockerfile field
2017-04-11 14:50:55 +09:00
Tonis Tiigi
33e07f41ad builder: add an option for specifying build target
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-04-10 16:21:43 -07:00
Daniel Nephin
f3e205ddc1 Remove the dockerfile field from Builder.
Return dockerfile from parseDockerfile and pass the dockerfile nodes
as an arg

Strip unused arg from builder.NewBuilder.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-10 18:09:17 -04:00
Daniel Nephin
239c53bf83 Refactor BuildArgs
Add MetaArgs for ARG that occur before the first FROM
Integration test for these cases.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-07 17:57:47 -04:00
Daniel Nephin
f0a9c2e3f4 Allow ARG to come before FROM to support variables in FROM.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-07 17:57:47 -04:00
Tõnis Tiigi
6abbc933ae Merge pull request #31750 from dnephin/some-builder-cleanup
Fix `docker build --label` when the label includes single quotes and a space
2017-04-04 09:37:00 -07:00
Alexandre Garnier
d005219830 Display created tags on successful build
Signed-off-by: Alexandre Garnier <alexandre.garnier@zenika.com>
Signed-off-by: Alexandre Garnier <zigarn@gmail.com>
2017-03-25 11:46:20 +01:00
Daniel Nephin
726fb269cf Fix --label on docker build when a single quote is used in the value
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-03-24 12:29:14 -04:00
Tõnis Tiigi
2fa8fe400d Merge pull request #31257 from tonistiigi/nested-build
build: add multi-stage build support
2017-03-23 19:14:13 -07:00