Closes#17781
This allows for env vars in EXPOSE to be parsed for spaces so that each
"word" is then treated independently instead of as a single word/arg.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Fixes#17290
Fixes following issues:
- Cache checksums turning off while walking a broken symlink.
- Cache checksums were taken from symlinks while targets were actually copied.
- Copying a symlink pointing to a file to a directory used the basename of the target as a destination basename, instead of basename of the symlink.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
The LXC driver was deprecated in Docker 1.8.
Following the deprecation rules, we can remove a deprecated feature
after two major releases. LXC won't be supported anymore starting on Docker 1.10.
Signed-off-by: David Calavera <david.calavera@gmail.com>
Side effects:
- Decouple daemon and container to start containers.
- Decouple daemon and container to copy files.
Signed-off-by: David Calavera <david.calavera@gmail.com>
This patch creates interfaces in builder/ for building Docker images.
It is a first step in a series of patches to remove the daemon
dependency on builder and later allow a client-side Dockerfile builder
as well as potential builder plugins.
It is needed because we cannot remove the /build API endpoint, so we
need to keep the server-side Dockerfile builder, but we also want to
reuse the same Dockerfile parser and evaluator for both server-side and
client-side.
builder/dockerfile/ and api/server/builder.go contain implementations
of those interfaces as a refactoring of the current code.
Signed-off-by: Tibor Vass <tibor@docker.com>
Although having a request ID available throughout the codebase is very
valuable, the impact of requiring a Context as an argument to every
function in the codepath of an API request, is too significant and was
not properly understood at the time of the review.
Furthermore, mixing API-layer code with non-API-layer code makes the
latter usable only by API-layer code (one that has a notion of Context).
This reverts commit de41640435, reversing
changes made to 7daeecd42d.
Signed-off-by: Tibor Vass <tibor@docker.com>
Conflicts:
api/server/container.go
builder/internals.go
daemon/container_unix.go
daemon/create.go
This reverts commit ff92f45be4, reversing
changes made to 80e31df3b6.
Reverting to make the next revert easier.
Signed-off-by: Tibor Vass <tibor@docker.com>
Avoid creating a global context object that will be used while the daemon is running.
Not only this object won't ever be garbage collected, but it won't ever be used for anything else than creating other contexts in each request. I think it's a bad practive to have something like this sprawling aroud the code.
This change removes that global object and initializes a context in the cases we don't have already one, like shutting down the server.
This also removes a bunch of context arguments from functions that did nothing with it.
Signed-off-by: David Calavera <david.calavera@gmail.com>
This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:
```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.
Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.
I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.
ping @icecrime @calavera @crosbymichael
Signed-off-by: Doug Davis <dug@us.ibm.com>
- The build-time variables are passed as environment-context for command(s)
run as part of the RUN primitve. These variables are not persisted in environment of
intermediate and final images when passed as context for RUN. The build environment
is prepended to the intermediate continer's command string for aiding cache lookups.
It also helps with build traceability. But this also makes the feature less secure from
point of view of passing build time secrets.
- The build-time variables also get used to expand the symbols used in certain
Dockerfile primitves like ADD, COPY, USER etc, without an explicit prior definiton using a
ENV primitive. These variables get persisted in the intermediate and final images
whenever they are expanded.
- The build-time variables are only expanded or passed to the RUN primtive if they
are defined in Dockerfile using the ARG primitive or belong to list of built-in variables.
HTTP_PROXY, HTTPS_PROXY, http_proxy, https_proxy, FTP_PROXY and NO_PROXY are built-in
variables that needn't be explicitly defined in Dockerfile to use this feature.
Signed-off-by: Madhav Puri <madhav.puri@gmail.com>
This is the first step in converting out static strings into well-defined
error types. This shows just a few examples of it to get a feel for how things
will look. Once we agree on the basic outline we can then work on converting
the rest of the code over.
Signed-off-by: Doug Davis <dug@us.ibm.com>
In particular I want to make sure that calling getEnv() when the same
var name appears more than once in the env list that we only pick up
the first one. PR #15182 counts on this
Signed-off-by: Doug Davis <dug@us.ibm.com>
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Windows: add support for images stored in alternate location.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Some structures use int for sizes and UNIX timestamps. On some
platforms, int is 32 bits, so this can lead to the year 2038 issues and
overflows when dealing with large containers or layers.
Consistently use int64 to store sizes and UNIX timestamps in
api/types/types.go. Update related to code accordingly (i.e.
strconv.FormatInt instead of strconv.Itoa).
Use int64 in progressreader package to avoid integer overflow when
dealing with large quantities. Update related code accordingly.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
If there's an error while unpacking the build context then we weren't erasing
the tmp dir created to persist the context.
Signed-off-by: Doug Davis <dug@us.ibm.com>
It was missing some variants and 'maintainer' isn't actually supported.
Also sorted the list of allowed cmds in the code just to make it easier
to diff with the docs.
Signed-off-by: Doug Davis <dug@us.ibm.com>
By convention /pkg is safe to use from outside the docker tree, for example
if you're building a docker orchestrator.
/nat currently doesn't have any dependencies outside of /pkg, so it seems
reasonable to move it there.
This rename was performed with:
```
gomvpkg -vcs_mv_cmd="git mv {{.Src}} {{.Dst}}" \
-from github.com/docker/docker/nat \
-to github.com/docker/docker/pkg/nat
```
Signed-off-by: Peter Waller <p@pwaller.net>
With the 1.7 release, we introduced a change to how we store registry
credentials, but the build API endpoint did not expect a change in the format
of that file. This patch fixes this problem so that you can again pull private
images during `docker build`.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Now that the default network mode is "default" and this mode is chosen
even if the mode is empty string, it is not correct to have builder
still pointing to "bridge" as default (though this is daemon default).
Signed-off-by: Madhu Venugopal <madhu@docker.com>
Build cache uses pgk/tarsum to get a digest of content which is
ADD'd or COPY'd during a build. The builder has always used v0 of
the tarsum algorithm which includes mtimes however since the whole
file is hashed anyway, the mtime doesn't really provide any extra
information about whether the file has changed and many version
control tools like Git strip mtime from files when they are cloned.
This patch updates the build subsystem to use v1 of Tarsum which
explicitly ignores mtime when calculating a digest. Now ADD and
COPY will result in a cache hit if only the mtime and not the file
contents have changed.
NOTE: Tarsum is NOT a meant to be a cryptographically secure hash
function. It is a best-effort approach to determining if two sets of
filesystem content are different.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Signed by all authors:
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
Signed-off-by: David Calavera <david.calavera@gmail.com>
Signed-off-by: Jeff Lindsay <progrium@gmail.com>
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
Signed-off-by: Luke Marsden <luke@clusterhq.com>
Signed-off-by: David Calavera <david.calavera@gmail.com>
Previous fix used %q which incorrectly go-escaped things. For example:
```
RUN echoo A \& B C
```
would result in the user seeing:
```
INFO[0000] The command '/bin/sh -c echoo A \\& B\tC' returned a non-zero code: 127
```
Note the double-\ and the \t instead of a tab character
The testcase had to double escape things due to logrus getting in the way
but I'm going to fix that in another PR because its a change to the UX.
Signed-off-by: Doug Davis <dug@us.ibm.com>
When RUN returns with a non-zero return code it prints the command
that was executed as a Go []string:
```
INFO[0000] The command &{[/bin/sh -c noop a1 a2]} returned a non-zero code: 127
```
instead it should look like this:
```
INFO[0000] The command "/bin/sh -c noop a1 a2" returned a non-zero code: 127
```
Signed-off-by: Doug Davis <dug@us.ibm.com>
This fixes an issue where the build output for the "Steps" would look like:
```
Step 1: RUN echo hi echo hi
```
instead of
```
Step 1: RUN echo hi
```
Also, I noticed that there were no checks to make sure invalid Dockerfile
cmd flags were caught on cmds that didn't use cmd flags at all. They would
have been caught on the cmds that had flags, but cmds that didn't bother
to add a new code for flags would have just ignored them. So, I added
checks to each cmd to flag it.
Added testcases for issues.
Signed-off-by: Doug Davis <dug@us.ibm.com>
This adds support for Dockerfile commands to have options - e.g:
COPY --user=john foo /tmp/
COPY --ignore-mtime foo /tmp/
Supports both booleans and strings.
Signed-off-by: Doug Davis <dug@us.ibm.com>
This only happens with the old git http dumb protocol, but that's what we use in our integration tests.
We check the Content-Type header advertised in http requests to make sure the http transport is the git smart transport:
See this commit as a reference:
4656bf47fc
Signed-off-by: David Calavera <david.calavera@gmail.com>
Several parts of the codebase didn't use the correct path sanitisation
wrappers. Now that the wrappers have been exposed, use those.
Signed-off-by: Aleksa Sarai <cyphar@cyphar.com> (github: cyphar)
This PR does the following:
- migrated ~/.dockerfg to ~/.docker/config.json. The data is migrated
but the old file remains in case its needed
- moves the auth json in that fie into an "auth" property so we can add new
top-level properties w/o messing with the auth stuff
- adds support for an HttpHeaders property in ~/.docker/config.json
which adds these http headers to all msgs from the cli
In a follow-on PR I'll move the config file process out from under
"registry" since it not specific to that any more. I didn't do it here
because I wanted the diff to be smaller so people can make sure I didn't
break/miss any auth code during my edits.
Signed-off-by: Doug Davis <dug@us.ibm.com>
This makes `registry.Service` a first class type and does not use jobs
to interact with this type.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Add the capability to cancel the build by disconnecting the client.
This adds a `cancelled` channel which is used to signal that a build
should halt. The build is halted by sending a Kill signal and noticing
that the cancellation channel is closed.
This first pass implementation does not allow cancellation during a
pull, but that will come in a subsequent PR.
* Add documentation of cancellation to cli and API
* Protect job cancellation with sync.Once
* Add TestBuildCancelationKillsSleep
* Add test case for build cancellation of RUN statements.
Signed-off-by: Peter Waller <p@pwaller.net>
With this Dockerfile
```
FROM ubuntu
ENV ABC=""
VOLUME $ABC
```
It builds ok but then at run time I get this error:
FATA[0002] Error response from daemon: Cannot start container 8902b4a7aaf5c4e4b11a38070d392db465fa97ad88c91c8b38dda5ab8149ccac: [8] System error: no such file or directory
Because the Volume config shows "" as the path. This PR checks for "" as
the path and stops it at build time.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Closes#10191
Allow `docker build` to set --cpu-shares, --cpuset, --memory,
--memory-swap for all containers created by the build.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
- command.Commands was missing "Label"
- used the correct error string in dispatcher when LABEL has no args, otherwise
the test TestBuildMissingArgs will not work
- removed the premature error msg in line_parser that was blocking the
label() func in dispatcher from showing the err msg in previous bullet
- since LABEL uses the env parser it needs to be added to the replaceEnvAllowed
list so that proper quote processing will be done. Especially once
PR #10431 is merged.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Instead of just printing a warning and going on, this will generate
an error and stop processing.
This used to be part of #10561 but I decided it might need its own
independent discussion/PR as to not derail #10561.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Save "LABEL" field in Dockerfile into image content.
This will allow a user to save user data into an image, which
can later be retrieved using:
docker inspect IMAGEID
I have copied this from the "Comment" handling in docker images.
We want to be able to add Name/Value data to an image to describe the image,
and then be able to use other tools to look at this data, to be able to do
security checks based on this data.
We are thinking about adding version names,
Perhaps listing the content of the dockerfile.
Descriptions of where the code came from etc.
This LABEL field should also be allowed to be specified in the
docker import --change LABEL:Name=Value
docker commit --change LABEL:Name=Value
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
FROM scratch
ADD echo /
RUN [ "echo" ]
will die with
exec: "/bin/sh": stat /bin/sh: no such file or directory
Signed-off-by: Doug Davis <dug@us.ibm.com>
parser_test only needed the directory-names for
the tests to run. This replaces f.Readdir() with
f.Readdirnames() to only return the names.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We could use EXPOSE ip:hostPort:containerPort,
but actually it did as EXPOSE ::containerPort
commit 2275c833 already warned user on daemon side.
This patch will print warning message on client side.
Signed-off-by: Chen Hanxiao <chenhanxiao@cn.fujitsu.com>
Closes#10807
Adds support for `dockerfile` ONLY when `Dockerfile` can't be found.
If we're building from a Dockerfile via stdin/URL then always download
it a `Dockerfile` and ignore the -f flag.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Instead of building the actual image, `build_config` will serialize a subset of
dockerfile ast into *runconfig.Config
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
`docker build -q .` where Dockerfile contains a RUN cmd will hang on the
RUN. It waits for the output stream to close but because of -q we never
attached to the container and end up waiting forever.
The fact that no one noticed this tells me that people may not actually
use -q and if so I wonder if it would make sense to make -q work the may
it does for other commands (like `docker ps`) and make it so it only
shows the container ID at the end. A -q/quiet option that only hides the
container RUN output apparently isn't really that useful since no one is
using it. See: https://github.com/docker/docker/issues/4094
Signed-off-by: Doug Davis <dug@us.ibm.com>
The most obvious use case is when one wants to make sure as fast
as possible that a command is a valid Dockerfile command.
Signed-off-by: kargakis <kargakis@users.noreply.github.com>
d1e9d07c introduces a dependency to libcontainer and other daemon
related packages through builder package. The only thing test needs
is set of the Dockerfile commands. Extracting them to a separate
package.
This was causing CI tests to not to compile on non-Linux platforms.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
