Commit graph

57 commits

Author SHA1 Message Date
David Calavera
27c76522de Define a context per request.
Avoid creating a global context object that will be used while the daemon is running.

Not only this object won't ever be garbage collected, but it won't ever be used for anything else than creating other contexts in each request. I think it's a bad practive to have something like this sprawling aroud the code.

This change removes that global object and initializes a context in the cases we don't have already one, like shutting down the server.
This also removes a bunch of context arguments from functions that did nothing with it.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-24 18:44:53 -04:00
Doug Davis
26b1064967 Add context.RequestID to event stream
This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:

```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.

Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.

I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.

ping @icecrime @calavera @crosbymichael

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-24 11:56:37 -07:00
Madhav Puri
54240f8da9 Support for passing build-time variables in build context
- The build-time variables are passed as environment-context for command(s)
run as part of the RUN primitve. These variables are not persisted in environment of
intermediate and final images when passed as context for RUN. The build environment
is prepended to the intermediate continer's command string for aiding cache lookups.
It also helps with build traceability. But this also makes the feature less secure from
point of view of passing build time secrets.

- The build-time variables also get used to expand the symbols used in certain
Dockerfile primitves like ADD, COPY, USER etc, without an explicit prior definiton using a
ENV primitive. These variables get persisted in the intermediate and final images
whenever they are expanded.

- The build-time variables are only expanded or passed to the RUN primtive if they
are defined in Dockerfile using the ARG primitive or belong to list of built-in variables.
HTTP_PROXY, HTTPS_PROXY, http_proxy, https_proxy, FTP_PROXY and NO_PROXY are built-in
variables that needn't be explicitly defined in Dockerfile to use this feature.

Signed-off-by: Madhav Puri <madhav.puri@gmail.com>
2015-09-16 03:31:15 -07:00
John Howard
4b933dd2ea Windows: Stop commit on running container
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-08-13 22:17:03 -07:00
Aaron Lehmann
1f61084d83 Fix uses of "int" where "int64" should be used instead
Some structures use int for sizes and UNIX timestamps. On some
platforms, int is 32 bits, so this can lead to the year 2038 issues and
overflows when dealing with large containers or layers.

Consistently use int64 to store sizes and UNIX timestamps in
api/types/types.go. Update related to code accordingly (i.e.
strconv.FormatInt instead of strconv.Itoa).

Use int64 in progressreader package to avoid integer overflow when
dealing with large quantities. Update related code accordingly.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-07-31 16:31:40 -07:00
Vincent Demeester
18c7c67308 Lint on pkg/* packages
- pkg/useragent
- pkg/units
- pkg/ulimit
- pkg/truncindex
- pkg/timeoutconn
- pkg/term
- pkg/tarsum
- pkg/tailfile
- pkg/systemd
- pkg/stringutils
- pkg/stringid
- pkg/streamformatter
- pkg/sockets
- pkg/signal
- pkg/proxy
- pkg/progressreader
- pkg/pools
- pkg/plugins
- pkg/pidfile
- pkg/parsers
- pkg/parsers/filters
- pkg/parsers/kernel
- pkg/parsers/operatingsystem

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-07-27 21:26:21 +02:00
Lei
877dbbbde8 Add ulimit to docker build.
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-07-23 10:26:06 +08:00
Qiang Huang
8c4a282a57 Fix golint warnings for builder
Addresses: #14756

Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2015-07-22 13:29:03 +08:00
Ma Shimiao
1b67c38f6f fix 8926: rmi dangling is unsafe when pulling
Signed-off-by: Ma Shimiao <mashimiao.fnst@cn.fujitsu.com>
Signed-off-by: Tibor Vass <tibor@docker.com>
2015-07-17 11:39:57 -04:00
Doug Davis
0bf0e5a6ef Align 'docker commit' docs with the code
It was missing some variants and 'maintainer' isn't actually supported.
Also sorted the list of allowed cmds in the code just to make it easier
to diff with the docs.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-07-09 13:17:47 -07:00
Alexander Morozov
53b897ce9c Merge pull request #13744 from turtlebender/feature/whitelist-label-for-commit
Support LABEL as change for `docker commit`
2015-07-01 10:57:33 -07:00
David Calavera
d543a01e17 Merge pull request #14061 from runcom/clean-builder-daemon-config
Add struct to configure Builder commit
2015-06-29 11:04:05 -07:00
Josh Hawn
02c7bbefb8 [api, builder] Fix build auth config
With the 1.7 release, we introduced a change to how we store registry
credentials, but the build API endpoint did not expect a change in the format
of that file. This patch fixes this problem so that you can again pull private
images during `docker build`.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-06-23 12:58:17 -07:00
Antonio Murdaca
7046651b87 Add struct to configure Builder commit instead of using one defined in daemon
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2015-06-20 12:53:47 +02:00
Moysés Borges
d48bface59 Support downloading remote tarball contexts in builder jobs.
Signed-off-by: Moysés Borges <moysesb@gmail.com>
2015-06-19 16:35:00 -03:00
Tom Howe
6e2662b3ba just adding label to whitelist for commit command (Fixes #13738)
Signed-off-by: Tom Howe <turtlebender@gmail.com>
2015-06-16 13:59:16 -05:00
Antonio Murdaca
4ce817796e Avoid nil pointer dereference while creating a container with an empty Config
Signed-off-by: Antonio Murdaca <runcom@linux.com>
2015-06-06 20:07:32 +02:00
Tibor Vass
3985f17812 Merge pull request #13131 from Microsoft/10662-loadonwindows
Windows: Build load.go
2015-05-12 18:04:33 -07:00
Antonio Murdaca
6b700bdaca Refactor pkg/stremformatter with custom constructors instead of passing a boolean
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-05-13 00:09:41 +02:00
John Howard
aed8f9f76e Windows: Build load.go
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-05-12 14:26:57 -07:00
Antonio Murdaca
7284b08204 Remove API codepaths < 1.12
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-05-12 20:09:49 +02:00
Ma Shimiao
dccb8b5c33 add cpu.cfs_period_us support
Signed-off-by: Ma Shimiao <mashimiao.fnst@cn.fujitsu.com>
2015-05-09 10:02:46 +08:00
Julien Barbier
bb41193998 Happy birthday Docker! cgroup-parent option for docker build. Thanks to Michael, Nathan and Jessie for their support! #42
Signed-off-by: Julien Barbier <write0@gmail.com>
2015-04-25 15:25:51 -04:00
Alexander Morozov
790c63a5ef Merge pull request #12502 from calavera/shallow_git_history
Shallow clone using git to build images.
2015-04-24 10:16:01 -07:00
David Calavera
1cfb307d70 Remove duplicated git clone logic.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-04-24 09:39:45 -07:00
David Calavera
9fb7204a41 Do not try to shallow git history when the protocol doesn't allow it.
This only happens with the old git http dumb protocol, but that's what we use in our integration tests.
We check the Content-Type header advertised in http requests to make sure the http transport is the git smart transport:

See this commit as a reference:
4656bf47fc

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-04-24 09:39:38 -07:00
David Calavera
36fbf4b864 Shallow clone using git to build images.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-04-24 09:39:38 -07:00
Lei Jitang
5f4fb8be00 Add cpu cfs quota to build
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-04-24 08:54:08 +08:00
Doug Davis
bb9da6ba92 Move CLI config processing out from under registry dir
No logic changes should be in here, just moving things around.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-04-23 10:18:38 -07:00
Alexander Morozov
a2f74aa4b4 Remove chain of engine passing from builder to loadManifest
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-04-21 14:55:23 -07:00
David Calavera
ae4063585e Remove engine.Job from builder.CmdBuildConfig.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-04-20 13:37:06 -07:00
David Calavera
9a2c009751 Remove engine.Job references from builder.CmdBuild
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-04-20 13:37:04 -07:00
Doug Davis
18c9b6c645 Add .docker/config.json and support for HTTP Headers
This PR does the following:
- migrated ~/.dockerfg to ~/.docker/config.json. The data is migrated
  but the old file remains in case its needed
- moves the auth json in that fie into an "auth" property so we can add new
  top-level properties w/o messing with the auth stuff
- adds support for an HttpHeaders property in ~/.docker/config.json
  which adds these http headers to all msgs from the cli

In a follow-on PR I'll move the config file process out from under
"registry" since it not specific to that any more. I didn't do it here
because I wanted the diff to be smaller so people can make sure I didn't
break/miss any auth code during my edits.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-04-20 13:05:24 -07:00
Doug Davis
b1d8ae3824 Merge pull request #12358 from ZJU-SEL/remove_job_from_tag
remove job from tag
2015-04-19 16:02:28 -04:00
Simei He
99f6309b97 remove job from tag
Signed-off-by: Simei He <hesimei@zju.edu.cn>
2015-04-19 18:36:56 +08:00
Qiang Huang
f8dc7e8754 Add cpuset-mems support for docker build
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2015-04-15 09:34:12 +08:00
Antonio Murdaca
c30a55f14d Refactor utils/utils, fixes #11923
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-04-14 01:37:36 +02:00
bobby abbott
0cd6c05d81 Fixes hacks from progressreader refactor
related to #10959

Signed-off-by: bobby abbott <ttobbaybbob@gmail.com>
2015-03-25 18:21:02 -07:00
Antonio Murdaca
c79b9bab54 Remove engine.Status and replace it with standard go error
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-03-25 22:32:08 +01:00
Peter Waller
671c12204c Implement build cancellation
Add the capability to cancel the build by disconnecting the client.

This adds a `cancelled` channel which is used to signal that a build
should halt. The build is halted by sending a Kill signal and noticing
that the cancellation channel is closed.

This first pass implementation does not allow cancellation during a
pull, but that will come in a subsequent PR.

* Add documentation of cancellation to cli and API

* Protect job cancellation with sync.Once

* Add TestBuildCancelationKillsSleep

* Add test case for build cancellation of RUN statements.

Signed-off-by: Peter Waller <p@pwaller.net>
2015-03-22 11:31:28 +00:00
Brian Goff
e6ae89a45a Allow setting resource constrains for build
Closes #10191

Allow `docker build` to set --cpu-shares, --cpuset, --memory,
--memory-swap for all containers created by the build.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-03-18 20:57:13 -04:00
Doug Davis
15924f2385 Support dockerfile and Dockerfile
Closes #10807

Adds support for `dockerfile` ONLY when `Dockerfile` can't be found.
If we're building from a Dockerfile via stdin/URL then always download
it a `Dockerfile` and ignore the -f flag.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-03-03 18:38:50 -08:00
Dan Walsh
17abfc3ddc pass --change changes to the import job
Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
2015-02-24 13:01:36 -05:00
Michael Crosby
3210d13fc8 Return error on invalid --change command
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>

Docker-DCO-1.1-Signed-off-by: Michael Crosby <crosbymichael@gmail.com> (github: rhatdan)
2015-02-24 13:01:35 -05:00
Dan Walsh
7f091eca70 build_config job: parse dockerfile ast into config
Instead of building the actual image, `build_config` will serialize a subset of
dockerfile ast into *runconfig.Config

Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)

Docker-DCO-1.1-Signed-off-by: Dan Walsh <dwalsh@redhat.com> (github: rhatdan)
2015-02-24 13:01:35 -05:00
Don Kjer
568f86eb18 Deprecating ResolveRepositoryName
Passing RepositoryInfo to ResolveAuthConfig, pullRepository, and pushRepository

Moving --registry-mirror configuration to registry config

Created resolve_repository job

Repo names with 'index.docker.io' or 'docker.io' are now synonymous with omitting an index name.

Adding test for RepositoryInfo

Adding tests for opts.StringSetOpts and registry.ValidateMirror

Fixing search term use of repoInfo

Adding integration tests for registry mirror configuration

Normalizing LookupImage image name to match LocalName parsing rules

Normalizing repository LocalName to avoid multiple references to an official image

Removing errorOut use in tests

Removing TODO comment

gofmt changes

golint comments cleanup.  renaming RegistryOptions => registry.Options, and RegistryServiceConfig => registry.ServiceConfig

Splitting out builtins.Registry and registry.NewService calls

Stray whitespace cleanup

Moving integration tests for Mirrors and InsecureRegistries into TestNewIndexInfo unit test

Factoring out ValidateRepositoryName from NewRepositoryInfo

Removing unused IndexServerURL

Allowing json marshaling of ServiceConfig.  Exposing ServiceConfig in /info

Switching to CamelCase for json marshaling

PR cleanup; removing 'Is' prefix from boolean members.  Removing unneeded json tags.

Removing non-cleanup related fix for 'localhost:[port]' in splitReposName

Merge fixes for gh9735

Fixing integration test

Reapplying #9754

Adding comment on config.IndexConfigs use from isSecureIndex

Remove unused error return value from isSecureIndex

Signed-off-by: Don Kjer <don.kjer@gmail.com>

Adding back comment in isSecureIndex

Signed-off-by: Don Kjer <don.kjer@gmail.com>
2015-01-08 20:14:58 +00:00
Doug Davis
eb3ea3b43c Allow for Dockerfile to be named something else.
Add a check to make sure Dockerfile is in the build context
Add docs and a testcase
Make -f relative to current dir, not build context

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-01-06 16:33:26 -08:00
Michael Crosby
5794b5373e Update code for use of urlutil pkg
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2014-11-24 18:47:42 -05:00
unclejack
054e57a622 build: add pull flag to force image pulling
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2014-11-21 19:59:44 +02:00
Jessie Frazelle
9cc71b4597 Merge pull request #8468 from laktek/master
Fix URL check in build from Git.
2014-10-31 14:43:21 -07:00