Commit graph

173 commits

Author SHA1 Message Date
John Howard
4ec9766a27 LCOW: Fix nits from 33241
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-27 11:59:49 -07:00
John Stephens
930e689668 Merge pull request #33241 from Microsoft/jjh/multi-layerstore
LCOW: Support most operations excluding remote filesystem
2017-06-21 15:45:23 -07:00
Brian Goff
e64c2d7122 Merge pull request #33755 from aaronlehmann/imagestore-locking
image: Improve store locking
2017-06-21 10:01:19 -07:00
John Howard
87abf34a3d LCOW: Store integrity checks
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:49:53 -07:00
John Howard
8f53780694 LCOW: commit to use image platform, not host
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:49:52 -07:00
John Howard
6c33684987 LCOW: Add platform to image store
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:49:51 -07:00
Aaron Lehmann
44e67adae5 image: Improve store locking
This showed up in a recent profile.

Search doesn't need to take a lock on the store, because digestset has
its own locking.

Some other methods can get by with a read lock instead of an exclusive
lock.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-06-20 18:31:29 -07:00
John Howard
42c5c1a9ec LCOW: Pass platform through into layer store
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 09:21:37 -07:00
Daniel Nephin
5136096520 Fix copy when used with scratch and images with empty RootFS
Commit the rwLayer to get the correct DiffID
Refacator copy in thebuilder
move more code into exportImage
cleanup some windows tests
Release the newly commited layer.
Set the imageID on the buildStage after exporting a new image.
Move archiver to BuildManager.
Have ReleaseableLayer.Commit return a layer
and store the Image from exportImage in the local imageSources cache
Remove NewChild from image interface.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-08 15:07:16 -04:00
Daniel Nephin
bd5f92d263 Remove CopyOnBuild from the daemon.
Add CreateImage() to the daemon
Refactor daemon.Comit() and expose a Image.NewChild()
Update copy to use IDMappings.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-08 15:06:54 -04:00
Brian Goff
c8141a1fb1 Merge pull request #32293 from moypray/save_delete
Fix delete a image while saving it, delete successfully but failed to save it
2017-06-05 09:55:56 -04:00
Wentao Zhang
4a014e6b0d Fix delete a image while saving it, delete successfully but failed to save it
Issue Description:
 * 1. Saving more than one images, `docker save -o a.tar aaa bbb`
 * 2. Delete the last image which in saving progress.  `docker rmi bbb`

Espected:
 Saving images operation shouldn't be disturbed. But the real result is that failed to
save image and get an error as below:
  `Error response from daemon: open
/var/lib/docker/image/devicemapper/imagedb/content/sha256/7c24e4d533a76e801662ad1b7e6e06bc1204f80110b5623e96ba2877c51479a1:
no such file or directory`

Analysis:
  1. While saving more than one images, it will get all the image info from reference/imagestore,
     and then using the `cached data` to save the images to a tar file.
  2. But this process doesn't have a resource lock, if a deletion operation comes, the image will be deleted,
     so saving operation will fail.

Solution:
 When begin to save an image, `Get` all the layers first. then the
deletion operation won't delete the layers.

Signed-off-by: Wentao Zhang <zhangwentao234@huawei.com>
2017-06-01 18:13:20 +08:00
Aaron Lehmann
08d7fad45d Merge pull request #32949 from dnephin/refactor-streamformatter-and-progress
Refacator pkg/streamformatter
2017-05-03 09:05:26 -07:00
Daniel Nephin
c87d67b0ad Refacator pkg/streamformatter
StreamFormatter suffered was two distinct structs mixed into a single struct
without any overlap.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-02 17:38:12 -04:00
Daniel Nephin
9f738cc574 Cleanup all the mutate + defer revert of b.runConfig in the builder
Instead of mutating and reverting, just create a copy and pass the copy
around.

Add a unit test for builder dispatcher.run

Fix two test failures

Fix image history by adding a CreatedBy to commit options. Previously the
createdBy field was being created by modifying a reference to the runConfig that
was held from when the container was created.

Fix a test that expected a trailing slash. Previously the runConfig was being
modified by container create. Now that we're creating a copy of runConfig
instead of sharing a reference the runConfig retains the trailing slash.

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-05-01 18:36:53 -04:00
Stephen J Day
c9beb417f4
image/spec: tags are 128 characters
Mostly, they always have been. Most regular expressions were implemented
as `[\w][\w.-]{0,127]`, which actually allows 128 characters, since we
start with a character in the first expression.

This "fact" has been backported to the existing specifications where
length is mentioned. For the most part, no ill-effects should come of
this, unless someone has optimized to hold the length of a tag in a 7
bit integer.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-04-24 15:23:34 -07:00
Aaron Lehmann
6052f2b396 Remove pkg/testutil/assert in favor of testify
I noticed that we're using a homegrown package for assertions. The
functions are extremely similar to testify, but with enough slight
differences to be confusing (for example, Equal takes its arguments in a
different order). We already vendor testify, and it's used in a few
places by tests.

I also found some problems with pkg/testutil/assert. For example, the
NotNil function seems to be broken. It checks the argument against
"nil", which only works for an interface. If you pass in a nil map or
slice, the equality check will fail.

In the interest of avoiding NIH, I'm proposing replacing
pkg/testutil/assert with testify. The test code looks almost the same,
but we avoid the confusion of having two similar but slightly different
assertion packages, and having to maintain our own package instead of
using a commonly-used one.

In the process, I found a few places where the tests should halt if an
assertion fails, so I've made those cases (that I noticed) use "require"
instead of "assert", and I've vendored the "require" package from
testify alongside the already-present "assert" package.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-04-14 12:03:21 -07:00
harche
5e1349e083 Fixing a typo in spelling
Wrong tense used in a sentence

Signed-off-by: Harshal Patil <harshal.patil@in.ibm.com>
2017-03-30 14:31:45 +05:30
Daniel Nephin
7abeb08a8c Wrap errors in digest package so they are consistent between platforms.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-03-08 11:48:46 -05:00
Daniel Nephin
69d7362058 Use assertions in image package unit tests.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-03-08 11:46:10 -05:00
Antonio Murdaca
1cf4b2b8bd
image/cache: fix isValidParent logic
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-02-21 19:32:04 +01:00
Derek McGowan
3a1279393f
Use distribution reference
Remove forked reference package. Use normalized named values
everywhere and familiar functions to convert back to familiar
strings for UX and storage compatibility.

Enforce that the source repository in the distribution metadata
is always a normalized string, ignore invalid values which are not.
Update distribution tests to use normalized values.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2017-02-07 11:08:37 -08:00
Xinbo Weng
5b258f6ac2
some grammatical errors
Signed-off-by: Xinbo Weng <xihuanbo_0521@zju.edu.cn>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-01-31 19:33:12 -08:00
Alexander Morozov
df5d909ac9 Merge pull request #29851 from vdemeester/daemon-extract-image-cache
Extract the daemon image cache to its own package
2017-01-26 10:50:35 -08:00
Alexander Morozov
effc91e7a5 Merge pull request #29830 from timthelion/issue29821
Docs: Be more clear when specifying valid formats for strings
2017-01-26 10:45:03 -08:00
Lei Jitang
ee5228e799 Follow #28926, fix docker save can't share layers in the tar archive.
After https://github.com/docker/docker/pull/28926, if saving multiple images
which have common layers at same time, the common layers can't share in the tar archive
because the hash ID changes because of the Create time. The Create time is used for
pre v1.9 which treat each layer as a image and make no sense for after v1.10.
To make the hash ID consistent and keep the image save from >1.10 working properly
on pre v1.9, using a constant Create time `time.Unix(0,0)`.

Signed-off-by: Lei Jitang <leijitang@huawei.com>
2017-01-17 05:40:59 -05:00
Antonio Murdaca
675f4140e7
image: tarexport: do not quote integers in format string
Basically, prevents outputs like:
```
invalid manifest, layers length mismatch: expected '\x02', got '\x01'
```

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-01-14 20:03:11 +01:00
Stephen J Day
7a85579917
*: use opencontainers/go-digest package
The `digest` data type, used throughout docker for image verification
and identity, has been broken out into `opencontainers/go-digest`. This
PR updates the dependencies and moves uses over to the new type.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2017-01-06 18:48:41 -08:00
Timothy Hobbs
b71b4e1fb9 Docs: Be more clear when specifying valid formats for strings
- Use the word letter rather than character to refer to letters ;) when trying to specify that only letters and numbers can be used, and not ANY character...
- Small corrections

Fixes #29821

Signed-off-by: Timothy Hobbs <timothy@hobbs.cz>
2017-01-03 21:40:50 +01:00
Vincent Demeester
be1df1ee13
Extract the daemon image cache to its own package
ImageCache is now independent of `Daemon` and is located in
`image/cache` package.

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-01-03 19:19:27 +01:00
Tonis Tiigi
9f3046f9a0 Move imageID validation to stringid pkg
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-12-26 18:53:22 -08:00
Vincent Demeester
ba1c20f8d6 Merge pull request #29345 from yuexiao-wang/fix-err-info
Fix incorrect words and formats in image
2016-12-21 21:50:11 +01:00
allencloud
7c3657065c change minor mistake of spelling
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-12-20 21:05:19 +08:00
yuexiao-wang
b2ec509a3f Fix incorrect info and format of error in image
Signed-off-by: yuexiao-wang <wang.yuexiao@zte.com.cn>
2016-12-20 18:20:15 +08:00
unclejack
2c187a24e0 return directly without ifs in remaining packages
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2016-12-14 23:28:27 +02:00
Tõnis Tiigi
dd20fa60b4 Merge pull request #28926 from coolljt0725/fix_save
Fix docker save with empty timestamp of layer created time
2016-11-30 11:23:19 -08:00
Stephen J Day
8616b0690a
tarexport: check symlink error when saving layer
Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-11-29 13:08:24 -08:00
Lei Jitang
e0b2c0af4d Fix docker save with empty timestamp of layer created time
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2016-11-29 03:58:23 -05:00
John Howard
7c29f5beed Windows: Use sequential file access
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-11-14 17:41:52 -08:00
Qiang Huang
e6866492c4 Fix bunch of typos
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
2016-10-29 15:03:26 +08:00
Tonis Tiigi
805223982c Generalize content addressable and reference storage
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-09-15 18:08:19 -07:00
Tõnis Tiigi
b890d475f7 Merge pull request #25806 from Microsoft/jjh-missingdigest
Don't fail daemon start on missing layer
2016-09-07 16:35:27 -07:00
John Howard
540c8e9b20 Windows: Skip layers+base images
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-09-07 12:16:45 -07:00
Michael Crosby
91e197d614 Add engine-api types to docker
This moves the types for the `engine-api` repo to the existing types
package.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2016-09-07 11:05:58 -07:00
Wenkai Yin
8f37abd056 fixes #25654
Signed-off-by: Wenkai Yin <yinw@vmware.com>
2016-08-31 15:37:52 +08:00
Thomas Leonard
9841cfe4c9 Document Healthcheck in image spec
Signed-off-by: Thomas Leonard <thomas.leonard@docker.com>
2016-08-18 15:53:22 +01:00
Vincent Demeester
353af0b3ec Merge pull request #25557 from yongtang/25529-image-load-api
Return `application/json` for image/load API (quiet=1)
2016-08-12 11:07:43 +02:00
Yong Tang
2f27632cde Return application/json for image/load API (quite=1)
This fix tries to address the issue raised in 25529 wehre the
image/load API returns `application/json` for quiet=0 and
`text/plain` for quite=1.

This fix makes the change so that `application/json` is returned
for both quiet=0 and quite=1.

This fix has been tested manually.

This fix fixes 25529.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2016-08-09 17:55:42 -07:00
allencloud
0ead624473 add defer file.Close to avoid potential fd leak
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-08-10 08:36:09 +08:00
Stefan J. Wernli
f342b27145 Removing old custom RootFS behavior on Windows.
Windows base layers are no longer the special "layers+base" type, so we can remove all the special handling for that.

Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
2016-07-29 14:15:59 -07:00