Commit graph

1061 commits

Author SHA1 Message Date
Alexander Morozov
65464d11f1 Merge pull request #21809 from Microsoft/jjh/tp4removal
Windows: Remove TP4 support from main codebase
2016-04-06 14:11:33 -07:00
Stefan J. Wernli
ef5bfad321 Adding readOnly parameter to graphdriver Create method
Since the layer store was introduced, the level above the graphdriver
now differentiates between read/write and read-only layers.  This
distinction is useful for graphdrivers that need to take special steps
when creating a layer based on whether it is read-only or not.
Adding this parameter allows the graphdrivers to differentiate, which
in the case of the Windows graphdriver, removes our dependence on parsing
the id of the parent for "-init" in order to infer this information.

This will also set the stage for unblocking some of the layer store
unit tests in the next preview build of Windows.

Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
2016-04-06 13:52:53 -07:00
Tibor Vass
8c361eb5da Merge pull request #21723 from devimc/master
Fix compilation errors with btrfs-progs-4.5
2016-04-06 15:56:42 -04:00
John Howard
331c8a86d4 Windows: Remove TP4 support from main code
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-04-06 12:12:20 -07:00
John Howard
fc9912fd00 Merge pull request #21272 from Microsoft/jstarks/manifest_updates
Add os_version and os_features to Image
2016-04-05 16:16:25 -07:00
John Starks
194eaa5c0f Add os_version and os_features to Image
These fields are needed to specify the exact version of Windows that an
image can run on. They may be useful for other platforms in the future.

This also changes image.store.Create to validate that the loaded image is
supported on the current machine. This change affects Linux as well, since
it now validates the architecture and OS fields.

Signed-off-by: John Starks <jostarks@microsoft.com>
2016-04-04 13:14:57 -07:00
Sebastiaan van Stijn
b8f38747e6 Improve udev unsupported error message
Show a different message if a dynamic binary
is running, but doesn't have udev sync support.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2016-04-01 13:31:44 -07:00
Julio Montes
a038cccf88 Fix compilation errors with btrfs-progs-4.5
btrfs-progs-4.5 introduces device delete by devid
for this reason btrfs_ioctl_vol_args_v2's name was encapsulated
in a union

this patch is for setting btrfs_ioctl_vol_args_v2's name
using a C function in order to preserve compatibility
with all btrfs-progs versions

Signed-off-by: Julio Montes <imc.coder@gmail.com>
2016-04-01 08:58:29 -06:00
Vincent Demeester
e6aa40a017 Merge pull request #19367 from shishir-a412ed/rootfs_size_configurable_cli
CLI flag for docker create(run) to change block device size.
2016-03-29 08:52:54 +02:00
Shishir Mahajan
b16decfccf CLI flag for docker create(run) to change block device size.
Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>
2016-03-28 10:05:18 -04:00
Tonis Tiigi
824c24e680 Protect aufs mounts with locks
Parallel aufs mount calls produce invalid argument error.

Fixes #21545


Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-03-26 22:53:47 -07:00
Brian Goff
65d79e3e5e Move layer mount refcounts to mountedLayer
Instead of implementing refcounts at each graphdriver, implement this in
the layer package which is what the engine actually interacts with now.
This means interacting directly with the graphdriver is no longer
explicitly safe with regard to Get/Put calls being refcounted.

In addition, with the containerd, layers may still be mounted after
a daemon restart since we will no longer explicitly kill containers when
we shutdown or startup engine.
Because of this ref counts would need to be repopulated.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-03-23 14:42:52 -07:00
Anusha Ragunathan
57ca2a2101 Fix use of mounted() in overlay.
Handle error and mounted case separately.

Signed-off-by: Anusha Ragunathan <anusha@docker.com>
2016-03-23 14:42:52 -07:00
Tonis Tiigi
e91de9fb9d Revert "Move layer mount refcounts to mountedLayer"
This reverts commit 563d0711f8.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-03-23 00:33:02 -07:00
Tõnis Tiigi
92a3ece35a Merge pull request #21107 from cpuguy83/one_ctr_to_rule_them_all
Move layer mount refcounts to mountedLayer
2016-03-22 21:19:00 -07:00
Brian Goff
563d0711f8 Move layer mount refcounts to mountedLayer
Instead of implementing refcounts at each graphdriver, implement this in
the layer package which is what the engine actually interacts with now.
This means interacting directly with the graphdriver is no longer
explicitly safe with regard to Get/Put calls being refcounted.

In addition, with the containerd, layers may still be mounted after
a daemon restart since we will no longer explicitly kill containers when
we shutdown or startup engine.
Because of this ref counts would need to be repopulated.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-03-22 11:36:28 -04:00
Kenfe-Mickael Laventure
8af4f89cba Remove unneeded references to execDriver
This includes:
 - updating the docs
 - removing dangling variables

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2016-03-21 13:06:08 -07:00
Jessica Frazelle
0e025b4bb1
fix variables that werent being called
Signed-off-by: Jessica Frazelle <acidburn@docker.com>
2016-03-17 13:19:55 -07:00
Vivek Goyal
4141a00921 Fix the assignment to wrong variable
We should be assigning value to minFreeMetadata instead of minFreeData. This
is copy/paste error.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2016-03-17 15:19:08 +00:00
Antonio Murdaca
f0d83c4cdb *: fix response body leaks
Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2016-03-16 17:15:42 +01:00
Brian Goff
37a1fadae6 Merge pull request #21097 from thaJeztah/dont-run-without-udev-sync
Fail when devicemapper doesn't support udev-sync
2016-03-14 21:18:01 -04:00
Vincent Batts
bfed97b688 Merge pull request #20786 from rhvgoyal/min-free-space
devmapper: Add a new option dm.min_free_space_percent
2016-03-14 20:10:43 -04:00
David Calavera
ed6e33eede Make sure we call every graph init with the same root path.
Remove O(n^2) check for several prior configured drivers.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2016-03-11 17:15:55 -05:00
Sebastiaan van Stijn
de64171510 Fail when devicemapper doesn't support udev-sync
Now what we provide dynamic binaries for all plaforms,
we shouldn't try to run docker without udev sync support.

This change changes the previous warning to an Error,
unless the user explicitly overrides the warning, in
which case they're at their own risk.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2016-03-10 19:13:44 +01:00
Vivek Goyal
2e222f69b3 devmapper: Add a new option dm.min_free_space
Once thin pool gets full, bad things can happen. Especially in case of xfs
it is possible that xfs keeps on retrying IO infinitely (for certain kind
of IO) and container hangs. 

One way to mitigate the problem is that once thin pool is about to get full,
start failing some of the docker operations like pulling new images or
creation of new containers. That way user will get warning ahead of time
and can try to rectify it by creating more free space in thin pool. This
can be done either by deleting existing images/containers or by adding more
free space to thin pool.

This patch adds a new option dm.min_free_space to devicemapper graph
driver. Say one specifies dm.min_free_space=10%. This means atleast
10% of data and metadata blocks should be free in pool before new device
creation is allowed, otherwise operation will fail.

By default min_free_space is 10%. User can change it by specifying
dm.min_free_space=X% on command line. A value of 0% will disable the
check.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
2016-03-07 20:27:39 +00:00
Tatsushi Inagaki
e8513675a2 Aufs: reduce redundant parsing of mountinfo
Check whether or not the file system type of a mountpoint is aufs
by calling statfs() instead of parsing mountinfo. This assumes
that aufs graph driver does not allow aufs as a backing file
system.

Signed-off-by: Tatsushi Inagaki <e29253@jp.ibm.com>
2016-03-04 11:39:59 +09:00
John Starks
5649030e25 Write Windows layer diffs to tar in standard format
Previously, Windows layer diffs were written using a Windows-internal
format based on the BackupRead/BackupWrite Win32 APIs. This caused
problems with tar-split and tarsum and led to performance problems
in implementing methods such as DiffPath. It also was just an
unnecessary differentiation point between Windows and Linux.

With this change, Windows layer diffs look much more like their
Linux counterparts. They use AUFS-style whiteout files for files
that have been removed, and they encode all metadata directly in
the tar file.

This change only affects Windows post-TP4, since changes to the Windows
container storage APIs were necessary to make this possible.

Signed-off-by: John Starks <jostarks@microsoft.com>
2016-03-02 16:13:40 -08:00
David Calavera
8f109829e2 Merge pull request #20475 from Microsoft/jstarks/filegetter
graphdriver: Replace DiffPath with DiffGetter
2016-03-02 08:36:36 -08:00
John Starks
58bec40d16 graphdriver: Replace DiffPath with DiffGetter
This allows a graph driver to provide a custom FileGetter for tar-split
to use. Windows will use this to provide a more efficient implementation
in a follow-up change.

Signed-off-by: John Starks <jostarks@microsoft.com>
2016-03-01 10:25:33 -08:00
hsinko
772f5495b7 folders->directories
Signed-off-by: hsinko <21551195@zju.edu.cn>
2016-02-29 21:32:30 -08:00
Brian Goff
e386dfc33f fix double-lock
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-27 09:49:21 -05:00
Brian Goff
c2f7777603 Revert "Add finer-grained locking for aufs"
This reverts commit f31014197c.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-27 08:01:19 -05:00
Brian Goff
f31014197c Add finer-grained locking for aufs
```
benchmark                       old ns/op       new ns/op     delta
BenchmarkConcurrentAccess-8     10269529748     26834747      -99.74%

benchmark                       old allocs     new allocs     delta
BenchmarkConcurrentAccess-8     309948         7232           -97.67%

benchmark                       old bytes     new bytes     delta
BenchmarkConcurrentAccess-8     23943576      1578441       -93.41%
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-25 18:06:41 -05:00
Brian Goff
55c91f2ab9 Fix some issues with concurrency in aufs.
Adds a benchmark to measure performance under concurrent actions.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-25 14:32:13 -05:00
Stefan Weil
2eee613326 Fix some typos in comments and strings
Most of them were found and fixed by codespell.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
2016-02-22 20:27:15 +01:00
Kai Qiang Wu(Kennan)
c33cdf9ee3 Fix the typo
Signed-off-by: Kai Qiang Wu(Kennan) <wkqwu@cn.ibm.com>
2016-02-16 07:00:01 +00:00
Phil Estes
922986b76e Add proper refcounting to zfs graphdriver
Fixes issues with layer remounting (e.g. a running container which then
has `docker cp` used to copy files in or out) by applying the same
refcounting implementation that exists in other graphdrivers like
overlay and aufs.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2016-02-11 01:00:54 -05:00
Stefan J. Wernli
041a9510c6 Fixing 'docker save' on Windows.
Save was failing file integrity checksums due to bugs in both
Windows and Docker. This commit includes fixes to file time handling
in tarexport and system.chtimes that are necessary along with
the Windows platform fixes to correctly support save. With this
change, sysfile_backups for windowsfilter driver are no longer
needed, so that code is removed.

Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
2016-02-08 18:08:49 -08:00
David Calavera
a93cb2b856 Merge pull request #20045 from estesp/zfs-userns-permissions-fix
Fix ZFS permissions bug with user namespaces
2016-02-05 15:32:43 -08:00
Phil Estes
aef0995b02 Fix ZFS permissions bug with user namespaces
Fix root directory of the mountpoint being owned by real root. This is
unique to ZFS because of the way file mountpoints are created using the
ZFS tooling, and the remapping that happens at layer unpack doesn't
impact this root (already created) holding directory for the layer.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2016-02-05 14:37:33 -05:00
Liu Bo
b2e27fee53 Graphdriver/btrfs: Avoid using single d.Get()
For btrfs driver, in d.Create(), Get() of parentDir is called but not followed
by Put().

If we apply SElinux mount label, we need to mount btrfs subvolumes in d.Get(),
without a Put() would end up with a later Remove() failure on
"Device resourse is busy".

This calls the subvolume helper function directly in d.Create().

Signed-off-by: Liu Bo <bo.li.liu@oracle.com>
2016-02-04 10:25:24 -08:00
Tonis Tiigi
b9a395c85d Remove case sensitive duplicate dir in vendor
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-02-03 13:06:32 -08:00
Kai Qiang Wu(Kennan)
feda5d7684 Make btrfs call same interface as others
Most storage drivers call graphdriver.GetFSMagic(home),
it is more clean to easy to maintain. So btrfs need to
adopt such change.

Signed-off-by: Kai Qiang Wu(Kennan) <wkqwu@cn.ibm.com>
2016-02-01 07:50:21 +00:00
Phil Estes
67d86d10d8 Merge pull request #19216 from scaleoutsean/master
Add GPFS filesystem IDs
2016-01-22 21:27:03 -05:00
scaleoutsean
bdc8241da9 Add GPFS
Signed-off-by: Sean Lee <seanlee@tw.ibm.com>
2016-01-22 21:12:47 +08:00
Tibor Vass
d2448a0c20 Merge pull request #19524 from LK4D4/iocopy_writeto
Use bufio.Reader in io.Copy source for overlay.copyRegular
2016-01-21 15:37:12 -05:00
Alexander Morozov
3f5e1c69b3 Use pools.Copy instead of io.Copy for overlay.copyRegular
That function is pretty heavy used on container start. Autoallocating
buffer can be painful.

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2016-01-21 08:53:37 -08:00
Tonis Tiigi
98d0997894 Revert "Copy aufs hardlinks to top layer"
This reverts commit ef05b83417.

Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-01-21 07:52:50 -08:00
Sebastiaan van Stijn
661d75f398 Merge pull request #19123 from shishir-a412ed/rootfs_size_configurable
daemon option (--storage-opt dm.basesize) for increasing the base device size on daemon restart
2016-01-13 13:22:08 -08:00
Shishir Mahajan
e47112d3e8 daemon option (--storage-opt dm.basesize) for increasing the base device size on daemon restart
Signed-off-by: Shishir Mahajan <shishir.mahajan@redhat.com>
2016-01-13 13:57:31 -05:00