beenull/moby
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Merge pull request #22001 from thaJeztah/more-docs-cherries

Browse source 
More docs cherry-picks
This commit is contained in:
Tibor Vass 2016-04-13 12:59:12 -04:00
commit 4a6f2274be
3 changed files with 185 additions and 116 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

42
docs/reference/commandline/daemon.md
View file

@ -195,17 +195,17 @@ options for `zfs` start with `zfs`.
to create and manage the thin-pool volume. This volume is then handed to Docker
to exclusively create snapshot volumes needed for images and containers.
Managing the thin-pool outside of Docker makes for the most feature-rich
Managing the thin-pool outside of Engine makes for the most feature-rich
method of having Docker utilize device mapper thin provisioning as the
backing storage for Docker's containers. The highlights of the lvm-based
backing storage for Docker containers. The highlights of the lvm-based
thin-pool management feature include: automatic or interactive thin-pool
resize support, dynamically changing thin-pool features, automatic thinp
metadata checking when lvm activates the thin-pool, etc.
As a fallback if no thin pool is provided, loopback files will be
As a fallback if no thin pool is provided, loopback files are
created. Loopback is very slow, but can be used without any
pre-configuration of storage. It is strongly recommended that you do
not use loopback in production. Ensure your Docker daemon has a
not use loopback in production. Ensure your Engine daemon has a
`--storage-opt dm.thinpooldev` argument provided.
Example use:
@ -441,29 +441,33 @@ options for `zfs` start with `zfs`.
* `dm.min_free_space`
Specifies the min free space percent in thin pool require for new device
Specifies the min free space percent in a thin pool require for new device
creation to succeed. This check applies to both free data space as well
as free metadata space. Valid values are from 0% - 99%. Value 0% disables
free space checking logic. If user does not specify a value for this optoin,
then default value for this option is 10%.
free space checking logic. If user does not specify a value for this option,
the Engine uses a default value of 10%.
Whenever a new thin pool device is created (during docker pull or
during container creation), docker will check minimum free space is
available as specified by this parameter. If that is not the case, then
device creation will fail and docker operation will fail.
Whenever a new a thin pool device is created (during `docker pull` or during
container creation), the Engine checks if the minimum free space is
available. If sufficient space is unavailable, then device creation fails
and any relevant `docker` operation fails.
One will have to create more free space in thin pool to recover from the
error. Either delete some of the images and containers from thin pool and
create free space or add more storage to thin pool.
To recover from this error, you must create more free space in the thin pool
to recover from the error. You can create free space by deleting some images
and containers from the thin pool. You can also add more storage to the thin
pool.
For lvm thin pool, one can add more storage to volume group container thin
pool and that should automatically resolve it. If loop devices are being
used, then stop docker, grow the size of loop files and restart docker and
that should resolve the issue.
To add more space to a LVM (logical volume management) thin pool, just add
more storage to the volume group container thin pool; this should automatically
resolve any errors. If your configuration uses loop devices, then stop the
Engine daemon, grow the size of loop files and restart the daemon to resolve
the issue.
Example use:
$ docker daemon --storage-opt dm.min_free_space=10%
```bash
$ docker daemon --storage-opt dm.min_free_space=10%
```
Currently supported options of `zfs`:

197
docs/userguide/storagedriver/device-mapper-driver.md
View file

@ -17,7 +17,9 @@ for image and container management. This article refers to the Device Mapper
storage driver as `devicemapper`, and the kernel framework as `Device Mapper`.
>**Note**: The [Commercially Supported Docker Engine (CS-Engine) running on RHEL and CentOS Linux](https://www.docker.com/compatibility-maintenance) requires that you use the `devicemapper` storage driver.
>**Note**: The [Commercially Supported Docker Engine (CS-Engine) running on RHEL
and CentOS Linux](https://www.docker.com/compatibility-maintenance) requires
that you use the `devicemapper` storage driver.
## An alternative to AUFS
@ -206,103 +208,155 @@ mode uses block devices to create the thin pool. The following procedure shows
you how to configure a Docker host to use the `devicemapper` storage driver in
a `direct-lvm` configuration.
> **Caution:** If you have already run the Docker daemon on your Docker host
> **Caution:** If you have already run the Engine daemon on your Docker host
> and have images you want to keep, `push` them Docker Hub or your private
> Docker Trusted Registry before attempting this procedure.
The procedure below will create a 90GB data volume and 4GB metadata volume to
use as backing for the storage pool. It assumes that you have a spare block
device at `/dev/xvdf` with enough free space to complete the task. The device
device at `/dev/sdd` with enough free space to complete the task. The device
identifier and volume sizes may be be different in your environment and you
should substitute your own values throughout the procedure. The procedure also
assumes that the Docker daemon is in the `stopped` state.
should substitute your own values throughout the procedure.
1. Log in to the Docker host you want to configure and stop the Docker daemon.
The procedure also assumes that the Engine daemon is in the `stopped` state.
Any existing images or data are lost by this process.
2. If it exists, delete your existing image store by removing the
`/var/lib/docker` directory.
1. Log in to the Docker host you want to configure.
2. If it is running, stop the Engine daemon.
3. Install the logical volume management version 2.
$ sudo rm -rf /var/lib/docker
```bash
$ yum install lvm2
```
4. Create a physical volume replacing `/dev/sdd` with your block device.
3. Create an LVM physical volume (PV) on your spare block device using the
`pvcreate` command.
```bash
$ pvcreate /dev/sdd
```
$ sudo pvcreate /dev/xvdf
Physical volume `/dev/xvdf` successfully created
5. Create a 'docker' volume group.
The device identifier may be different on your system. Remember to
substitute your value in the command above.
```bash
$ vgcreate docker /dev/sdd
```
4. Create a new volume group (VG) called `vg-docker` using the PV created in
the previous step.
6. Create a thin pool named `thinpool`.
$ sudo vgcreate vg-docker /dev/xvdf
Volume group `vg-docker` successfully created
In this example, the data logical is 95% of the 'docker' volume group size.
Leaving this free space allows for auto expanding of either the data or
metadata if space runs low as a temporary stopgap.
5. Create a new 90GB logical volume (LV) called `data` from space in the
`vg-docker` volume group.
```bash
$ lvcreate --wipesignatures y -n thinpool docker -l 95%VG
$ lvcreate --wipesignatures y -n thinpoolmeta docker -l 1%VG
```
$ sudo lvcreate -L 90G -n data vg-docker
Logical volume `data` created.
7. Convert the pool to a thin pool.
The command creates an LVM logical volume called `data` and an associated
block device file at `/dev/vg-docker/data`. In a later step, you instruct the
`devicemapper` storage driver to use this block device to store image and
container data.
```bash
$ lvconvert -y --zero n -c 512K --thinpool docker/thinpool --poolmetadata docker/thinpoolmeta
```
If you receive a signature detection warning, make sure you are working on
the correct devices before continuing. Signature warnings indicate that the
device you're working on is currently in use by LVM or has been used by LVM in
the past.
8. Configure autoextension of thin pools via an `lvm` profile.
6. Create a new logical volume (LV) called `metadata` from space in the
`vg-docker` volume group.
```bash
$ vi /etc/lvm/profile/docker-thinpool.profile
```
$ sudo lvcreate -L 4G -n metadata vg-docker
Logical volume `metadata` created.
9. Specify 'thin_pool_autoextend_threshold' value.
This creates an LVM logical volume called `metadata` and an associated
block device file at `/dev/vg-docker/metadata`. In the next step you instruct
the `devicemapper` storage driver to use this block device to store image and
container metadata.
The value should be the percentage of space used before `lvm` attempts
to autoextend the available space (100 = disabled).
7. Start the Docker daemon with the `devicemapper` storage driver and the
`--storage-opt` flags.
```
thin_pool_autoextend_threshold = 80
```
The `data` and `metadata` devices that you pass to the `--storage-opt`
options were created in the previous steps.
10. Modify the `thin_pool_autoextend_percent` for when thin pool autoextension occurs.
$ sudo docker daemon --storage-driver=devicemapper --storage-opt dm.datadev=/dev/vg-docker/data --storage-opt dm.metadatadev=/dev/vg-docker/metadata &
[1] 2163
[root@ip-10-0-0-75 centos]# INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
INFO[0027] Option DefaultDriver: bridge
INFO[0027] Option DefaultNetwork: bridge
<output truncated>
INFO[0027] Daemon has completed initialization
INFO[0027] Docker daemon commit=1b09a95-unsupported graphdriver=aufs version=1.11.0-dev
The value's setting is the perentage of space to increase the thin pool (100 =
disabled)
It is also possible to set the `--storage-driver` and `--storage-opt` flags
in the Docker config file and start the daemon normally using the `service` or
`systemd` commands.
```
thin_pool_autoextend_percent = 20
```
8. Use the `docker info` command to verify that the daemon is using `data` and
`metadata` devices you created.
11. Check your work, your `docker-thinpool.profile` file should appear similar to the following:
$ sudo docker info
INFO[0180] GET /v1.20/info
Containers: 0
Images: 0
Storage Driver: devicemapper
Pool Name: docker-202:1-1032-pool
Pool Blocksize: 65.54 kB
Backing Filesystem: xfs
Data file: /dev/vg-docker/data
Metadata file: /dev/vg-docker/metadata
[...]
An example `/etc/lvm/profile/docker-thinpool.profile` file:
```
activation {
thin_pool_autoextend_threshold=80
thin_pool_autoextend_percent=20
}
```
12. Apply your new lvm profile
```bash
$ lvchange --metadataprofile docker-thinpool docker/thinpool
```
13. Verify the `lv` is monitored.
```bash
$ lvs -o+seg_monitor
```
14. If Engine was previously started, clear your graph driver directory.
Clearing your graph driver removes any images and containers in your Docker
installation.
```bash
$ rm -rf /var/lib/docker/*
```
14. Configure the Engine daemon with specific devicemapper options.
There are two ways to do this. You can set options on the commmand line if you start the daemon there:
```bash
--storage-driver=devicemapper --storage-opt=dm.thinpooldev=/dev/mapper/docker-thinpool --storage-opt dm.use_deferred_removal=true
```
You can also set them for startup in the `daemon.json` configuration, for example:
```json
{
"storage-driver": "devicemapper",
"storage-opts": [
"dm.thinpooldev=/dev/mapper/docker-thinpool",
"dm.use_deferred_removal=true"
]
}
```
15. Start the Engine daemon.
```bash
$ systemctl start docker
```
After you start the Engine daemon, ensure you monitor your thin pool and volume
group free space. While the volume group will auto-extend, it can still fill
up. To monitor logical volumes, use `lvs` without options or `lvs -a` to see tha
data and metadata sizes. To monitor volume group free space, use the `vgs` command.
Logs can show the auto-extension of the thin pool when it hits the threshold, to
view the logs use:
```bash
journalctl -fu dm-event.service
```
If you run into repeated problems with thin pool, you can use the
`dm.min_free_space` option to tune the Engine behavior. This value ensures that
operations fail with a warning when the free space is at or near the minimum.
For information, see <a
href="https://docs.docker.com/engine/reference/commandline/daemon/#storage-driver-options"
target="_blank">the storage driver options in the Engine daemon reference</a>.
The output of the command above shows the storage driver as `devicemapper`.
The last two lines also confirm that the correct devices are being used for
the `Data file` and the `Metadata file`.
### Examine devicemapper structures on the host
@ -336,7 +390,7 @@ Docker-MAJ:MIN-INO-pool
Because Device Mapper operates at the block level it is more difficult to see
diffs between image layers and containers. Docker 1.10 and later no longer
matches image layer IDs with directory names in `/var/lib/docker`. However,
matches image layer IDs with directory names in `/var/lib/docker`. However,
there are two key directories. The `/var/lib/docker/devicemapper/mnt` directory
contains the mount points for image and container layers. The
`/var/lib/docker/devicemapper/metadata`directory contains one file for every
@ -410,3 +464,4 @@ data volumes.
* [Select a storage driver](selectadriver.md)
* [AUFS storage driver in practice](aufs-driver.md)
* [Btrfs storage driver in practice](btrfs-driver.md)
* [daemon reference](../../reference/commandline/daemon#storage-driver-options)

62
man/docker-daemon.8.md
View file

@ -269,19 +269,27 @@ allocating images and container snapshots.
Specifies a custom block storage device to use for the thin pool.
If using a block device for device mapper storage, it is best to use
`lvm` to create and manage the thin-pool volume. This volume is then
handed to Docker to create snapshot volumes needed for images and
containers.
If using a block device for device mapper storage, it is best to use `lvm`
to create and manage the thin-pool volume. This volume is then handed to Docker
to exclusively create snapshot volumes needed for images and containers.
Managing the thin-pool outside of Docker makes for the most feature-rich method
of having Docker utilize device mapper thin provisioning as the backing storage
for Docker's containers. The highlights of the LVM-based thin-pool management
feature include: automatic or interactive thin-pool resize support, dynamically
changing thin-pool features, automatic thinp metadata checking when lvm activates
the thin-pool, etc.
Managing the thin-pool outside of Engine makes for the most feature-rich
method of having Docker utilize device mapper thin provisioning as the
backing storage for Docker containers. The highlights of the lvm-based
thin-pool management feature include: automatic or interactive thin-pool
resize support, dynamically changing thin-pool features, automatic thinp
metadata checking when lvm activates the thin-pool, etc.
Example use: `docker daemon --storage-opt dm.thinpooldev=/dev/mapper/thin-pool`
As a fallback if no thin pool is provided, loopback files are
created. Loopback is very slow, but can be used without any
pre-configuration of storage. It is strongly recommended that you do
not use loopback in production. Ensure your Engine daemon has a
`--storage-opt dm.thinpooldev` argument provided.
Example use:
$ docker daemon \
--storage-opt dm.thinpooldev=/dev/mapper/thin-pool
#### dm.basesize
@ -471,27 +479,29 @@ daemon with a supported environment.
#### dm.min_free_space
Specifies the min free space percent in thin pool require for new device
Specifies the min free space percent in a thin pool require for new device
creation to succeed. This check applies to both free data space as well
as free metadata space. Valid values are from 0% - 99%. Value 0% disables
free space checking logic. If user does not specify a value for this optoin,
then default value for this option is 10%.
free space checking logic. If user does not specify a value for this option,
the Engine uses a default value of 10%.
Whenever a new thin pool device is created (during docker pull or
during container creation), docker will check minimum free space is
available as specified by this parameter. If that is not the case, then
device creation will fail and docker operation will fail.
Whenever a new a thin pool device is created (during `docker pull` or during
container creation), the Engine checks if the minimum free space is
available. If the space is unavailable, then device creation fails and any
relevant `docker` operation fails.
One will have to create more free space in thin pool to recover from the
error. Either delete some of the images and containers from thin pool and
create free space or add more storage to thin pool.
To recover from this error, you must create more free space in the thin pool to
recover from the error. You can create free space by deleting some images
and containers from tge thin pool. You can also add
more storage to the thin pool.
For lvm thin pool, one can add more storage to volume group container thin
pool and that should automatically resolve it. If loop devices are being
used, then stop docker, grow the size of loop files and restart docker and
that should resolve the issue.
To add more space to an LVM (logical volume management) thin pool, just add
more storage to the group container thin pool; this should automatically
resolve any errors. If your configuration uses loop devices, then stop the
Engine daemon, grow the size of loop files and restart the daemon to resolve
the issue.
Example use: `docker daemon --storage-opt dm.min_free_space=10%`
Example use:: `docker daemon --storage-opt dm.min_free_space=10%`
## ZFS options