Fixes #21701 devicemapper docs
Copy edit the content Updates to existing material Adding mbentley's comments Updating with last minute comments Update with Seb's comments Signed-off-by: Mary Anthony <mary@docker.com>
This commit is contained in:
Mary Anthony
parent
1687f77db2
commit
783ebebff4
3 changed files with 185 additions and 116 deletions
|
|
@ -195,17 +195,17 @@ options for `zfs` start with `zfs`.
|
|||
to create and manage the thin-pool volume. This volume is then handed to Docker
|
||||
to exclusively create snapshot volumes needed for images and containers.
|
||||
|
||||
Managing the thin-pool outside of Docker makes for the most feature-rich
|
||||
Managing the thin-pool outside of Engine makes for the most feature-rich
|
||||
method of having Docker utilize device mapper thin provisioning as the
|
||||
backing storage for Docker's containers. The highlights of the lvm-based
|
||||
backing storage for Docker containers. The highlights of the lvm-based
|
||||
thin-pool management feature include: automatic or interactive thin-pool
|
||||
resize support, dynamically changing thin-pool features, automatic thinp
|
||||
metadata checking when lvm activates the thin-pool, etc.
|
||||
|
||||
As a fallback if no thin pool is provided, loopback files will be
|
||||
As a fallback if no thin pool is provided, loopback files are
|
||||
created. Loopback is very slow, but can be used without any
|
||||
pre-configuration of storage. It is strongly recommended that you do
|
||||
not use loopback in production. Ensure your Docker daemon has a
|
||||
not use loopback in production. Ensure your Engine daemon has a
|
||||
`--storage-opt dm.thinpooldev` argument provided.
|
||||
|
||||
Example use:
|
||||
|
|
@ -441,29 +441,33 @@ options for `zfs` start with `zfs`.
|
|||
|
||||
* `dm.min_free_space`
|
||||
|
||||
Specifies the min free space percent in thin pool require for new device
|
||||
Specifies the min free space percent in a thin pool require for new device
|
||||
creation to succeed. This check applies to both free data space as well
|
||||
as free metadata space. Valid values are from 0% - 99%. Value 0% disables
|
||||
free space checking logic. If user does not specify a value for this optoin,
|
||||
then default value for this option is 10%.
|
||||
free space checking logic. If user does not specify a value for this option,
|
||||
the Engine uses a default value of 10%.
|
||||
|
||||
Whenever a new thin pool device is created (during docker pull or
|
||||
during container creation), docker will check minimum free space is
|
||||
available as specified by this parameter. If that is not the case, then
|
||||
device creation will fail and docker operation will fail.
|
||||
Whenever a new a thin pool device is created (during `docker pull` or during
|
||||
container creation), the Engine checks if the minimum free space is
|
||||
available. If sufficient space is unavailable, then device creation fails
|
||||
and any relevant `docker` operation fails.
|
||||
|
||||
One will have to create more free space in thin pool to recover from the
|
||||
error. Either delete some of the images and containers from thin pool and
|
||||
create free space or add more storage to thin pool.
|
||||
To recover from this error, you must create more free space in the thin pool
|
||||
to recover from the error. You can create free space by deleting some images
|
||||
and containers from the thin pool. You can also add more storage to the thin
|
||||
pool.
|
||||
|
||||
For lvm thin pool, one can add more storage to volume group container thin
|
||||
pool and that should automatically resolve it. If loop devices are being
|
||||
used, then stop docker, grow the size of loop files and restart docker and
|
||||
that should resolve the issue.
|
||||
To add more space to a LVM (logical volume management) thin pool, just add
|
||||
more storage to the volume group container thin pool; this should automatically
|
||||
resolve any errors. If your configuration uses loop devices, then stop the
|
||||
Engine daemon, grow the size of loop files and restart the daemon to resolve
|
||||
the issue.
|
||||
|
||||
Example use:
|
||||
|
||||
$ docker daemon --storage-opt dm.min_free_space=10%
|
||||
```bash
|
||||
$ docker daemon --storage-opt dm.min_free_space=10%
|
||||
```
|
||||
|
||||
Currently supported options of `zfs`:
|
||||
|
||||
|
|
|
|||
|
|
@ -17,7 +17,9 @@ for image and container management. This article refers to the Device Mapper
|
|||
storage driver as `devicemapper`, and the kernel framework as `Device Mapper`.
|
||||
|
||||
|
||||
>**Note**: The [Commercially Supported Docker Engine (CS-Engine) running on RHEL and CentOS Linux](https://www.docker.com/compatibility-maintenance) requires that you use the `devicemapper` storage driver.
|
||||
>**Note**: The [Commercially Supported Docker Engine (CS-Engine) running on RHEL
|
||||
and CentOS Linux](https://www.docker.com/compatibility-maintenance) requires
|
||||
that you use the `devicemapper` storage driver.
|
||||
|
||||
|
||||
## An alternative to AUFS
|
||||
|
|
@ -206,103 +208,155 @@ mode uses block devices to create the thin pool. The following procedure shows
|
|||
you how to configure a Docker host to use the `devicemapper` storage driver in
|
||||
a `direct-lvm` configuration.
|
||||
|
||||
> **Caution:** If you have already run the Docker daemon on your Docker host
|
||||
> **Caution:** If you have already run the Engine daemon on your Docker host
|
||||
> and have images you want to keep, `push` them Docker Hub or your private
|
||||
> Docker Trusted Registry before attempting this procedure.
|
||||
|
||||
The procedure below will create a 90GB data volume and 4GB metadata volume to
|
||||
use as backing for the storage pool. It assumes that you have a spare block
|
||||
device at `/dev/xvdf` with enough free space to complete the task. The device
|
||||
device at `/dev/sdd` with enough free space to complete the task. The device
|
||||
identifier and volume sizes may be be different in your environment and you
|
||||
should substitute your own values throughout the procedure. The procedure also
|
||||
assumes that the Docker daemon is in the `stopped` state.
|
||||
should substitute your own values throughout the procedure.
|
||||
|
||||
1. Log in to the Docker host you want to configure and stop the Docker daemon.
|
||||
The procedure also assumes that the Engine daemon is in the `stopped` state.
|
||||
Any existing images or data are lost by this process.
|
||||
|
||||
2. If it exists, delete your existing image store by removing the
|
||||
`/var/lib/docker` directory.
|
||||
1. Log in to the Docker host you want to configure.
|
||||
2. If it is running, stop the Engine daemon.
|
||||
3. Install the logical volume management version 2.
|
||||
|
||||
$ sudo rm -rf /var/lib/docker
|
||||
```bash
|
||||
$ yum install lvm2
|
||||
```
|
||||
4. Create a physical volume replacing `/dev/sdd` with your block device.
|
||||
|
||||
3. Create an LVM physical volume (PV) on your spare block device using the
|
||||
`pvcreate` command.
|
||||
```bash
|
||||
$ pvcreate /dev/sdd
|
||||
```
|
||||
|
||||
$ sudo pvcreate /dev/xvdf
|
||||
Physical volume `/dev/xvdf` successfully created
|
||||
5. Create a 'docker' volume group.
|
||||
|
||||
The device identifier may be different on your system. Remember to
|
||||
substitute your value in the command above.
|
||||
```bash
|
||||
$ vgcreate docker /dev/sdd
|
||||
```
|
||||
|
||||
4. Create a new volume group (VG) called `vg-docker` using the PV created in
|
||||
the previous step.
|
||||
6. Create a thin pool named `thinpool`.
|
||||
|
||||
$ sudo vgcreate vg-docker /dev/xvdf
|
||||
Volume group `vg-docker` successfully created
|
||||
In this example, the data logical is 95% of the 'docker' volume group size.
|
||||
Leaving this free space allows for auto expanding of either the data or
|
||||
metadata if space runs low as a temporary stopgap.
|
||||
|
||||
5. Create a new 90GB logical volume (LV) called `data` from space in the
|
||||
`vg-docker` volume group.
|
||||
```bash
|
||||
$ lvcreate --wipesignatures y -n thinpool docker -l 95%VG
|
||||
$ lvcreate --wipesignatures y -n thinpoolmeta docker -l 1%VG
|
||||
```
|
||||
|
||||
$ sudo lvcreate -L 90G -n data vg-docker
|
||||
Logical volume `data` created.
|
||||
7. Convert the pool to a thin pool.
|
||||
|
||||
The command creates an LVM logical volume called `data` and an associated
|
||||
block device file at `/dev/vg-docker/data`. In a later step, you instruct the
|
||||
`devicemapper` storage driver to use this block device to store image and
|
||||
container data.
|
||||
```bash
|
||||
$ lvconvert -y --zero n -c 512K --thinpool docker/thinpool --poolmetadata docker/thinpoolmeta
|
||||
```
|
||||
|
||||
If you receive a signature detection warning, make sure you are working on
|
||||
the correct devices before continuing. Signature warnings indicate that the
|
||||
device you're working on is currently in use by LVM or has been used by LVM in
|
||||
the past.
|
||||
8. Configure autoextension of thin pools via an `lvm` profile.
|
||||
|
||||
6. Create a new logical volume (LV) called `metadata` from space in the
|
||||
`vg-docker` volume group.
|
||||
```bash
|
||||
$ vi /etc/lvm/profile/docker-thinpool.profile
|
||||
```
|
||||
|
||||
$ sudo lvcreate -L 4G -n metadata vg-docker
|
||||
Logical volume `metadata` created.
|
||||
9. Specify 'thin_pool_autoextend_threshold' value.
|
||||
|
||||
This creates an LVM logical volume called `metadata` and an associated
|
||||
block device file at `/dev/vg-docker/metadata`. In the next step you instruct
|
||||
the `devicemapper` storage driver to use this block device to store image and
|
||||
container metadata.
|
||||
The value should be the percentage of space used before `lvm` attempts
|
||||
to autoextend the available space (100 = disabled).
|
||||
|
||||
7. Start the Docker daemon with the `devicemapper` storage driver and the
|
||||
`--storage-opt` flags.
|
||||
```
|
||||
thin_pool_autoextend_threshold = 80
|
||||
```
|
||||
|
||||
The `data` and `metadata` devices that you pass to the `--storage-opt`
|
||||
options were created in the previous steps.
|
||||
10. Modify the `thin_pool_autoextend_percent` for when thin pool autoextension occurs.
|
||||
|
||||
$ sudo docker daemon --storage-driver=devicemapper --storage-opt dm.datadev=/dev/vg-docker/data --storage-opt dm.metadatadev=/dev/vg-docker/metadata &
|
||||
[1] 2163
|
||||
[root@ip-10-0-0-75 centos]# INFO[0000] Listening for HTTP on unix (/var/run/docker.sock)
|
||||
INFO[0027] Option DefaultDriver: bridge
|
||||
INFO[0027] Option DefaultNetwork: bridge
|
||||
<output truncated>
|
||||
INFO[0027] Daemon has completed initialization
|
||||
INFO[0027] Docker daemon commit=1b09a95-unsupported graphdriver=aufs version=1.11.0-dev
|
||||
The value's setting is the perentage of space to increase the thin pool (100 =
|
||||
disabled)
|
||||
|
||||
It is also possible to set the `--storage-driver` and `--storage-opt` flags
|
||||
in the Docker config file and start the daemon normally using the `service` or
|
||||
`systemd` commands.
|
||||
```
|
||||
thin_pool_autoextend_percent = 20
|
||||
```
|
||||
|
||||
8. Use the `docker info` command to verify that the daemon is using `data` and
|
||||
`metadata` devices you created.
|
||||
11. Check your work, your `docker-thinpool.profile` file should appear similar to the following:
|
||||
|
||||
$ sudo docker info
|
||||
INFO[0180] GET /v1.20/info
|
||||
Containers: 0
|
||||
Images: 0
|
||||
Storage Driver: devicemapper
|
||||
Pool Name: docker-202:1-1032-pool
|
||||
Pool Blocksize: 65.54 kB
|
||||
Backing Filesystem: xfs
|
||||
Data file: /dev/vg-docker/data
|
||||
Metadata file: /dev/vg-docker/metadata
|
||||
[...]
|
||||
An example `/etc/lvm/profile/docker-thinpool.profile` file:
|
||||
|
||||
```
|
||||
activation {
|
||||
thin_pool_autoextend_threshold=80
|
||||
thin_pool_autoextend_percent=20
|
||||
}
|
||||
```
|
||||
|
||||
12. Apply your new lvm profile
|
||||
|
||||
```bash
|
||||
$ lvchange --metadataprofile docker-thinpool docker/thinpool
|
||||
```
|
||||
|
||||
13. Verify the `lv` is monitored.
|
||||
|
||||
```bash
|
||||
$ lvs -o+seg_monitor
|
||||
```
|
||||
|
||||
14. If Engine was previously started, clear your graph driver directory.
|
||||
|
||||
Clearing your graph driver removes any images and containers in your Docker
|
||||
installation.
|
||||
|
||||
```bash
|
||||
$ rm -rf /var/lib/docker/*
|
||||
```
|
||||
|
||||
14. Configure the Engine daemon with specific devicemapper options.
|
||||
|
||||
There are two ways to do this. You can set options on the commmand line if you start the daemon there:
|
||||
|
||||
```bash
|
||||
--storage-driver=devicemapper --storage-opt=dm.thinpooldev=/dev/mapper/docker-thinpool --storage-opt dm.use_deferred_removal=true
|
||||
```
|
||||
|
||||
You can also set them for startup in the `daemon.json` configuration, for example:
|
||||
|
||||
```json
|
||||
{
|
||||
"storage-driver": "devicemapper",
|
||||
"storage-opts": [
|
||||
"dm.thinpooldev=/dev/mapper/docker-thinpool",
|
||||
"dm.use_deferred_removal=true"
|
||||
]
|
||||
}
|
||||
```
|
||||
15. Start the Engine daemon.
|
||||
|
||||
```bash
|
||||
$ systemctl start docker
|
||||
```
|
||||
|
||||
After you start the Engine daemon, ensure you monitor your thin pool and volume
|
||||
group free space. While the volume group will auto-extend, it can still fill
|
||||
up. To monitor logical volumes, use `lvs` without options or `lvs -a` to see tha
|
||||
data and metadata sizes. To monitor volume group free space, use the `vgs` command.
|
||||
|
||||
Logs can show the auto-extension of the thin pool when it hits the threshold, to
|
||||
view the logs use:
|
||||
|
||||
```bash
|
||||
journalctl -fu dm-event.service
|
||||
```
|
||||
|
||||
If you run into repeated problems with thin pool, you can use the
|
||||
`dm.min_free_space` option to tune the Engine behavior. This value ensures that
|
||||
operations fail with a warning when the free space is at or near the minimum.
|
||||
For information, see <a
|
||||
href="https://docs.docker.com/engine/reference/commandline/daemon/#storage-driver-options"
|
||||
target="_blank">the storage driver options in the Engine daemon reference</a>.
|
||||
|
||||
The output of the command above shows the storage driver as `devicemapper`.
|
||||
The last two lines also confirm that the correct devices are being used for
|
||||
the `Data file` and the `Metadata file`.
|
||||
|
||||
### Examine devicemapper structures on the host
|
||||
|
||||
|
|
@ -336,7 +390,7 @@ Docker-MAJ:MIN-INO-pool
|
|||
|
||||
Because Device Mapper operates at the block level it is more difficult to see
|
||||
diffs between image layers and containers. Docker 1.10 and later no longer
|
||||
matches image layer IDs with directory names in `/var/lib/docker`. However,
|
||||
matches image layer IDs with directory names in `/var/lib/docker`. However,
|
||||
there are two key directories. The `/var/lib/docker/devicemapper/mnt` directory
|
||||
contains the mount points for image and container layers. The
|
||||
`/var/lib/docker/devicemapper/metadata`directory contains one file for every
|
||||
|
|
@ -410,3 +464,4 @@ data volumes.
|
|||
* [Select a storage driver](selectadriver.md)
|
||||
* [AUFS storage driver in practice](aufs-driver.md)
|
||||
* [Btrfs storage driver in practice](btrfs-driver.md)
|
||||
* [daemon reference](../../reference/commandline/daemon#storage-driver-options)
|
||||
|
|
|
|||
|
|
@ -269,19 +269,27 @@ allocating images and container snapshots.
|
|||
|
||||
Specifies a custom block storage device to use for the thin pool.
|
||||
|
||||
If using a block device for device mapper storage, it is best to use
|
||||
`lvm` to create and manage the thin-pool volume. This volume is then
|
||||
handed to Docker to create snapshot volumes needed for images and
|
||||
containers.
|
||||
If using a block device for device mapper storage, it is best to use `lvm`
|
||||
to create and manage the thin-pool volume. This volume is then handed to Docker
|
||||
to exclusively create snapshot volumes needed for images and containers.
|
||||
|
||||
Managing the thin-pool outside of Docker makes for the most feature-rich method
|
||||
of having Docker utilize device mapper thin provisioning as the backing storage
|
||||
for Docker's containers. The highlights of the LVM-based thin-pool management
|
||||
feature include: automatic or interactive thin-pool resize support, dynamically
|
||||
changing thin-pool features, automatic thinp metadata checking when lvm activates
|
||||
the thin-pool, etc.
|
||||
Managing the thin-pool outside of Engine makes for the most feature-rich
|
||||
method of having Docker utilize device mapper thin provisioning as the
|
||||
backing storage for Docker containers. The highlights of the lvm-based
|
||||
thin-pool management feature include: automatic or interactive thin-pool
|
||||
resize support, dynamically changing thin-pool features, automatic thinp
|
||||
metadata checking when lvm activates the thin-pool, etc.
|
||||
|
||||
Example use: `docker daemon --storage-opt dm.thinpooldev=/dev/mapper/thin-pool`
|
||||
As a fallback if no thin pool is provided, loopback files are
|
||||
created. Loopback is very slow, but can be used without any
|
||||
pre-configuration of storage. It is strongly recommended that you do
|
||||
not use loopback in production. Ensure your Engine daemon has a
|
||||
`--storage-opt dm.thinpooldev` argument provided.
|
||||
|
||||
Example use:
|
||||
|
||||
$ docker daemon \
|
||||
--storage-opt dm.thinpooldev=/dev/mapper/thin-pool
|
||||
|
||||
#### dm.basesize
|
||||
|
||||
|
|
@ -471,27 +479,29 @@ daemon with a supported environment.
|
|||
|
||||
#### dm.min_free_space
|
||||
|
||||
Specifies the min free space percent in thin pool require for new device
|
||||
Specifies the min free space percent in a thin pool require for new device
|
||||
creation to succeed. This check applies to both free data space as well
|
||||
as free metadata space. Valid values are from 0% - 99%. Value 0% disables
|
||||
free space checking logic. If user does not specify a value for this optoin,
|
||||
then default value for this option is 10%.
|
||||
free space checking logic. If user does not specify a value for this option,
|
||||
the Engine uses a default value of 10%.
|
||||
|
||||
Whenever a new thin pool device is created (during docker pull or
|
||||
during container creation), docker will check minimum free space is
|
||||
available as specified by this parameter. If that is not the case, then
|
||||
device creation will fail and docker operation will fail.
|
||||
Whenever a new a thin pool device is created (during `docker pull` or during
|
||||
container creation), the Engine checks if the minimum free space is
|
||||
available. If the space is unavailable, then device creation fails and any
|
||||
relevant `docker` operation fails.
|
||||
|
||||
One will have to create more free space in thin pool to recover from the
|
||||
error. Either delete some of the images and containers from thin pool and
|
||||
create free space or add more storage to thin pool.
|
||||
To recover from this error, you must create more free space in the thin pool to
|
||||
recover from the error. You can create free space by deleting some images
|
||||
and containers from tge thin pool. You can also add
|
||||
more storage to the thin pool.
|
||||
|
||||
For lvm thin pool, one can add more storage to volume group container thin
|
||||
pool and that should automatically resolve it. If loop devices are being
|
||||
used, then stop docker, grow the size of loop files and restart docker and
|
||||
that should resolve the issue.
|
||||
To add more space to an LVM (logical volume management) thin pool, just add
|
||||
more storage to the group container thin pool; this should automatically
|
||||
resolve any errors. If your configuration uses loop devices, then stop the
|
||||
Engine daemon, grow the size of loop files and restart the daemon to resolve
|
||||
the issue.
|
||||
|
||||
Example use: `docker daemon --storage-opt dm.min_free_space=10%`
|
||||
Example use:: `docker daemon --storage-opt dm.min_free_space=10%`
|
||||
|
||||
## ZFS options
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue