Merge pull request #4463 from tianon/update-packagers-doc-and-lxc-dep

Update PACKAGERS.md and hack/make/ubuntu

docs/sources/installation/binaries.rst

@@ -26,10 +26,7 @@ Check runtime dependencies
 
 To run properly, docker needs the following software to be installed at runtime:
 
-- iproute2 version 3.5 or later (build after 2012-05-21), and
-  specifically the "ip" utility
 - iptables version 1.4 or later
-- The LXC utility scripts (http://lxc.sourceforge.net) version 0.8 or later
 - Git version 1.7 or later
 - XZ Utils 4.9 or later
 
@@ -41,7 +38,7 @@ Docker in daemon mode has specific kernel requirements. For details,
 check your distribution in :ref:`installation_list`.
 
 Note that Docker also has a client mode, which can run on virtually
-any linux kernel (it even builds on OSX!).
+any Linux kernel (it even builds on OSX!).
 
 
 Get the docker binary:

hack/PACKAGERS.md

@@ -1,65 +1,94 @@
-Dear packager.
+# Dear Packager,
 
-If you are looking to make docker available on your favorite software distribution,
-this document is for you. It summarizes the requirements for building and running
-docker.
+If you are looking to make Docker available on your favorite software
+distribution, this document is for you. It summarizes the requirements for
+building and running the Docker client and the Docker daemon.
 
-## Getting started
+## Getting Started
 
-We really want to help you package Docker successfully. Before anything, a good first step
-is to introduce yourself on the [docker-dev mailing list](https://groups.google.com/forum/?fromgroups#!forum/docker-dev)
-, explain what you''re trying to achieve, and tell us how we can help. Don''t worry, we don''t bite!
-There might even be someone already working on packaging for the same distro!
+We want to help you package Docker successfully. Before doing any packaging, a
+good first step is to introduce yourself on the [docker-dev mailing
+list](https://groups.google.com/d/forum/docker-dev), explain what you're trying
+to achieve, and tell us how we can help. Don't worry, we don't bite! There might
+even be someone already working on packaging for the same distro!
 
-You can also join the IRC channel - #docker and #docker-dev on Freenode are both active and friendly.
+You can also join the IRC channel - #docker and #docker-dev on Freenode are both
+active and friendly.
 
-## Package name
+We like to refer to Tianon ("@tianon" on GitHub and "tianon" on IRC) as our
+"Packagers Relations", since he's always working to make sure our packagers have
+a good, healthy upstream to work with (both in our communication and in our
+build scripts). If you're having any kind of trouble, feel free to ping him
+directly. He also likes to keep track of what distributions we have packagers
+for, so feel free to reach out to him even just to say "Hi!"
 
-If possible, your package should be called "docker". If that name is already taken, a second
-choice is "lxc-docker".
+## Package Name
 
-## Official build vs distro build
+If possible, your package should be called "docker". If that name is already
+taken, a second choice is "lxc-docker", but with the caveat that "LXC" is now an
+optional dependency (as noted below). Another possible choice is "docker.io".
 
-The Docker project maintains its own build and release toolchain. It is pretty neat and entirely
-based on Docker (surprise!). This toolchain is the canonical way to build Docker, and the only
-method supported by the development team. We encourage you to give it a try, and if the circumstances
+## Official Build vs Distro Build
+
+The Docker project maintains its own build and release toolchain. It is pretty
+neat and entirely based on Docker (surprise!). This toolchain is the canonical
+way to build Docker. We encourage you to give it a try, and if the circumstances
 allow you to use it, we recommend that you do.
 
-You might not be able to use the official build toolchain - usually because your distribution has a
-toolchain and packaging policy of its own. We get it! Your house, your rules. The rest of this document
-should give you the information you need to package Docker your way, without denaturing it in
-the process.
+You might not be able to use the official build toolchain - usually because your
+distribution has a toolchain and packaging policy of its own. We get it! Your
+house, your rules. The rest of this document should give you the information you
+need to package Docker your way, without denaturing it in the process.
 
-## System build dependencies
+## Build Dependencies
 
-To build docker, you will need the following system dependencies
+To build Docker, you will need the following:
 
-* An amd64 machine
 * A recent version of git and mercurial
 * Go version 1.2 or later
+* A clean checkout of the source added to a valid [Go
+  workspace](http://golang.org/doc/code.html#Workspaces) under the path
+  *src/github.com/dotcloud/docker* (unless you plan to use `AUTO_GOPATH`,
+  explained in more detail below).
+
+To build the Docker daemon, you will additionally need:
+
+* An amd64/x86_64 machine running Linux
 * SQLite version 3.7.9 or later
-* libdevmapper version 1.02.68-cvs (2012-01-26) or later from lvm2 version 2.02.89 or later
-* btrfs-progs version 3.8 or later (including commit e5cb128 from 2013-01-07) for the necessary btrfs headers
-* A clean checkout of the source must be added to a valid Go [workspace](http://golang.org/doc/code.html#Workspaces)
-under the path *src/github.com/dotcloud/docker*.
+* libdevmapper version 1.02.68-cvs (2012-01-26) or later from lvm2 version
+  2.02.89 or later
+* btrfs-progs version 3.8 or later (including commit e5cb128 from 2013-01-07)
+  for the necessary btrfs headers
 
-## Go dependencies
+Be sure to also check out Docker's Dockerfile for the most up-to-date list of
+these build-time dependencies.
 
-All Go dependencies are vendored under ./vendor. They are used by the official build,
-so the source of truth for the current version is whatever is in ./vendor.
+### Go Dependencies
 
-To use the vendored dependencies, simply make sure the path to ./vendor is included in $GOPATH.
+All Go dependencies are vendored under "./vendor". They are used by the official
+build, so the source of truth for the current version of each dependency is
+whatever is in "./vendor".
 
-If you would rather package these dependencies yourself, take a look at ./hack/vendor.sh for an
-easy-to-parse list of the exact version for each.
+To use the vendored dependencies, simply make sure the path to "./vendor" is
+included in `GOPATH` (or use `AUTO_GOPATH`, as explained below).
 
-NOTE: if you''re not able to package the exact version (to the exact commit) of a given dependency,
-please get in touch so we can remediate! Who knows what discrepancies can be caused by even the
-slightest deviation. We promise to do our best to make everybody happy.
+If you would rather (or must, due to distro policy) package these dependencies
+yourself, take a look at "./hack/vendor.sh" for an easy-to-parse list of the
+exact version for each.
+
+NOTE: if you're not able to package the exact version (to the exact commit) of a
+given dependency, please get in touch so we can remediate! Who knows what
+discrepancies can be caused by even the slightest deviation. We promise to do
+our best to make everybody happy.
 
 ## Stripping Binaries
 
-Please, please, please do not strip any compiled binaries. This is really important.
+Please, please, please do not strip any compiled binaries. This is really
+important.
+
+In our own testing, stripping the resulting binaries sometimes results in a
+binary that appears to work, but more often causes random panics, segfaults, and
+other issues. Even if the binary appears to work, please don't strip.
 
 See the following quotes from Dave Cheney, which explain this position better
 from the upstream Golang perspective.
@@ -94,79 +123,157 @@ from the upstream Golang perspective.
 
 ## Building Docker
 
-To build the docker binary, run the following command with the source checkout as the
-working directory:
+Please use our build script ("./hack/make.sh") for all your compilation of
+Docker. If there's something you need that it isn't doing, or something it could
+be doing to make your life as a packager easier, please get in touch with Tianon
+and help us rectify the situation. Chances are good that other packagers have
+probably run into the same problems and a fix might already be in the works, but
+none of us will know for sure unless you harass Tianon about it. :)
+
+All the commands listed within this section should be run with the Docker source
+checkout as the current working directory.
+
+### `AUTO_GOPATH`
+
+If you'd rather not be bothered with the hassles that setting up `GOPATH`
+appropriately can be, and prefer to just get a "build that works", you should
+add something similar to this to whatever script or process you're using to
+build Docker:
 
 ```bash
-./hack/make.sh binary
+export AUTO_GOPATH=1
 ```
 
-This will create a static binary under *./bundles/$VERSION/binary/docker-$VERSION*, where
-*$VERSION* is the contents of the file *./VERSION*.
+This will cause the build scripts to set up a reasonable `GOPATH` that
+automatically and properly includes both dotcloud/docker from the local
+directory, and the local "./vendor" directory as necessary.
 
-You are encouraged to use ./hack/make.sh without modification. If you must absolutely write
-your own script (are you really, really sure you need to? make.sh is really not that complicated),
-then please take care the respect the following:
+### Static Daemon
 
-* In *./hack/make.sh*: $LDFLAGS, $BUILDFLAGS, $VERSION and $GITCOMMIT
-* In *./hack/make/binary*: the exact build command to run
+If it is feasible within the constraints of your distribution, you should
+seriously consider packaging Docker as a single static binary. A good comparison
+is Busybox, which is often packaged statically as a feature to enable mass
+portability. Because of the unique way Docker operates, being similarly static
+is a "feature".
+
+To build a static Docker daemon binary, run the following command (first
+ensuring that all the necessary libraries are available in static form for
+linking - see the "Build Dependencies" section above, and the relevant lines
+within Docker's own Dockerfile that set up our official build environment):
+
+```bash
+./hack/make.sh binary
+```
 
-You may be tempted to tweak these settings. In particular, being a rigorous maintainer, you may want
-to disable static linking. Please don''t! Docker *needs* to be statically linked to function properly.
-You would do the users of your distro a disservice and "void the docker warranty" by changing the flags.
+This will create a static binary under
+"./bundles/$VERSION/binary/docker-$VERSION", where "$VERSION" is the contents of
+the file "./VERSION". This binary is usually installed somewhere like
+"/usr/bin/docker".
 
-A good comparison is Busybox: all distros package it as a statically linked binary, because it just
-makes sense. Docker is the same way.
+### Dynamic Daemon / Client-only Binary
 
-If you *must* have a non-static Docker binary, please use:
+If you need to (due to distro policy, distro library availability, or for other
+reasons) create a dynamically compiled daemon binary, or if you are only
+interested in creating a client binary for Docker, use something similar to the
+following:
 
 ```bash
 ./hack/make.sh dynbinary
 ```
 
-This will create *./bundles/$VERSION/dynbinary/docker-$VERSION* and *./bundles/$VERSION/binary/dockerinit-$VERSION*.
-The first of these would usually be installed at */usr/bin/docker*, while the second must be installed
-at */usr/libexec/docker/dockerinit*.
+This will create "./bundles/$VERSION/dynbinary/docker-$VERSION", which for
+client-only builds is the important file to grab and install as appropriate.
+
+For daemon builds, you will also need to grab and install
+"./bundles/$VERSION/dynbinary/dockerinit-$VERSION", which is created from the
+minimal set of Docker's codebase that _must_ be compiled statically (and is thus
+a pure static binary). The acceptable locations Docker will search for this file
+are as follows (in order):
 
-## Testing Docker
+* as "dockerinit" in the same directory as the daemon binary (ie, if docker is
+  installed at "/usr/bin/docker", then "/usr/bin/dockerinit" will be the first
+  place this file is searched for)
+* "/usr/libexec/docker/dockerinit" or "/usr/local/libexec/docker/dockerinit"
+  ([FHS 3.0 Draft](http://www.linuxbase.org/betaspecs/fhs/fhs.html#usrlibexec))
+* "/usr/lib/docker/dockerinit" or "/usr/local/lib/docker/dockerinit" ([FHS
+  2.3](http://refspecs.linuxfoundation.org/FHS_2.3/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA))
 
-Before releasing your binary, make sure to run the tests! Run the following command with the source
-checkout as the working directory:
+If (and please, only if) one of the paths above is insufficient due to distro
+policy or similar issues, you may use the `DOCKER_INITPATH` environment variable
+at compile-time as follows to set a different path for Docker to search:
 
 ```bash
-./hack/make.sh test
+export DOCKER_INITPATH=/usr/lib/docker.io/dockerinit
 ```
 
-The test suite includes both live integration tests and unit tests, so you will need all runtime
-dependencies to be installed (see below).
+If you find yourself needing this, please don't hesitate to reach out to Tianon
+to see if it would be reasonable or helpful to add more paths to Docker's list,
+especially if there's a relevant standard worth referencing (such as the FHS).
 
-The test suite will also download a small test container, so you will need internet connectivity.
+Also, it goes without saying, but for the purposes of the daemon please consider
+these two binaries ("docker" and "dockerinit") as if they were a single unit.
+Mixing and matching can cause undesired consequences, and will fail to run
+properly.
 
-## Runtime dependencies
+## System Dependencies
 
-To run properly, docker needs the following software to be installed at runtime:
+### Runtime Dependencies
+
+To function properly, the Docker daemon needs the following software to be
+installed and available at runtime:
 
-* iproute2 version 3.5 or later (build after 2012-05-21), and specifically the "ip" utility
 * iptables version 1.4 or later
-* The LXC utility scripts (http://lxc.sourceforge.net) version 0.8 or later
+* XZ Utils version 4.9 or later
+
+Additionally, the Docker client needs the following software to be installed and
+available at runtime:
+
 * Git version 1.7 or later
-* XZ Utils 4.9 or later
 
-## Kernel dependencies
+### Kernel Requirements
 
-Docker in daemon mode has specific kernel requirements. For details, see
-http://docs.docker.io/en/latest/installation/kernel/
+The Docker daemon has very specific kernel requirements. Most pre-packaged
+kernels already include the necessary options enabled. If you are building your
+own kernel, you will either need to discover the options necessary via trial and
+error, or check out the [Gentoo
+ebuild](https://github.com/tianon/docker-overlay/blob/master/app-emulation/docker/docker-9999.ebuild),
+in which a list is maintained (and if there are any issues or discrepancies in
+that list, please contact Tianon so they can be rectified).
 
-Note that Docker also has a client mode, which can run on virtually any linux kernel (it even builds
-on OSX!).
+Note that in client mode, there are no specific kernel requirements, and that
+the client will even run on alternative platforms such as Mac OS X / Darwin.
 
-## Init script
+### Optional Dependencies
 
-Docker expects to run as a daemon at machine startup. Your package will need to include a script
-for your distro''s process supervisor of choice.
+Some of Docker's features are activated by using optional command-line flags or
+by having support for them in the kernel or userspace. A few examples include:
 
-Docker should be run as root, with the following arguments:
+* LXC execution driver (requires version 0.8 or later of the LXC utility scripts)
+* AUFS graph driver (requires AUFS patches/support enabled in the kernel, and at
+  least the "auplink" utility from aufs-tools)
+* experimental BTRFS graph driver (requires BTRFS support enabled in the kernel)
+
+## Daemon Init Script
+
+Docker expects to run as a daemon at machine startup. Your package will need to
+include a script for your distro's process supervisor of choice. Be sure to
+check out the "contrib/init" folder in case a suitable init script already
+exists (and if one does not, contact Tianon about whether it might be
+appropriate for your distro's init script to live there too!).
+
+In general, Docker should be run as root, similar to the following:
 
 ```bash
 docker -d
 ```
+
+Generally, a `DOCKER_OPTS` variable of some kind is available for adding more
+flags (such as changing the graph driver to use BTRFS, switching the location of
+"/var/lib/docker", etc).
+
+## Communicate
+
+As a final note, please do feel free to reach out to Tianon at any time for
+pretty much anything. He really does love hearing from our packagers and wants
+to make sure we're not being a "hostile upstream". As should be a given, we
+appreciate the work our packagers do to make sure we have broad distribution!

hack/make/ubuntu

@@ -112,10 +112,10 @@ EOF
 		    --after-remove /tmp/postrm \
 		    --architecture "$PACKAGE_ARCHITECTURE" \
 		    --prefix / \
-		    --depends lxc \
-		    --depends aufs-tools \
 		    --depends iptables \
+		    --deb-recommends aufs-tools \
 		    --deb-recommends ca-certificates \
+		    --deb-recommends git \
 		    --deb-recommends xz-utils \
 		    --description "$PACKAGE_DESCRIPTION" \
 		    --maintainer "$PACKAGE_MAINTAINER" \