moby/pkg
John Howard 85ad4b16c1 Windows: Experimental: Allow containerd for runtime
Signed-off-by: John Howard <jhoward@microsoft.com>

This is the first step in refactoring moby (dockerd) to use containerd on Windows.
Similar to the current model in Linux, this adds the option to enable it for runtime.
It does not switch the graphdriver to containerd snapshotters.

 - Refactors libcontainerd to a series of subpackages so that either a
  "local" containerd (1) or a "remote" (2) containerd can be loaded as opposed
  to conditional compile as "local" for Windows and "remote" for Linux.

 - Updates libcontainerd such that Windows has an option to allow the use of a
   "remote" containerd. Here, it communicates over a named pipe using GRPC.
   This is currently guarded behind the experimental flag, an environment variable,
   and the providing of a pipename to connect to containerd.

 - Infrastructure pieces such as under pkg/system to have helper functions for
   determining whether containerd is being used.

(1) "local" containerd is what the daemon on Windows has used since inception.
It's not really containerd at all - it's simply local invocation of HCS APIs
directly in-process from the daemon through the Microsoft/hcsshim library.

(2) "remote" containerd is what docker on Linux uses for it's runtime. It means
that there is a separate containerd service running, and docker communicates over
GRPC to it.

To try this out, you will need to start with something like the following:

Window 1:
	containerd --log-level debug

Window 2:
	$env:DOCKER_WINDOWS_CONTAINERD=1
	dockerd --experimental -D --containerd \\.\pipe\containerd-containerd

You will need the following binary from github.com/containerd/containerd in your path:
 - containerd.exe

You will need the following binaries from github.com/Microsoft/hcsshim in your path:
 - runhcs.exe
 - containerd-shim-runhcs-v1.exe

For LCOW, it will require and initrd.img and kernel in `C:\Program Files\Linux Containers`.
This is no different to the current requirements. However, you may need updated binaries,
particularly initrd.img built from Microsoft/opengcs as (at the time of writing), Linuxkit
binaries are somewhat out of date.

Note that containerd and hcsshim for HCS v2 APIs do not yet support all the required
functionality needed for docker. This will come in time - this is a baby (although large)
step to migrating Docker on Windows to containerd.

Note that the HCS v2 APIs are only called on RS5+ builds. RS1..RS4 will still use
HCS v1 APIs as the v2 APIs were not fully developed enough on these builds to be usable.
This abstraction is done in HCSShim. (Referring specifically to runtime)

Note the LCOW graphdriver still uses HCS v1 APIs regardless.

Note also that this does not migrate docker to use containerd snapshotters
rather than graphdrivers. This needs to be done in conjunction with Linux also
doing the same switch.
2019-03-12 18:41:55 -07:00
..
aaparser Add canonical import comment 2018-02-05 16:51:57 -05:00
archive pkg/archive:CopyTo(): fix for long dest filename 2019-01-24 18:10:52 -08:00
authorization Format code with gofmt -s from go-1.11beta1 2018-09-06 15:24:16 -07:00
broadcaster Various code-cleanup 2018-05-23 17:50:54 +02:00
chrootarchive Add ADD/COPY --chown flag support to Windows 2018-08-13 21:59:11 -07:00
containerfs Add ADD/COPY --chown flag support to Windows 2018-08-13 21:59:11 -07:00
devicemapper Various code-cleanup 2018-05-23 17:50:54 +02:00
directory Extract volume interaction to a volumes service 2018-05-25 14:21:07 -04:00
discovery Add canonical import comment 2018-02-05 16:51:57 -05:00
dmesg Add canonical import comment 2018-02-05 16:51:57 -05:00
filenotify pkg/filenotify/poller: fix Close() 2018-08-29 22:16:04 -07:00
fileutils Remove duplicated words in pkg files 2018-10-05 22:32:14 +08:00
fsutils Add canonical import comment 2018-02-05 16:51:57 -05:00
homedir allow running dockerd in an unprivileged user namespace (rootless mode) 2019-02-04 00:24:27 +09:00
idtools Add ADD/COPY --chown flag support to Windows 2018-08-13 21:59:11 -07:00
ioutils Update tests to use gotest.tools 👼 2018-06-13 09:04:30 +02:00
jsonmessage replace gotty with aec, since gotty hasn't been updated since very long time and aec can drop in for gotty 2019-01-16 22:10:39 +05:30
locker Add canonical import comment 2018-02-05 16:51:57 -05:00
longpath Add canonical import comment 2018-02-05 16:51:57 -05:00
loopback Add canonical import comment 2018-02-05 16:51:57 -05:00
mount Fix some go_vet issues 2018-12-19 23:57:06 +01:00
namesgenerator Makes a few modifications to the name generator. 2018-12-23 10:22:28 +00:00
parsers Skip kernel-memory tests on RHEL/CentOS daemons 2019-01-04 13:27:46 +01:00
pidfile Add canonical import comment 2018-02-05 16:51:57 -05:00
platform Add canonical import comment 2018-02-05 16:51:57 -05:00
plugingetter Move plugin client to separate interface 2018-05-30 15:22:10 -04:00
plugins Remove duplicated words in pkg files 2018-10-05 22:32:14 +08:00
pools Update tests to use gotest.tools 👼 2018-06-13 09:04:30 +02:00
progress pkg/progress: work around closing closed channel panic 2018-09-06 20:20:12 +00:00
pubsub Various code-cleanup 2018-05-23 17:50:54 +02:00
reexec fix build on OpenBSD by defining Self() 2018-06-16 19:09:04 +02:00
signal pkg/signal.TestTrap: use a subtest 2019-01-12 18:04:44 +01:00
stdcopy Merge pull request #37088 from ohbarye/fix-typos-duplicated-the 2018-05-19 20:57:47 +02:00
streamformatter Remove duplicated words in pkg files 2018-10-05 22:32:14 +08:00
stringid Add canonical import comment 2018-02-05 16:51:57 -05:00
symlink Update LICENSE 2018-09-12 14:27:53 +01:00
sysinfo allow running dockerd in an unprivileged user namespace (rootless mode) 2019-02-04 00:24:27 +09:00
system Windows: Experimental: Allow containerd for runtime 2019-03-12 18:41:55 -07:00
tailfile Use assert.NilError() instead of assert.Assert() 2019-01-21 13:16:02 +01:00
tarsum pkg/tarsum: fix unit test for Go 1.11+ 2018-07-19 10:38:45 +03:00
term Fix #28814 2018-08-07 10:20:13 +02:00
truncindex fixes display text in Multiple IDs found with provided prefix 2018-09-24 12:17:31 +08:00
urlutil Be explicit about github.com prefix being a legacy feature 2018-05-30 11:09:14 +02:00
useragent Add canonical import comment 2018-02-05 16:51:57 -05:00
README.md Rename a few docker to moby 2017-10-25 13:56:12 +02:00

pkg/ is a collection of utility packages used by the Moby project without being specific to its internals.

Utility packages are kept separate from the moby core codebase to keep it as small and concise as possible. If some utilities grow larger and their APIs stabilize, they may be moved to their own repository under the Moby organization, to facilitate re-use by other projects. However that is not the priority.

The directory pkg is named after the same directory in the camlistore project. Since Brad is a core Go maintainer, we thought it made sense to copy his methods for organizing Go code :) Thanks Brad!

Because utility packages are small and neatly separated from the rest of the codebase, they are a good place to start for aspiring maintainers and contributors. Get in touch if you want to help maintain them!