beenull/moby
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
moby/hack/integration-cli-on-swarm
History
Kir Kolyshkin bdcd81d330 go {build,test}: rm -i option, add go cache volume 
Looks like -i (together with DOCKER_INCREMENTAL_BINARY etc)
were used to get faster incremental builds.

Nowdays (since Go 1.10) this is no longer the case, as
go build cache is used [1]. Here's a quote:

> You do not have to use "go test -i" or "go build -i" or
> "go install" just to get fast incremental builds. We will
> not have to teach new users those workarounds anymore.
> Everything will just be fast.

To enable go cache between builds, add a volume for /root/.cache.

[1] https://groups.google.com/forum/#!msg/golang-dev/qfa3mHN4ZPA/X2UzjNV1BAAJ

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-09-26 10:10:42 -07:00
..
agent Various code-cleanup 2018-05-23 17:50:54 +02:00
host Fix swagger volume type generation 2018-05-14 13:46:20 -04:00
README.md go {build,test}: rm -i option, add go cache volume 2018-09-26 10:10:42 -07:00

README.md

Integration Testing on Swarm

IT on Swarm allows you to execute integration test in parallel across a Docker Swarm cluster

Architecture

Master service

  • Works as a funker caller
  • Calls a worker funker (-worker-service) with a chunk of -check.f filter strings (passed as a file via -input flag, typically /mnt/input)

Worker service

  • Works as a funker callee
  • Executes an equivalent of TESTFLAGS=-check.f TestFoo|TestBar|TestBaz ... make test-integration using the bind-mounted API socket (docker.sock)

Client

  • Controls master and workers via docker stack
  • No need to have a local daemon

Typically, the master and workers are supposed to be running on a cloud environment, while the client is supposed to be running on a laptop, e.g. Docker for Mac/Windows.

Requirement

  • Docker daemon 1.13 or later
  • Private registry for distributed execution with multiple nodes

Usage

Step 1: Prepare images

$ make build-integration-cli-on-swarm

Following environment variables are known to work in this step:

  • BUILDFLAGS

Note: during the transition into Moby Project, you might need to create a symbolic link $GOPATH/src/github.com/docker/docker to $GOPATH/src/github.com/moby/moby.

Step 2: Execute tests

$ ./hack/integration-cli-on-swarm/integration-cli-on-swarm -replicas 40 -push-worker-image YOUR_REGISTRY.EXAMPLE.COM/integration-cli-worker:latest

Following environment variables are known to work in this step:

  • DOCKER_GRAPHDRIVER
  • DOCKER_EXPERIMENTAL

Flags

Basic flags:

  • -replicas N: the number of worker service replicas. i.e. degree of parallelism.
  • -chunks N: the number of chunks. By default, chunks == replicas.
  • -push-worker-image REGISTRY/IMAGE:TAG: push the worker image to the registry. Note that if you have only single node and hence you do not need a private registry, you do not need to specify -push-worker-image.

Experimental flags for mitigating makespan nonuniformity:

  • -shuffle: Shuffle the test filter strings

Flags for debugging IT on Swarm itself:

  • -rand-seed N: the random seed. This flag is useful for deterministic replaying. By default(0), the timestamp is used.
  • -filters-file FILE: the file contains -check.f strings. By default, the file is automatically generated.
  • -dry-run: skip the actual workload
  • keep-executor: do not auto-remove executor containers, which is used for running privileged programs on Swarm