The Moby Project - a collaborative project for the container ecosystem to assemble container-based systems
Find a file
Lukas Heeren ce61a1ed98
Adding ability to change max download attempts
Moby works perfectly when you are in a situation when one has a good and stable
internet connection. Operating in area's where internet connectivity is likely
to be lost in undetermined intervals, like a satellite connection or 4G/LTE in
rural area's, can become a problem when pulling a new image. When connection is
lost while image layers are being pulled, Moby will try to reconnect up to 5 times.
If this fails, the incompletely downloaded layers are lost will need to be completely
downloaded again during the next pull request. This means that we are using more
data than we might have to.

Pulling a layer multiple times from the start can become costly over a satellite
or 4G/LTE connection. As these techniques (especially 4G) quite common in IoT and
Moby is used to run Azure IoT Edge devices, I would like to add a settable maximum
download attempts. The maximum download attempts is currently set at 5
(distribution/xfer/download.go). I would like to change this constant to a variable
that the user can set. The default will still be 5, so nothing will change from
the current version unless specified when starting the daemon with the added flag
or in the config file.

I added a default value of 5 for DefaultMaxDownloadAttempts and a settable
max-download-attempts in the daemon config file. It is also added to the config
of dockerd so it can be set with a flag when starting the daemon. This value gets
stored in the imageService of the daemon when it is initiated and can be passed
to the NewLayerDownloadManager as a parameter. It will be stored in the
LayerDownloadManager when initiated. This enables us to set the max amount of
retries in makeDownoadFunc equal to the max download attempts.

I also added some tests that are based on maxConcurrentDownloads/maxConcurrentUploads.

You can pull this version and test in a development container. Either create a config
`file /etc/docker/daemon.json` with `{"max-download-attempts"=3}``, or use
`dockerd --max-download-attempts=3 -D &` to start up the dockerd. Start downloading
a container and disconnect from the internet whilst downloading. The result would
be that it stops pulling after three attempts.

Signed-off-by: Lukas Heeren <lukas-heeren@hotmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-19 13:51:40 +02:00
.github hack: remove integration-cli-on-swarm 2019-06-01 04:45:26 +09:00
api api/server: explicitly ignore unhandled errors (errcheck) 2019-09-18 12:57:46 +02:00
builder builder-next: explicitely ignore unhandled errors (errcheck) 2019-09-18 12:57:47 +02:00
cli allow running dockerd in an unprivileged user namespace (rootless mode) 2019-02-04 00:24:27 +09:00
client client/hijack: suppress SA1019: httputil.ErrPersistEOF is deprecated (staticcheck) 2019-09-18 12:57:54 +02:00
cmd/dockerd Adding ability to change max download attempts 2019-09-19 13:51:40 +02:00
container goimports: fix imports 2019-09-18 12:56:54 +02:00
contrib Fix some inefassign warnings 2019-09-18 12:57:29 +02:00
daemon Adding ability to change max download attempts 2019-09-19 13:51:40 +02:00
distribution Adding ability to change max download attempts 2019-09-19 13:51:40 +02:00
dockerversion Remove version-checks for containerd and runc 2018-10-04 23:17:13 +02:00
docs Merge pull request from vikramhh/move_windows_gopath_out_of_goroot 2019-09-10 14:33:12 -07:00
errdefs Merge pull request from thaJeztah/pull_platform_regression 2019-07-16 03:29:16 +02:00
hack Merge pull request from thaJeztah/fix_TESTFLAGS 2019-09-18 17:56:23 -07:00
image govet: suppress warning about Parent field that repeated json tag "parent" 2019-09-18 12:57:40 +02:00
integration TestGraphdriverPluginV2: SA5001: should check returned error before deferring (staticcheck) 2019-09-18 12:57:55 +02:00
integration-cli misspell: fix some spelling issues 2019-09-18 12:57:48 +02:00
internal/test/suite testutil: make testing packages public 2019-09-11 07:47:23 -05:00
layer Remove unused functions, variables, fields 2019-09-18 12:57:12 +02:00
libcontainerd goimports: fix imports 2019-09-18 12:56:54 +02:00
oci oci: fix SA4009: argument e is overwritten before first use (staticcheck) 2019-09-18 12:57:50 +02:00
opts TestUlimitOpt: fix composite literal uses unkeyed fields (govet) 2019-09-18 12:57:34 +02:00
pkg TestPollerEvent: fix filemode (staticcheck) 2019-09-18 12:57:56 +02:00
plugin plugin/setupNewPlugin: rm dead code 2019-09-18 12:57:27 +02:00
profiles profiles: Fix file permissions on json files 2019-09-16 11:15:37 +07:00
project Add support for setting GOARM in cross target. 2019-04-17 13:19:14 -07:00
reference Remove unused functions, variables, fields 2019-09-18 12:57:12 +02:00
registry registry/newV1EndpointFromStr: rm dead code 2019-09-18 12:57:28 +02:00
reports Fix typos 2018-05-16 09:15:43 +08:00
restartmanager unconvert: remove unnescessary conversions 2019-09-18 12:57:33 +02:00
rootless goimports: fix imports 2019-09-18 12:56:54 +02:00
runconfig runconfig/config_test.go: remove unused test-utilities 2019-09-18 12:57:11 +02:00
testutil golangci-lint: enable gosec linter 2019-09-18 12:57:58 +02:00
vendor vendor: remove vdemeester/shakers and go-check/check 2019-09-09 21:09:57 +00:00
volume Remove unused functions, variables, fields 2019-09-18 12:57:12 +02:00
.DEREK.yml Update .DEREK.yml 2019-05-06 13:27:04 -07:00
.dockerignore hack: remove integration-cli-on-swarm 2019-06-01 04:45:26 +09:00
.gitignore run unit tests and generate junit report 2019-08-01 06:08:35 +00:00
.mailmap Update AUTHORS and mailmap 2019-04-27 16:50:47 -07:00
AUTHORS Update AUTHORS and mailmap 2019-04-27 16:50:47 -07:00
CHANGELOG.md Fix some typos 2018-09-07 13:13:47 +08:00
codecov.yml Add code coverage report and codecov config 2018-01-16 16:50:56 -05:00
CONTRIBUTING.md Update CONTRIBUTING.md to have an option to keep name anonymous if requested 2019-07-15 16:04:11 +02:00
Dockerfile Replace gometalinter with golangci-lint 2019-09-18 12:57:57 +02:00
Dockerfile.e2e Bump Golang 1.12.9 2019-08-16 10:02:22 +08:00
Dockerfile.simple Bump Golang 1.12.9 2019-08-16 10:02:22 +08:00
Dockerfile.windows Merge pull request from thaJeztah/fix_go_version_check_again 2019-09-11 17:34:45 -07:00
Jenkinsfile Jenkinsfile: ensure all containers are cleaned up 2019-09-18 21:48:01 +00:00
LICENSE Update LICENSE 2018-09-12 14:27:53 +01:00
MAINTAINERS fix some spelling mistakes 2019-08-13 22:57:55 +08:00
Makefile integration: simplify parallel run destination 2019-09-11 17:18:38 +02:00
NOTICE switch kr/pty to creack/pty v1.1.7 2019-07-29 16:59:08 -07:00
poule.yml Poule: remove random assign 2019-09-05 20:57:17 +02:00
README.md Add "Lego set" back in README.md 2017-10-10 14:10:39 +00:00
ROADMAP.md Fix some typos in ROADMAP.md 2019-01-25 14:27:13 +08:00
SECURITY.md Update SECURITY.md to have an option to keep name anonymous if requested. 2019-06-18 16:37:16 +00:00
TESTING.md Bump golang 1.12.8 (CVE-2019-9512, CVE-2019-9514) 2019-08-14 02:27:05 +02:00
vendor.conf vendor: remove vdemeester/shakers and go-check/check 2019-09-09 21:09:57 +00:00
VENDORING.md fix the bare url and the Summary of http://semver.org 2017-01-17 16:20:11 +08:00

The Moby Project

Moby Project logo

Moby is an open-source project created by Docker to enable and accelerate software containerization.

It provides a "Lego set" of toolkit components, the framework for assembling them into custom container-based systems, and a place for all container enthusiasts and professionals to experiment and exchange ideas. Components include container build tools, a container registry, orchestration tools, a runtime and more, and these can be used as building blocks in conjunction with other tools and projects.

Principles

Moby is an open project guided by strong principles, aiming to be modular, flexible and without too strong an opinion on user experience. It is open to the community to help set its direction.

  • Modular: the project includes lots of components that have well-defined functions and APIs that work together.
  • Batteries included but swappable: Moby includes enough components to build fully featured container system, but its modular architecture ensures that most of the components can be swapped by different implementations.
  • Usable security: Moby provides secure defaults without compromising usability.
  • Developer focused: The APIs are intended to be functional and useful to build powerful tools. They are not necessarily intended as end user tools but as components aimed at developers. Documentation and UX is aimed at developers not end users.

Audience

The Moby Project is intended for engineers, integrators and enthusiasts looking to modify, hack, fix, experiment, invent and build systems based on containers. It is not for people looking for a commercially supported system, but for people who want to work and learn with open source code.

Relationship with Docker

The components and tools in the Moby Project are initially the open source components that Docker and the community have built for the Docker Project. New projects can be added if they fit with the community goals. Docker is committed to using Moby as the upstream for the Docker Product. However, other projects are also encouraged to use Moby as an upstream, and to reuse the components in diverse ways, and all these uses will be treated in the same way. External maintainers and contributors are welcomed.

The Moby project is not intended as a location for support or feature requests for Docker products, but as a place for contributors to work on open source code, fix bugs, and make the code more useful. The releases are supported by the maintainers, community and users, on a best efforts basis only, and are not intended for customers who want enterprise or commercial support; Docker EE is the appropriate product for these use cases.


Legal

Brought to you courtesy of our legal counsel. For more context, please see the NOTICE document in this repo.

Use and transfer of Moby may be subject to certain restrictions by the United States and other governments.

It is your responsibility to ensure that your use and/or transfer does not violate applicable laws.

For more information, please see https://www.bis.doc.gov

Licensing

Moby is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.