Commit graph

36 commits

Author SHA1 Message Date
Tonis Tiigi
a47cd63915 Fix closing attach streams on lost tcp connection
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2016-04-06 21:27:47 -07:00
Brian Goff
9a25b1d942 Improve performance/reduce allocs of bytespipe
Creates a `fixedBuffer` type that is used to encapsulate functionality
for reading/writing from the underlying byte slices.

Uses lazily-loaded set of sync.Pools for storing buffers that are no
longer needed so they can be re-used.

```
benchmark                     old ns/op     new ns/op     delta
BenchmarkBytesPipeWrite-8     138469        48985         -64.62%
BenchmarkBytesPipeRead-8      130922        56601         -56.77%

benchmark                     old allocs     new allocs     delta
BenchmarkBytesPipeWrite-8     18             8              -55.56%
BenchmarkBytesPipeRead-8      0              0              +0.00%

benchmark                     old bytes     new bytes     delta
BenchmarkBytesPipeWrite-8     66903         1649          -97.54%
BenchmarkBytesPipeRead-8      0             1             +Inf%
```

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-04-05 13:06:04 -04:00
Alexander Morozov
07b568cb53 pkg: cleanup some unused code
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2016-03-31 11:39:59 -07:00
Brian Goff
96f1a1a10b Cleanup WriteFlusher
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2016-02-09 14:02:26 -05:00
Justas Brazauskas
927b334ebf Fix typos found across repository
Signed-off-by: Justas Brazauskas <brazauskasjustas@gmail.com>
2015-12-13 18:04:12 +02:00
Aaron Lehmann
572ce80230 Improved push and pull with upload manager and download manager
This commit adds a transfer manager which deduplicates and schedules
transfers, and also an upload manager and download manager that build on
top of the transfer manager to provide high-level interfaces for uploads
and downloads. The push and pull code is modified to use these building
blocks.

Some benefits of the changes:

- Simplification of push/pull code
- Pushes can upload layers concurrently
- Failed downloads and uploads are retried after backoff delays
- Cancellation is supported, but individual transfers will only be
  cancelled if all pushes or pulls using them are cancelled.
- The distribution code is decoupled from Docker Engine packages and API
  conventions (i.e. streamformatter), which will make it easier to split
  out.

This commit also includes unit tests for the new distribution/xfer
package. The tests cover 87.8% of the statements in the package.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-12-09 19:13:35 -08:00
Aaron Lehmann
64f8ee444d Cap the amount of buffering done by BytesPipe
Turn BytesPipe's Read and Write functions into blocking, goroutine-safe
functions. Add a CloseWithError function to propagate an error code to
the Read function.

Adjust tests to work with the blocking Read and Write functions.

Remove BufReader, since now its users can use BytesPipe directly.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2015-11-12 10:11:29 -08:00
Stephen J Day
ec2289b2d9 Avoid panic on write after close in http
By adding a (*WriteFlusher).Close, we limit the Write calls to possibly
deallocated http response buffers to the lifetime of an http request.
Typically, this is seen as a very confusing panic, the cause is usually a
situation where an http.ResponseWriter is held after request completion. We
avoid the panic by disallowing further writes to the response writer after the
request is completed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2015-11-02 18:14:43 -08:00
Tibor Vass
e0ef11a4c2 Abstract builder and implement server-side dockerfile builder
This patch creates interfaces in builder/ for building Docker images.
It is a first step in a series of patches to remove the daemon
dependency on builder and later allow a client-side Dockerfile builder
as well as potential builder plugins.

It is needed because we cannot remove the /build API endpoint, so we
need to keep the server-side Dockerfile builder, but we also want to
reuse the same Dockerfile parser and evaluator for both server-side and
client-side.

builder/dockerfile/ and api/server/builder.go contain implementations
of those interfaces as a refactoring of the current code.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-10-06 19:10:19 -04:00
Brian Goff
252af0ae2f Merge pull request #16570 from duglin/ReaderFix
Make Close() on simpleReaderCloser actually close the reader
2015-09-29 12:55:03 -04:00
Srini Brahmaroutu
7c52fcce85 goroutine fairness is not guaranteed causing the hang with GCCGO
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-09-25 14:40:16 +00:00
Doug Davis
b5265fe7ad Make Close() on simpleReaderCloser actually close the reader
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-24 20:22:59 -07:00
Jess Frazelle
8c33c6c737 Merge pull request #16190 from LK4D4/drain_refactor
Refactoring of bufReader
2015-09-24 01:54:40 -07:00
Tonis Tiigi
c5b23337c3 Make bytesPipe use linear allocations
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-09-23 17:12:54 -07:00
Alexander Morozov
56b70bf84e Refactor bufReader to use BytesPipe
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-09-23 16:00:28 -07:00
Alexander Morozov
24310b5b4a Add BytesPipe datastructure to ioutils
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-09-23 16:00:27 -07:00
Hu Keping
141e91c480 Format output of docker info
Format those info which will only be displayed when daemon is
in debug mode.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-23 14:35:13 +08:00
unclejack
f9df813a99 pkg/ioutils: lower bufReader reset timeout
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2015-08-28 23:13:02 +03:00
Veres Lajos
5146232723 typofix - https://github.com/vlajos/misspell_fixer
Signed-off-by: Veres Lajos <vlajos@gmail.com>
2015-08-07 23:25:49 +01:00
Lei Jitang
27a8f9937e Enable golint part of #14756
pkg/broadcastwriter
pkg/graphdb
pkg/httputils
pkg/ioutils

Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-08-03 09:45:05 +08:00
David Calavera
40ea67a596 Fix reset timeout for buffer readers.
Use our goroutine-safe random source.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-07-28 14:30:18 -07:00
Brian Goff
c0391bf554 Split reader interface from logger interface
Implement new reader interface on jsonfile.
Moves jsonlog decoding from daemon to jsonfile logger.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-07-21 20:47:31 -04:00
David Calavera
7b83b0e15c Merge pull request #14605 from brahmaroutu/gccgo_scheduler
Go Scheduler issue with sync.Mutex
2015-07-17 08:16:32 -07:00
root
9ca913d0f1 Go Scheduler issue with sync.Mutex using gccgo
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-07-17 01:33:58 +00:00
Tibor Vass
276c640be4 remove pkg/transport and use the one from distribution
Signed-off-by: Tibor Vass <tibor@docker.com>
2015-07-16 13:13:46 -04:00
Victor Vieux
c790aa36ea no not print empty keys in docker info
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2015-06-01 15:05:20 -07:00
vagrant
d2a2c974d9 fix to gccgo compilation error
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
2015-05-14 16:28:23 +00:00
Tianon Gravi
223d6de728 Move WriteFlusher out of utils into ioutils
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
2015-05-08 12:33:33 -06:00
Vincent Demeester
815b472a02 Add more ioutils tests.
Closes #11595

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-05-04 19:56:10 +02:00
Antonio Murdaca
c30a55f14d Refactor utils/utils, fixes #11923
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-04-14 01:37:36 +02:00
unclejack
e5ea2b2357 pkg/ioutils: avoid huge Buffer growth in bufreader
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2015-03-18 19:51:55 +02:00
unclejack
028f7987fe pkg/ioutils: add tests for BufReader
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2015-03-18 19:32:46 +02:00
unclejack
bd130e72a0 pkg/ioutils: add ReaderErrWrapper to readers
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00
unclejack
c93b9e81c0 pkg/ioutils: add NewBufReaderWithDrainbufAndBuffer
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00
unclejack
7b6f34057c pkg/ioutils: add WriteCloserWrapper
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00
unclejack
76212635b5 move some io related utils to pkg/ioutils
Docker-DCO-1.1-Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com> (github: unclejack)
2014-09-03 11:36:21 +03:00