Commit graph

35227 commits

Author SHA1 Message Date
Arash Deshmeh
0e3012cfff Added network package to integration/internal to refactor integration tests calls to client.NetworkCreate
Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
2018-06-05 06:32:24 -04:00
Sebastiaan van Stijn
ca25df3b54
Merge pull request #37194 from AntaresS/fix-test-failure
fix a failed test
2018-06-05 02:38:04 +02:00
Sebastiaan van Stijn
5037c5a8ce
Merge pull request #36688 from cpuguy83/volumes_service
Extract volume interaction to a volumes service
2018-06-05 02:16:20 +02:00
Sebastiaan van Stijn
d7e94d6ea7
Merge pull request #37191 from adshmh/use-unique-names-for-integration-tests-secret
use unique names for resources in integration tests of secret package
2018-06-05 00:06:50 +02:00
Anda Xu
1d9973c0d4 fix a failed test
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-06-04 14:29:00 -07:00
Vincent Demeester
e466f999aa
Merge pull request #37197 from tonistiigi/vendor-buildkit
vendor: dockerfile parser from buildkit
2018-06-04 14:53:22 +02:00
Akihiro Suda
641e2c01cc
Merge pull request #37190 from vdemeester/allow-make-run-with-args
Allow to add any args when doing a make run
2018-06-03 03:42:02 +09:00
Tonis Tiigi
ad77dee8ba vendor: add new opentracing deps for session tracing
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-02 11:10:38 -07:00
Tonis Tiigi
c9ebd2f13b vendor: use dockerfile parser from buildkit
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-06-02 11:10:34 -07:00
Vincent Demeester
d2af0d96e3
Allow to add any args when doing a make run
`make DOCKERD_ARGS=--init binary run` should start the daemon with
`--init` as flags (with any other "automagically" added ones).

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-01 19:19:20 +02:00
Arash Deshmeh
a70794aff0 use unique names for resources in integration tests of integration/secret package
Signed-off-by: Arash Deshmeh <adeshmeh@ca.ibm.com>
2018-06-01 06:46:52 -04:00
Anusha Ragunathan
71cd53e4a1
Merge pull request #37177 from cpuguy83/minor_plugin_addr_change
Move plugin client to separate interface
2018-05-31 08:22:04 -07:00
Vincent Demeester
2ea4ece674
Merge pull request #37174 from thaJeztah/no_additional_magic_matching
Be explicit about github.com prefix being a legacy feature
2018-05-31 09:45:59 +02:00
Vincent Demeester
3e494c4b93
Merge pull request #37178 from tonistiigi/layer-leak
builder: fix layer leak on multi-stage wildcard copy
2018-05-31 09:39:28 +02:00
Tibor Vass
b94d12be0c
Merge pull request #37179 from AntaresS/fix-skip
fix mis-used skip condition
2018-05-30 20:04:39 -10:00
Anda Xu
d8e6f273b5 fix the mis-used skip condition
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-05-30 17:13:20 -07:00
Tõnis Tiigi
b2719e35f5
Merge pull request #37134 from thaJeztah/fix-env-substitution
builder: fix processing of invalid substitusion syntax
2018-05-30 16:52:01 -07:00
Tonis Tiigi
4a18c11bdc builder: fix layer leak on multi-stage wildcard copy
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2018-05-30 15:18:40 -07:00
Tibor Vass
0a94468a5b
Merge pull request #37169 from AntaresS/add-version-checking
add api version checking for tests from new feature
2018-05-30 09:58:45 -10:00
Brian Goff
7c77df8acc Move plugin client to separate interface
This makes it a bit simpler to remove this interface for v2 plugins
and not break external projects (libnetwork and swarmkit).

Note that before we remove the `Client()` interface from `CompatPlugin`
libnetwork and swarmkit must be updated to explicitly check for the v1
client interface as is done int his PR.

This is just a minor tweak that I realized is needed after trying to
implement the needed changes on libnetwork.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-30 15:22:10 -04:00
Brian Goff
b85799b63f
Merge pull request #36874 from kolyshkin/stop-timeout
daemon.ContainerStop(): fix for a negative timeout
2018-05-30 13:38:42 -04:00
Anusha Ragunathan
cbbe0ff511
Merge pull request #37017 from cpuguy83/plugins_addr
Move plugin client creation to the extension point
2018-05-30 09:54:46 -07:00
Sebastiaan van Stijn
babb0c14fd
Be explicit about github.com prefix being a legacy feature
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-30 11:09:14 +02:00
Sebastiaan van Stijn
65bd038fc5
Merge pull request #37131 from kolyshkin/top-opt
Optimize ContainerTop() aka docker top
2018-05-30 03:28:07 +02:00
Anda Xu
8ed0fdebe7 add api version checking for tests from new feature
Signed-off-by: Anda Xu <anda.xu@docker.com>
2018-05-29 15:25:33 -07:00
Vincent Demeester
864b36255d
Merge pull request #37166 from yongtang/05282018-unique-service-name
Add `t.Name()` to tests so that service names are unique
2018-05-29 12:04:21 +02:00
Vincent Demeester
2d7ae3e7e5
Merge pull request #37143 from arm64b/copy-after-command-exit
Alternative failure mitigation of `TestExecInteractiveStdinClose`
2018-05-29 12:03:46 +02:00
Yong Tang
76195f8047 Add t.Name() to tests so that service names are unique
This fix adds `t.Name()` to tests in integration/network
so that services created in those tests have unique names.

Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
2018-05-28 22:57:52 +00:00
Dennis Chen
386e0f36c4 Alternative failure mitigation of TestExecInteractiveStdinClose
Begin to copy the data until the command to exit and any coping to
stdin or copy from stdout/stderr has completed.
Also adding defense code to trim the possible '\x00' null value.

Signed-off-by: Dennis Chen <dennis.chen@arm.com>
2018-05-28 11:25:31 +08:00
Brian Goff
f51a96c016 Move plugin client creation to the extension point
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 15:18:53 -04:00
Brian Goff
e4b6adc88e Extract volume interaction to a volumes service
This cleans up some of the package API's used for interacting with
volumes, and simplifies management.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2018-05-25 14:21:07 -04:00
Sebastiaan van Stijn
2628896b5e
Handle multi-digit positional parameters
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-25 18:51:11 +02:00
Sebastiaan van Stijn
9654e9b6f8
Add detection of "special parameters" for substitution
Detect Special parameters as defined in
http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_05_02

Treat these as parameters that are not set, instead of
producing an error that a modifier is missing.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-25 18:20:22 +02:00
Sebastiaan van Stijn
8687a3f4b8
Add more test-cases for positional parameters
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-25 18:20:10 +02:00
Sebastiaan van Stijn
b80e0309d2
Add line-numbers to asserts
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-25 18:20:01 +02:00
Sebastiaan van Stijn
334bf3ea76
Fix detection for missing parameter in substitution
`${}`, `${:}` and so on are invalid because there's
no parameter within the brackets; fix detection for
this situation and add/update tests.

There were some existing test-cases that were testing
for the wrong behavior, which are now updated.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-25 18:19:32 +02:00
Sebastiaan van Stijn
955a6ad95f
builder: fix processing of invalid substitusion syntax
The builder did not detect syntax errors in substitusions in the
Dockerfile, causing those values to be processed incorrectly instead of
producing an error.

Example 1: missing `}`

    docker build --no-cache -<<'EOF'
    FROM busybox
    ARG var=${aaa:-bbb
    RUN echo $var
    EOF

Before:

    Step 3/3 : RUN echo $var
     ---> Running in f06571e77146
    bbb

After:

    Step 2/3 : ARG var=${aaa:-bbb
    failed to process "${aaa:-bbb": syntax error: missing '}'

Example 2: missing closing `}`, no default value

    docker build --no-cache -<<'EOF'
    FROM busybox
    ARG var=${aaa
    RUN echo $var
    EOF

Before:

    Step 2/3 : ARG var=${aaa
    failed to process "${aaa": missing ':' in substitution

After:

    Step 2/3 : ARG var=${aaa
    failed to process "${aaa": syntax error: missing '}'

Example 3: double opening bracket (`{`)

    docker build --no-cache -<<'EOF'
    FROM busybox
    ARG var=${{aaa:-bbb}
    RUN echo $var
    EOF

Before:

    Step 2/3 : ARG var=${{aaa:-bbb}
    failed to process "${{aaa:-bbb}": missing ':' in substitution

After:

    Step 2/3 : ARG var=${{aaa:-bbb}
    failed to process "${{aaa:-bbb}": syntax error: bad substitution

Example 4: double opening bracket (`{`), no default value

    docker build --no-cache -<<'EOF'
    FROM busybox
    ARG var=${{aaa}
    RUN echo $var
    EOF

Before:

    Step 2/3 : ARG var=${{aaa}
    failed to process "${{aaa}": missing ':' in substitution

After:

    Step 2/3 : ARG var=${{aaa}
    failed to process "${{aaa}": syntax error: bad substitution

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-25 18:19:17 +02:00
Tibor Vass
9c2c887b12
Merge pull request #37141 from thaJeztah/bump_api_v1_38
Update API version to v1 38
2018-05-24 17:54:05 -10:00
Kir Kolyshkin
a41328d570 ContainerTop: improve error message
If "ps" fails, in many cases it prints a meaningful error message
which a user can benefit from. Let's use it.

While at it, let's use errdefs.System to classify the error,
as well as errors.Wrap.

Before:

> $ docker top $CT <any bad ps options>
> Error response from daemon: Error running ps: exit status 1

After:

> $ docker top $CT auxm
> Error response from daemon: ps: error: thread display conflicts with forest display

or

> $ docker top $CT saur
> Error response from daemon: ps: error: conflicting format options

or, if there's no meaningful error on stderr, same as before:

> $ docker top $CT 1234
> Error response from daemon: ps: exit status 1

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-24 18:24:36 -07:00
Kir Kolyshkin
a076badb8b ContainerTop: speed up
Current ContainerTop (a.k.a. docker top) implementation uses "ps"
to get the info about *all* running processes, then parses it, then
filters the results to only contain PIDs used by the container.
Collecting data only to throw most of it away is inefficient,
especially on a system running many containers (or processes).
For example, "docker top" on a container with a single process
can take up to 0.5 seconds to execute (on a mostly idle system)
which is noticeably slow.

Since the containers PIDs are known beforehand, let's use ps's
"-q" option to provide it with a list of PIDs we want info about.

The problem with this approach is, some ps options can't be used
with "-q" (the only one I'm aware of is "f" ("forest view") but
there might be more). As the list of such options is not known,
in case ps fails, it is executed again without "q" (retaining
the old behavior).

Next, the data produced by "ps" is filtered in the same way as before.
The difference here is, in case "-q" worked, the list is much shorter.

I ran some benchmarks on my laptop, with about 8000 "sleep" processes
running to amplify the savings.

The improvement in "docker top" execution times is 5x to 10x (roughly
0.05s vs 0.5s).

The improvement in ContainerTop() execution time is up to 100x
(roughly 3ms vs 300ms).

I haven't measured the memory or the CPU time savings, guess those
are not that critical.

NOTE that busybox ps does not implement -q so the fallback is always
used, but AFAIK it is not usable anyway and Docker expects a normal
ps to be on the system (say the list of fields produced by
"busybox ps -ef" differs from normal "ps -ef" etc.).

Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
2018-05-24 18:24:27 -07:00
Sebastiaan van Stijn
80bfcc3a7a
Merge pull request #37136 from muesli/sdnotify-api-const
Use go-systemd const instead of magic string in Linux version of dockerd
2018-05-24 14:30:28 +02:00
Sebastiaan van Stijn
aa2d23a735
Merge pull request #37133 from cpuguy83/bump_continuity
Bump continuity
2018-05-24 13:56:59 +02:00
Sebastiaan van Stijn
a061b1e2d8
Adjust API version to match correct release
This fix was not yet included in Docker 17.05, so
API version v1.37 was not the right selector (Docker
18.03, 18.04 and 18.05 all support API v1.37).

We should change these checks for engine versions,
or use a different method to skip tests when running
against older engines.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-24 02:39:56 +02:00
Sebastiaan van Stijn
df5175e1ee
Bump API version to v1.38
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-24 02:39:28 +02:00
Sebastiaan van Stijn
8e2f920306
Merge pull request #37108 from thaJeztah/remove_unneeded_aliases
Various code-cleanup
2018-05-24 01:38:56 +02:00
Christian Muehlhaeuser
703c3c5735
Adapt listeners to upstream API changes in go-systemd
Signed-off-by: Christian Muehlhaeuser <muesli@gmail.com>
2018-05-23 21:57:30 +02:00
Brian Goff
5a68e2617d
Merge pull request #35246 from dperny/attachment-inspect
Output network attachment task information
2018-05-23 15:44:39 -04:00
Christian Muehlhaeuser
d393774a53
Use go-systemd const instead of magic string in Linux version of dockerd
Signed-off-by: Christian Muehlhaeuser <muesli@gmail.com>
2018-05-23 19:06:34 +02:00
Christian Muehlhaeuser
8991bac2e0
Bump go-systemd dependency to v17
Signed-off-by: Christian Muehlhaeuser <muesli@gmail.com>
2018-05-23 19:06:13 +02:00
Sebastiaan van Stijn
f23c00d870
Various code-cleanup
remove unnescessary import aliases, brackets, and so on.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2018-05-23 17:50:54 +02:00