Having this information can help debugging issues in CI (which could
be caused by missing/incorrect configuration of the machines).
We ping to a fixed version of the script, because this script is ran
directly on the host, and we don't want pull-requests modifying this
script to have direct access to the machines.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Both of these tests are fairly short, and shouldn't interfer with
eachother, so we can combine them and re-use the same dev-image
(so that it'll only be built once).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This patch removes the manual steps to resolve the Git commit, and
instead, uses the `GIT_COMMIT` that's set by Jenkins's Git plugin.
Behavior changes slightly, because `GIT_PLUGIN` contains the full
commit-sha, not the short one.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This is basically taking some stuff that make a custom shell function
for.
This takes a test filter, builds the appropriate TESTFLAGS, and sets the
integration API test dirs that match the given filter to avoid building
all test dirs.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Container and image names are already unique because they have
the git-sha or build-number, and a single machine won't be running
tests for multiple architectures.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
From the code style guidelines;
https://wiki.jenkins.io/display/JENKINS/Code+Style+Guidelines
> 1. Use spaces. Tabs are banned.
> 2. Java blocks are 4 spaces. JavaScript blocks as for Java. XML nesting is 2 spaces
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
1. Use "in-place" variables for if statements to limit their scope to
the respectful `if` block.
2. Report the error returned from sd_journal_* by using CErr().
3. Use errors.New() instead of fmt.Errorf().
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
From the first glance, `docker logs --tail 0` does not make sense,
as it is supposed to produce no output, but `tail -n 0` from GNU
coreutils is working like that, plus there is even a test case
(`TestLogsTail` in integration-cli/docker_cli_logs_test.go).
Now, something like `docker logs --follow --tail 0` makes total
sense, so let's make it work.
(NOTE if --tail is not used, config.Tail is set to -1)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
If we take a long time to process log messages, and during that time
journal file rotation occurs, the journald client library will keep
those rotated files open until sd_journal_process() is called.
By periodically calling sd_journal_process() during the processing
loop we shrink the window of time a client instance has open file
descriptors for rotated (deleted) journal files.
This code is modelled after that of journalctl [1]; the above explanation
as well as the value of 1024 is taken from there.
[v2: fix CErr() argument]
[1] https://github.com/systemd/systemd/blob/dc16327c48d/src/journal/journalctl.c#L2676
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
TL;DR: simplify the code, fix --follow hanging indefinitely
Do the following to simplify the followJournal() code:
1. Use Go-native select instead of C-native polling.
2. Use Watch{Producer,Consumer}Gone(), eliminating the need
to have journald.closed variable, and an extra goroutine.
3. Use sd_journal_wait(). In the words of its own man page:
> A synchronous alternative for using sd_journal_get_fd(),
> sd_journal_get_events(), sd_journal_get_timeout() and
> sd_journal_process() is sd_journal_wait().
Unfortunately, the logic is still not as simple as it
could be; the reason being, once the container has exited,
journald might still be writing some logs from its internal
buffers onto journal file(s), and there is no way to
figure out whether it's done so we are guaranteed to
read all of it back. This bug can be reproduced with
something like
> $ ID=$(docker run -d busybox seq 1 150000); docker logs --follow $ID
> ...
> 128123
> $
(The last expected output line should be `150000`).
To avoid exiting from followJournal() early, add the
following logic: once the container is gone, keep trying
to drain the journal until there's no new data for at
least `waitTimeout` time period.
Should fix https://github.com/docker/for-linux/issues/575
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. The journald client library initializes inotify watch(es)
during the first call to sd_journal_get_fd(), and it make sense
to open it earlier in order to not lose any journal file rotation
events.
2. It only makes sense to call this if we're going to use it
later on -- so add a check for config.Follow.
3. Remove the redundant call to sd_journal_get_fd().
NOTE that any subsequent calls to sd_journal_get_fd() return
the same file descriptor, so there's no real need to save it
for later use in wait_for_data_cancelable().
Based on earlier patch by Nalin Dahyabhai <nalin@redhat.com>.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In case the LogConsumer is gone, the code that sends the message can
stuck forever. Wrap the code in select case, as all other loggers do.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In case Tail=N parameter is requested, we need to show N lines.
It does not make sense to walk backwards one by one if we can
do it at once. Now, if Since=T is also provided, make sure we
haven't jumped too far (before T), and if we did, move forward.
The primary motivation for this was to make the code simpler.
This also fixes a tiny bug in the "since" implementation.
Before this commit:
> $ docker logs -t --tail=6000 --since="2019-03-10T03:54:25.00" $ID | head
> 2019-03-10T03:54:24.999821000Z 95981
After:
> $ docker logs -t --tail=6000 --since="2019-03-10T03:54:25.00" $ID | head
> 2019-03-10T03:54:25.000013000Z 95982
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
When cleaning up IPC mounts, the daemon could log a warning if the IPC mount was not found;
```
cleanup: failed to unmount IPC: umount /var/lib/docker/containers/90f408e26e205d30676655a08504dddc0d17f5713c1dd4654cf67ded7d3bbb63/mounts/shm, flags: 0x2: no such file or directory"
```
These warnings are safe to ignore, but can cause some confusion; `container.UnmountIpcMount()`
already attempted to suppress these warnings, however, `mount.Unmount()` returns a `mountError`,
which nests the original error, therefore detecting failed.
This parch uses `errors.Cause()` to get the _underlying_ error to detect if it's a "is not exist".
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Protect access to q.quotas map, and lock around changing nextProjectID.
Techinically, the lock in findNextProjectID() is not needed as it is
only called during initialization, but one can never be too careful.
Fixes: 52897d1c09 ("projectquota: utility class for project quota controls")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
1. Use `go list` to get list of integration dirs to build. This means we
do not need to have a valid `.go` in every subdirectory and also
filters out other dirs like "bundles" which may have been created.
2. Add option to specify custom flags for integration and
integration-cli. This is needed so both suites can be run AND set
custom flags... since the cli suite does not support standard go
flags.
3. Add options to skip an entire integration suite.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This was inadvertedly removed in 7bfe48cc00,
because it was documented as a dependency for docker-py, but
actually used to validate the swagger file.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
