The intent of this check is to not add integration tests that use the
CLI to test API functionality, so check for changes in "API" tests as
well, now that the `integration/` suite has matured enough to develop
new tests, or migrate existing ones.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The golang-ci-lint takes longest to run, so do the other
checks first; this also makes the output a bit more readable.
While at it, fix some consistency issues in the output of the
other checks.
Before this change:
TEST_FORCE_VALIDATE=1 hack/validate/default
Congratulations! All commits are properly signed with the DCO!
Congratulations! Seccomp profile generation is done correctly.
INFO [config_reader] Used config file hack/validate/golangci-lint.yml
INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
...
INFO Execution took 2m12.4287302s
Congratulations! "./pkg/..." is safely isolated from internal code.
The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
Congratulations! All api changes are done the right way.
Congratulations! All toml source files changed here have valid syntax.
Congratulations! Changelog CHANGELOG.md is well-formed.
Congratulations! Changelog CHANGELOG.md dates are in descending order.
After this change:
TEST_FORCE_VALIDATE=1 hack/validate/default
Congratulations! All commits are properly signed with the DCO!
Congratulations! Seccomp profile generation is done correctly.
Congratulations! Packages in "./pkg/..." are safely isolated from internal code.
Congratulations! The swagger spec at "api/swagger.yaml" is valid against swagger specification 2.0
Congratulations! All API changes are done the right way.
Congratulations! All TOML source files changed here have valid syntax.
Congratulations! Changelog CHANGELOG.md is well-formed.
Congratulations! Changelog CHANGELOG.md dates are in descending order.
Congratulations! No new tests were added to integration-cli.
INFO Start validation with golang-ci-lint
INFO [config_reader] Used config file hack/validate/golangci-lint.yml
INFO [lintersdb] Active 15 linters: [deadcode gofmt goimports golint gosec gosimple govet ineffassign misspell staticcheck structcheck typecheck unconvert unused varcheck]
...
INFO Execution took 2m12.4287302s
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This introduces a FORCE_VALIDATE environment variable, which allows
forcing some validation steps, even if no changes were detected.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The `hack/validate/deprecate-integration-cli` script exited on
success. As a result, validation steps to run afterwards would
not be executed.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It was suggested that we use '.*\(COPYING\|LICENSE\|COPYRIGHT\).*'
as an argument to `find -iregex`, and this is how it all started.
Next thing, there is no COPYRIGHT in any of the vendored packages,
so it can be removed for good.
Next, we should not look too deep inside the package directory, as
the license should be in its root directory, so add `-maxdepth 1`
to `find`. This should also speed things up.
Finally, since we're not using the recursion feature of `find`,
it can be replaced with `echo | grep`.
While at it,
* avoid temporary $pkgs variable as it is only used once;
* replace `ls -d "vendor/$f" > /dev/null 2>&1` with `test -d`.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
The export statement is definitely not needed. The rest is obvious.
> In hack/validate/vendor line 3:
> export SCRIPTDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
> ^-- SC2155: Declare and assign separately to avoid masking return values.
>
>
> In hack/validate/vendor line 43:
> if ls -d vendor/$f > /dev/null 2>&1; then
> ^-- SC2086: Double quote to prevent globbing and word splitting.
>
>
> In hack/validate/vendor line 44:
> found=$(find vendor/$f -iregex '.*LICENSE.*' -or -iregex '.*COPYRIGHT.*' -or -iregex '.*COPYING.*' | wc -l)
> ^-- SC2086: Double quote to prevent globbing and word splitting.
>
>
> In hack/validate/vendor line 45:
> if [ $found -eq 0 ]; then
> ^-- SC2086: Double quote to prevent globbing and word splitting.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
In case some files were modified (rather than merely added
or removed), we're curious to see the diff for those.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Rewrite sockRequestHijack to requestHijack which use writable
Transport's Response.Body to replace deprecated hijacked httputil.ClientConn.
```
// As of Go 1.12, the Body will also implement io.Writer
// on a successful "101 Switching Protocols" response,
// as used by WebSockets and HTTP/2's "h2c" mode.
Body io.ReadCloser
```.
TestPostContainersAttach and TestExecResizeImmediatelyAfterExecStart
replace all sockRequestHijack to requestHijack.
Signed-off-by: HuanHuan Ye <logindaveye@gmail.com>
This makes it clearer from the configuration what linters
are enabled.
Also disables the `gofmt` linter, because it's superseded
by the `goimports` linter.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
The default exclusion rules were too permissive; disable
them, but copy the relevant ones to the configuation.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
1.Change base.Dial to base.DailContext.
2.Remove proxyDialer that was previously used to configure a
net.Dialer to route proxy.Dialer which will route the connections
through the proxy using the connections through a SOCKS proxy.
SOCKS proxies are now supported by configuring only http.Transport.Proxy,
and no longer require changing http.Transport.Dial.
Signed-off-by: HuanHuan Ye <logindaveye@gmail.com>
```
distribution/registry.go:84:3: SA1019: base.Dial is deprecated: Use DialContext instead, which allows the transport to cancel dials as soon as they are no longer needed. If both are set, DialContext takes priority. (staticcheck)
registry/registry.go:188:3: SA1019: base.Dial is deprecated: Use DialContext instead, which allows the transport to cancel dials as soon as they are no longer needed. If both are set, DialContext takes priority. (staticcheck)
client/hijack.go:85:16: SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead. (staticcheck)
integration-cli/docker_api_attach_test.go:245:12: SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead. (staticcheck)
integration/plugin/authz/authz_plugin_test.go:180:7: SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead. (staticcheck)
integration/plugin/authz/authz_plugin_test.go:479:12: SA1019: httputil.NewClientConn is deprecated: Use the Client or Transport in package net/http instead. (staticcheck)
integration-cli/docker_api_attach_test.go:239:134: SA1019: httputil.ClientConn is deprecated: Use Client or Transport in package net/http instead. (staticcheck)
daemon/daemon.go:885:3: SA1019: grpc.WithDialer is deprecated: use WithContextDialer instead. Will be supported throughout 1.x. (staticcheck)
pkg/authorization/response.go:18:2: SA1019: http.CloseNotifier is deprecated: the CloseNotifier interface predates Go's context package. New code should use Request.Context instead. (staticcheck)
pkg/authorization/response.go:160:30: SA1019: http.CloseNotifier is deprecated: the CloseNotifier interface predates Go's context package. New code should use Request.Context instead. (staticcheck)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
```
pkg/archive/archive.go:407:3: SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead. (staticcheck)
pkg/archive/archive.go:408:3: SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead. (staticcheck)
pkg/archive/archive.go:661:26: SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead. (staticcheck)
pkg/archive/archive_linux.go:47:7: SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead. (staticcheck)
pkg/archive/archive_linux.go:48:12: SA1019: hdr.Xattrs is deprecated: Use PAXRecords instead. (staticcheck)
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
This `ENV` was added to the Dockerfile in b96093fa56,
when the repository used per-architecture Dockerfiles, and some architectures needed
a different configuration.
Now that we use a multi-arch Dockerfile, and CI uses a Jenkinsfile, we can remove
this `ENV` from the Dockerfile, and set it in CI instead if needed.
Also updated the wording and fixed linting issues in hack/validate/gometalinter
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
- remove -check.* flags
- use (per-test) -timeout flag
- allow user to override TEST_SKIP_* regardless of TESTFLAGS
- remove test-imports validation
Signed-off-by: Tibor Vass <tibor@docker.com>
Attempting to fix;
```
21:16:00 Traceback (most recent call last):
21:16:00 File "/usr/local/bin/yamllint", line 11, in <module>
21:16:00 sys.exit(run())
21:16:00 File "/usr/local/lib/python3.5/dist-packages/yamllint/cli.py", line 170, in run
21:16:00 problems = linter.run(f, conf, filepath)
21:16:00 File "/usr/local/lib/python3.5/dist-packages/yamllint/linter.py", line 233, in run
21:16:00 content = input.read()
21:16:00 File "/usr/lib/python3.5/encodings/ascii.py", line 26, in decode
21:16:00 return codecs.ascii_decode(input, self.errors)[0]
21:16:00 UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 123522: ordinal not in range(128)
21:16:00 Build step 'Execute shell' marked build as failure
```
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
It has been declared deprecated by the author, and has a knack for
false-positives (as well as giving bad advice when it comes to APIs --
which is quite clear when looking at "nolint: interfacer" comments).
Signed-off-by: Aleksa Sarai <asarai@suse.de>
CIs are assumed to do a git fetch and git merge before running tests.
Therefore, no need for a git fetch inside our validate scripts in CI.
If VALIDATE_ORIGIN_BRANCH is set, then git fetch is skipped and
VALIDATE_ORIGIN_BRANCH is used in validate scripts.
Otherwise, behavior is unchanged.
Signed-off-by: Tibor Vass <tibor@docker.com>
When running CI in other repositories (e.g. Docker's downstream
docker/engine repository), or other branches, the validation
scripts were calculating the list of changes based on the wrong
information.
This lead to weird failures in CI in a branch where these values
were not updated ':-) (CI on a pull request failed because it detected
that new tests were added to the deprecated `integration-cli` test-suite,
but the pull request did not actually make changes in that area).
This patch allows overriding the target repository (and branch)
to compare to (without having to edit the scripts).
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
These scripts explicitly use Bash, so we should be able to use
`[[` instead of `[` (which seems to be recommended).
Also added curly brackets to some bare variables, and quoted some paths.
This makes my IDE a bit more silent :-)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
We can do that now as we're no longer carrying archive/tar.
Note that latest vndr removes vendor/ subdir so we don't have to,
thus the change in hack/validate/vendor.
While at it, re-run a new vndr version to make sure everything
that should be there is.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
It's that time of year again! Go 1.11 is released, time to use it.
This commit also
* removes our archive/tar fork, since upstream archive/tar
is fixed for static builds, and osusergo build tag is set.
* removes ENV GO_VERSION from Dockerfile as it's not needed
anymore since PR #37592 is merged.
[v2: switch to beta2]
[v3: switch to beta3]
[v4: rc1]
[v5: remove ENV GO_VERSION as PR #37592 is now merged]
[v6: rc2]
[v7: final!]
[v8: use 1.11.0]
[v9: back to 1.11]
[v8: use 1.11.0]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Previously adding files to vendor/ without adding to vendor.conf would not fail the
validation.
Also be consistent with indentation and use tabs.
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
1. Add = between the option and the argument, otherwise the argument
appears to be passed on to the linters directly, as in:
> DEBUG: [golint.8]: executing /home/kir/go/bin/golint
> -min_confidence 0.800000 ./10m ./api ./api/errdefs <...>
2. Fix setting the default for GOMETALINTER_OPTS -- the default
was -deadline (rather than --deadline).
Fixes: b96093fa56 ("gometalinter: add per-platform configurable options")
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
I have run into two separate issues while doing 'make all' on armhf
(a Scaleway C1 machine, same as used in CI). This commit fixes both.
1. There were a lot of "not enough memory" errors, and after that
in a few runs gometalinter just stuck forever on FUTEX_WAIT with
no children left.
Looking into docs, I found the --enable-gc option which solved the issue.
[Update: this has already been added]
2. Timeout of 2 minutes is not enough for the abovementioned platform.
The longest running linter is goimports which takes almost 6 minutes to run.
Set the timeout to the observable run time roughly doubled.
In addition, ARM platforms does not have too much RAM (2GB), so
running too many processes in parallel might be problematic. Limit
it by using -j2
[v2: make the timeout arch-dependent, also tested on aarch64 (2m15s)]
[v3: moved timeout setting to Dockerfiles]
[v4: generalized to GOMETALINTER_OPTS, added -j2 for ARM platforms]
[v5: rebase to master]
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Logging plugins use the same HTTP interface as other plugins for basic
command operations meanwhile actual logging operations are handled (on
Unix) via a fifo.
The plugin interface looks like so:
```go
type loggingPlugin interface {
StartLogging(fifoPath string, loggingContext Context) error
StopLogging(fifoPath)
```
This means a plugin must implement `LoggingDriver.StartLogging` and
`LoggingDriver.StopLogging` endpoints and be able to consume the passed
in fifo.
Logs are sent via stream encoder to the fifo encoded with protobuf.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This is especially important for distributions like NixOS where `/bin/bash` doesn't exist, or for MacOS users who've installed a newer version of Bash than the one that comes with their OS.
Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
- yamllint to ensure it is a valid YAML file
- go-swagger validate to ensure it is a valid swagger file
Signed-off-by: Ben Firshman <ben@firshman.co.uk>
The validate-lint script excludes any package names that match
api/types. However, the only subpackage that appears to cause issues is
api/types/container (due to stuttering names). Tighten the filtering so
that other code inside api/types is validated.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
Allow each script to run directly without the hack/make.sh wrapper. These
scripts do not produce artifacts and do not benefit from the "bundles"
framework.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
