Commit graph

56 commits

Author SHA1 Message Date
Sebastiaan van Stijn
e59f7fba6a
pkg/fileutils: remove aliases for deprecated functions and types
commit 3c69b9f2c5 replaced these functions
and types with github.com/moby/patternmatcher. That commit has shipped with
docker 23.0, and BuildKit v0.11 no longer uses the old functions, so we can
remove these.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-03 15:00:51 +02:00
Sebastiaan van Stijn
4fa853f5de
pkg/fileutils: ReadSymlinkedDirectory: preserve underlying error
We were discarding the underlying error, which made it impossible for
callers to detect (e.g.) an os.ErrNotExist.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-16 20:15:08 +02:00
Sebastiaan van Stijn
24e371c812
pkg/fileutils: improve tests
- use t.TempDir() to make sure we're testing from a clean state
- improve checks for errors to have the correct error-type where possible

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-10-16 20:15:08 +02:00
Sebastiaan van Stijn
3c69b9f2c5
replace pkg/fileutils Matching funcs with github.com/moby/patternmatcher
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 23:25:28 +02:00
Sebastiaan van Stijn
0440ca07ba
pkg/fileutils: remove gotest.tools as dependency, use t.TempDir()
In preparation of moving this package separate.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-30 17:14:38 +02:00
Sebastiaan van Stijn
412c650e05
pkg/*: fix "empty-lines" (revive)
pkg/directory/directory.go:9:49: empty-lines: extra empty line at the start of a block (revive)
    pkg/pubsub/publisher.go:8:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/loopback/attach_loopback.go:96:69: empty-lines: extra empty line at the start of a block (revive)
    pkg/devicemapper/devmapper_wrapper.go:136:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/devicemapper/devmapper.go:391:35: empty-lines: extra empty line at the end of a block (revive)
    pkg/devicemapper/devmapper.go:676:35: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/changes_posix_test.go:15:38: empty-lines: extra empty line at the end of a block (revive)
    pkg/devicemapper/devmapper.go:241:51: empty-lines: extra empty line at the start of a block (revive)
    pkg/fileutils/fileutils_test.go:17:47: empty-lines: extra empty line at the end of a block (revive)
    pkg/fileutils/fileutils_test.go:34:48: empty-lines: extra empty line at the end of a block (revive)
    pkg/fileutils/fileutils_test.go:318:32: empty-lines: extra empty line at the end of a block (revive)
    pkg/tailfile/tailfile.go:171:6: empty-lines: extra empty line at the end of a block (revive)
    pkg/tarsum/fileinfosums_test.go:16:41: empty-lines: extra empty line at the end of a block (revive)
    pkg/tarsum/tarsum_test.go:198:42: empty-lines: extra empty line at the start of a block (revive)
    pkg/tarsum/tarsum_test.go:294:25: empty-lines: extra empty line at the start of a block (revive)
    pkg/tarsum/tarsum_test.go:407:34: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/fswriters_test.go:52:45: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/writers_test.go:24:39: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/bytespipe_test.go:78:26: empty-lines: extra empty line at the end of a block (revive)
    pkg/sysinfo/sysinfo_linux_test.go:13:37: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive_linux_test.go:57:64: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/changes.go:248:72: empty-lines: extra empty line at the start of a block (revive)
    pkg/archive/changes_posix_test.go:15:38: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/copy.go:248:124: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/diff_test.go:198:44: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:304:12: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:749:37: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:812:81: empty-lines: extra empty line at the start of a block (revive)
    pkg/archive/copy_unix_test.go:347:34: empty-lines: extra empty line at the end of a block (revive)
    pkg/system/path.go:11:39: empty-lines: extra empty line at the end of a block (revive)
    pkg/system/meminfo_linux.go:29:21: empty-lines: extra empty line at the end of a block (revive)
    pkg/plugins/plugins.go:135:32: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:71:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/api_test.go:18:51: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_test.go:23:44: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_unix_test.go:17:46: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/api_test.go:57:45: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:83:50: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/api_test.go:66:47: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_unix_test.go:45:48: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:145:75: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/middleware_unix_test.go:56:51: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:49 +02:00
Aaron Lehmann
38e2e756c9 Avoid platform-specific NewPatternMatcher function in TestCompile
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-01-20 09:08:16 -08:00
Aaron Lehmann
8d250be008 Add unit test for (*Pattern).compile
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2022-01-12 10:31:35 -08:00
Aaron Lehmann
dd66dcad9c fileutils: Avoid compiling a regexp for simple patterns
If we detect that a pattern is either an exact match, prefix match, or
suffix match, use an optimized code path instead of compiling a regexp.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-12-20 16:32:13 -08:00
Aaron Lehmann
55da5245de Fix missing parent info case in MatchesUsingParentResults
Unfortunately, this check was missing in the original version. It could
cause a positive match to be overwritten by checking parent dirs.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-11-26 10:22:19 -08:00
Aaron Lehmann
0f1b68df16 Adjust deprecation comments
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-11-24 07:53:10 -08:00
Aaron Lehmann
4555d3aa54 pkg/fileutils: Track incremental pattern match results against each pattern
The existing code does not correctly handle the case where a file
matches one of the patterns, but should not match overall because of an
exclude pattern that applied to a parent directory (see
https://github.com/docker/buildx/issues/850).

Fix this by independently tracking the results of matching against each
pattern. A file should be considered to match any pattern that matched a
parent dir.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-11-19 15:58:13 -08:00
Milas Bowman
195558de5a pkg/fileutils: escape additional regex meta characters
There were a couple characters being explicitly escaped, but it
wasn't comprehensive.

This is now the set difference between the Golang regex meta
characters and the `filepath` match meta characters with the
exception of `\`, which already has special logic due to being
the path separator on Windows.

Signed-off-by: Milas Bowman <milasb@gmail.com>
2021-10-22 11:31:11 -04:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
Sebastiaan van Stijn
686be57d0a
Update to Go 1.17.0, and gofmt with Go 1.17
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-08-24 23:33:27 +02:00
Aaron Lehmann
c44b90f3bf Test fix for Windows compatibility
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-12 20:03:41 -07:00
Aaron Lehmann
97ede9df26 Rename Matches to MatchesOrParentMatches
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-12 18:10:04 -07:00
Aaron Lehmann
9bae4f2f24 Add more optimal MatchesUsingParentResult method, use it in pkg/archive
Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-08-12 13:57:50 -07:00
Aaron Lehmann
90f8d1b675 fileutils: Fix incorrect handling of "**/foo" pattern
(*PatternMatcher).Matches includes a special case for when the pattern
matches a parent dir, even though it doesn't match the current path.
However, it assumes that the parent dir which would match the pattern
must have the same number of separators as the pattern itself. This
doesn't hold true with a patern like "**/foo". A file foo/bar would have
len(parentPathDirs) == 1, which is less than the number of path
len(pattern.dirs) == 2... therefore this check would be skipped.

Given that "**/foo" matches "foo", I think it's a bug that the "parent
subdir matches" check is being skipped in this case.

It seems safer to loop over the parent subdirs and check each against
the pattern. It's possible there is a safe optimization to check only a
certain subset, but the existing logic seems unsafe.

Signed-off-by: Aaron Lehmann <alehmann@netflix.com>
2021-07-26 11:28:10 -07:00
Sebastiaan van Stijn
bc8acc5b08
pkg/fileutils: PatternMatcher.Matches(): remove debug logging
Trying to avoid logging code in "libraries" used elsewhere.
If this debug log is important, it should be easy to add in code
that's calling it.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-06-09 11:56:00 +02:00
Sebastiaan van Stijn
2842639e0e
pkg/fileutils: TestMatches: remove cases no longer valid for go1.16
These tests were no longer valid on Go 1.16; related to https://tip.golang.org/doc/go1.16#path/filepath

> The Match and Glob functions now return an error if the unmatched part of
> the pattern has a syntax error. Previously, the functions returned early on
> a failed match, and thus did not report any later syntax error in the pattern.

Causing the test to fail:

    === RUN   TestMatches
        fileutils_test.go:388: assertion failed: error is not nil: syntax error in pattern: pattern="a\\" text="a"
    --- FAIL: TestMatches (0.00s)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2021-05-18 09:37:13 +02:00
Sebastiaan van Stijn
9f0b3f5609
bump gotest.tools v3.0.1 for compatibility with Go 1.14
full diff: https://github.com/gotestyourself/gotest.tools/compare/v2.3.0...v3.0.1

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2020-02-11 00:06:42 +01:00
Sebastiaan van Stijn
9419024554
gosec: add ignore comments for reported issues that can be ignored
```
builder/remotecontext/remote.go:48:        G107: Potential HTTP request made with variable url (gosec)
builder/remotecontext/git/gitutils.go:145: G107: Potential HTTP request made with variable url (gosec)
builder/remotecontext/git/gitutils.go:147: G107: Potential HTTP request made with variable url (gosec)
pkg/fileutils/fileutils_test.go:185:       G303: File creation in shared tmp directory without using ioutil.Tempfile (gosec)
pkg/tarsum/tarsum_test.go:7:               G501: Blacklisted import `crypto/md5`: weak cryptographic primitive (gosec)
pkg/tarsum/tarsum_test.go:9:               G505: Blacklisted import `crypto/sha1`: weak cryptographic primitive (gosec)
```

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2019-09-18 12:57:43 +02:00
mooncake
35739fd290 Remove duplicated words in pkg files
Signed-off-by: mooncake <xcoder@tenxcloud.com>
2018-10-05 22:32:14 +08:00
Vincent Demeester
3845728524
Update tests to use gotest.tools 👼
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2018-06-13 09:04:30 +02:00
chenyuzhu
5ac0d2f0b8 Fix typos
Signed-off-by: chenyuzhu <chenyuzhi@oschina.cn>
2018-05-16 15:31:12 +08:00
Daniel Nephin
6be0f70983 Automated migration using
gty-migrate-from-testify --ignore-build-tags

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-03-16 11:03:43 -04:00
Daniel Nephin
4f0d95fa6e Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
Michael Crosby
5a9b5f10cf Remove solaris files
For obvious reasons that it is not really supported now.

Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-10-24 15:39:34 -04:00
Daniel Nephin
372670b507 Add goimports to linters.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-21 18:15:08 -04:00
Derek McGowan
1009e6a40b
Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-31 13:16:46 -07:00
Daniel Nephin
9484c3bd81 Enable a unit test on windows.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-04-21 12:31:16 -04:00
unclejack
20d6f5c2a9 pkg/file{utils,notify}: don't compare to bool
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2017-03-30 16:38:01 +03:00
Tonis Tiigi
adeb3e3684 Fix inefficient file paths filter
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2017-03-08 18:59:25 -08:00
Doug Davis
376bb84c5c Fix use of **/ in .dockerignore
.dockerignore pattern of **/.foo incorrectly matched **/bar.foo
because **/.foo was getting converted into a .*\.foo regex
instead of (.*/)*\.foo

Closes #29014

Signed-off-by: Doug Davis <dug@us.ibm.com>
2016-12-02 03:58:54 -08:00
Elena Morozova
a37027f6bd all: use strings.Contains instead Index
Signed-off-by: Elena Morozova <lelenanam@gmail.com>
2016-10-13 15:55:41 -07:00
allencloud
4e959ef2f7 fix typos
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-07-23 11:32:23 +08:00
allencloud
d2dd1a1f7d add fileutils_darwin.go in pkg/fileutils to support darwin platform
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-07-01 13:02:26 +08:00
Amit Krishnan
86d8758e2b Get the Docker Engine to build clean on Solaris
Signed-off-by: Amit Krishnan <krish.amit@gmail.com>
2016-05-23 16:37:12 -07:00
allencloud
3f77baf2ce fix typos
Signed-off-by: allencloud <allen.sun@daocloud.io>
2016-05-06 18:23:11 +08:00
John Howard
3e78ad7be2 Windows CI: Turn off failing unit test pkg\fileutils
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-03-02 19:05:33 -08:00
John Howard
b368a9f9b7 Windows CI: Fixes panic in test-unit for FileUtils
Signed-off-by: John Howard <jhoward@microsoft.com>
2016-02-23 13:31:45 -08:00
Justas Brazauskas
927b334ebf Fix typos found across repository
Signed-off-by: Justas Brazauskas <brazauskasjustas@gmail.com>
2015-12-13 18:04:12 +02:00
Doug Davis
eddb14a44e Support multi-dir wildcards in .dockerignore
Closes #13113

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-11-13 13:44:07 -08:00
John Howard
8d56108ffb Windows: Fix warning on info
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-09-18 13:39:12 -07:00
Vincent Demeester
09adf87f23 Add missing tests and docs for pkg/fileutils
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-07-12 22:43:42 +02:00
Lei Jitang
289ee90b04 Fix copy from a "created" container. Fixes #14420
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-07-08 11:15:09 +08:00
unclejack
2f6e4fdb29 linting changes
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2015-06-17 01:16:57 +03:00
Doug Davis
82ea6ed2bc Allow .dockerignore to ignore everything
Change CLI error msg because it was too specific and didn't make sense
when there were errors not related to inaccessible files.

Removed some log.Error() calls since they're not really errors we should
log. Returning the error will be enough.

Closes: #13417

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-05-30 04:16:00 -07:00
Vincent Demeester
8454e1a3b2 Add coverage on pkg/fileutils
Should fix #11598

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2015-04-29 16:27:12 +02:00