Commit graph

37 commits

Author SHA1 Message Date
Sebastiaan van Stijn
f7552f1de4
pkg/archive: format code with gofumpt
Formatting the code with https://github.com/mvdan/gofumpt

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-06-29 00:25:09 +02:00
Brian Goff
74da6a6363 Switch all logging to use containerd log pkg
This unifies our logging and allows us to propagate logging and trace
contexts together.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2023-06-24 00:23:44 +00:00
Sebastiaan van Stijn
412c650e05
pkg/*: fix "empty-lines" (revive)
pkg/directory/directory.go:9:49: empty-lines: extra empty line at the start of a block (revive)
    pkg/pubsub/publisher.go:8:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/loopback/attach_loopback.go:96:69: empty-lines: extra empty line at the start of a block (revive)
    pkg/devicemapper/devmapper_wrapper.go:136:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/devicemapper/devmapper.go:391:35: empty-lines: extra empty line at the end of a block (revive)
    pkg/devicemapper/devmapper.go:676:35: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/changes_posix_test.go:15:38: empty-lines: extra empty line at the end of a block (revive)
    pkg/devicemapper/devmapper.go:241:51: empty-lines: extra empty line at the start of a block (revive)
    pkg/fileutils/fileutils_test.go:17:47: empty-lines: extra empty line at the end of a block (revive)
    pkg/fileutils/fileutils_test.go:34:48: empty-lines: extra empty line at the end of a block (revive)
    pkg/fileutils/fileutils_test.go:318:32: empty-lines: extra empty line at the end of a block (revive)
    pkg/tailfile/tailfile.go:171:6: empty-lines: extra empty line at the end of a block (revive)
    pkg/tarsum/fileinfosums_test.go:16:41: empty-lines: extra empty line at the end of a block (revive)
    pkg/tarsum/tarsum_test.go:198:42: empty-lines: extra empty line at the start of a block (revive)
    pkg/tarsum/tarsum_test.go:294:25: empty-lines: extra empty line at the start of a block (revive)
    pkg/tarsum/tarsum_test.go:407:34: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/fswriters_test.go:52:45: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/writers_test.go:24:39: empty-lines: extra empty line at the end of a block (revive)
    pkg/ioutils/bytespipe_test.go:78:26: empty-lines: extra empty line at the end of a block (revive)
    pkg/sysinfo/sysinfo_linux_test.go:13:37: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive_linux_test.go:57:64: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/changes.go:248:72: empty-lines: extra empty line at the start of a block (revive)
    pkg/archive/changes_posix_test.go:15:38: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/copy.go:248:124: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/diff_test.go:198:44: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:304:12: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:749:37: empty-lines: extra empty line at the end of a block (revive)
    pkg/archive/archive.go:812:81: empty-lines: extra empty line at the start of a block (revive)
    pkg/archive/copy_unix_test.go:347:34: empty-lines: extra empty line at the end of a block (revive)
    pkg/system/path.go:11:39: empty-lines: extra empty line at the end of a block (revive)
    pkg/system/meminfo_linux.go:29:21: empty-lines: extra empty line at the end of a block (revive)
    pkg/plugins/plugins.go:135:32: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:71:48: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/api_test.go:18:51: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_test.go:23:44: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_unix_test.go:17:46: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/api_test.go:57:45: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:83:50: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/api_test.go:66:47: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/middleware_unix_test.go:45:48: empty-lines: extra empty line at the end of a block (revive)
    pkg/authorization/response.go:145:75: empty-lines: extra empty line at the start of a block (revive)
    pkg/authorization/middleware_unix_test.go:56:51: empty-lines: extra empty line at the end of a block (revive)

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2022-09-28 01:58:49 +02:00
Cory Snider
098a44c07f Finish refactor of UID/GID usage to a new struct
Finish the refactor which was partially completed with commit
34536c498d, passing around IdentityMapping structs instead of pairs of
[]IDMap slices.

Existing code which uses []IDMap relies on zero-valued fields to be
valid, empty mappings. So in order to successfully finish the
refactoring without introducing bugs, their replacement therefore also
needs to have a useful zero value which represents an empty mapping.
Change IdentityMapping to be a pass-by-value type so that there are no
nil pointers to worry about.

The functionality provided by the deprecated NewIDMappingsFromMaps
function is required by unit tests to to construct arbitrary
IdentityMapping values. And the daemon will always need to access the
mappings to pass them to the Linux kernel. Accommodate these use cases
by exporting the struct fields instead. BuildKit currently depends on
the UIDs and GIDs methods so we cannot get rid of them yet.

Signed-off-by: Cory Snider <csnider@mirantis.com>
2022-03-14 16:28:57 -04:00
Eng Zer Jun
c55a4ac779
refactor: move from io/ioutil to io and os package
The io/ioutil package has been deprecated in Go 1.16. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-08-27 14:56:57 +08:00
John Howard
56b732058e pkg/archive fixes, and port most unit tests to Windows
Signed-off-by: John Howard <jhoward@microsoft.com>

If fixes an error in sameFsTime which was using `==` to compare two times. The correct way is to use go's built-in timea.Equals(timeb).

In changes_windows, it uses sameFsTime to compare mTim of a `system.StatT` to allow TestChangesDirsMutated to operate correctly now.

Note there is slight different between the Linux and Windows implementations of detecting changes. Due to https://github.com/moby/moby/issues/9874,
and the fix at https://github.com/moby/moby/pull/11422, Linux does not consider a change to the directory time as a change. Windows on NTFS
does. See https://github.com/moby/moby/pull/37982 for more information. The result in `TestChangesDirsMutated`, `dir3` is NOT considered a change
in Linux, but IS considered a change on Windows. The test mutates dir3 to have a mtime of +1 second.

With a handful of tests still outstanding, this change ports most of the unit tests under pkg/archive to Windows.

It provides an implementation of `copyDir` in tests for Windows. To make a copy similar to Linux's `cp -a` while preserving timestamps
and links to both valid and invalid targets, xcopy isn't sufficient. So I used robocopy, but had to circumvent certain exit codes that
robocopy exits with which are warnings. Link to article describing this is in the code.
2018-11-26 10:20:40 -08:00
Daniel Nephin
4f0d95fa6e Add canonical import comment
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2018-02-05 16:51:57 -05:00
Sébastien HOUZÉ
8a34c67a7e
archive: add ChownOpts support to TarWithOptions
Signed-off-by: Sébastien HOUZÉ <cto@verylastroom.com>
2017-08-23 20:52:29 +02:00
Derek McGowan
1009e6a40b
Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-31 13:16:46 -07:00
Daniel Nephin
5672eeb5e0 Convert tarAppender to the newIDMappings.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-07 11:44:34 -04:00
unclejack
087f7307a6 pkg/{chroot,}archive: clean up a few small issues
Signed-off-by: Cristian Staretu <cristian.staretu@gmail.com>
2017-03-30 16:37:54 +03:00
Stephen J Day
aa2cc18745
pkg/archive: remove unnecessary Archive and Reader type
The `archive` package defines aliases for `io.ReadCloser` and
`io.Reader`. These don't seem to provide an benefit other than type
decoration. Per this change, several unnecessary type cases were
removed.

Signed-off-by: Stephen J Day <stephen.day@docker.com>
2016-10-20 19:31:24 -07:00
Derek McGowan
8222c86360 Update archive package to support overlay whiteouts
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2016-06-08 00:10:16 -07:00
Justas Brazauskas
927b334ebf Fix typos found across repository
Signed-off-by: Justas Brazauskas <brazauskasjustas@gmail.com>
2015-12-13 18:04:12 +02:00
Alexander Morozov
7aa28b6bdd Fix go vet warnings
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-11-02 08:28:34 -08:00
Dmitry Vorobev
4102537cfd Fixes #9283. Consider hardlinks in image size.
Based on #8984. This patch fixes behavior when image size calculation
didn't consider hardlinks.

Signed-off-by: Dmitry Vorobev <dimahabr@gmail.com>
2015-10-17 12:24:05 +02:00
Phil Estes
442b45628e Add user namespace (mapping) support to the Docker engine
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.

Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2015-10-09 17:47:37 -04:00
Tonis Tiigi
2fb5d0c323 Add constants for AUFS whiteout files
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-09-29 11:21:45 -07:00
Zhang Wei
7e420ad850 fix golint warnings/errors on pkg/system and pkg/stdcopy
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
2015-08-13 18:47:13 +08:00
Lei Jitang
ba332b7d12 Enable golint in pkg/arcive
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-08-04 09:52:54 +08:00
John Howard
3c177dc877 Windows: Docker build starting to work
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-06-17 10:41:28 -07:00
Arnaud Porterie
6a7a7570bb Merge pull request #12833 from burke/faster-changes
Optimize archive.ChangesDirs on Linux
2015-06-12 17:05:34 -07:00
Shijiang Wei
e2c6a8be7c Add the parent directory to changes set if new files are generated
The "TestChangesWithChanges" case randomlly fails on my development
VM with the following errors:
```
--- FAIL: TestChangesWithChanges (0.00s)
        changes_test.go:201: no change for expected change C /dir1/subfolder != A /dir1/subfolder/newFile
```

If I apply the following patch to changes_test.go, the test passes.

```diff
diff --git a/pkg/archive/changes_test.go b/pkg/archive/changes_test.go
index 290b2dd..ba1aca0 100644
--- a/pkg/archive/changes_test.go
+++ b/pkg/archive/changes_test.go
@@ -156,6 +156,7 @@ func TestChangesWithChanges(t *testing.T) {
        }
        defer os.RemoveAll(layer)
        createSampleDir(t, layer)
+       time.Sleep(5 * time.Millisecond)
        os.MkdirAll(path.Join(layer, "dir1/subfolder"), 0740)

        // Let's modify modtime for dir1 to be sure it's the same for the two layer (to not having false positive)
```

It seems that if a file is created immediately after the directory is created,
the `archive.Changes` function could't recognize that the parent directory of
the new file is modified.

Perhaps the problem may reproduce on machines with low time precision?
I had successfully reproduced the failure on my development VM as well as
a VM on DigitalOcean.

Signed-off-by: Shijiang Wei <mountkin@gmail.com>
2015-06-10 20:29:29 +08:00
John Howard
8228ee4b0f Windows: Archive package changes for Windows daemon
Signed-off-by: John Howard <jhoward@microsoft.com>
2015-05-20 08:55:54 -07:00
Tianon Gravi
576985a1dc Finally remove our copy of "archive/tar" now that Go 1.4 is the minimum!
IT'S ABOUT TIME. 🎉

Signed-off-by: Andrew "Tianon" Page <admwiggin@gmail.com>
2015-05-01 16:01:10 -06:00
Burke Libbey
45c45a2c9a
archive: Optimize ChangesDirs on Linux
If we tear through a few layers of abstraction, we can get at the inodes
contained in a directory without having to stat all the files. This
allows us to eliminate identical files much earlier in the changelist
generation process.

Signed-off-by: Burke Libbey <burke@libbey.me>
2015-04-27 21:26:13 -04:00
Antonio Murdaca
6f4d847046 Replace aliased imports of logrus, fixes #11762
Signed-off-by: Antonio Murdaca <me@runcom.ninja>
2015-03-26 23:22:04 +01:00
Jimmy Puckett
ec5e22efe3 Changing bitflag checking style to preferred style. Fixes #11668
Signed-off-by: Jimmy Puckett <jimmy.puckett@spinen.com>
2015-03-24 21:09:25 -04:00
Vincent Batts
2ce37f6616 pkg/archive: ignore mtime changes on directories
on overlay fs, the mtime of directories changes in a container where new
files are added in an upper layer (e.g. '/etc'). This flags the
directory as a change where there was none.

Closes #9874

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-03-17 10:53:34 -04:00
Rik Nijessen
1ec2eac50d Make utils_daemon and volumes cross-platform compileable.
Signed-off-by: Rik Nijessen <riknijessen@gmail.com>
2015-03-16 11:46:07 +01:00
Vincent Batts
32d6d49539 pkg/archive: sort files
sort changes found and exported.

Sorting the files before appending them to the tar archive
would mean a dependable ordering for types like hardlinks.

Also, combine sort logic used

Signed-off-by: Vincent Batts <vbatts@redhat.com>
2015-02-06 18:13:57 +01:00
Ahmet Alp Balkan
2180aa4f6f Refactor pkg/archive with a platform-independent stat struct
pkg/archive contains code both invoked from cli (cross platform) and
daemon (linux only) and Unix-specific dependencies break compilation on
Windows. We extracted those stat-related funcs into platform specific
implementations at pkg/system and added unit tests.

Signed-off-by: Ahmet Alp Balkan <ahmetb@microsoft.com>
2014-11-14 18:20:54 -08:00
Ahmet Alp Balkan
3d2fae353f Extract mknod, umask, lstat to pkg/system
Some parts of pkg/archive is called on both client/daemon code. To get
it compiling on Windows, these funcs are extracted into files with
build tags.

Signed-off-by: Ahmet Alp Balkan <ahmetb@microsoft.com>
2014-11-14 18:20:54 -08:00
Vincent Batts
f14a9ed011 archive: cleanup and more information
Signed-off-by: Vincent Batts <vbatts@redhat.com>
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2014-10-29 14:52:59 -04:00
Vincent Batts
f9f8044363 archive: preserve hardlinks in Tar and Untar
* integration test for preserving hardlinks

Signed-off-by: Vincent Batts <vbatts@redhat.com>
Signed-off-by: Vincent Batts <vbatts@hashbangbash.com>
2014-10-29 14:52:52 -04:00
Alexandr Morozov
7c62cee51e Use logrus everywhere for logging
Fixed #8761

Signed-off-by: Alexandr Morozov <lk4d4@docker.com>
2014-10-24 15:03:06 -07:00
Rafe Colton
30d5a42c1f Move archive package into pkg/archive
Now that the archive package does not depend on any docker-specific
packages, only those in pkg and vendor, it can be safely moved into pkg.

Signed-off-by: Rafe Colton <rafael.colton@gmail.com>
2014-09-29 23:23:36 -07:00
Renamed from archive/changes.go (Browse further)