beenull/moby
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Merge pull request #34089 from tonistiigi/builder-report-2017-07-10

Browse source 
Add builder dev report for 2017-07-10
This commit is contained in:
Sebastiaan van Stijn 2017-07-13 23:25:57 -07:00 committed by GitHub
commit 93494237df
1 changed files with 65 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

65
reports/builder/2017-07-10.md Normal file
View file

@ -0,0 +1,65 @@
# Development Report for July 10, 2017
### BuildKit
[Repo](https://github.com/moby/buildkit)
[Proposal](https://github.com/moby/moby/issues/32925)
Many new features have been added since the last report.
The build definition solver was updated to detect the identical parts of the graph sent by different clients and synchronize their processing. This is important when multiple targets of the same project are built at the same time and removes any duplication of work.
Running build jobs now has support for graceful canceling and clear error reporting in case some build steps fail or are canceled. Bugs that may have left state dir in an inconsistent state of server shutdown were fixed.
`buildctl du` command now shows all the information about allocated and in-use snapshots. It also shows the total space used and total reclaimable space. All snapshots are now persistent, and state is not lost with server restarts.
New metadata package was implemented that other packages can use to add persistent and searchable metadata to individual snapshots. First users of that feature are the content blobs mapping on pull, size cache for `du` and instruction cache. There is also a new debug command `buildctl debug dump-metadata` to inspect what data is being stored.
The first version of instruction cache was implemented. This caching scheme has many benefits compared to the current `docker build` caching as it doesn't require all data to be locally available to determine the cache match. The interface for the cache implementation is much simpler and could be implemented remotely as it only needs to store the cache keys and doesn't need to understand or compare their values. Content-based caching will be implemented on top of this work later.
Separate source implementation for git repositories is currently in review. Using this source for accessing source code in git repositories has many performance and caching advantages. All the build jobs using the same git remote will use a shared local repository where updates will be pulled. All the nodes based on a git source will be cached using the commit ID of the current checkout.
Next areas to be worked on will be implementing first exporters for getting access to the build artifacts and porting over the client session/incremental-send feature from `17.07-ce`.
### Typed Dockerfile parsing
[PR](https://github.com/moby/moby/pull/33492)
The PR is in code review and waiting for feedback. Hopefully ready to be merged this week.
### Quality: Dependency interface switch
No updates for this week. Metadata commands need to be updated but it is probably easier to do it after https://github.com/moby/moby/pull/33492 has been merged.
### Proposals for new Dockerfile features that need design feedback:
[Add IMPORT/EXPORT commands to Dockerfile](https://github.com/moby/moby/issues/32100)
[Add `DOCKEROS/DOCKERARCH` default ARG to Dockerfile](https://github.com/moby/moby/issues/32487)
[Add support for `RUN --mount`](https://github.com/moby/moby/issues/32507)
[DAG image builder](https://github.com/moby/moby/issues/32550)
[Option to export the hash of the build context](https://github.com/moby/moby/issues/32963) (new)
[Allow --cache-from=*](https://github.com/moby/moby/issues/33002#issuecomment-299041162) (new)
[Provide advanced .dockeringore use-cases](https://github.com/moby/moby/issues/12886) [2](https://github.com/moby/moby/issues/12886#issuecomment-306247989)
New: [RFC: Distributed BuildKit](https://github.com/moby/buildkit/issues/62)
If you are interested in implementing any of them, leave a comment on the specific issues.
### Other builder PRs merged last week
[build: fix add from remote url](https://github.com/moby/moby/pull/33851)
### Builder features currently in code-review:
[Fix shallow git clone in docker-build](https://github.com/moby/moby/pull/33704)
### Backlog
[Build secrets](https://github.com/moby/moby/issues/33343) has not got much traction. If you want this feature to become a reality, please make yourself heard.