beenull/moby
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

update to master

Browse source
This commit is contained in:
Victor Vieux 2013-05-07 19:23:50 +02:00
commit 10c0e99037
39 changed files with 1973 additions and 564 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

16
CHANGELOG.md
View file

@ -1,5 +1,14 @@
# Changelog
## 0.3.0 (2013-05-06)
+ Registry: Implement the new registry
+ Documentation: new example: sharing data between 2 couchdb databases
- Runtime: Fix the command existance check
- Runtime: strings.Split may return an empty string on no match
- Runtime: Fix an index out of range crash if cgroup memory is not
* Documentation: Various improvments
* Vagrant: Use only one deb line in /etc/apt
## 0.2.2 (2013-05-03)
+ Support for data volumes ('docker run -v=PATH')
+ Share data volumes between containers ('docker run -volumes-from')
@ -8,7 +17,7 @@
* Various upgrades to the dev environment for contributors
## 0.2.1 (2013-05-01)
+ 'docker commit -run' bundles a layer with default runtime options: command, ports etc.
+ 'docker commit -run' bundles a layer with default runtime options: command, ports etc.
* Improve install process on Vagrant
+ New Dockerfile operation: "maintainer"
+ New Dockerfile operation: "expose"
@ -25,13 +34,12 @@
+ Add a changelog
- Various bugfixes
## 0.1.8 (2013-04-22)
- Dynamically detect cgroup capabilities
- Issue stability warning on kernels <3.8
- 'docker push' buffers on disk instead of memory
- Fix 'docker diff' for removed files
- Fix 'docker stop' for ghost containers
- Fix 'docker stop' for ghost containers
- Fix handling of pidfile
- Various bugfixes and stability improvements
@ -52,7 +60,7 @@
- Improve diagnosis of missing system capabilities
- Allow disabling memory limits at compile time
- Add debian packaging
- Documentation: installing on Arch Linux
- Documentation: installing on Arch Linux
- Documentation: running Redis on docker
- Fixed lxc 0.9 compatibility
- Automatically load aufs module

91
README.md
View file

@ -1,37 +1,94 @@
Docker: the Linux container runtime
===================================
Docker: the Linux container engine
==================================
Docker complements LXC with a high-level API which operates at the process level. It runs unix processes with strong guarantees of isolation and repeatability across servers.
Docker is an open-source engine which automates the deployment of applications as highly portable, self-sufficient containers.
Docker is a great building block for automating distributed systems: large-scale web deployments, database clusters, continuous deployment systems, private PaaS, service-oriented architectures, etc.
Docker containers are both *hardware-agnostic* and *platform-agnostic*. This means that they can run anywhere, from your
laptop to the largest EC2 compute instance and everything in between - and they don't require that you use a particular
language, framework or packaging system. That makes them great building blocks for deploying and scaling web apps, databases
and backend services without depending on a particular stack or provider.
Docker is an open-source implementation of the deployment engine which powers [dotCloud](http://dotcloud.com), a popular Platform-as-a-Service.
It benefits directly from the experience accumulated over several years of large-scale operation and support of hundreds of thousands
of applications and databases.
![Docker L](docs/sources/static_files/lego_docker.jpg "Docker")
* *Heterogeneous payloads*: any combination of binaries, libraries, configuration files, scripts, virtualenvs, jars, gems, tarballs, you name it. No more juggling between domain-specific tools. Docker can deploy and run them all.
## Better than VMs
* *Any server*: docker can run on any x64 machine with a modern linux kernel - whether it's a laptop, a bare metal server or a VM. This makes it perfect for multi-cloud deployments.
A common method for distributing applications and sandbox their execution is to use virtual machines, or VMs. Typical VM formats
are VMWare's vmdk, Oracle Virtualbox's vdi, and Amazon EC2's ami. In theory these formats should allow every developer to
automatically package their application into a "machine" for easy distribution and deployment. In practice, that almost never
happens, for a few reasons:
* *Isolation*: docker isolates processes from each other and from the underlying host, using lightweight containers.
* *Size*: VMs are very large which makes them impractical to store and transfer.
* *Performance*: running VMs consumes significant CPU and memory, which makes them impractical in many scenarios, for example local development of multi-tier applications, and
large-scale deployment of cpu and memory-intensive applications on large numbers of machines.
* *Portability*: competing VM environments don't play well with each other. Although conversion tools do exist, they are limited and add even more overhead.
* *Hardware-centric*: VMs were designed with machine operators in mind, not software developers. As a result, they offer very limited tooling for what developers need most:
building, testing and running their software. For example, VMs offer no facilities for application versioning, monitoring, configuration, logging or service discovery.
* *Repeatability*: because containers are isolated in their own filesystem, they behave the same regardless of where, when, and alongside what they run.
By contrast, Docker relies on a different sandboxing method known as *containerization*. Unlike traditional virtualization,
containerization takes place at the kernel level. Most modern operating system kernels now support the primitives necessary
for containerization, including Linux with [openvz](http://openvz.org), [vserver](http://linux-vserver.org) and more recently [lxc](http://lxc.sourceforge.net),
Solaris with [zones](http://docs.oracle.com/cd/E26502_01/html/E29024/preface-1.html#scrolltoc) and FreeBSD with [Jails](http://www.freebsd.org/doc/handbook/jails.html).
Docker builds on top of these low-level primitives to offer developers a portable format and runtime environment that solves
all 4 problems. Docker containers are small (and their transfer can be optimized with layers), they have basically zero memory and cpu overhead,
the are completely portable and are designed from the ground up with an application-centric design.
The best part: because docker operates at the OS level, it can still be run inside a VM!
## Plays well with others
Docker does not require that you buy into a particular programming language, framework, packaging system or configuration language.
Is your application a unix process? Does it use files, tcp connections, environment variables, standard unix streams and command-line
arguments as inputs and outputs? Then docker can run it.
Can your application's build be expressed a sequence of such commands? Then docker can build it.
Notable features
-----------------
## Escape dependency hell
* Filesystem isolation: each process container runs in a completely separate root filesystem.
A common problem for developers is the difficulty of managing all their application's dependencies in a simple and automated way.
* Resource isolation: system resources like cpu and memory can be allocated differently to each process container, using cgroups.
This is usually difficult for several reasons:
* Network isolation: each process container runs in its own network namespace, with a virtual interface and IP address of its own.
* *Cross-platform dependencies*. Modern applications often depend on a combination of system libraries and binaries, language-specific packages, framework-specific modules,
internal components developed for another project, etc. These dependencies live in different "worlds" and require different tools - these tools typically don't work
well with each other, requiring awkward custom integrations.
* Copy-on-write: root filesystems are created using copy-on-write, which makes deployment extremely fast, memory-cheap and disk-cheap.
* Conflicting dependencies. Different applications may depend on different versions of the same dependency. Packaging tools handle these situations with various degrees of ease -
but they all handle them in different and incompatible ways, which again forces the developer to do extra work.
* Custom dependencies. A developer may need to prepare a custom version of his application's dependency. Some packaging systems can handle custom versions of a dependency,
others can't - and all of them handle it differently.
* Logging: the standard streams (stdout/stderr/stdin) of each process container are collected and logged for real-time or batch retrieval.
* Change management: changes to a container's filesystem can be committed into a new image and re-used to create more containers. No templating or manual configuration required.
Docker solves dependency hell by giving the developer a simple way to express *all* his application's dependencies in one place,
and streamline the process of assembling them. If this makes you think of [XKCD 927](http://xkcd.com/927/), don't worry. Docker doesn't
*replace* your favorite packaging systems. It simply orchestrates their use in a simple and repeatable way. How does it do that? With layers.
Docker defines a build as running a sequence unix commands, one after the other, in the same container. Build commands modify the contents of the container
(usually by installing new files on the filesystem), the next command modifies it some more, etc. Since each build command inherits the result of the previous
commands, the *order* in which the commands are executed expresses *dependencies*.
Here's a typical docker build process:
```bash
from ubuntu:12.10
run apt-get update
run apt-get install python
run apt-get install python-pip
run pip install django
run apt-get install curl
run curl http://github.com/shykes/helloflask/helloflask/master.tar.gz | tar -zxv
run cd master && pip install -r requirements.txt
```
Note that Docker doesn't care *how* dependencies are built - as long as they can be built by running a unix command in a container.
* Interactive shell: docker can allocate a pseudo-tty and attach to the standard input of any container, for example to run a throwaway interactive shell.
Install instructions
==================

85
api.go
View file

@ -6,6 +6,7 @@ import (
"fmt"
"github.com/dotcloud/docker/auth"
"github.com/gorilla/mux"
"github.com/shin-/cookiejar"
"log"
"net"
"net/http"
@ -75,6 +76,7 @@ func ListenAndServe(addr string, srv *Server) error {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
} else {
srv.runtime.graph.getHttpClient().Jar = cookiejar.NewCookieJar()
srv.runtime.authConfig = newAuthConfig
}
if status != "" {
@ -141,6 +143,27 @@ func ListenAndServe(addr string, srv *Server) error {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
viz := r.Form.Get("viz")
if viz == "1" {
file, rwc, err := hijackServer(w)
if file != nil {
defer file.Close()
}
if rwc != nil {
defer rwc.Close()
}
if err != nil {
httpError(w, err)
return
}
fmt.Fprintf(file, "HTTP/1.1 200 OK\r\nContent-Type: raw-stream-hijack\r\n\r\n")
if err := srv.ImagesViz(file); err != nil {
fmt.Fprintln(file, "Error: "+err.Error())
}
return
}
all := r.Form.Get("all")
filter := r.Form.Get("filter")
quiet := r.Form.Get("quiet")
@ -331,7 +354,8 @@ func ListenAndServe(addr string, srv *Server) error {
}
fmt.Fprintf(file, "HTTP/1.1 200 OK\r\nContent-Type: raw-stream-hijack\r\n\r\n")
if image != "" { //pull
if err := srv.ImagePull(image, file); err != nil {
registry := r.Form.Get("registry")
if err := srv.ImagePull(image, tag, registry, file); err != nil {
fmt.Fprintln(file, "Error: "+err.Error())
}
} else { //import
@ -341,8 +365,15 @@ func ListenAndServe(addr string, srv *Server) error {
}
})
r.Path("/images/{name:*.}/push").Methods("POST").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
r.Path("/images/{name:*.}/insert").Methods("POST").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
log.Println(r.Method, r.RequestURI)
if err := r.ParseForm(); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
url := r.Form.Get("url")
path := r.Form.Get("path")
vars := mux.Vars(r)
name := vars["name"]
@ -358,7 +389,55 @@ func ListenAndServe(addr string, srv *Server) error {
return
}
fmt.Fprintf(file, "HTTP/1.1 200 OK\r\nContent-Type: raw-stream-hijack\r\n\r\n")
if err := srv.ImagePush(name, file); err != nil {
if err := srv.ImageInsert(name, url, path, file); err != nil {
fmt.Fprintln(file, "Error: "+err.Error())
}
})
r.Path("/images/{name:*.}/push").Methods("POST").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
log.Println(r.Method, r.RequestURI)
if err := r.ParseForm(); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
registry := r.Form.Get("registry")
vars := mux.Vars(r)
name := vars["name"]
file, rwc, err := hijackServer(w)
if file != nil {
defer file.Close()
}
if rwc != nil {
defer rwc.Close()
}
if err != nil {
httpError(w, err)
return
}
fmt.Fprintf(file, "HTTP/1.1 200 OK\r\nContent-Type: raw-stream-hijack\r\n\r\n")
if err := srv.ImagePush(name, registry, file); err != nil {
fmt.Fprintln(file, "Error: "+err.Error())
}
})
r.Path("/build").Methods("POST").HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
log.Println(r.Method, r.RequestURI)
file, rwc, err := hijackServer(w)
if file != nil {
defer file.Close()
}
if rwc != nil {
defer rwc.Close()
}
if err != nil {
httpError(w, err)
return
}
fmt.Fprintf(file, "HTTP/1.1 200 OK\r\nContent-Type: raw-stream-hijack\r\n\r\n")
if err := srv.ImageCreateFormFile(file); err != nil {
fmt.Fprintln(file, "Error: "+err.Error())
}
})

52
auth/auth.go
View file

@ -3,7 +3,6 @@ package auth
import (
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io/ioutil"
"net/http"
@ -16,7 +15,7 @@ import (
const CONFIGFILE = ".dockercfg"
// the registry server we want to login against
const REGISTRY_SERVER = "https://registry.docker.io"
const INDEX_SERVER = "https://index.docker.io"
type AuthConfig struct {
Username string `json:"username"`
@ -76,6 +75,9 @@ func LoadConfig(rootPath string) (*AuthConfig, error) {
return nil, err
}
arr := strings.Split(string(b), "\n")
if len(arr) < 2 {
return nil, fmt.Errorf("The Auth config file is empty")
}
origAuth := strings.Split(arr[0], " = ")
origEmail := strings.Split(arr[1], " = ")
authConfig, err := DecodeAuth(origAuth[1])
@ -89,9 +91,14 @@ func LoadConfig(rootPath string) (*AuthConfig, error) {
// save the auth config
func saveConfig(rootPath, authStr string, email string) error {
confFile := path.Join(rootPath, CONFIGFILE)
if len(email) == 0 {
os.Remove(confFile)
return nil
}
lines := "auth = " + authStr + "\n" + "email = " + email + "\n"
b := []byte(lines)
err := ioutil.WriteFile(path.Join(rootPath, CONFIGFILE), b, 0600)
err := ioutil.WriteFile(confFile, b, 0600)
if err != nil {
return err
}
@ -101,40 +108,38 @@ func saveConfig(rootPath, authStr string, email string) error {
// try to register/login to the registry server
func Login(authConfig *AuthConfig) (string, error) {
storeConfig := false
client := &http.Client{}
reqStatusCode := 0
var status string
var errMsg string
var reqBody []byte
jsonBody, err := json.Marshal(authConfig)
if err != nil {
errMsg = fmt.Sprintf("Config Error: %s", err)
return "", errors.New(errMsg)
return "", fmt.Errorf("Config Error: %s", err)
}
// using `bytes.NewReader(jsonBody)` here causes the server to respond with a 411 status.
b := strings.NewReader(string(jsonBody))
req1, err := http.Post(REGISTRY_SERVER+"/v1/users", "application/json; charset=utf-8", b)
req1, err := http.Post(INDEX_SERVER+"/v1/users/", "application/json; charset=utf-8", b)
if err != nil {
errMsg = fmt.Sprintf("Server Error: %s", err)
return "", errors.New(errMsg)
return "", fmt.Errorf("Server Error: %s", err)
}
reqStatusCode = req1.StatusCode
defer req1.Body.Close()
reqBody, err = ioutil.ReadAll(req1.Body)
if err != nil {
errMsg = fmt.Sprintf("Server Error: [%#v] %s", reqStatusCode, err)
return "", errors.New(errMsg)
return "", fmt.Errorf("Server Error: [%#v] %s", reqStatusCode, err)
}
if reqStatusCode == 201 {
status = "Account Created\n"
status = "Account created. Please use the confirmation link we sent" +
" to your e-mail to activate it.\n"
storeConfig = true
} else if reqStatusCode == 403 {
return "", fmt.Errorf("Login: Your account hasn't been activated. " +
"Please check your e-mail for a confirmation link.")
} else if reqStatusCode == 400 {
// FIXME: This should be 'exists', not 'exist'. Need to change on the server first.
if string(reqBody) == "Username or email already exist" {
client := &http.Client{}
req, err := http.NewRequest("GET", REGISTRY_SERVER+"/v1/users", nil)
if string(reqBody) == "\"Username or email already exists\"" {
req, err := http.NewRequest("GET", INDEX_SERVER+"/v1/users/", nil)
req.SetBasicAuth(authConfig.Username, authConfig.Password)
resp, err := client.Do(req)
if err != nil {
@ -148,17 +153,18 @@ func Login(authConfig *AuthConfig) (string, error) {
if resp.StatusCode == 200 {
status = "Login Succeeded\n"
storeConfig = true
} else if resp.StatusCode == 401 {
saveConfig(authConfig.rootPath, "", "")
return "", fmt.Errorf("Wrong login/password, please try again")
} else {
status = fmt.Sprintf("Login: %s", body)
return "", errors.New(status)
return "", fmt.Errorf("Login: %s (Code: %d; Headers: %s)", body,
resp.StatusCode, resp.Header)
}
} else {
status = fmt.Sprintf("Registration: %s", reqBody)
return "", errors.New(status)
return "", fmt.Errorf("Registration: %s", reqBody)
}
} else {
status = fmt.Sprintf("[%s] : %s", reqStatusCode, reqBody)
return "", errors.New(status)
return "", fmt.Errorf("Unexpected status code [%d] : %s", reqStatusCode, reqBody)
}
if storeConfig {
authStr := EncodeAuth(authConfig)

463
builder.go Normal file
View file

@ -0,0 +1,463 @@
package docker
import (
"bufio"
"encoding/json"
"fmt"
"io"
"os"
"path"
"strings"
"time"
)
type Builder struct {
runtime *Runtime
repositories *TagStore
graph *Graph
}
func NewBuilder(runtime *Runtime) *Builder {
return &Builder{
runtime: runtime,
graph: runtime.graph,
repositories: runtime.repositories,
}
}
func (builder *Builder) mergeConfig(userConf, imageConf *Config) {
if userConf.Hostname != "" {
userConf.Hostname = imageConf.Hostname
}
if userConf.User != "" {
userConf.User = imageConf.User
}
if userConf.Memory == 0 {
userConf.Memory = imageConf.Memory
}
if userConf.MemorySwap == 0 {
userConf.MemorySwap = imageConf.MemorySwap
}
if userConf.PortSpecs == nil || len(userConf.PortSpecs) == 0 {
userConf.PortSpecs = imageConf.PortSpecs
}
if !userConf.Tty {
userConf.Tty = userConf.Tty
}
if !userConf.OpenStdin {
userConf.OpenStdin = imageConf.OpenStdin
}
if !userConf.StdinOnce {
userConf.StdinOnce = imageConf.StdinOnce
}
if userConf.Env == nil || len(userConf.Env) == 0 {
userConf.Env = imageConf.Env
}
if userConf.Cmd == nil || len(userConf.Cmd) == 0 {
userConf.Cmd = imageConf.Cmd
}
if userConf.Dns == nil || len(userConf.Dns) == 0 {
userConf.Dns = imageConf.Dns
}
}
func (builder *Builder) Create(config *Config) (*Container, error) {
// Lookup image
img, err := builder.repositories.LookupImage(config.Image)
if err != nil {
return nil, err
}
if img.Config != nil {
builder.mergeConfig(config, img.Config)
}
if config.Cmd == nil {
return nil, fmt.Errorf("No command specified")
}
// Generate id
id := GenerateId()
// Generate default hostname
// FIXME: the lxc template no longer needs to set a default hostname
if config.Hostname == "" {
config.Hostname = id[:12]
}
container := &Container{
// FIXME: we should generate the ID here instead of receiving it as an argument
Id: id,
Created: time.Now(),
Path: config.Cmd[0],
Args: config.Cmd[1:], //FIXME: de-duplicate from config
Config: config,
Image: img.Id, // Always use the resolved image id
NetworkSettings: &NetworkSettings{},
// FIXME: do we need to store this in the container?
SysInitPath: sysInitPath,
}
container.root = builder.runtime.containerRoot(container.Id)
// Step 1: create the container directory.
// This doubles as a barrier to avoid race conditions.
if err := os.Mkdir(container.root, 0700); err != nil {
return nil, err
}
// If custom dns exists, then create a resolv.conf for the container
if len(config.Dns) > 0 {
container.ResolvConfPath = path.Join(container.root, "resolv.conf")
f, err := os.Create(container.ResolvConfPath)
if err != nil {
return nil, err
}
defer f.Close()
for _, dns := range config.Dns {
if _, err := f.Write([]byte("nameserver " + dns + "\n")); err != nil {
return nil, err
}
}
} else {
container.ResolvConfPath = "/etc/resolv.conf"
}
// Step 2: save the container json
if err := container.ToDisk(); err != nil {
return nil, err
}
// Step 3: register the container
if err := builder.runtime.Register(container); err != nil {
return nil, err
}
return container, nil
}
// Commit creates a new filesystem image from the current state of a container.
// The image can optionally be tagged into a repository
func (builder *Builder) Commit(container *Container, repository, tag, comment, author string, config *Config) (*Image, error) {
// FIXME: freeze the container before copying it to avoid data corruption?
// FIXME: this shouldn't be in commands.
rwTar, err := container.ExportRw()
if err != nil {
return nil, err
}
// Create a new image from the container's base layers + a new layer from container changes
img, err := builder.graph.Create(rwTar, container, comment, author, config)
if err != nil {
return nil, err
}
// Register the image if needed
if repository != "" {
if err := builder.repositories.Set(repository, tag, img.Id, true); err != nil {
return img, err
}
}
return img, nil
}
func (builder *Builder) clearTmp(containers, images map[string]struct{}) {
for c := range containers {
tmp := builder.runtime.Get(c)
builder.runtime.Destroy(tmp)
Debugf("Removing container %s", c)
}
for i := range images {
builder.runtime.graph.Delete(i)
Debugf("Removing image %s", i)
}
}
func (builder *Builder) getCachedImage(image *Image, config *Config) (*Image, error) {
// Retrieve all images
images, err := builder.graph.All()
if err != nil {
return nil, err
}
// Store the tree in a map of map (map[parentId][childId])
imageMap := make(map[string]map[string]struct{})
for _, img := range images {
if _, exists := imageMap[img.Parent]; !exists {
imageMap[img.Parent] = make(map[string]struct{})
}
imageMap[img.Parent][img.Id] = struct{}{}
}
// Loop on the children of the given image and check the config
for elem := range imageMap[image.Id] {
img, err := builder.graph.Get(elem)
if err != nil {
return nil, err
}
if CompareConfig(&img.ContainerConfig, config) {
return img, nil
}
}
return nil, nil
}
func (builder *Builder) Build(dockerfile io.Reader, stdout io.Writer) (*Image, error) {
var (
image, base *Image
config *Config
maintainer string
env map[string]string = make(map[string]string)
tmpContainers map[string]struct{} = make(map[string]struct{})
tmpImages map[string]struct{} = make(map[string]struct{})
)
defer builder.clearTmp(tmpContainers, tmpImages)
file := bufio.NewReader(dockerfile)
for {
line, err := file.ReadString('\n')
if err != nil {
if err == io.EOF {
break
}
return nil, err
}
line = strings.Replace(strings.TrimSpace(line), " ", " ", 1)
// Skip comments and empty line
if len(line) == 0 || line[0] == '#' {
continue
}
tmp := strings.SplitN(line, " ", 2)
if len(tmp) != 2 {
return nil, fmt.Errorf("Invalid Dockerfile format")
}
instruction := strings.Trim(tmp[0], " ")
arguments := strings.Trim(tmp[1], " ")
switch strings.ToLower(instruction) {
case "from":
fmt.Fprintf(stdout, "FROM %s\n", arguments)
image, err = builder.runtime.repositories.LookupImage(arguments)
if err != nil {
if builder.runtime.graph.IsNotExist(err) {
var tag, remote string
if strings.Contains(arguments, ":") {
remoteParts := strings.Split(arguments, ":")
tag = remoteParts[1]
remote = remoteParts[0]
} else {
remote = arguments
}
if err := builder.runtime.graph.PullRepository(stdout, remote, tag, builder.runtime.repositories, builder.runtime.authConfig); err != nil {
return nil, err
}
image, err = builder.runtime.repositories.LookupImage(arguments)
if err != nil {
return nil, err
}
} else {
return nil, err
}
}
config = &Config{}
break
case "maintainer":
fmt.Fprintf(stdout, "MAINTAINER %s\n", arguments)
maintainer = arguments
break
case "run":
fmt.Fprintf(stdout, "RUN %s\n", arguments)
if image == nil {
return nil, fmt.Errorf("Please provide a source image with `from` prior to run")
}
config, _, err := ParseRun([]string{image.Id, "/bin/sh", "-c", arguments}, builder.runtime.capabilities)
if err != nil {
return nil, err
}
for key, value := range env {
config.Env = append(config.Env, fmt.Sprintf("%s=%s", key, value))
}
if cache, err := builder.getCachedImage(image, config); err != nil {
return nil, err
} else if cache != nil {
image = cache
fmt.Fprintf(stdout, "===> %s\n", image.ShortId())
break
}
Debugf("Env -----> %v ------ %v\n", config.Env, env)
// Create the container and start it
c, err := builder.Create(config)
if err != nil {
return nil, err
}
if os.Getenv("DEBUG") != "" {
out, _ := c.StdoutPipe()
err2, _ := c.StderrPipe()
go io.Copy(os.Stdout, out)
go io.Copy(os.Stdout, err2)
}
if err := c.Start(); err != nil {
return nil, err
}
tmpContainers[c.Id] = struct{}{}
// Wait for it to finish
if result := c.Wait(); result != 0 {
return nil, fmt.Errorf("!!! '%s' return non-zero exit code '%d'. Aborting.", arguments, result)
}
// Commit the container
base, err = builder.Commit(c, "", "", "", maintainer, nil)
if err != nil {
return nil, err
}
tmpImages[base.Id] = struct{}{}
fmt.Fprintf(stdout, "===> %s\n", base.ShortId())
// use the base as the new image
image = base
break
case "env":
tmp := strings.SplitN(arguments, " ", 2)
if len(tmp) != 2 {
return nil, fmt.Errorf("Invalid ENV format")
}
key := strings.Trim(tmp[0], " ")
value := strings.Trim(tmp[1], " ")
fmt.Fprintf(stdout, "ENV %s %s\n", key, value)
env[key] = value
if image != nil {
fmt.Fprintf(stdout, "===> %s\n", image.ShortId())
} else {
fmt.Fprintf(stdout, "===> <nil>\n")
}
break
case "cmd":
fmt.Fprintf(stdout, "CMD %s\n", arguments)
// Create the container and start it
c, err := builder.Create(&Config{Image: image.Id, Cmd: []string{"", ""}})
if err != nil {
return nil, err
}
if err := c.Start(); err != nil {
return nil, err
}
tmpContainers[c.Id] = struct{}{}
cmd := []string{}
if err := json.Unmarshal([]byte(arguments), &cmd); err != nil {
return nil, err
}
config.Cmd = cmd
// Commit the container
base, err = builder.Commit(c, "", "", "", maintainer, config)
if err != nil {
return nil, err
}
tmpImages[base.Id] = struct{}{}
fmt.Fprintf(stdout, "===> %s\n", base.ShortId())
image = base
break
case "expose":
ports := strings.Split(arguments, " ")
fmt.Fprintf(stdout, "EXPOSE %v\n", ports)
if image == nil {
return nil, fmt.Errorf("Please provide a source image with `from` prior to copy")
}
// Create the container and start it
c, err := builder.Create(&Config{Image: image.Id, Cmd: []string{"", ""}})
if err != nil {
return nil, err
}
if err := c.Start(); err != nil {
return nil, err
}
tmpContainers[c.Id] = struct{}{}
config.PortSpecs = append(ports, config.PortSpecs...)
// Commit the container
base, err = builder.Commit(c, "", "", "", maintainer, config)
if err != nil {
return nil, err
}
tmpImages[base.Id] = struct{}{}
fmt.Fprintf(stdout, "===> %s\n", base.ShortId())
image = base
break
case "insert":
if image == nil {
return nil, fmt.Errorf("Please provide a source image with `from` prior to copy")
}
tmp = strings.SplitN(arguments, " ", 2)
if len(tmp) != 2 {
return nil, fmt.Errorf("Invalid INSERT format")
}
sourceUrl := strings.Trim(tmp[0], " ")
destPath := strings.Trim(tmp[1], " ")
fmt.Fprintf(stdout, "COPY %s to %s in %s\n", sourceUrl, destPath, base.ShortId())
file, err := Download(sourceUrl, stdout)
if err != nil {
return nil, err
}
defer file.Body.Close()
config, _, err := ParseRun([]string{base.Id, "echo", "insert", sourceUrl, destPath}, builder.runtime.capabilities)
if err != nil {
return nil, err
}
c, err := builder.Create(config)
if err != nil {
return nil, err
}
if err := c.Start(); err != nil {
return nil, err
}
// Wait for echo to finish
if result := c.Wait(); result != 0 {
return nil, fmt.Errorf("!!! '%s' return non-zero exit code '%d'. Aborting.", arguments, result)
}
if err := c.Inject(file.Body, destPath); err != nil {
return nil, err
}
base, err = builder.Commit(c, "", "", "", maintainer, nil)
if err != nil {
return nil, err
}
fmt.Fprintf(stdout, "===> %s\n", base.ShortId())
image = base
break
default:
fmt.Fprintf(stdout, "Skipping unknown instruction %s\n", strings.ToUpper(instruction))
}
}
if image != nil {
// The build is successful, keep the temporary containers and images
for i := range tmpImages {
delete(tmpImages, i)
}
for i := range tmpContainers {
delete(tmpContainers, i)
}
fmt.Fprintf(stdout, "Build finished. image id: %s\n", image.ShortId())
return image, nil
}
return nil, fmt.Errorf("An error occured during the build\n")
}

88
builder_test.go Normal file
View file

@ -0,0 +1,88 @@
package docker
import (
"strings"
"testing"
)
const Dockerfile = `
# VERSION 0.1
# DOCKER-VERSION 0.2
from ` + unitTestImageName + `
run sh -c 'echo root:testpass > /tmp/passwd'
run mkdir -p /var/run/sshd
insert https://raw.github.com/dotcloud/docker/master/CHANGELOG.md /tmp/CHANGELOG.md
`
func TestBuild(t *testing.T) {
runtime, err := newTestRuntime()
if err != nil {
t.Fatal(err)
}
defer nuke(runtime)
builder := NewBuilder(runtime)
img, err := builder.Build(strings.NewReader(Dockerfile), &nopWriter{})
if err != nil {
t.Fatal(err)
}
container, err := builder.Create(
&Config{
Image: img.Id,
Cmd: []string{"cat", "/tmp/passwd"},
},
)
if err != nil {
t.Fatal(err)
}
defer runtime.Destroy(container)
output, err := container.Output()
if err != nil {
t.Fatal(err)
}
if string(output) != "root:testpass\n" {
t.Fatalf("Unexpected output. Read '%s', expected '%s'", output, "root:testpass\n")
}
container2, err := builder.Create(
&Config{
Image: img.Id,
Cmd: []string{"ls", "-d", "/var/run/sshd"},
},
)
if err != nil {
t.Fatal(err)
}
defer runtime.Destroy(container2)
output, err = container2.Output()
if err != nil {
t.Fatal(err)
}
if string(output) != "/var/run/sshd\n" {
t.Fatal("/var/run/sshd has not been created")
}
container3, err := builder.Create(
&Config{
Image: img.Id,
Cmd: []string{"cat", "/tmp/CHANGELOG.md"},
},
)
if err != nil {
t.Fatal(err)
}
defer runtime.Destroy(container3)
output, err = container3.Output()
if err != nil {
t.Fatal(err)
}
if len(output) == 0 {
t.Fatal("/tmp/CHANGELOG.md has not been copied")
}
}

151
commands.go
View file

@ -22,7 +22,7 @@ import (
"unicode"
)
const VERSION = "0.2.2"
const VERSION = "0.3.0"
var (
GIT_COMMIT string
@ -58,6 +58,7 @@ func ParseCommands(args ...string) error {
"export": CmdExport,
"images": CmdImages,
"info": CmdInfo,
"insert": CmdInsert,
"inspect": CmdInspect,
"import": CmdImport,
"history": CmdHistory,
@ -94,6 +95,7 @@ func cmdHelp(args ...string) error {
help := "Usage: docker COMMAND [arg...]\n\nA self-sufficient runtime for linux containers.\n\nCommands:\n"
for _, cmd := range [][]string{
{"attach", "Attach to a running container"},
{"build", "Build a container from Dockerfile via stdin"},
{"commit", "Create a new image from a container's changes"},
{"diff", "Inspect changes on a container's filesystem"},
{"export", "Stream the contents of a container as a tar archive"},
@ -101,7 +103,8 @@ func cmdHelp(args ...string) error {
{"images", "List images"},
{"import", "Create a new filesystem image from the contents of a tarball"},
{"info", "Display system-wide information"},
{"inspect", "Return low-level information on a container/image"},
{"insert", "Insert a file in an image"},
{"inspect", "Return low-level information on a container"},
{"kill", "Kill a running container"},
{"login", "Register or Login to the docker registry server"},
{"logs", "Fetch the logs of a container"},
@ -125,6 +128,40 @@ func cmdHelp(args ...string) error {
return nil
}
func CmdInsert(args ...string) error {
cmd := Subcmd("insert", "IMAGE URL PATH", "Insert a file from URL in the IMAGE at PATH")
if err := cmd.Parse(args); err != nil {
return nil
}
if cmd.NArg() != 3 {
cmd.Usage()
return nil
}
v := url.Values{}
v.Set("url", cmd.Arg(1))
v.Set("path", cmd.Arg(2))
err := hijack("POST", "/images/"+cmd.Arg(0)+"?"+v.Encode(), false)
if err != nil {
return err
}
return nil
}
func CmdBuild(args ...string) error {
cmd := Subcmd("build", "-", "Build an image from Dockerfile via stdin")
if err := cmd.Parse(args); err != nil {
return nil
}
err := hijack("POST", "/build", false)
if err != nil {
return err
}
return nil
}
// 'docker login': login / register a user to registry service.
func CmdLogin(args ...string) error {
var readStringOnRawTerminal = func(stdin io.Reader, stdout io.Writer, echo bool) string {
@ -565,7 +602,8 @@ func CmdImport(args ...string) error {
}
func CmdPush(args ...string) error {
cmd := Subcmd("push", "NAME", "Push an image or a repository to the registry")
cmd := Subcmd("push", "[OPTION] NAME", "Push an image or a repository to the registry")
registry := cmd.String("registry", "", "Registry host to push the image to")
if err := cmd.Parse(args); err != nil {
return nil
}
@ -587,8 +625,8 @@ func CmdPush(args ...string) error {
return err
}
// If the login failed, abort
if out.Username == "" {
// If the login failed AND we're using the index, abort
if *registry == "" && out.Username == "" {
if err := CmdLogin(args...); err != nil {
return err
}
@ -608,12 +646,13 @@ func CmdPush(args ...string) error {
}
if len(strings.SplitN(name, "/", 2)) == 1 {
return fmt.Errorf(
"Impossible to push a \"root\" repository. Please rename your repository in <user>/<repo> (ex: %s/%s)",
out.Username, name)
return fmt.Errorf("Impossible to push a \"root\" repository. Please rename your repository in <user>/<repo> (ex: %s/%s)", out.Username, name)
}
if err := hijack("POST", "/images"+name+"/push", false); err != nil {
v := url.Values{}
v.Set("registry", *registry)
if err := hijack("POST", "/images"+name+"/push?"+v.Encode(), false); err != nil {
return err
}
return nil
@ -621,6 +660,8 @@ func CmdPush(args ...string) error {
func CmdPull(args ...string) error {
cmd := Subcmd("pull", "NAME", "Pull an image or a repository from the registry")
tag := cmd.String("t", "", "Download tagged image in repository")
registry := cmd.String("registry", "", "Registry to download from. Necessary if image is pulled by ID")
if err := cmd.Parse(args); err != nil {
return nil
}
@ -629,8 +670,18 @@ func CmdPull(args ...string) error {
cmd.Usage()
return nil
}
remote := cmd.Arg(0)
if strings.Contains(remote, ":") {
remoteParts := strings.Split(remote, ":")
tag = &remoteParts[1]
remote = remoteParts[0]
}
v := url.Values{}
v.Set("fromImage", cmd.Arg(0))
v.Set("fromImage", remote)
v.Set("tag", *tag)
v.Set("registry", *registry)
if err := hijack("POST", "/images?"+v.Encode(), false); err != nil {
return err
@ -643,6 +694,7 @@ func CmdImages(args ...string) error {
cmd := Subcmd("images", "[OPTIONS] [NAME]", "List images")
quiet := cmd.Bool("q", false, "only show numeric IDs")
all := cmd.Bool("a", false, "show all images")
flViz := cmd.Bool("viz", false, "output graph in graphviz format")
if err := cmd.Parse(args); err != nil {
return nil
@ -651,42 +703,50 @@ func CmdImages(args ...string) error {
cmd.Usage()
return nil
}
v := url.Values{}
if cmd.NArg() == 1 {
v.Set("filter", cmd.Arg(0))
}
if *quiet {
v.Set("quiet", "1")
}
if *all {
v.Set("all", "1")
}
body, _, err := call("GET", "/images?"+v.Encode(), nil)
if err != nil {
return err
}
var outs []ApiImages
err = json.Unmarshal(body, &outs)
if err != nil {
return err
}
w := tabwriter.NewWriter(os.Stdout, 20, 1, 3, ' ', 0)
if !*quiet {
fmt.Fprintln(w, "REPOSITORY\tTAG\tID\tCREATED")
}
for _, out := range outs {
if !*quiet {
fmt.Fprintf(w, "%s\t%s\t%s\t%s ago\n", out.Repository, out.Tag, out.Id, HumanDuration(time.Now().Sub(time.Unix(out.Created, 0))))
} else {
fmt.Fprintln(w, out.Id)
if *flViz {
if err := hijack("GET", "/images?viz=1", false); err != nil {
return err
}
} else {
v := url.Values{}
if cmd.NArg() == 1 {
v.Set("filter", cmd.Arg(0))
}
if *quiet {
v.Set("quiet", "1")
}
if *all {
v.Set("all", "1")
}
}
if !*quiet {
w.Flush()
body, _, err := call("GET", "/images?"+v.Encode(), nil)
if err != nil {
return err
}
var outs []ApiImages
err = json.Unmarshal(body, &outs)
if err != nil {
return err
}
w := tabwriter.NewWriter(os.Stdout, 20, 1, 3, ' ', 0)
if !*quiet {
fmt.Fprintln(w, "REPOSITORY\tTAG\tID\tCREATED")
}
for _, out := range outs {
if !*quiet {
fmt.Fprintf(w, "%s\t%s\t%s\t%s ago\n", out.Repository, out.Tag, out.Id, HumanDuration(time.Now().Sub(time.Unix(out.Created, 0))))
} else {
fmt.Fprintln(w, out.Id)
}
}
if !*quiet {
w.Flush()
}
}
return nil
}
@ -775,7 +835,6 @@ func CmdCommit(args ...string) error {
return err
}
}
body, _, err := call("POST", "/commit?"+v.Encode(), config)
if err != nil {
return err
@ -975,7 +1034,7 @@ func CmdTag(args ...string) error {
}
func CmdRun(args ...string) error {
config, cmd, err := ParseRun(args)
config, cmd, err := ParseRun(args, nil)
if err != nil {
return err
}
@ -1004,7 +1063,6 @@ func CmdRun(args ...string) error {
return err
}
return nil
}
if err != nil {
return err
@ -1102,6 +1160,7 @@ func hijack(method, path string, setRawTerminal bool) error {
if err != nil {
return err
}
req.Header.Set("Content-Type", "plain/text")
dial, err := net.Dial("tcp", "0.0.0.0:4243")
if err != nil {
return err

73
commands_test.go
View file

@ -75,6 +75,77 @@ func cmdWait(srv *Server, container *Container) error {
return closeWrap(stdout, stdoutPipe)
}
func cmdImages(srv *Server, args ...string) (string, error) {
stdout, stdoutPipe := io.Pipe()
go func() {
if err := srv.CmdImages(nil, stdoutPipe, args...); err != nil {
return
}
// force the pipe closed, so that the code below gets an EOF
stdoutPipe.Close()
}()
output, err := ioutil.ReadAll(stdout)
if err != nil {
return "", err
}
// Cleanup pipes
return string(output), closeWrap(stdout, stdoutPipe)
}
// TestImages checks that 'docker images' displays information correctly
func TestImages(t *testing.T) {
runtime, err := newTestRuntime()
if err != nil {
t.Fatal(err)
}
defer nuke(runtime)
srv := &Server{runtime: runtime}
output, err := cmdImages(srv)
if !strings.Contains(output, "REPOSITORY") {
t.Fatal("'images' should have a header")
}
if !strings.Contains(output, "docker-ut") {
t.Fatal("'images' should show the docker-ut image")
}
if !strings.Contains(output, "e9aa60c60128") {
t.Fatal("'images' should show the docker-ut image id")
}
output, err = cmdImages(srv, "-q")
if strings.Contains(output, "REPOSITORY") {
t.Fatal("'images -q' should not have a header")
}
if strings.Contains(output, "docker-ut") {
t.Fatal("'images' should not show the docker-ut image name")
}
if !strings.Contains(output, "e9aa60c60128") {
t.Fatal("'images' should show the docker-ut image id")
}
output, err = cmdImages(srv, "-viz")
if !strings.HasPrefix(output, "digraph docker {") {
t.Fatal("'images -v' should start with the dot header")
}
if !strings.HasSuffix(output, "}\n") {
t.Fatal("'images -v' should end with a '}'")
}
if !strings.Contains(output, "base -> \"e9aa60c60128\" [style=invis]") {
t.Fatal("'images -v' should have the docker-ut image id node")
}
// todo: add checks for -a
}
// TestRunHostname checks that 'docker run -h' correctly sets a custom hostname
func TestRunHostname(t *testing.T) {
runtime, err := newTestRuntime()
@ -341,7 +412,7 @@ func TestAttachDisconnect(t *testing.T) {
srv := &Server{runtime: runtime}
container, err := runtime.Create(
container, err := NewBuilder(runtime).Create(
&Config{
Image: GetTestImage(runtime).Id,
Memory: 33554432,

37
container.go
View file

@ -71,7 +71,7 @@ type Config struct {
VolumesFrom string
}
func ParseRun(args []string) (*Config, *flag.FlagSet, error) {
func ParseRun(args []string, capabilities *Capabilities) (*Config, *flag.FlagSet, error) {
cmd := Subcmd("run", "[OPTIONS] IMAGE COMMAND [ARG...]", "Run a command in a new container")
if len(args) > 0 && args[0] != "--help" {
cmd.SetOutput(ioutil.Discard)
@ -86,6 +86,11 @@ func ParseRun(args []string) (*Config, *flag.FlagSet, error) {
flTty := cmd.Bool("t", false, "Allocate a pseudo-tty")
flMemory := cmd.Int64("m", 0, "Memory limit (in bytes)")
if capabilities != nil && *flMemory > 0 && !capabilities.MemoryLimit {
//fmt.Fprintf(stdout, "WARNING: Your kernel does not support memory limit capabilities. Limitation discarded.\n")
*flMemory = 0
}
var flPorts ListOpts
cmd.Var(&flPorts, "p", "Expose a container's port to the host (use 'docker port' to see the actual mapping)")
@ -143,6 +148,11 @@ func ParseRun(args []string) (*Config, *flag.FlagSet, error) {
VolumesFrom: *flVolumesFrom,
}
if capabilities != nil && *flMemory > 0 && !capabilities.SwapLimit {
//fmt.Fprintf(stdout, "WARNING: Your kernel does not support swap limit capabilities. Limitation discarded.\n")
config.MemorySwap = -1
}
// When allocating stdin in attached mode, close stdin at client disconnect
if config.OpenStdin && config.AttachStdin {
config.StdinOnce = true
@ -168,6 +178,23 @@ func (settings *NetworkSettings) PortMappingHuman() string {
return strings.Join(mapping, ", ")
}
// Inject the io.Reader at the given path. Note: do not close the reader
func (container *Container) Inject(file io.Reader, pth string) error {
// Make sure the directory exists
if err := os.MkdirAll(path.Join(container.rwPath(), path.Dir(pth)), 0755); err != nil {
return err
}
// FIXME: Handle permissions/already existing dest
dest, err := os.Create(path.Join(container.rwPath(), pth))
if err != nil {
return err
}
if _, err := io.Copy(dest, file); err != nil {
return err
}
return nil
}
func (container *Container) Cmd() *exec.Cmd {
return container.cmd
}
@ -726,6 +753,14 @@ func (container *Container) ExportRw() (Archive, error) {
return Tar(container.rwPath(), Uncompressed)
}
func (container *Container) RwChecksum() (string, error) {
rwData, err := Tar(container.rwPath(), Xz)
if err != nil {
return "", err
}
return HashData(rwData)
}
func (container *Container) Export() (Archive, error) {
if err := container.EnsureMounted(); err != nil {
return nil, err

74
container_test.go
View file

@ -20,7 +20,7 @@ func TestIdFormat(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container1, err := runtime.Create(
container1, err := NewBuilder(runtime).Create(
&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"/bin/sh", "-c", "echo hello world"},
@ -44,7 +44,7 @@ func TestMultipleAttachRestart(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(
container, err := NewBuilder(runtime).Create(
&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"/bin/sh", "-c",
@ -148,8 +148,10 @@ func TestDiff(t *testing.T) {
}
defer nuke(runtime)
builder := NewBuilder(runtime)
// Create a container and remove a file
container1, err := runtime.Create(
container1, err := builder.Create(
&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"/bin/rm", "/etc/passwd"},
@ -190,7 +192,7 @@ func TestDiff(t *testing.T) {
}
// Create a new container from the commited image
container2, err := runtime.Create(
container2, err := builder.Create(
&Config{
Image: img.Id,
Cmd: []string{"cat", "/etc/passwd"},
@ -223,7 +225,9 @@ func TestCommitAutoRun(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container1, err := runtime.Create(
builder := NewBuilder(runtime)
container1, err := builder.Create(
&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"/bin/sh", "-c", "echo hello > /world"},
@ -254,8 +258,7 @@ func TestCommitAutoRun(t *testing.T) {
}
// FIXME: Make a TestCommit that stops here and check docker.root/layers/img.id/world
container2, err := runtime.Create(
container2, err := builder.Create(
&Config{
Image: img.Id,
},
@ -301,7 +304,10 @@ func TestCommitRun(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container1, err := runtime.Create(
builder := NewBuilder(runtime)
container1, err := builder.Create(
&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"/bin/sh", "-c", "echo hello > /world"},
@ -333,7 +339,7 @@ func TestCommitRun(t *testing.T) {
// FIXME: Make a TestCommit that stops here and check docker.root/layers/img.id/world
container2, err := runtime.Create(
container2, err := builder.Create(
&Config{
Image: img.Id,
Cmd: []string{"cat", "/world"},
@ -380,7 +386,7 @@ func TestStart(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(
container, err := NewBuilder(runtime).Create(
&Config{
Image: GetTestImage(runtime).Id,
Memory: 33554432,
@ -419,7 +425,7 @@ func TestRun(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(
container, err := NewBuilder(runtime).Create(
&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"ls", "-al"},
@ -447,7 +453,7 @@ func TestOutput(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(
container, err := NewBuilder(runtime).Create(
&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"echo", "-n", "foobar"},
@ -472,7 +478,7 @@ func TestKillDifferentUser(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"tail", "-f", "/etc/resolv.conf"},
User: "daemon",
@ -520,7 +526,7 @@ func TestKill(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"cat", "/dev/zero"},
},
@ -566,7 +572,9 @@ func TestExitCode(t *testing.T) {
}
defer nuke(runtime)
trueContainer, err := runtime.Create(&Config{
builder := NewBuilder(runtime)
trueContainer, err := builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"/bin/true", ""},
})
@ -581,7 +589,7 @@ func TestExitCode(t *testing.T) {
t.Errorf("Unexpected exit code %d (expected 0)", trueContainer.State.ExitCode)
}
falseContainer, err := runtime.Create(&Config{
falseContainer, err := builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"/bin/false", ""},
})
@ -603,7 +611,7 @@ func TestRestart(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"echo", "-n", "foobar"},
},
@ -636,7 +644,7 @@ func TestRestartStdin(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"cat"},
@ -715,8 +723,10 @@ func TestUser(t *testing.T) {
}
defer nuke(runtime)
builder := NewBuilder(runtime)
// Default user must be root
container, err := runtime.Create(&Config{
container, err := builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"id"},
},
@ -734,7 +744,7 @@ func TestUser(t *testing.T) {
}
// Set a username
container, err = runtime.Create(&Config{
container, err = builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"id"},
@ -754,7 +764,7 @@ func TestUser(t *testing.T) {
}
// Set a UID
container, err = runtime.Create(&Config{
container, err = builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"id"},
@ -774,7 +784,7 @@ func TestUser(t *testing.T) {
}
// Set a different user by uid
container, err = runtime.Create(&Config{
container, err = builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"id"},
@ -796,7 +806,7 @@ func TestUser(t *testing.T) {
}
// Set a different user by username
container, err = runtime.Create(&Config{
container, err = builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"id"},
@ -823,7 +833,9 @@ func TestMultipleContainers(t *testing.T) {
}
defer nuke(runtime)
container1, err := runtime.Create(&Config{
builder := NewBuilder(runtime)
container1, err := builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"cat", "/dev/zero"},
},
@ -833,7 +845,7 @@ func TestMultipleContainers(t *testing.T) {
}
defer runtime.Destroy(container1)
container2, err := runtime.Create(&Config{
container2, err := builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"cat", "/dev/zero"},
},
@ -879,7 +891,7 @@ func TestStdin(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"cat"},
@ -926,7 +938,7 @@ func TestTty(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"cat"},
@ -973,7 +985,7 @@ func TestEnv(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"/usr/bin/env"},
},
@ -1047,7 +1059,7 @@ func TestLXCConfig(t *testing.T) {
memMin := 33554432
memMax := 536870912
mem := memMin + rand.Intn(memMax-memMin)
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"/bin/true"},
@ -1074,7 +1086,7 @@ func BenchmarkRunSequencial(b *testing.B) {
}
defer nuke(runtime)
for i := 0; i < b.N; i++ {
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"echo", "-n", "foo"},
},
@ -1109,7 +1121,7 @@ func BenchmarkRunParallel(b *testing.B) {
complete := make(chan error)
tasks = append(tasks, complete)
go func(i int, complete chan error) {
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"echo", "-n", "foo"},
},

2
contrib/docker-build/docker-build
View file

@ -89,7 +89,7 @@ def main():
# Skip comments and empty lines
if line == "" or line[0] == "#":
continue
op, param = line.split(" ", 1)
op, param = line.split(None, 1)
print op.upper() + " " + param
if op == "from":
base = param

4
contrib/install.sh
View file

@ -36,9 +36,9 @@ else
fi
echo "Downloading docker binary and uncompressing into /usr/local/bin..."
curl -s http://get.docker.io/builds/$(uname -s)/$(uname -m)/docker-master.tgz |
curl -s http://get.docker.io/builds/$(uname -s)/$(uname -m)/docker-latest.tgz |
tar -C /usr/local/bin --strip-components=1 -zxf- \
docker-master/docker
docker-latest/docker
if [ -f /etc/init/dockerd.conf ]
then

2
docs/requirements.txt Normal file
View file

@ -0,0 +1,2 @@
Sphinx==1.1.3
sphinxcontrib-httpdomain==1.1.8

130
docs/sources/builder/basics.rst Normal file
View file

@ -0,0 +1,130 @@
==============
Docker Builder
==============
.. contents:: Table of Contents
1. Format
=========
The Docker builder format is quite simple:
``instruction arguments``
The first instruction must be `FROM`
All instruction are to be placed in a file named `Dockerfile`
In order to place comments within a Dockerfile, simply prefix the line with "`#`"
2. Instructions
===============
Docker builder comes with a set of instructions:
1. FROM: Set from what image to build
2. RUN: Execute a command
3. INSERT: Insert a remote file (http) into the image
2.1 FROM
--------
``FROM <image>``
The `FROM` instruction must be the first one in order for Builder to know from where to run commands.
`FROM` can also be used in order to build multiple images within a single Dockerfile
2.2 MAINTAINER
--------------
``MAINTAINER <name>``
The `MAINTAINER` instruction allow you to set the Author field of the generated images.
This instruction is never automatically reset.
2.3 RUN
-------
``RUN <command>``
The `RUN` instruction is the main one, it allows you to execute any commands on the `FROM` image and to save the results.
You can use as many `RUN` as you want within a Dockerfile, the commands will be executed on the result of the previous command.
2.4 CMD
-------
``CMD <command>``
The `CMD` instruction sets the command to be executed when running the image.
It is equivalent to do `docker commit -run '{"Cmd": <command>}'` outside the builder.
.. note::
Do not confuse `RUN` with `CMD`. `RUN` actually run a command and save the result, `CMD` does not execute anything.
2.5 EXPOSE
----------
``EXPOSE <port> [<port>...]``
The `EXPOSE` instruction sets ports to be publicly exposed when running the image.
This is equivalent to do `docker commit -run '{"PortSpecs": ["<port>", "<port2>"]}'` outside the builder.
2.6 ENV
-------
``ENV <key> <value>``
The `ENV` instruction set as environment variable `<key>` with the value `<value>`. This value will be passed to all future ``RUN`` instructions.
.. note::
The environment variables are local to the Dockerfile, they will not be set as autorun.
2.7 INSERT
----------
``INSERT <file url> <path>``
The `INSERT` instruction will download the file at the given url and place it within the image at the given path.
.. note::
The path must include the file name.
3. Dockerfile Examples
======================
::
# Nginx
#
# VERSION 0.0.1
# DOCKER-VERSION 0.2
from ubuntu
maintainer Guillaume J. Charmes "guillaume@dotcloud.com"
# make sure the package repository is up to date
run echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list
run apt-get update
run apt-get install -y inotify-tools nginx apache openssh-server
insert https://raw.github.com/creack/docker-vps/master/nginx-wrapper.sh /usr/sbin/nginx-wrapper
::
# Firefox over VNC
#
# VERSION 0.3
# DOCKER-VERSION 0.2
from ubuntu
# make sure the package repository is up to date
run echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list
run apt-get update
# Install vnc, xvfb in order to create a 'fake' display and firefox
run apt-get install -y x11vnc xvfb firefox
run mkdir /.vnc
# Setup a password
run x11vnc -storepasswd 1234 ~/.vnc/passwd
# Autostart firefox (might not be the best way to do it, but it does the trick)
run bash -c 'echo "firefox" >> /.bashrc'
expose 5900
cmd ["x11vnc", "-forever", "-usepw", "-create"]

14
docs/sources/builder/index.rst Normal file
View file

@ -0,0 +1,14 @@
:title: docker documentation
:description: Documentation for docker builder
:keywords: docker, builder, dockerfile
Builder
=======
Contents:
.. toctree::
:maxdepth: 2
basics

1
docs/sources/commandline/cli.rst
View file

@ -27,6 +27,7 @@ Available Commands
:maxdepth: 1
command/attach
command/build
command/commit
command/diff
command/export

9
docs/sources/commandline/command/build.rst Normal file
View file

@ -0,0 +1,9 @@
===========================================
``build`` -- Build a container from Dockerfile via stdin
===========================================
::
Usage: docker build -
Example: cat Dockerfile | docker build -
Build a new image from the Dockerfile passed via stdin

10
docs/sources/commandline/command/images.rst
View file

@ -10,3 +10,13 @@
-a=false: show all images
-q=false: only show numeric IDs
-viz=false: output in graphviz format
Displaying images visually
--------------------------
::
docker images -viz | dot -Tpng -o docker.png
.. image:: images/docker_images.gif

BIN
docs/sources/commandline/command/images/docker_images.gif Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 35 KiB

2
docs/sources/commandline/index.rst
View file

@ -9,7 +9,7 @@ Commands
Contents:
.. toctree::
:maxdepth: 2
:maxdepth: 3
basics
workingwithrepository

28
docs/sources/index.html
View file

@ -7,7 +7,7 @@
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>Docker - the Linux container runtime</title>
<title>Docker - the Linux container engine</title>
<meta name="description" content="Docker encapsulates heterogeneous payloads in standard containers">
<meta name="viewport" content="width=device-width">
@ -71,18 +71,28 @@
<div style="text-align: center; padding: 50px 30px 50px 30px;">
<h1>Docker</h1>
<h2>The Linux container runtime</h2>
<h2>The Linux container engine</h2>
</div>
<div style="display: block; text-align: center; padding: 10px 30px 50px 30px;">
<p>
Docker complements LXC with a high-level API which operates at the process level.
It runs unix processes with strong guarantees of isolation and repeatability across servers.
</p>
<p>
Docker is a great building block for automating distributed systems: large-scale web deployments, database clusters, continuous deployment systems, private PaaS, service-oriented architectures, etc.
</p>
<p>
Docker is an open-source engine which automates the deployment of applications as highly portable, self-sufficient containers.
</p>
<p>
Docker containers are both <string>hardware-agnostic</strong> and <strong>platform-agnostic</strong>. This means that they can run anywhere, from your
laptop to the largest EC2 compute instance and everything in between - and they don't require that you use a particular
language, framework or packaging system. That makes them great building blocks for deploying and scaling web apps, databases
and backend services without depending on a particular stack or provider.
</p>
<p>
Docker is an open-source implementation of the deployment engine which powers <a href="http://dotcloud.com">dotCloud</a>, a popular Platform-as-a-Service.
It benefits directly from the experience accumulated over several years of large-scale operation and support of hundreds of thousands
of applications and databases.
</p>
</div>

2
docs/sources/index.rst
View file

@ -16,6 +16,8 @@ This documentation has the following resources:
contributing/index
commandline/index
registry/index
index/index
builder/index
remote-api/index
faq

15
docs/sources/index/index.rst Normal file
View file

@ -0,0 +1,15 @@
:title: Docker Index documentation
:description: Documentation for docker Index
:keywords: docker, index, api
Index
=====
Contents:
.. toctree::
:maxdepth: 2
search

38
docs/sources/index/search.rst Normal file
View file

@ -0,0 +1,38 @@
=======================
Docker Index Search API
=======================
Search
------
.. http:get:: /v1/search
Search the Index given a search term. It accepts :http:method:`get` only.
**Example request**:
.. sourcecode:: http
GET /v1/search?q=search_term HTTP/1.1
Host: example.com
Accept: application/json
**Example response**:
.. sourcecode:: http
HTTP/1.1 200 OK
Vary: Accept
Content-Type: application/json
{"query":"search_term",
"num_results": 2,
"results" : [
{"name": "dotcloud/base", "description": "A base ubuntu64 image..."},
{"name": "base2", "description": "A base ubuntu64 image..."},
]
}
:query q: what you want to search for
:statuscode 200: no error
:statuscode 500: server error

8
docs/sources/installation/binaries.rst
View file

@ -26,9 +26,9 @@ Install the docker binary:
::
wget http://get.docker.io/builds/Linux/x86_64/docker-master.tgz
tar -xf docker-master.tgz
sudo cp ./docker-master /usr/local/bin
wget http://get.docker.io/builds/Linux/x86_64/docker-latest.tgz
tar -xf docker-latest.tgz
sudo cp ./docker-latest/docker /usr/local/bin
Note: docker currently only supports 64-bit Linux hosts.
@ -50,4 +50,4 @@ Run your first container!
Continue with the :ref:`hello_world` example.
Continue with the :ref:`hello_world` example.

4
docs/sources/installation/ubuntulinux.rst
View file

@ -18,7 +18,7 @@ The linux-image-extra package is only needed on standard Ubuntu EC2 AMIs in orde
.. code-block:: bash
sudo apt-get install linux-image-extra-`uname -r`
sudo apt-get install linux-image-extra-`uname -r` lxc bsdtar
Installation
@ -48,7 +48,7 @@ Now install it, you will see another warning that the package cannot be authenti
.. code-block:: bash
sudo apt-get install lxc-docker
curl get.docker.io | sudo sh -x
Verify it worked

6
docs/sources/installation/upgrading.rst
View file

@ -11,7 +11,7 @@ Get the latest docker binary:
::
wget http://get.docker.io/builds/$(uname -s)/$(uname -m)/docker-master.tgz
wget http://get.docker.io/builds/$(uname -s)/$(uname -m)/docker-latest.tgz
@ -19,7 +19,7 @@ Unpack it to your current dir
::
tar -xf docker-master.tgz
tar -xf docker-latest.tgz
Stop your current daemon. How you stop your daemon depends on how you started it.
@ -38,4 +38,4 @@ Now start the daemon
sudo ./docker -d &
Alternatively you can replace the docker binary in ``/usr/local/bin``
Alternatively you can replace the docker binary in ``/usr/local/bin``

6
docs/sources/registry/api.rst
View file

@ -84,7 +84,9 @@ It’s possible to run docker pull https://<registry>/repositories/samalba/busyb
Currently registry redirects to s3 urls for downloads, going forward all downloads need to be streamed through the registry. The Registry will then abstract the calls to S3 by a top-level class which implements sub-classes for S3 and local storage.
Token is only returned when it is a private repo, public repos do not require tokens to be returned. The Registry will still contact the Index to make sure the pull is authorized (“is it ok to download this repos without a Token?”).
Token is only returned when the 'X-Docker-Token' header is sent with request.
Basic Auth is required to pull private repos. Basic auth isn't required for pulling public repos, but if one is provided, it needs to be valid and for an active account.
API (pulling repository foo/bar):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -426,6 +428,8 @@ You have 3 options:
- X-Docker-Token: true
In this case, along with the 200 response, you’ll get a new token (if user auth is ok):
If authorization isn't correct you get a 401 response.
If account isn't active you will get a 403 response.
**Response**:
- 200 OK

2
docs/sources/remote-api/api.rst
View file

@ -533,6 +533,7 @@ Create an image
:query fromSrc: source to import, - means stdin
:query repo: repository
:query tag: tag
:query registry: the registry to pull from
:statuscode 200: no error
:statuscode 500: server error
@ -646,6 +647,7 @@ Push an image on the registry
{{ STREAM }}
:query registry: the registry you wan to push, optional
:statuscode 200: no error
:statuscode 404: no such image
:statuscode 500: server error

12
graph.go
View file

@ -4,6 +4,7 @@ import (
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
"path"
"path/filepath"
@ -13,8 +14,9 @@ import (
// A Graph is a store for versioned filesystem images and the relationship between them.
type Graph struct {
Root string
idIndex *TruncIndex
Root string
idIndex *TruncIndex
httpClient *http.Client
}
// NewGraph instantiates a new graph at the given root path in the filesystem.
@ -97,15 +99,11 @@ func (graph *Graph) Create(layerData Archive, container *Container, comment, aut
img.Parent = container.Image
img.Container = container.Id
img.ContainerConfig = *container.Config
if config == nil {
if parentImage, err := graph.Get(container.Image); err == nil && parentImage != nil {
img.Config = parentImage.Config
}
}
}
if err := graph.Register(layerData, img); err != nil {
return nil, err
}
img.Checksum()
return img, nil
}

61
image.go
View file

@ -2,6 +2,7 @@ package docker
import (
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
@ -51,6 +52,7 @@ func LoadImage(root string) (*Image, error) {
} else if !stat.IsDir() {
return nil, fmt.Errorf("Couldn't load image %s: %s is not a directory", img.Id, layerPath(root))
}
return &img, nil
}
@ -257,3 +259,62 @@ func (img *Image) layer() (string, error) {
}
return layerPath(root), nil
}
func (img *Image) Checksum() (string, error) {
root, err := img.root()
if err != nil {
return "", err
}
checksumDictPth := path.Join(root, "..", "..", "checksums")
checksums := new(map[string]string)
if checksumDict, err := ioutil.ReadFile(checksumDictPth); err == nil {
if err := json.Unmarshal(checksumDict, checksums); err != nil {
return "", err
}
if checksum, ok := (*checksums)[img.Id]; ok {
return checksum, nil
}
}
layer, err := img.layer()
if err != nil {
return "", err
}
jsonData, err := ioutil.ReadFile(jsonPath(root))
if err != nil {
return "", err
}
layerData, err := Tar(layer, Xz)
if err != nil {
return "", err
}
h := sha256.New()
if _, err := h.Write(jsonData); err != nil {
return "", err
}
if _, err := h.Write([]byte("\n")); err != nil {
return "", err
}
if _, err := io.Copy(h, layerData); err != nil {
return "", err
}
hash := "sha256:" + hex.EncodeToString(h.Sum(nil))
if *checksums == nil {
*checksums = map[string]string{}
}
(*checksums)[img.Id] = hash
checksumJson, err := json.Marshal(checksums)
if err != nil {
return hash, err
}
if err := ioutil.WriteFile(checksumDictPth, checksumJson, 0600); err != nil {
return hash, err
}
return hash, nil
}

11
packaging/ubuntu/Vagrantfile vendored
View file

@ -1,12 +1,15 @@
BUILDBOT_IP = '192.168.33.32'
GOPHERS_KEY = "308C15A29AD198E9"
Vagrant::Config.run do |config|
config.vm.box = 'precise64'
config.vm.box_url = 'http://files.vagrantup.com/precise64.box'
config.vm.share_folder 'v-data', '/data/docker', "#{File.dirname(__FILE__)}/../.."
config.vm.network :hostonly,BUILDBOT_IP
# Add docker PPA key to the local repository and install docker
pkg_cmd = "apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys #{GOPHERS_KEY}; " \
"echo 'deb http://ppa.launchpad.net/gophers/go/ubuntu precise main' >/etc/apt/sources.list.d/gophers-go.list; " \
# Install ubuntu packaging dependencies and create ubuntu packages
config.vm.provision :shell, :inline => 'export DEBIAN_FRONTEND=noninteractive; apt-get -qq update; apt-get install -qq -y git debhelper autotools-dev devscripts golang'
config.vm.provision :shell, :inline => "export GPG_KEY='#{ENV['GPG_KEY']}'; cd /data/docker/packaging/ubuntu; make ubuntu"
pkg_cmd << "export DEBIAN_FRONTEND=noninteractive; apt-get -qq update; apt-get install -qq -y git debhelper autotools-dev devscripts golang-stable; " \
"export GPG_KEY='#{ENV['GPG_KEY']}'; cd /data/docker/packaging/ubuntu; make ubuntu"
config.vm.provision :shell, :inline => pkg_cmd
end

24
packaging/ubuntu/changelog
View file

@ -1,16 +1,28 @@
lxc-docker (0.3.0-1) precise; urgency=low
- Registry: Implement the new registry
- Documentation: new example: sharing data between 2 couchdb databases
- Runtime: Fix the command existance check
- Runtime: strings.Split may return an empty string on no match
- Runtime: Fix an index out of range crash if cgroup memory is not
- Documentation: Various improvments
- Vagrant: Use only one deb line in /etc/apt
-- dotCloud <ops@dotcloud.com> Fri, 5 May 2013 00:00:00 -0700
lxc-docker (0.2.2-1) precise; urgency=low
- Support for data volumes ('docker run -v=PATH')
- Share data volumes between containers ('docker run -volumes-from')
- Improved documentation
- Upgrade to Go 1.0.3
- Various upgrades to the dev environment for contributors
- Support for data volumes ('docker run -v=PATH')
- Share data volumes between containers ('docker run -volumes-from')
- Improved documentation
- Upgrade to Go 1.0.3
- Various upgrades to the dev environment for contributors
-- dotCloud <ops@dotcloud.com> Fri, 3 May 2013 00:00:00 -0700
lxc-docker (0.2.1-1) precise; urgency=low
- 'docker commit -run' bundles a layer with default runtime options: command, ports etc.
- 'docker commit -run' bundles a layer with default runtime options: command, ports etc.
- Improve install process on Vagrant
- New Dockerfile operation: "maintainer"
- New Dockerfile operation: "expose"

2
packaging/ubuntu/control
View file

@ -2,7 +2,7 @@ Source: lxc-docker
Section: misc
Priority: extra
Maintainer: Daniel Mizyrycki <daniel@dotcloud.com>
Build-Depends: debhelper,autotools-dev,devscripts,golang
Build-Depends: debhelper,autotools-dev,devscripts,golang-stable
Standards-Version: 3.9.3
Homepage: http://github.com/dotcloud/docker

666
registry.go
View file

@ -1,20 +1,20 @@
package docker
import (
"bytes"
"encoding/json"
"fmt"
"github.com/dotcloud/docker/auth"
"github.com/shin-/cookiejar"
"io"
"io/ioutil"
"net/http"
"os"
"path"
"strings"
)
//FIXME: Set the endpoint in a conf file or via commandline
//const REGISTRY_ENDPOINT = "http://registry-creack.dotcloud.com/v1"
const REGISTRY_ENDPOINT = auth.REGISTRY_SERVER + "/v1"
const INDEX_ENDPOINT = auth.INDEX_SERVER + "/v1"
// Build an Image object from raw json data
func NewImgJson(src []byte) (*Image, error) {
@ -28,34 +28,23 @@ func NewImgJson(src []byte) (*Image, error) {
return ret, nil
}
// Build an Image object list from a raw json data
// FIXME: Do this in "stream" mode
func NewMultipleImgJson(src []byte) ([]*Image, error) {
ret := []*Image{}
dec := json.NewDecoder(strings.NewReader(string(src)))
for {
m := &Image{}
if err := dec.Decode(m); err == io.EOF {
break
} else if err != nil {
return nil, err
}
ret = append(ret, m)
func doWithCookies(c *http.Client, req *http.Request) (*http.Response, error) {
for _, cookie := range c.Jar.Cookies(req.URL) {
req.AddCookie(cookie)
}
return ret, nil
return c.Do(req)
}
// Retrieve the history of a given image from the Registry.
// Return a list of the parent's json (requested image included)
func (graph *Graph) getRemoteHistory(imgId string, authConfig *auth.AuthConfig) ([]*Image, error) {
client := &http.Client{}
func (graph *Graph) getRemoteHistory(imgId, registry string, token []string) ([]string, error) {
client := graph.getHttpClient()
req, err := http.NewRequest("GET", REGISTRY_ENDPOINT+"/images/"+imgId+"/history", nil)
req, err := http.NewRequest("GET", registry+"/images/"+imgId+"/ancestry", nil)
if err != nil {
return nil, err
}
req.SetBasicAuth(authConfig.Username, authConfig.Password)
req.Header.Set("Authorization", "Token "+strings.Join(token, ", "))
res, err := client.Do(req)
if err != nil || res.StatusCode != 200 {
if res != nil {
@ -70,41 +59,83 @@ func (graph *Graph) getRemoteHistory(imgId string, authConfig *auth.AuthConfig)
return nil, fmt.Errorf("Error while reading the http response: %s\n", err)
}
history, err := NewMultipleImgJson(jsonString)
if err != nil {
return nil, fmt.Errorf("Error while parsing the json: %s\n", err)
Debugf("Ancestry: %s", jsonString)
history := new([]string)
if err := json.Unmarshal(jsonString, history); err != nil {
return nil, err
}
return history, nil
return *history, nil
}
func (graph *Graph) getHttpClient() *http.Client {
if graph.httpClient == nil {
graph.httpClient = &http.Client{}
graph.httpClient.Jar = cookiejar.NewCookieJar()
}
return graph.httpClient
}
// Check if an image exists in the Registry
func (graph *Graph) LookupRemoteImage(imgId string, authConfig *auth.AuthConfig) bool {
func (graph *Graph) LookupRemoteImage(imgId, registry string, authConfig *auth.AuthConfig) bool {
rt := &http.Transport{Proxy: http.ProxyFromEnvironment}
req, err := http.NewRequest("GET", REGISTRY_ENDPOINT+"/images/"+imgId+"/json", nil)
req, err := http.NewRequest("GET", registry+"/images/"+imgId+"/json", nil)
if err != nil {
return false
}
req.SetBasicAuth(authConfig.Username, authConfig.Password)
res, err := rt.RoundTrip(req)
if err != nil || res.StatusCode != 307 {
return false
return err == nil && res.StatusCode == 307
}
func (graph *Graph) getImagesInRepository(repository string, authConfig *auth.AuthConfig) ([]map[string]string, error) {
u := INDEX_ENDPOINT + "/repositories/" + repository + "/images"
req, err := http.NewRequest("GET", u, nil)
if err != nil {
return nil, err
}
return res.StatusCode == 307
if authConfig != nil && len(authConfig.Username) > 0 {
req.SetBasicAuth(authConfig.Username, authConfig.Password)
}
res, err := graph.getHttpClient().Do(req)
if err != nil {
return nil, err
}
defer res.Body.Close()
// Repository doesn't exist yet
if res.StatusCode == 404 {
return nil, nil
}
jsonData, err := ioutil.ReadAll(res.Body)
if err != nil {
return nil, err
}
imageList := []map[string]string{}
err = json.Unmarshal(jsonData, &imageList)
if err != nil {
Debugf("Body: %s (%s)\n", res.Body, u)
return nil, err
}
return imageList, nil
}
// Retrieve an image from the Registry.
// Returns the Image object as well as the layer as an Archive (io.Reader)
func (graph *Graph) getRemoteImage(stdout io.Writer, imgId string, authConfig *auth.AuthConfig) (*Image, Archive, error) {
client := &http.Client{}
func (graph *Graph) getRemoteImage(stdout io.Writer, imgId, registry string, token []string) (*Image, Archive, error) {
client := graph.getHttpClient()
fmt.Fprintf(stdout, "Pulling %s metadata\r\n", imgId)
// Get the Json
req, err := http.NewRequest("GET", REGISTRY_ENDPOINT+"/images/"+imgId+"/json", nil)
req, err := http.NewRequest("GET", registry+"/images/"+imgId+"/json", nil)
if err != nil {
return nil, nil, fmt.Errorf("Failed to download json: %s", err)
}
req.SetBasicAuth(authConfig.Username, authConfig.Password)
req.Header.Set("Authorization", "Token "+strings.Join(token, ", "))
res, err := client.Do(req)
if err != nil {
return nil, nil, fmt.Errorf("Failed to download json: %s", err)
@ -127,11 +158,11 @@ func (graph *Graph) getRemoteImage(stdout io.Writer, imgId string, authConfig *a
// Get the layer
fmt.Fprintf(stdout, "Pulling %s fs layer\r\n", imgId)
req, err = http.NewRequest("GET", REGISTRY_ENDPOINT+"/images/"+imgId+"/layer", nil)
req, err = http.NewRequest("GET", registry+"/images/"+imgId+"/layer", nil)
if err != nil {
return nil, nil, fmt.Errorf("Error while getting from the server: %s\n", err)
}
req.SetBasicAuth(authConfig.Username, authConfig.Password)
req.Header.Set("Authorization", "Token "+strings.Join(token, ", "))
res, err = client.Do(req)
if err != nil {
return nil, nil, err
@ -139,16 +170,87 @@ func (graph *Graph) getRemoteImage(stdout io.Writer, imgId string, authConfig *a
return img, ProgressReader(res.Body, int(res.ContentLength), stdout, "Downloading %v/%v (%v)"), nil
}
func (graph *Graph) PullImage(stdout io.Writer, imgId string, authConfig *auth.AuthConfig) error {
history, err := graph.getRemoteHistory(imgId, authConfig)
func (graph *Graph) getRemoteTags(stdout io.Writer, registries []string, repository string, token []string) (map[string]string, error) {
client := graph.getHttpClient()
if strings.Count(repository, "/") == 0 {
// This will be removed once the Registry supports auto-resolution on
// the "library" namespace
repository = "library/" + repository
}
for _, host := range registries {
endpoint := fmt.Sprintf("https://%s/v1/repositories/%s/tags", host, repository)
req, err := http.NewRequest("GET", endpoint, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Token "+strings.Join(token, ", "))
res, err := client.Do(req)
defer res.Body.Close()
Debugf("Got status code %d from %s", res.StatusCode, endpoint)
if err != nil || (res.StatusCode != 200 && res.StatusCode != 404) {
continue
} else if res.StatusCode == 404 {
return nil, fmt.Errorf("Repository not found")
}
result := new(map[string]string)
rawJson, err := ioutil.ReadAll(res.Body)
if err != nil {
return nil, err
}
if err = json.Unmarshal(rawJson, result); err != nil {
return nil, err
}
return *result, nil
}
return nil, fmt.Errorf("Could not reach any registry endpoint")
}
func (graph *Graph) getImageForTag(stdout io.Writer, tag, remote, registry string, token []string) (string, error) {
client := graph.getHttpClient()
registryEndpoint := "https://" + registry + "/v1"
repositoryTarget := registryEndpoint + "/repositories/" + remote + "/tags/" + tag
req, err := http.NewRequest("GET", repositoryTarget, nil)
if err != nil {
return "", err
}
req.Header.Set("Authorization", "Token "+strings.Join(token, ", "))
res, err := client.Do(req)
if err != nil {
return "", fmt.Errorf("Error while retrieving repository info: %v", err)
}
defer res.Body.Close()
if res.StatusCode == 403 {
return "", fmt.Errorf("You aren't authorized to access this resource")
} else if res.StatusCode != 200 {
return "", fmt.Errorf("HTTP code: %d", res.StatusCode)
}
var imgId string
rawJson, err := ioutil.ReadAll(res.Body)
if err != nil {
return "", err
}
if err = json.Unmarshal(rawJson, &imgId); err != nil {
return "", err
}
return imgId, nil
}
func (graph *Graph) PullImage(stdout io.Writer, imgId, registry string, token []string) error {
history, err := graph.getRemoteHistory(imgId, registry, token)
if err != nil {
return err
}
// FIXME: Try to stream the images?
// FIXME: Lunch the getRemoteImage() in goroutines
for _, j := range history {
if !graph.Exists(j.Id) {
img, layer, err := graph.getRemoteImage(stdout, j.Id, authConfig)
// FIXME: Launch the getRemoteImage() in goroutines
for _, id := range history {
if !graph.Exists(id) {
img, layer, err := graph.getRemoteImage(stdout, id, registry, token)
if err != nil {
// FIXME: Keep goging in case of error?
return err
@ -161,165 +263,195 @@ func (graph *Graph) PullImage(stdout io.Writer, imgId string, authConfig *auth.A
return nil
}
// FIXME: Handle the askedTag parameter
func (graph *Graph) PullRepository(stdout io.Writer, remote, askedTag string, repositories *TagStore, authConfig *auth.AuthConfig) error {
client := &http.Client{}
client := graph.getHttpClient()
fmt.Fprintf(stdout, "Pulling repository %s\r\n", remote)
var repositoryTarget string
// If we are asking for 'root' repository, lookup on the Library's registry
if strings.Index(remote, "/") == -1 {
repositoryTarget = REGISTRY_ENDPOINT + "/library/" + remote
} else {
repositoryTarget = REGISTRY_ENDPOINT + "/users/" + remote
}
fmt.Fprintf(stdout, "Pulling repository %s from %s\r\n", remote, INDEX_ENDPOINT)
repositoryTarget := INDEX_ENDPOINT + "/repositories/" + remote + "/images"
req, err := http.NewRequest("GET", repositoryTarget, nil)
if err != nil {
return err
}
req.SetBasicAuth(authConfig.Username, authConfig.Password)
if authConfig != nil && len(authConfig.Username) > 0 {
req.SetBasicAuth(authConfig.Username, authConfig.Password)
}
req.Header.Set("X-Docker-Token", "true")
res, err := client.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
if res.StatusCode == 401 {
return fmt.Errorf("Please login first (HTTP code %d)", res.StatusCode)
}
// TODO: Right now we're ignoring checksums in the response body.
// In the future, we need to use them to check image validity.
if res.StatusCode != 200 {
return fmt.Errorf("HTTP code: %d", res.StatusCode)
}
rawJson, err := ioutil.ReadAll(res.Body)
var token, endpoints []string
if res.Header.Get("X-Docker-Token") != "" {
token = res.Header["X-Docker-Token"]
}
if res.Header.Get("X-Docker-Endpoints") != "" {
endpoints = res.Header["X-Docker-Endpoints"]
} else {
return fmt.Errorf("Index response didn't contain any endpoints")
}
var tagsList map[string]string
if askedTag == "" {
tagsList, err = graph.getRemoteTags(stdout, endpoints, remote, token)
if err != nil {
return err
}
} else {
tagsList = map[string]string{askedTag: ""}
}
for askedTag, imgId := range tagsList {
fmt.Fprintf(stdout, "Resolving tag \"%s:%s\" from %s\n", remote, askedTag, endpoints)
success := false
for _, registry := range endpoints {
if imgId == "" {
imgId, err = graph.getImageForTag(stdout, askedTag, remote, registry, token)
if err != nil {
fmt.Fprintf(stdout, "Error while retrieving image for tag: %v (%v) ; "+
"checking next endpoint", askedTag, err)
continue
}
}
if err := graph.PullImage(stdout, imgId, "https://"+registry+"/v1", token); err != nil {
return err
}
if err = repositories.Set(remote, askedTag, imgId, true); err != nil {
return err
}
success = true
}
if !success {
return fmt.Errorf("Could not find repository on any of the indexed registries.")
}
}
if err = repositories.Save(); err != nil {
return err
}
return nil
}
func pushImageRec(graph *Graph, stdout io.Writer, img *Image, registry string, token []string) error {
if parent, err := img.GetParent(); err != nil {
return err
} else if parent != nil {
if err := pushImageRec(graph, stdout, parent, registry, token); err != nil {
return err
}
}
client := graph.getHttpClient()
jsonRaw, err := ioutil.ReadFile(path.Join(graph.Root, img.Id, "json"))
if err != nil {
return fmt.Errorf("Error while retreiving the path for {%s}: %s", img.Id, err)
}
fmt.Fprintf(stdout, "Pushing %s metadata\r\n", img.Id)
// FIXME: try json with UTF8
jsonData := strings.NewReader(string(jsonRaw))
req, err := http.NewRequest("PUT", registry+"/images/"+img.Id+"/json", jsonData)
if err != nil {
return err
}
t := map[string]string{}
if err = json.Unmarshal(rawJson, &t); err != nil {
req.Header.Add("Content-type", "application/json")
req.Header.Set("Authorization", "Token "+strings.Join(token, ","))
checksum, err := img.Checksum()
if err != nil {
return fmt.Errorf("Error while retrieving checksum for %s: %v", img.Id, err)
}
req.Header.Set("X-Docker-Checksum", checksum)
res, err := doWithCookies(client, req)
if err != nil {
return fmt.Errorf("Failed to upload metadata: %s", err)
}
defer res.Body.Close()
if len(res.Cookies()) > 0 {
client.Jar.SetCookies(req.URL, res.Cookies())
}
if res.StatusCode != 200 {
errBody, err := ioutil.ReadAll(res.Body)
if err != nil {
return fmt.Errorf("HTTP code %d while uploading metadata and error when"+
" trying to parse response body: %v", res.StatusCode, err)
}
var jsonBody map[string]string
if err := json.Unmarshal(errBody, &jsonBody); err != nil {
errBody = []byte(err.Error())
} else if jsonBody["error"] == "Image already exists" {
fmt.Fprintf(stdout, "Image %v already uploaded ; skipping\n", img.Id)
return nil
}
return fmt.Errorf("HTTP code %d while uploading metadata: %s", res.StatusCode, errBody)
}
fmt.Fprintf(stdout, "Pushing %s fs layer\r\n", img.Id)
layerData, err := graph.TempLayerArchive(img.Id, Xz, stdout)
if err != nil {
return fmt.Errorf("Failed to generate layer archive: %s", err)
}
req3, err := http.NewRequest("PUT", registry+"/images/"+img.Id+"/layer",
ProgressReader(layerData, -1, stdout, ""))
if err != nil {
return err
}
for tag, rev := range t {
fmt.Fprintf(stdout, "Pulling tag %s:%s\r\n", remote, tag)
if err = graph.PullImage(stdout, rev, authConfig); err != nil {
return err
}
if err = repositories.Set(remote, tag, rev, true); err != nil {
return err
}
req3.ContentLength = -1
req3.TransferEncoding = []string{"chunked"}
req3.Header.Set("Authorization", "Token "+strings.Join(token, ","))
res3, err := doWithCookies(client, req3)
if err != nil {
return fmt.Errorf("Failed to upload layer: %s", err)
}
if err = repositories.Save(); err != nil {
return err
res3.Body.Close()
if res3.StatusCode != 200 {
return fmt.Errorf("Received HTTP code %d while uploading layer", res3.StatusCode)
}
return nil
}
// Push a local image to the registry with its history if needed
func (graph *Graph) PushImage(stdout io.Writer, imgOrig *Image, authConfig *auth.AuthConfig) error {
client := &http.Client{}
// FIXME: Factorize the code
// FIXME: Do the puts in goroutines
if err := imgOrig.WalkHistory(func(img *Image) error {
jsonRaw, err := ioutil.ReadFile(path.Join(graph.Root, img.Id, "json"))
if err != nil {
return fmt.Errorf("Error while retreiving the path for {%s}: %s", img.Id, err)
}
fmt.Fprintf(stdout, "Pushing %s metadata\r\n", img.Id)
// FIXME: try json with UTF8
jsonData := strings.NewReader(string(jsonRaw))
req, err := http.NewRequest("PUT", REGISTRY_ENDPOINT+"/images/"+img.Id+"/json", jsonData)
if err != nil {
return err
}
req.Header.Add("Content-type", "application/json")
req.SetBasicAuth(authConfig.Username, authConfig.Password)
res, err := client.Do(req)
if err != nil {
return fmt.Errorf("Failed to upload metadata: %s", err)
}
defer res.Body.Close()
if res.StatusCode != 200 {
switch res.StatusCode {
case 204:
// Case where the image is already on the Registry
// FIXME: Do not be silent?
return nil
default:
errBody, err := ioutil.ReadAll(res.Body)
if err != nil {
errBody = []byte(err.Error())
}
return fmt.Errorf("HTTP code %d while uploading metadata: %s", res.StatusCode, errBody)
}
}
fmt.Fprintf(stdout, "Pushing %s fs layer\r\n", img.Id)
req2, err := http.NewRequest("PUT", REGISTRY_ENDPOINT+"/images/"+img.Id+"/layer", nil)
req2.SetBasicAuth(authConfig.Username, authConfig.Password)
res2, err := client.Do(req2)
if err != nil {
return fmt.Errorf("Registry returned error: %s", err)
}
res2.Body.Close()
if res2.StatusCode != 307 {
return fmt.Errorf("Registry returned unexpected HTTP status code %d, expected 307", res2.StatusCode)
}
url, err := res2.Location()
if err != nil || url == nil {
return fmt.Errorf("Failed to retrieve layer upload location: %s", err)
}
// FIXME: stream the archive directly to the registry instead of buffering it on disk. This requires either:
// a) Implementing S3's proprietary streaming logic, or
// b) Stream directly to the registry instead of S3.
// I prefer option b. because it doesn't lock us into a proprietary cloud service.
tmpLayer, err := graph.TempLayerArchive(img.Id, Xz, stdout)
if err != nil {
return err
}
defer os.Remove(tmpLayer.Name())
req3, err := http.NewRequest("PUT", url.String(), ProgressReader(tmpLayer, int(tmpLayer.Size), stdout, "Uploading %v/%v (%v)"))
if err != nil {
return err
}
req3.ContentLength = int64(tmpLayer.Size)
req3.TransferEncoding = []string{"none"}
res3, err := client.Do(req3)
if err != nil {
return fmt.Errorf("Failed to upload layer: %s", err)
}
res3.Body.Close()
if res3.StatusCode != 200 {
return fmt.Errorf("Received HTTP code %d while uploading layer", res3.StatusCode)
}
return nil
}); err != nil {
return err
}
return nil
func (graph *Graph) PushImage(stdout io.Writer, imgOrig *Image, registry string, token []string) error {
registry = "https://" + registry + "/v1"
return pushImageRec(graph, stdout, imgOrig, registry, token)
}
// push a tag on the registry.
// Remote has the format '<user>/<repo>
func (graph *Graph) pushTag(remote, revision, tag string, authConfig *auth.AuthConfig) error {
// Keep this for backward compatibility
if tag == "" {
tag = "lastest"
}
func (graph *Graph) pushTag(remote, revision, tag, registry string, token []string) error {
// "jsonify" the string
revision = "\"" + revision + "\""
registry = "https://" + registry + "/v1"
Debugf("Pushing tags for rev [%s] on {%s}\n", revision, REGISTRY_ENDPOINT+"/users/"+remote+"/"+tag)
Debugf("Pushing tags for rev [%s] on {%s}\n", revision, registry+"/users/"+remote+"/"+tag)
client := &http.Client{}
req, err := http.NewRequest("PUT", REGISTRY_ENDPOINT+"/users/"+remote+"/"+tag, strings.NewReader(revision))
client := graph.getHttpClient()
req, err := http.NewRequest("PUT", registry+"/repositories/"+remote+"/tags/"+tag, strings.NewReader(revision))
if err != nil {
return err
}
req.Header.Add("Content-type", "application/json")
req.SetBasicAuth(authConfig.Username, authConfig.Password)
res, err := client.Do(req)
req.Header.Set("Authorization", "Token "+strings.Join(token, ","))
req.ContentLength = int64(len(revision))
res, err := doWithCookies(client, req)
if err != nil {
return err
}
@ -327,62 +459,25 @@ func (graph *Graph) pushTag(remote, revision, tag string, authConfig *auth.AuthC
if res.StatusCode != 200 && res.StatusCode != 201 {
return fmt.Errorf("Internal server error: %d trying to push tag %s on %s", res.StatusCode, tag, remote)
}
Debugf("Result of push tag: %d\n", res.StatusCode)
switch res.StatusCode {
default:
return fmt.Errorf("Error %d\n", res.StatusCode)
case 200:
case 201:
}
return nil
}
func (graph *Graph) LookupRemoteRepository(remote string, authConfig *auth.AuthConfig) bool {
rt := &http.Transport{Proxy: http.ProxyFromEnvironment}
var repositoryTarget string
// If we are asking for 'root' repository, lookup on the Library's registry
if strings.Index(remote, "/") == -1 {
repositoryTarget = REGISTRY_ENDPOINT + "/library/" + remote + "/lookup"
} else {
repositoryTarget = REGISTRY_ENDPOINT + "/users/" + remote + "/lookup"
}
Debugf("Checking for permissions on: %s", repositoryTarget)
req, err := http.NewRequest("PUT", repositoryTarget, strings.NewReader("\"\""))
if err != nil {
Debugf("%s\n", err)
return false
}
req.SetBasicAuth(authConfig.Username, authConfig.Password)
req.Header.Add("Content-type", "application/json")
res, err := rt.RoundTrip(req)
if err != nil || res.StatusCode != 404 {
errBody, err := ioutil.ReadAll(res.Body)
if err != nil {
errBody = []byte(err.Error())
}
Debugf("Lookup status code: %d (body: %s)", res.StatusCode, errBody)
return false
}
return true
}
// FIXME: this should really be PushTag
func (graph *Graph) pushPrimitive(stdout io.Writer, remote, tag, imgId string, authConfig *auth.AuthConfig) error {
func (graph *Graph) pushPrimitive(stdout io.Writer, remote, tag, imgId, registry string, token []string) error {
// Check if the local impage exists
img, err := graph.Get(imgId)
if err != nil {
fmt.Fprintf(stdout, "Skipping tag %s:%s: %s does not exist\r\n", remote, tag, imgId)
return nil
}
fmt.Fprintf(stdout, "Pushing tag %s:%s\r\n", remote, tag)
fmt.Fprintf(stdout, "Pushing image %s:%s\r\n", remote, tag)
// Push the image
if err = graph.PushImage(stdout, img, authConfig); err != nil {
if err = graph.PushImage(stdout, img, registry, token); err != nil {
return err
}
fmt.Fprintf(stdout, "Registering tag %s:%s\r\n", remote, tag)
// And then the tag
if err = graph.pushTag(remote, imgId, tag, authConfig); err != nil {
if err = graph.pushTag(remote, imgId, tag, registry, token); err != nil {
return err
}
return nil
@ -391,18 +486,155 @@ func (graph *Graph) pushPrimitive(stdout io.Writer, remote, tag, imgId string, a
// Push a repository to the registry.
// Remote has the format '<user>/<repo>
func (graph *Graph) PushRepository(stdout io.Writer, remote string, localRepo Repository, authConfig *auth.AuthConfig) error {
// Check if the remote repository exists/if we have the permission
if !graph.LookupRemoteRepository(remote, authConfig) {
return fmt.Errorf("Permission denied on repository %s\n", remote)
client := graph.getHttpClient()
checksums, err := graph.Checksums(stdout, localRepo)
if err != nil {
return err
}
fmt.Fprintf(stdout, "Pushing repository %s (%d tags)\r\n", remote, len(localRepo))
// For each image within the repo, push them
for tag, imgId := range localRepo {
if err := graph.pushPrimitive(stdout, remote, tag, imgId, authConfig); err != nil {
// FIXME: Continue on error?
return err
imgList := make([]map[string]string, len(checksums))
checksums2 := make([]map[string]string, len(checksums))
uploadedImages, err := graph.getImagesInRepository(remote, authConfig)
if err != nil {
return fmt.Errorf("Error occured while fetching the list: %s", err)
}
// Filter list to only send images/checksums not already uploaded
i := 0
for _, obj := range checksums {
found := false
for _, uploadedImg := range uploadedImages {
if obj["id"] == uploadedImg["id"] && uploadedImg["checksum"] != "" {
found = true
break
}
}
if !found {
imgList[i] = map[string]string{"id": obj["id"]}
checksums2[i] = obj
i += 1
}
}
checksums = checksums2[:i]
imgList = imgList[:i]
imgListJson, err := json.Marshal(imgList)
if err != nil {
return err
}
req, err := http.NewRequest("PUT", INDEX_ENDPOINT+"/repositories/"+remote+"/", bytes.NewReader(imgListJson))
if err != nil {
return err
}
req.SetBasicAuth(authConfig.Username, authConfig.Password)
req.ContentLength = int64(len(imgListJson))
req.Header.Set("X-Docker-Token", "true")
res, err := client.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
for res.StatusCode >= 300 && res.StatusCode < 400 {
Debugf("Redirected to %s\n", res.Header.Get("Location"))
req, err = http.NewRequest("PUT", res.Header.Get("Location"), bytes.NewReader(imgListJson))
if err != nil {
return err
}
req.SetBasicAuth(authConfig.Username, authConfig.Password)
req.ContentLength = int64(len(imgListJson))
req.Header.Set("X-Docker-Token", "true")
res, err = client.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
}
if res.StatusCode != 200 && res.StatusCode != 201 {
return fmt.Errorf("Error: Status %d trying to push repository %s", res.StatusCode, remote)
}
var token, endpoints []string
if res.Header.Get("X-Docker-Token") != "" {
token = res.Header["X-Docker-Token"]
Debugf("Auth token: %v", token)
} else {
return fmt.Errorf("Index response didn't contain an access token")
}
if res.Header.Get("X-Docker-Endpoints") != "" {
endpoints = res.Header["X-Docker-Endpoints"]
} else {
return fmt.Errorf("Index response didn't contain any endpoints")
}
for _, registry := range endpoints {
fmt.Fprintf(stdout, "Pushing repository %s to %s (%d tags)\r\n", remote, registry,
len(localRepo))
// For each image within the repo, push them
for tag, imgId := range localRepo {
if err := graph.pushPrimitive(stdout, remote, tag, imgId, registry, token); err != nil {
// FIXME: Continue on error?
return err
}
}
}
checksumsJson, err := json.Marshal(checksums)
if err != nil {
return err
}
req2, err := http.NewRequest("PUT", INDEX_ENDPOINT+"/repositories/"+remote+"/images", bytes.NewReader(checksumsJson))
if err != nil {
return err
}
req2.SetBasicAuth(authConfig.Username, authConfig.Password)
req2.Header["X-Docker-Endpoints"] = endpoints
req2.ContentLength = int64(len(checksumsJson))
res2, err := client.Do(req2)
if err != nil {
return err
}
res2.Body.Close()
if res2.StatusCode != 204 {
return fmt.Errorf("Error: Status %d trying to push checksums %s", res.StatusCode, remote)
}
return nil
}
func (graph *Graph) Checksums(output io.Writer, repo Repository) ([]map[string]string, error) {
var result []map[string]string
checksums := map[string]string{}
for _, id := range repo {
img, err := graph.Get(id)
if err != nil {
return nil, err
}
err = img.WalkHistory(func(image *Image) error {
fmt.Fprintf(output, "Computing checksum for image %s\n", image.Id)
if _, exists := checksums[image.Id]; !exists {
checksums[image.Id], err = image.Checksum()
if err != nil {
return err
}
}
return nil
})
if err != nil {
return nil, err
}
}
i := 0
result = make([]map[string]string, len(checksums))
for id, sum := range checksums {
result[i] = map[string]string{
"id": id,
"checksum": sum,
}
i++
}
return result, nil
}

136
runtime.go
View file

@ -12,7 +12,6 @@ import (
"path"
"sort"
"strings"
"time"
)
type Capabilities struct {
@ -79,114 +78,6 @@ func (runtime *Runtime) containerRoot(id string) string {
return path.Join(runtime.repository, id)
}
func (runtime *Runtime) mergeConfig(userConf, imageConf *Config) {
if userConf.Hostname == "" {
userConf.Hostname = imageConf.Hostname
}
if userConf.User == "" {
userConf.User = imageConf.User
}
if userConf.Memory == 0 {
userConf.Memory = imageConf.Memory
}
if userConf.MemorySwap == 0 {
userConf.MemorySwap = imageConf.MemorySwap
}
if userConf.PortSpecs == nil || len(userConf.PortSpecs) == 0 {
userConf.PortSpecs = imageConf.PortSpecs
}
if !userConf.Tty {
userConf.Tty = userConf.Tty
}
if !userConf.OpenStdin {
userConf.OpenStdin = imageConf.OpenStdin
}
if !userConf.StdinOnce {
userConf.StdinOnce = imageConf.StdinOnce
}
if userConf.Env == nil || len(userConf.Env) == 0 {
userConf.Env = imageConf.Env
}
if userConf.Cmd == nil || len(userConf.Cmd) == 0 {
userConf.Cmd = imageConf.Cmd
}
if userConf.Dns == nil || len(userConf.Dns) == 0 {
userConf.Dns = imageConf.Dns
}
}
func (runtime *Runtime) Create(config *Config) (*Container, error) {
// Lookup image
img, err := runtime.repositories.LookupImage(config.Image)
if err != nil {
return nil, err
}
if img.Config != nil {
runtime.mergeConfig(config, img.Config)
}
if config.Cmd == nil || len(config.Cmd) == 0 {
return nil, fmt.Errorf("No command specified")
}
// Generate id
id := GenerateId()
// Generate default hostname
// FIXME: the lxc template no longer needs to set a default hostname
if config.Hostname == "" {
config.Hostname = id[:12]
}
container := &Container{
// FIXME: we should generate the ID here instead of receiving it as an argument
Id: id,
Created: time.Now(),
Path: config.Cmd[0],
Args: config.Cmd[1:], //FIXME: de-duplicate from config
Config: config,
Image: img.Id, // Always use the resolved image id
NetworkSettings: &NetworkSettings{},
// FIXME: do we need to store this in the container?
SysInitPath: sysInitPath,
}
container.root = runtime.containerRoot(container.Id)
// Step 1: create the container directory.
// This doubles as a barrier to avoid race conditions.
if err := os.Mkdir(container.root, 0700); err != nil {
return nil, err
}
// If custom dns exists, then create a resolv.conf for the container
if len(config.Dns) > 0 {
container.ResolvConfPath = path.Join(container.root, "resolv.conf")
f, err := os.Create(container.ResolvConfPath)
if err != nil {
return nil, err
}
defer f.Close()
for _, dns := range config.Dns {
if _, err := f.Write([]byte("nameserver " + dns + "\n")); err != nil {
return nil, err
}
}
} else {
container.ResolvConfPath = "/etc/resolv.conf"
}
// Step 2: save the container json
if err := container.ToDisk(); err != nil {
return nil, err
}
// Step 3: register the container
if err := runtime.Register(container); err != nil {
return nil, err
}
return container, nil
}
func (runtime *Runtime) Load(id string) (*Container, error) {
container := &Container{root: runtime.containerRoot(id)}
if err := container.FromDisk(); err != nil {
@ -311,33 +202,6 @@ func (runtime *Runtime) Destroy(container *Container) error {
return nil
}
// Commit creates a new filesystem image from the current state of a container.
// The image can optionally be tagged into a repository
func (runtime *Runtime) Commit(id, repository, tag, comment, author string, config *Config) (*Image, error) {
container := runtime.Get(id)
if container == nil {
return nil, fmt.Errorf("No such container: %s", id)
}
// FIXME: freeze the container before copying it to avoid data corruption?
// FIXME: this shouldn't be in commands.
rwTar, err := container.ExportRw()
if err != nil {
return nil, err
}
// Create a new image from the container's base layers + a new layer from container changes
img, err := runtime.graph.Create(rwTar, container, comment, author, config)
if err != nil {
return nil, err
}
// Register the image if needed
if repository != "" {
if err := runtime.repositories.Set(repository, tag, img.Id, true); err != nil {
return img, err
}
}
return img, nil
}
func (runtime *Runtime) restore() error {
dir, err := ioutil.ReadDir(runtime.repository)
if err != nil {

21
runtime_test.go
View file

@ -118,7 +118,7 @@ func TestRuntimeCreate(t *testing.T) {
if len(runtime.List()) != 0 {
t.Errorf("Expected 0 containers, %v found", len(runtime.List()))
}
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"ls", "-al"},
},
@ -165,7 +165,7 @@ func TestDestroy(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"ls", "-al"},
},
@ -212,7 +212,10 @@ func TestGet(t *testing.T) {
t.Fatal(err)
}
defer nuke(runtime)
container1, err := runtime.Create(&Config{
builder := NewBuilder(runtime)
container1, err := builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"ls", "-al"},
},
@ -222,7 +225,7 @@ func TestGet(t *testing.T) {
}
defer runtime.Destroy(container1)
container2, err := runtime.Create(&Config{
container2, err := builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"ls", "-al"},
},
@ -232,7 +235,7 @@ func TestGet(t *testing.T) {
}
defer runtime.Destroy(container2)
container3, err := runtime.Create(&Config{
container3, err := builder.Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"ls", "-al"},
},
@ -262,7 +265,7 @@ func TestAllocatePortLocalhost(t *testing.T) {
if err != nil {
t.Fatal(err)
}
container, err := runtime.Create(&Config{
container, err := NewBuilder(runtime).Create(&Config{
Image: GetTestImage(runtime).Id,
Cmd: []string{"sh", "-c", "echo well hello there | nc -l -p 5555"},
PortSpecs: []string{"5555"},
@ -325,8 +328,10 @@ func TestRestore(t *testing.T) {
t.Fatal(err)
}
builder := NewBuilder(runtime1)
// Create a container with one instance of docker
container1, err := runtime1.Create(&Config{
container1, err := builder.Create(&Config{
Image: GetTestImage(runtime1).Id,
Cmd: []string{"ls", "-al"},
},
@ -337,7 +342,7 @@ func TestRestore(t *testing.T) {
defer runtime1.Destroy(container1)
// Create a second container meant to be killed
container2, err := runtime1.Create(&Config{
container2, err := builder.Create(&Config{
Image: GetTestImage(runtime1).Id,
Cmd: []string{"/bin/cat"},
OpenStdin: true,

105
server.go
View file

@ -43,6 +43,80 @@ func (srv *Server) ContainerExport(name string, file *os.File) error {
return fmt.Errorf("No such container: %s", name)
}
func (srv *Server) ImageInsert(name, url, path string, stdout *os.File) error {
img, err := srv.runtime.repositories.LookupImage(name)
if err != nil {
return err
}
file, err := Download(url, stdout)
if err != nil {
return err
}
defer file.Body.Close()
config, _, err := ParseRun([]string{img.Id, "echo", "insert", url, path}, srv.runtime.capabilities)
if err != nil {
return err
}
b := NewBuilder(srv.runtime)
c, err := b.Create(config)
if err != nil {
return err
}
if err := c.Inject(ProgressReader(file.Body, int(file.ContentLength), stdout, "Downloading %v/%v (%v)"), path); err != nil {
return err
}
// FIXME: Handle custom repo, tag comment, author
img, err = b.Commit(c, "", "", img.Comment, img.Author, nil)
if err != nil {
return err
}
fmt.Fprintf(stdout, "%s\n", img.Id)
return nil
}
func (srv *Server) ImagesViz(file *os.File) error {
images, _ := srv.runtime.graph.All()
if images == nil {
return nil
}
fmt.Fprintf(file, "digraph docker {\n")
var parentImage *Image
var err error
for _, image := range images {
parentImage, err = image.GetParent()
if err != nil {
return fmt.Errorf("Error while getting parent image: %v", err)
}
if parentImage != nil {
fmt.Fprintf(file, " \"%s\" -> \"%s\"\n", parentImage.ShortId(), image.ShortId())
} else {
fmt.Fprintf(file, " base -> \"%s\" [style=invis]\n", image.ShortId())
}
}
reporefs := make(map[string][]string)
for name, repository := range srv.runtime.repositories.Repositories {
for tag, id := range repository {
reporefs[TruncateId(id)] = append(reporefs[TruncateId(id)], fmt.Sprintf("%s:%s", name, tag))
}
}
for id, repos := range reporefs {
fmt.Fprintf(file, " \"%s\" [label=\"%s\\n%s\",shape=box,fillcolor=\"paleturquoise\",style=\"filled,rounded\"];\n", id, id, strings.Join(repos, "\\n"))
}
fmt.Fprintf(file, " base [style=invisible]\n")
fmt.Fprintf(file, "}\n")
return nil
}
func (srv *Server) Images(all, filter, quiet string) ([]ApiImages, error) {
var allImages map[string]*Image
var err error
@ -188,7 +262,11 @@ func (srv *Server) Containers(all, notrunc, quiet string, n int) []ApiContainers
}
func (srv *Server) ContainerCommit(name, repo, tag, author, comment string, config *Config) (string, error) {
img, err := srv.runtime.Commit(name, repo, tag, comment, author, config)
container := srv.runtime.Get(name)
if container == nil {
return "", fmt.Errorf("No such container: %s", name)
}
img, err := NewBuilder(srv.runtime).Commit(container, repo, tag, comment, author, config)
if err != nil {
return "", err
}
@ -202,19 +280,20 @@ func (srv *Server) ContainerTag(name, repo, tag string, force bool) error {
return nil
}
func (srv *Server) ImagePull(name string, file *os.File) error {
if srv.runtime.graph.LookupRemoteImage(name, srv.runtime.authConfig) {
if err := srv.runtime.graph.PullImage(file, name, srv.runtime.authConfig); err != nil {
func (srv *Server) ImagePull(name, tag, registry string, file *os.File) error {
if registry != "" {
if err := srv.runtime.graph.PullImage(file, name, registry, nil); err != nil {
return err
}
return nil
}
if err := srv.runtime.graph.PullRepository(file, name, "", srv.runtime.repositories, srv.runtime.authConfig); err != nil {
if err := srv.runtime.graph.PullRepository(file, name, tag, srv.runtime.repositories, srv.runtime.authConfig); err != nil {
return err
}
return nil
}
func (srv *Server) ImagePush(name string, file *os.File) error {
func (srv *Server) ImagePush(name, registry string, file *os.File) error {
img, err := srv.runtime.graph.Get(name)
if err != nil {
Debugf("The push refers to a repository [%s] (len: %d)\n", name, len(srv.runtime.repositories.Repositories[name]))
@ -228,7 +307,7 @@ func (srv *Server) ImagePush(name string, file *os.File) error {
return err
}
err = srv.runtime.graph.PushImage(file, img, srv.runtime.authConfig)
err = srv.runtime.graph.PushImage(file, img, registry, nil)
if err != nil {
return err
}
@ -294,7 +373,8 @@ func (srv *Server) ContainerCreate(config Config) (string, bool, bool, error) {
log.Println("WARNING: Your kernel does not support swap limit capabilities. Limitation discarded.")
config.MemorySwap = -1
}
container, err := srv.runtime.Create(&config)
b := NewBuilder(srv.runtime)
container, err := b.Create(&config)
if err != nil {
if srv.runtime.graph.IsNotExist(err) {
return "", false, false, fmt.Errorf("No such image: %s", config.Image)
@ -304,6 +384,15 @@ func (srv *Server) ContainerCreate(config Config) (string, bool, bool, error) {
return container.ShortId(), memoryW, swapW, nil
}
func (srv *Server) ImageCreateFormFile(file *os.File) error {
img, err := NewBuilder(srv.runtime).Build(file, file)
if err != nil {
return err
}
fmt.Fprintf(file, "%s\n", img.ShortId())
return nil
}
func (srv *Server) ContainerRestart(name string, t int) error {
if container := srv.runtime.Get(name); container != nil {
if err := container.Restart(t); err != nil {

86
utils.go
View file

@ -2,6 +2,8 @@ package docker
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"github.com/dotcloud/docker/term"
@ -151,6 +153,13 @@ func SelfPath() string {
return path
}
type nopWriter struct {
}
func (w *nopWriter) Write(buf []byte) (int, error) {
return len(buf), nil
}
type nopWriteCloser struct {
io.Writer
}
@ -412,6 +421,14 @@ func RestoreTerminal(state *term.State) {
term.Restore(int(os.Stdin.Fd()), state)
}
func HashData(src io.Reader) (string, error) {
h := sha256.New()
if _, err := io.Copy(h, src); err != nil {
return "", err
}
return "sha256:" + hex.EncodeToString(h.Sum(nil)), nil
}
type KernelVersionInfo struct {
Kernel int
Major int
@ -474,36 +491,49 @@ func FindCgroupMountpoint(cgroupType string) (string, error) {
return "", fmt.Errorf("cgroup mountpoint not found for %s", cgroupType)
}
/*
func ReadStringOnTerminal(prompt string) (string, error) {
fmt.Print(prompt)
in := bufio.NewReader(os.Stdin);
return in.ReadString('\n');
}
// Compare two Config struct. Do not compare the "Image" nor "Hostname" fields
// If OpenStdin is set, then it differs
func CompareConfig(a, b *Config) bool {
if a == nil || b == nil ||
a.OpenStdin || b.OpenStdin {
return false
}
if a.AttachStdout != b.AttachStdout ||
a.AttachStderr != b.AttachStderr ||
a.User != b.User ||
a.Memory != b.Memory ||
a.MemorySwap != b.MemorySwap ||
a.OpenStdin != b.OpenStdin ||
a.Tty != b.Tty {
return false
}
if len(a.Cmd) != len(b.Cmd) ||
len(a.Dns) != len(b.Dns) ||
len(a.Env) != len(b.Env) ||
len(a.PortSpecs) != len(b.PortSpecs) {
return false
}
func ReadPasswdOnTerminal(prompt string) (string, error) {
fmt.Print(prompt);
const stty_arg0 = "/bin/stty";
stty_argv_e_off := []string{"stty","-echo"};
stty_argv_e_on := []string{"stty","echo"};
const exec_cwdir = "";
fd := []*os.File{os.Stdin,os.Stdout,os.Stderr};
pid, err := os.StartProcess(stty_arg0,stty_argv_e_off,nil,exec_cwdir,fd);
if err != nil {
return "", err
for i := 0; i < len(a.Cmd); i++ {
if a.Cmd[i] != b.Cmd[i] {
return false
}
}
rd := bufio.NewReader(os.Stdin);
os.Wait(pid,0);
line, err := rd.ReadString('\n');
if err != nil {
return "", err
for i := 0; i < len(a.Dns); i++ {
if a.Dns[i] != b.Dns[i] {
return false
}
}
passwd := strings.TrimSpace(line)
pid, e := os.StartProcess(stty_arg0,stty_argv_e_on,nil,exec_cwdir,fd);
if e =! nil {
return "", err
for i := 0; i < len(a.Env); i++ {
if a.Env[i] != b.Env[i] {
return false
}
}
os.Wait(pid,0)
return passwd, err
for i := 0; i < len(a.PortSpecs); i++ {
if a.PortSpecs[i] != b.PortSpecs[i] {
return false
}
}
return true
}
*/