Commit graph

255 commits

Author SHA1 Message Date
John Howard
71651e0b80 Fixes LCOW after containerd 1.0 introduced regressions
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-10-27 09:55:43 -07:00
Kenfe-Mickael Laventure
ddae20c032
Update libcontainerd to use containerd 1.0
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-10-20 07:11:37 -07:00
Sebastiaan van Stijn
7cbbbb9509 Merge pull request #34846 from Microsoft/jjh/debuggcs
LCOW: Add UVM debugability by grabbing logs before tear-down
2017-09-19 22:33:46 +02:00
John Howard
2798576b37 Revendor Microsoft/opengcs @ v0.3.4
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-09-19 09:56:23 -07:00
Simon Ferquel
e89b6e8c2d Volume refactoring for LCOW
Signed-off-by: Simon Ferquel <simon.ferquel@docker.com>
2017-09-14 12:33:31 -07:00
John Howard
5a0e2beac3 LCOW: Add GCS debugging
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-09-13 14:54:34 -07:00
Daniel Nephin
f7f101d57e Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-12 12:09:59 -04:00
John Howard
17a24034bb LCOW: VHDX boot to read-only
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-09-06 10:55:19 -07:00
Yong Tang
cb952bf006 Merge pull request #34625 from dnephin/more-linters
Add interfacer and unconvert linters
2017-09-01 08:46:08 -07:00
John Howard
285bc99731 Merge pull request #34356 from mlaventure/update-containerd
Update containerd to 06b9cb35161009dcb7123345749fef02f7cea8e0
2017-08-24 14:25:44 -07:00
Daniel Nephin
2f5f0af3fd Add unconvert linter
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-24 15:08:31 -04:00
Yong Tang
e5cdaf1bd9 Merge pull request #34451 from Microsoft/jjh/bootvhdx
LCOW: Additional flags for VHD boot
2017-08-24 08:38:19 -07:00
Darren Stahl
7c29103ad9
Update Windows and LCOW to use v1.0.0 runtime-spec
Signed-off-by: Darren Stahl <darst@microsoft.com>
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-21 15:19:31 -07:00
Daniel Nephin
372670b507 Add goimports to linters.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-21 18:15:08 -04:00
Kenfe-Mickael Laventure
45d85c9913
Update containerd to 06b9cb35161009dcb7123345749fef02f7cea8e0
This also update:
 - runc to 3f2f8b84a77f73d38244dd690525642a72156c64
 - runtime-specs to v1.0.0

Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-08-21 12:04:07 -07:00
John Howard
ee0587a45d jhoward/opengcs --> Microsoft/opengcs
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-08 14:43:43 -07:00
John Howard
71328a7af5 LCOW: Additional flags for VHD boot
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-08 14:40:33 -07:00
John Starks
54354db850 Windows: Add named pipe mount support
Current insider builds of Windows have support for mounting individual
named pipe servers from the host to the guest. This allows, for example,
exposing the docker engine's named pipe to a container.

This change allows the user to request such a mount via the normal bind
mount syntax in the CLI:

  docker run -v \\.\pipe\docker_engine:\\.\pipe\docker_engine <args>

Signed-off-by: John Starks <jostarks@microsoft.com>
2017-08-07 11:34:36 -07:00
John Howard
ffdef6255e LCOW: Remove hard-coding
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-08-03 16:10:25 -07:00
Derek McGowan
1009e6a40b
Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-31 13:16:46 -07:00
Christopher Jones
069fdc8a08
[project] change syscall to /x/sys/unix|windows
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>

[s390x] switch utsname from unsigned to signed

per 33267e036f
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2017-07-11 08:00:32 -04:00
Vincent Demeester
359ed99352 Merge pull request #33835 from Microsoft/jjh/changedir
LCOW: Change directory from lcow to "Linux Containers"
2017-06-27 09:33:38 +02:00
John Howard
2c43cbe0d5 LCOW: Change to c:\Program Files\Linux Containers
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-26 21:26:18 -07:00
John Starks
19645521a9 LCOW: pass command arguments without extra quoting
Signed-off-by: John Starks <jostarks@microsoft.com>
2017-06-24 10:23:17 -07:00
John Howard
25a65a185b Merge pull request #33785 from Microsoft/jjh/hcsschemaupdate
LCOW: Updates necessary due to platform schema change
2017-06-23 13:08:13 -07:00
John Howard
330913cadf LCOW: Update for platforms latest schema
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-22 14:07:23 -07:00
Sebastiaan van Stijn
f88626b270 Merge pull request #33764 from keloyang/fix-queue-mem-leak
Fix mem leak in libcontainerd/queue/append
2017-06-22 10:57:07 -07:00
Vincent Demeester
c85f92de15 Merge pull request #33774 from Microsoft/jjh/lcow-networking
LCOW: owner and network endpoints
2017-06-22 16:40:50 +02:00
yangshukui
5425a5ab84 fix mem leak in libcontainerd/queue/append
Signed-off-by: yangshukui <yangshukui@huawei.com>
2017-06-22 16:47:47 +08:00
Sebastiaan van Stijn
4fc2710dc7 Merge pull request #33772 from cpuguy83/optimizations
Don't json marshal then immediately unmarshal
2017-06-22 01:19:08 -07:00
John Howard
e99a633720 LCOW: owner and network endpoints
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-21 22:35:30 -07:00
Brian Goff
8d588d9c5b Don't json marshal then immediately unmarshal
During container startup we end up spending a fair amount of time
encoding/decoding json.
This cuts out some of that since we already have the decoded object in
memory.

The old flow looked like:

1. Start container request
2. Create file
3. Encode container spec to json
4. Write to file
5. Close file
6. Open file
7. Read file
8. Decode container spec
9. Close file
10. Send to containerd.

The new flow cuts out steps 6-9 completely, and with it a lot of time
spent in reflect and file IO.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-06-21 15:18:01 -07:00
John Howard
ed10ac6ee9 LCOW: Create layer folders with correct ACL
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:50:12 -07:00
John Howard
f154588226 LCOW: OCI Spec and Environment for container start
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-20 19:50:11 -07:00
Daniel Nephin
6150ebf7b4 Remove MkdirAllNewAs and update tests.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-06-07 11:44:34 -04:00
Sebastiaan van Stijn
56da020e6b Merge pull request #33496 from Microsoft/jjh/removedummy
Windows: Correct comment
2017-06-03 01:07:26 +02:00
John Howard
6e33c4158c Windows: Correct comment
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-06-02 11:51:30 -07:00
Wentao Zhang
d3d8c77d19 Limit max backoff delay to 2 seconds for GRPC connection
Docker use default GRPC backoff strategy to reconnect to containerd when
connection is lost. and the delay time grows exponentially, until reaches 120s.

So Change the max delay time to 2s to avoid docker and containerd
connection failure.

Signed-off-by: Wentao Zhang <zhangwentao234@huawei.com>
2017-06-02 18:19:09 +08:00
Evan Hazlett
d7c125791a Merge pull request #32590 from moypray/containerd
Fix when containerd restarted, event handler may exit
2017-06-01 08:16:24 -04:00
Antonio Murdaca
27087eacbf
libcontainerd: fix reaper goroutine position
It has observed defunct containerd processes accumulating over
time while dockerd was permanently failing to restart containerd.
Due to a bug in the runContainerdDaemon() function, dockerd does not clean up
its child process if containerd already exits very soon after the (re)start.

The reproducer and analysis below comes from docker 1.12.x but bug
still applies on latest master.

- from libcontainerd/remote_linux.go:

  329 func (r *remote) runContainerdDaemon() error {
   :
   :      // start the containerd child process
   :
  403     if err := cmd.Start(); err != nil {
  404             return err
  405     }
   :
   :      // If containerd exits very soon after (re)start, it is
possible
   :      // that containerd is already in defunct state at the time
when
   :      // dockerd gets here. The setOOMScore() function tries to
write
   :      // to /proc/PID_OF_CONTAINERD/oom_score_adj. However, this
fails
   :      // with errno EINVAL because containerd is defunct. Please see
   :      // snippets of kernel source code and further explanation
below.
   :
  407     if err := setOOMScore(cmd.Process.Pid, r.oomScore); err != nil
{
  408             utils.KillProcess(cmd.Process.Pid)
   :
   :              // Due to the error from write() we return here. As
the
   :              // goroutine that would clean up the child has not
been
   :              // started yet, containerd remains in the defunct
state
   :              // and never gets reaped.
   :
  409             return err
  410     }
   :
  417     go func() {
  418             cmd.Wait()
  419             close(r.daemonWaitCh)
  420     }() // Reap our child when needed
   :
  423 }

This is the kernel function that gets invoked when dockerd tries to
write
to /proc/PID_OF_CONTAINERD/oom_score_adj.

- from fs/proc/base.c:

 1197 static ssize_t oom_score_adj_write(struct file *file, ...
 1198                                         size_t count, loff_t
*ppos)
 1199 {
   :
 1223         task = get_proc_task(file_inode(file));
   :
   :          // The defunct containerd process does not have a virtual
   :          // address space anymore, i.e. task->mm is NULL. Thus the
   :          // following code returns errno EINVAL to dockerd.
   :
 1230         if (!task->mm) {
 1231                 err = -EINVAL;
 1232                 goto err_task_lock;
 1233         }
   :
 1253 err_task_lock:
   :
 1257         return err < 0 ? err : count;
 1258 }

The purpose of the following program is to demonstrate the behavior of
the oom_score_adj_write() function in connection with a defunct process.

$ cat defunct_test.c

\#include <unistd.h>

main()
{
    pid_t pid = fork();

    if (pid == 0)
        // child
        _exit(0);

    // parent
    pause();
}

$ make defunct_test
cc     defunct_test.c   -o defunct_test

$ ./defunct_test &
[1] 3142

$ ps -f | grep defunct_test | grep -v grep
root      3142  2956  0 13:04 pts/0    00:00:00 ./defunct_test
root      3143  3142  0 13:04 pts/0    00:00:00 [defunct_test] <defunct>

$ echo "ps 3143" | crash -s
  PID    PPID  CPU       TASK        ST  %MEM     VSZ    RSS  COMM
  3143   3142   2  ffff880035def300  ZO   0.0       0      0
defunct_test

$ echo "px ((struct task_struct *)0xffff880035def300)->mm" | crash -s
$1 = (struct mm_struct *) 0x0
                          ^^^ task->mm is NULL

$ cat /proc/3143/oom_score_adj
0

$ echo 0 > /proc/3143/oom_score_adj
-bash: echo: write error: Invalid argument"

---

This patch fixes the above issue by making sure we start the reaper
goroutine as soon as possible.

Signed-off-by: Antonio Murdaca <runcom@redhat.com>
2017-05-27 15:13:59 +02:00
Wentao Zhang
02ce73f62e Fix when containerd restarted, event handler may exit
Description:
Kill docker-containerd continuously, and use kill -SIGUSR1 <dockerpid>
to check docker callstacks. And we will find that event
handler: startEventsMonitor or handleEventStream will exit.

This will only happen when system is busy, containerd need more time to
startup, and the monitor gorotine maybe exit.

Signed-off-by: Wentao Zhang <zhangwentao234@huawei.com>
2017-05-25 17:32:05 +08:00
John Howard
2f038c2586 Windows: Remove unused SandboxPath
Signed-off-by: John Howard <jhoward@microsoft.com>
2017-05-24 13:44:35 -07:00
Darren Stahl
425973cbb8 Use CpuMaximum instead of CpuPercent for more precision
Signed-off-by: Darren Stahl <darst@microsoft.com>
2017-05-19 12:33:14 -07:00
Tõnis Tiigi
e103125883 Merge pull request #32986 from moypray/containerd_close
fix when rpc reports "transport is closing" error, health check go routine will exit
2017-05-17 17:04:05 -07:00
Kenfe-Mickael Laventure
0ea0b2becf Use containerd Status variable when checking container state
Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com>
2017-05-15 10:53:51 -07:00
Deng Guangxing
9771780a01 fix inconsistent state string with containerd
should be `stopped` according to containerd:
  https://github.com/containerd/containerd/blob/v0.2.x/runtime/runtime.go#L104

Signed-off-by: Deng Guangxing <dengguangxing@huawei.com>
2017-05-15 10:53:51 -07:00
Michael Crosby
005506d36c Update moby to runc and oci 1.0 runtime final rc
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
2017-05-05 13:45:45 -07:00
Wentao Zhang
60742f9a95 fix when rpc reports "transport is closing" error, health check go routine will exit
Signed-off-by: Wentao Zhang <zhangwentao234@huawei.com>
2017-05-04 00:52:10 +08:00
Darren Stahl
dbdc8bbee4 Wait to delete container when restoring on Windows
Signed-off-by: Darren Stahl <darst@microsoft.com>
2017-03-31 10:59:00 -07:00
Tibor Vass
caf8d884aa Merge pull request #31629 from darrenstahlmsft/ShutdownLock
Windows: Stop holding client container lock during shutdown
2017-03-23 18:16:56 -07:00