瀏覽代碼

Sleep before restarting event processing

This prevents restarting event processing in a tight loop.
You can see this with the following steps:

```terminal
$ containerd &
$ dockerd --containerd=/run/containerd/containerd.sock &
$ pkill -9 containerd
```

At this point you will be spammed with logs such as:

```
ERRO[2019-07-12T22:29:37.318761400Z] failed to get event                           error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: Error while dialing dial unix /run/containerd/containerd.sock: connect: connection refused\"" module=libcontainerd namespace=plugins.moby
```

Without this change you can quickly end up with gigabytes of log data.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
(cherry picked from commit 1acaf2aabe000c6101501af321969df7c0ca5413)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Brian Goff 6 年之前
父節點
當前提交
3833f2a60b
共有 1 個文件被更改,包括 9 次插入3 次删除
  1. 9 3
      libcontainerd/remote/client.go

+ 9 - 3
libcontainerd/remote/client.go

@@ -703,10 +703,16 @@ func (c *client) processEventStream(ctx context.Context, ns string) {
 				errStatus, ok := status.FromError(err)
 				if !ok || errStatus.Code() != codes.Canceled {
 					c.logger.WithError(err).Error("failed to get event")
-					go c.processEventStream(ctx, ns)
-				} else {
-					c.logger.WithError(ctx.Err()).Info("stopping event stream following graceful shutdown")
+
+					// rate limit
+					select {
+					case <-time.After(time.Second):
+						go c.processEventStream(ctx, ns)
+						return
+					case <-ctx.Done():
+					}
 				}
+				c.logger.WithError(ctx.Err()).Info("stopping event stream following graceful shutdown")
 			}
 			return
 		case ev = <-eventStream: