Commit graph

258 commits

Author SHA1 Message Date
Daniel Nephin
035604cca6 Move jsonlog to a subpackage of jsonfilelog
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
a06ad2792a Fix benchmarks and remove more unnecessary code.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
231c5cbd50 Remove unused JSONLog marshaling
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
7de92de636 Unexport FastTimeMarshalJSON
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:25 -04:00
Daniel Nephin
27cfa68af1 Move RFC3339NanoFixed to a more appropriate package.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-25 16:07:24 -04:00
Vincent Demeester
5c57ca17d5 Merge pull request #34520 from fnoeding/fixed-raw-splunk-logger
Fixed `raw` mode splunk logger
2017-09-19 10:53:01 +02:00
Daniel Nephin
f7f101d57e Add gosimple linter
Update gometalinter

Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-09-12 12:09:59 -04:00
Vincent Demeester
b3b630655d Merge pull request #31727 from sascha-andres/31726-le-lion-only
Log payload only [logentries]
2017-08-22 14:59:28 +02:00
Daniel Nephin
9b47b7b151 Fix golint errors.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2017-08-18 14:23:44 -04:00
Florian Noeding
5f6d6a5093 Fixed raw mode splunk logger
Splunk HEC does not accept log events with an empty string or a
whitespace-only string.

Signed-off-by: Florian Noeding <florian@noeding.com>
2017-08-18 12:03:43 +02:00
Brian Goff
ebcb7d6b40 Remove string checking in API error handling
Use strongly typed errors to set HTTP status codes.
Error interfaces are defined in the api/errors package and errors
returned from controllers are checked against these interfaces.

Errors can be wraeped in a pkg/errors.Causer, as long as somewhere in the
line of causes one of the interfaces is implemented. The special error
interfaces take precedence over Causer, meaning if both Causer and one
of the new error interfaces are implemented, the Causer is not
traversed.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-08-15 16:01:11 -04:00
Vincent Demeester
9ef3b53597
Move pkg/templates away
- Remove unused function and variables from the package
- Remove usage of it from `profiles/apparmor` where it wasn't required
- Move the package to `daemon/logger/templates` where it's only used

Signed-off-by: Vincent Demeester <vincent@sbr.pm>
2017-08-08 18:16:41 +02:00
Sascha Andres
6362843e67 Log payload only [logentries]
If you want to use the advanced query in logentries
logging in json is very useful. If you do not need
the advanced information with
--log-opt line-only=true just logs the payload

Signed-off-by: Sascha Andres <sascha.andres@outlook.com>
2017-08-05 09:54:21 +02:00
Derek McGowan
1009e6a40b
Update logrus to v1.0.1
Fixes case sensitivity issue

Signed-off-by: Derek McGowan <derek@mcgstyle.net>
2017-07-31 13:16:46 -07:00
Justin Menga
0fd5a0bab7 Fix awslogs driver repeating last event - #34292
Signed-off-by: Justin Menga <justin.menga@gmail.com>
2017-07-28 20:33:14 +12:00
Aaron Lehmann
b642b3f21f Avoid using a map for log attributes
Having a map per log entry seemed heavier than necessary. These
attributes end up being sorted and serialized, so storing them in a map
doesn't add anything (there's no random access element). In SwarmKit,
they originate as a slice, so there's an unnecessary conversion to a map
and back.

This also fixes the sort comparator, which used to inefficiently split
the string on each comparison.

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
2017-07-18 19:01:20 -07:00
tim
cb972b1515 Replaces fluentd-address string by constant
Signed-off-by: tim <tim@fewagainstmany.com>
2017-07-16 23:06:57 -07:00
Christopher Jones
069fdc8a08
[project] change syscall to /x/sys/unix|windows
Changes most references of syscall to golang.org/x/sys/
Ones aren't changes include, Errno, Signal and SysProcAttr
as they haven't been implemented in /x/sys/.

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>

[s390x] switch utsname from unsigned to signed

per 33267e036f
char in s390x in the /x/sys/unix package is now signed, so
change the buildtags

Signed-off-by: Christopher Jones <tophj@linux.vnet.ibm.com>
2017-07-11 08:00:32 -04:00
Aaron Lehmann
3be2273d03 Merge pull request #33713 from x1022as/error_fd
fd leak and error handling
2017-07-06 16:47:34 -07:00
Sebastiaan van Stijn
db8c2651b8 Merge pull request #33897 from cpuguy83/jsonfile_log_blocking
Fix log readers can block writes indefinitely
2017-07-05 18:21:14 -07:00
Brian Goff
e2209185ed Fix log readers can block writes indefinitely
Before this patch, a log reader is able to block all log writes
indefinitely (and other operations) by simply opening the log stream and
not consuming all the messages.

The reason for this is we protect the read stream from corruption by
ensuring there are no new writes while the log stream is consumed (and
caught up with the live entries).

We can get around this issue because log files are append only, so we
can limit reads to only the section of the file that was written to when
the log stream was first requested.

Now logs are only blocked until all files are opened, rather than
streamed to the client.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-07-05 14:04:52 -04:00
Vincent Demeester
ff4f700f74 Merge pull request #33322 from jsoref/spelling
Spelling
2017-07-04 15:46:34 +02:00
Brian Goff
c2ad549533 Merge pull request #33213 from gkze/master
GCP Logs: send log text as jsonPayload.message instead of jsonPayload.data
2017-07-03 16:18:18 -04:00
Josh Soref
39bcaee47b
Spelling fixes
* additional
* ambiguous
* anonymous
* anything
* application
* because
* before
* building
* capabilities
* circumstances
* commit
* committer
* compresses
* concatenated
* config
* container
* container's
* current
* definition
* delimiter
* disassociates
* discovery
* distributed
* doesnotexist
* downloads
* duplicates
* either
* enhancing
* enumerate
* escapable
* exactly
* expect
* expectations
* expected
* explicitly
* false
* filesystem
* following
* forbidden
* git with
* healthcheck
* ignore
* independent
* inheritance
* investigating
* irrelevant
* it
* logging
* looking
* membership
* mimic
* minimum
* modify
* mountpoint
* multiline
* notifier
* outputting
* outside
* overridden
* override
* parsable
* plugins
* precedence
* propagation
* provided
* provides
* registries
* repositories
* returning
* settings
* should
* signals
* someone
* something
* specifically
* successfully
* synchronize
* they've
* thinking
* uninitialized
* unintentionally
* unmarshaling
* unnamed
* unreferenced
* verify

Signed-off-by: Josh Soref <jsoref@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-07-03 13:13:09 -07:00
Lei Jitang
96ea8eaa15 Fix wrong filemode for rotate log files
the filemode should be 0640 but not 06400

Signed-off-by: Lei Jitang <leijitang@huawei.com>
2017-07-03 03:49:22 -04:00
David Glasser
917050c572 Fix stderr logging for journald and syslog
logger.PutMessage, added in #28762 (v17.04.0-ce), clears msg.Source. So journald
and syslog were treating stderr messages as if they were stdout.

Signed-off-by: David Glasser <glasser@davidglasser.net>
2017-06-26 14:37:43 -07:00
Deng Guangxing
0042f992d8 fd leak and error handling
Signed-off-by: Deng Guangxing <dengguangxing@huawei.com>
2017-06-23 15:38:35 +08:00
Royce Remer
b3add005d5 * standardize timeouts for log reads and writes for logger adapter tests
* use an assertion framework in logger adapter tests

Signed-off-by: Royce Remer <royceremer@gmail.com>
2017-06-18 18:47:45 -07:00
Peter Bücker
e908e1a357 Logging driver should receive same file in start/stop request
Signed-off-by: Peter Bücker <peter.buecker@gmail.com>
2017-06-08 10:05:52 +02:00
Victor Vieux
2445e6b99d move multireader out of /pkg
Signed-off-by: Victor Vieux <victorvieux@gmail.com>
2017-06-01 16:22:00 -07:00
Sebastiaan van Stijn
07b51ed300 Don't log error if file is already closed
When closing the log-file, and the file is already
closed, there's no need to log an error.

This patch adds a `closed` boolean to check if the
file was closed, and if so, skip closing the file.
This prevents errors like this being logged:

    level=error msg="Error closing logger: invalid argument"

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2017-05-29 17:42:37 +02:00
YAMASAKI Masahide
524f306340 Improved poor memory efficiency of awslogs
Signed-off-by: YAMASAKI Masahide <masahide.y@gmail.com>
2017-05-18 23:30:00 +09:00
gkze
d42c268fc3 GCP Logs: send log line as jsonPayload.message instead of jsonPayload.data
Signed-off-by: gkze <george@bugsnag.com>
2017-05-15 16:35:53 -07:00
Justin Menga
dc87490b63 Update comments describing new behaviour
Signed-off-by: Justin Menga <justin.menga@gmail.com>
2017-05-15 11:08:16 +12:00
Justin Menga
3a4cf2b076 Code review changes
Signed-off-by: Justin Menga <justin.menga@gmail.com>
2017-05-15 10:28:18 +12:00
Justin Menga
bf1c377f60 Add missing month in datetime map
Signed-off-by: Justin Menga <justin.menga@gmail.com>
2017-05-13 22:10:51 +12:00
Justin Menga
4374f16667 Append new line to multiline events
Signed-off-by: Justin Menga <justin.menga@gmail.com>
2017-05-13 17:54:04 +12:00
Justin Menga
8ef9c19ecd Code review changes
Signed-off-by: Justin Menga <justin.menga@gmail.com>
2017-05-13 17:54:04 +12:00
Tibor Vass
f775005a17 awslogs: use github.com/pkg/errors to wrap an error
Signed-off-by: Tibor Vass <tibor@docker.com>
2017-05-13 17:54:04 +12:00
Justin Menga
6f073e3522 Formatting
Signed-off-by: Justin Menga <justin.menga@gmail.com>
2017-05-13 17:54:04 +12:00
Justin Menga
84b03660da Add awslogs benchmarks
Signed-off-by: Justin Menga <justin.menga@gmail.com>
2017-05-13 17:54:04 +12:00
Justin Menga
ab74038df9 Add awslogs multiline support
Signed-off-by: Justin Menga <justin.menga@gmail.com>
2017-05-13 17:54:04 +12:00
Grzegorz Jaśkiewicz
d925e50470 gcplogging driver MRPB set
Signed-off-by: Grzegorz Jaśkiewicz <gj.jaskiewicz@gmail.com>
2017-05-09 14:24:43 +02:00
Tõnis Tiigi
5eca7f7c5d Merge pull request #31856 from jim-minter/more_races
Resolve a set of race conditions in logging and attaching code
2017-04-26 16:04:02 -07:00
Jim Minter
4fdb17c777 Prevent ContainerLogs from hanging if container doesn't run for long
Signed-off-by: Jim Minter <jminter@redhat.com>
2017-04-20 12:27:51 +01:00
Brian Goff
17abacb894 Add logdrivers to /info
This is required for swarmkit to be able to filter based on log driver.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-04-11 18:07:15 -04:00
Brian Goff
27bd6842f8 Implement plugins for logging drivers
Logging plugins use the same HTTP interface as other plugins for basic
command operations meanwhile actual logging operations are handled (on
Unix) via a fifo.

The plugin interface looks like so:

```go
type loggingPlugin interface {
  StartLogging(fifoPath string, loggingContext Context) error
  StopLogging(fifoPath)
```

This means a plugin must implement `LoggingDriver.StartLogging` and
`LoggingDriver.StopLogging` endpoints and be able to consume the passed
in fifo.

Logs are sent via stream encoder to the fifo encoded with protobuf.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2017-04-10 13:17:20 -04:00
Drew Erny
1044093bb0 refactor logs and support service logs /w tty
Refactor container logs system to make communicating log messages
internally much simpler. Move responsibility for marshalling log
messages into the REST server. Support TTY logs. Pave the way for fixing
the ambiguous bytestream format. Pave the way for fixing details.

Signed-off-by: Drew Erny <drew.erny@docker.com>
2017-04-06 17:54:11 -07:00
Brian Goff
1325f667ee Merge pull request #27565 from rothrock/env-by-regex
[splunk] Use a regex to match environment variables
2017-03-31 10:46:48 -04:00
Sebastiaan van Stijn
57fd478169 Merge pull request #31083 from xulike666/fight-for-readability
fix some typos from module contrib to man
2017-03-03 15:13:01 +01:00