The json decoder starts to decode immediately an inotify event is
received.
But at the time the inotify event is trigged, the json log
entry might haven't been fully written to the disk.
In this case the decoder will return an "io.UnexpectedEOF" error, but
there is still data remaining in the decoder's buffer. And the data
should be passed to the decoder when the next inotify event is
triggered.
Signed-off-by: Shijiang Wei <mountkin@gmail.com>
this allows jsonfile logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`.
Extra attributes are saved into `attrs` attributes for each log data.
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
this allows journald logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
this allows fluentd logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
this allows gelf logger to collect extra metadata from containers with
`--log-opt labels=label1,label2 --log-opt env=env1,env2`
Additional log field will be prefixed with `_` as per gelf protocol
https://www.graylog.org/resources/gelf/
Signed-off-by: Daniel Dao <dqminh@cloudflare.com>
If an invalid logger address is provided on daemon start it will
silently fail. As syslog driver is doing, this check should be done on
daemon start and prevent it from starting even in other drivers.
This patch also adds integration tests for this behavior.
Signed-off-by: Antonio Murdaca <runcom@linux.com>
@noxiouz points out that we don't need to check for a nil result from
C.CString(), since an out-of-memory condition causes a runtime panic
instead.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
If a logdriver doesn't register a callback function to validate log
options, it won't be usable. Fix the journald driver by adding a dummy
validator.
Teach the client and the daemon's "logs" logic that the server can also
supply "logs" data via the "journald" driver. Update documentation and
tests that depend on error messages.
Add support for reading log data from the systemd journal to the
journald log driver. The internal logic uses a goroutine to scan the
journal for matching entries after any specified cutoff time, formats
the messages from those entries as JSONLog messages, and stuffs the
results down a pipe whose reading end we hand back to the caller.
If we are missing any of the 'linux', 'cgo', or 'journald' build tags,
however, we don't implement a reader, so the 'logs' endpoint will still
return an error.
Make the necessary changes to the build setup to ensure that support for
reading container logs from the systemd journal is built.
Rename the Jmap member of the journald logdriver's struct to "vars" to
make it non-public, and to make it easier to tell that it's just there
to hold additional variable values that we want journald to record along
with log data that we're sending to it.
In the client, don't assume that we know which logdrivers the server
implements, and remove the check that looks at the server. It's
redundant because the server already knows, and the check also makes
using older clients with newer servers (which may have new logdrivers in
them) unnecessarily hard.
When we try to "logs" and have to report that the container's logdriver
doesn't support reading, send the error message through the
might-be-a-multiplexer so that clients which are expecting multiplexed
data will be able to properly display the error, instead of tripping
over the data and printing a less helpful "Unrecognized input header"
error.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
After tailing a file, if the number of lines requested is > the number
of lines in the file, this would cause a json unmarshalling error to
occur when we later try to go follow the file.
So brute force set it to the end if any tailing occurred.
There is potential that there could be some missing log messages if logs
are being written very quickly, however I was not able to make this
happen even with `while true; do echo hello; done`, so this is probably
acceptable.
While testing this I also found a panic in LogWatcher.Close can be
called twice due to a race. Fix channel close to only close when there
has been no signal to the channel.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
- downcase and privatize exported variables that were unused
- make accurate an error message
- added package comments
- remove unused var ReadLogsNotSupported
- enable linter
- some spelling corrections
Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
There is no option validation for "journald" log-driver, so it makes no
sense to fail in that case.
Signed-off-by: Alexander Morozov <lk4d4@docker.com>
When using a scanner, log lines over 64K will crash the Copier with
bufio.ErrTooLong. Subsequently, the ioutils.bufReader will grow without
bound as the logs are no longer being flushed to disk.
Signed-off-by: Burke Libbey <burke.libbey@shopify.com>
- noplog driver pkg for '--log-driver=none' (null object pattern)
- centralized factory for log drivers (instead of case/switch)
- logging drivers registers themselves to factory upon import
(easy plug/unplug of drivers in daemon/logdrivers.go)
- daemon now doesn't start with an invalid log driver
- Name() method of loggers is actually now their cli names (made it useful)
- generalized Read() logic, made it unsupported except json-file (preserves
existing behavior)
Spotted some duplication code around processing of legacy json-file
format, didn't touch that and refactored in both places.
Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
This patch modifies the journald log driver to store the container ID in
a field named CONTAINER_ID, rather than (ab)using the MESSAGE_ID field.
Additionally, this adds the CONTAINER_ID_FULL field containing the
complete container ID and CONTAINER_NAME, containing the container name.
When using the journald log driver, this permits you to see log messages
from a particular container like this:
# journalctl CONTAINER_ID=a9238443e193
Example output from "journalctl -o verbose" includes the following:
CONTAINER_ID=27aae7361e67
CONTAINER_ID_FULL=27aae7361e67e2b4d3864280acd2b80e78daf8ec73786d8b68f3afeeaabbd4c4
CONTAINER_NAME=web
Closes: #12864
Signed-off-by: Lars Kellogg-Stedman <lars@redhat.com>
This patch changes two things
1. Set facility to LOG_DAEMON
2. Remove ": " from tag so that the tag + pid become a single column in
the log
Signed-off-by: Darren Shepherd <darren@rancher.com>
