Streaming logs via rsyslog

[Deepak Kodihalli]

Hi,

We have a requirement to forward BMC logs (the systemd journal) to an 
external logging server.

We're proposing doing this by making use of rsyslog. Here are notes 
based on some initial investigation.

Forwarding the journal logs to a syslog daemon :
Journald has this feature of being able to write journal logs to a 
socket which then can be read by a syslog daemon. I see that (on 
Witherspoon at least) the syslog daemon running by default (busybox 
syslogd) doesn't support reading logs forwarded by the journald to due 
to this commit : 
https://git.busybox.net/busybox/commit/?id=accd9eeb719916da974584b33b1aeced5f3bb346. 
However, I could get the forwarding working with the rsyslog daemon. I 
found an existing rsyslog recipe in the meta-oe layer.

rsyslog configuration :
rsyslog is configurable in various ways. For example, I can turn off 
local logging altogether via a config file, if I'm interested only in 
using rsyslog for streaming journal logs out. Various optional features 
can be turned off at build time to save flash space.

rsyslog server provision :
The BMC would need to be made aware of the external rsyslog server. This 
can just be an admin updating the rsyslog config file on the BMC, and 
there could also be a REST API if need be. I think a small app can 
implement the 
https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/xyz/openbmc_project/Network/Client.interface.yaml 
interface to update the rsyslog config.

streaming :
rsyslog can be configured to make use of TCP, or UDP, and optionally 
TLS. I think TCP is the default.


Thoughts on this proposal? If this sounds okay, I'm planning to get some 
commits up in Gerrit for review. I can pull rsyslog in a phosphor layer 
(I mean as the default choice of a syslog daemon, replacing busybox 
syslog) if there's interest, as opposed to making it only a 
meta-witherspoon package. Specific rsyslog package config and config 
file overrides are possible in system specific layers.

Regards,
Deepak