Streaming logs via rsyslog

[Deepak Kodihalli]

On 21/08/18 5:48 PM, Brad Bishop wrote:
> 
>> On Aug 16, 2018, at 6:21 AM, Deepak Kodihalli <dkodihal at linux.vnet.ibm.com> wrote:
>>
>> Hi,
>>
>> We have a requirement to forward BMC logs (the systemd journal) to an external logging server.
>>
>> We're proposing doing this by making use of rsyslog. Here are notes based on some initial investigation.
>>
>> Forwarding the journal logs to a syslog daemon :
>> Journald has this feature of being able to write journal logs to a socket which then can be read by a syslog daemon. I see that (on Witherspoon at least) the syslog daemon running by default (busybox syslogd) doesn't support reading logs forwarded by the journald to due to this commit : https://git.busybox.net/busybox/commit/?id=accd9eeb719916da974584b33b1aeced5f3bb346. However, I could get the forwarding working with the rsyslog daemon. I found an existing rsyslog recipe in the meta-oe layer.
>>
>> rsyslog configuration :
>> rsyslog is configurable in various ways. For example, I can turn off local logging altogether via a config file, if I'm interested only in using rsyslog for streaming journal logs out. Various optional features can be turned off at build time to save flash space.
>>
>> rsyslog server provision :
>> The BMC would need to be made aware of the external rsyslog server. This can just be an admin updating the rsyslog config file on the BMC, and there could also be a REST API if need be. I think a small app can implement the https://github.com/openbmc/phosphor-dbus-interfaces/blob/master/xyz/openbmc_project/Network/Client.interface.yaml interface to update the rsyslog config.
>>
>> streaming :
>> rsyslog can be configured to make use of TCP, or UDP, and optionally TLS. I think TCP is the default.
>>
>>
>> Thoughts on this proposal? If this sounds okay, I'm planning to get some commits up in Gerrit for review. I can pull rsyslog in a phosphor layer (I mean as the default choice of a syslog daemon, replacing busybox syslog) if there's interest, as opposed to making it only a meta-witherspoon package. Specific rsyslog package config and config file overrides are possible in system specific layers.
>>
>> Regards,
>> Deepak
> 
> Hi Deepak
> 
> Plain-old syslogd can forward log entries over the network to another
> syslogd instance.  Did we evaluate that?
> 
> -brad

Hi Brad,

The problem with syslogd was not being able to read logs forwarded by 
journald (due to the commit I noted in my original email). Besides that, 
rsyslog's filtering capabilities are much more impressive.

Regards,
Deepak