Link phosphor-hostlogger and bmcweb

[Nan Zhou]

>
> > If there are too many logs in a boot cycle, won't the current hostlogger
> > lose some earlier logs (boot logs) as well?
> That's the point!
> Hostlogger does not lose these logs. It writes the boot messages, then
> skips
> the middle and writes the last 3000 lines when the host shuts down.
> We have two log files per host session: start and end.
> It is too expensive to store all host output, so mid-session messages are
> skipped.
> It can be easily implemented with a buffer, but I am not sure we can
> achieve
> this with logrotate.

Thanks for the information. I am not aware of this functionality in the
current hostlogger. Are you saying it will be implemented in the future or
I miss it in the current codes.
One of our options for log rotations is writing our own codes, I am sure we
can implement the logic you mentioned above without too much effort.
The linux logrotate also has "prerotate scripts", we can carefully name the
compressed log file and keep the oldest several ones (which have host boot
logs) out of rotation.

> Or did me missing something?
> >
> > Also, we already talked about it: there's a problem that if BMC loses the
> > power before it sends out a signal to hostlogger, data in memory won't be
> > persisted.
> Yes, I agree that this is a problem. But there are ways to fix it without
> breaking the current functionality of Hostlogger.
> We can use rsyslog with external log server, or increase the buffer size
> in obmc-console-server, or use systemd-cat with logrotate.
> We can even add a new mode to Hostlogger that will not use the buffer, but
> as I said earlier, there are not many common parts.

I guess you are arguing we need a new daemon rather than modify Hostlogger,
right? +Ed Tanous <edtanous at google.com> here to see what his opinion is.

On Wed, May 26, 2021 at 11:22 AM Artem Senichev <artemsen at gmail.com> wrote:

> On Wed, May 26, 2021 at 09:20:38AM -0700, Nan Zhou wrote:
> > >
> > > > We plan to implement something similar to rotate count
> > > > <https://linux.die.net/man/8/logrotate> in linux logrotate. It is
> > > basically
> > > > like a ring buffer in the file system. We keep N log files. The
> latest
> > > log
> > > > file is in plain text and the writer keeps appending data to it. The
> rest
> > > > N-1 files are compressed.
> > > In this case, you will keep full logs without gaps:
> > > ```
> > > Host start <- log is empty, start logging
> > > |
> > > [...] <- write file, compress and rotate file
> > > |
> > > Host reboot or shut down
> > > ```
> > > If there are too many logs, logrotate removes the oldest one and we
> lose
> > > the
> > > boot log (form host start).
> > > This is the default Hostlogger mode:
> > > ```
> > > Host start <- log is empty, start logging
> > > |
> > > [line 3000] <- flush 3000 lines to the persistent file
> > > |
> > > [...] <- these logs are skipped (the last 3000 lines are in memory)
> > > |
> > > Host reboot or shut down <- flush last 3000 lines to the file
> >
> >
> > Thanks for your explanation, but I didn't get it. Are you arguing that
> one
> > can keep more logs in memory rather than on disk?
>
> Of course not! =)
>
> > If there are too many logs in a boot cycle, won't the current hostlogger
> > lose some earlier logs (boot logs) as well?
>
> That's the point!
> Hostlogger does not lose these logs. It writes the boot messages, then
> skips
> the middle and writes the last 3000 lines when the host shuts down.
> We have two log files per host session: start and end.
> It is too expensive to store all host output, so mid-session messages are
> skipped.
> It can be easily implemented with a buffer, but I am not sure we can
> achieve
> this with logrotate.
>
> > Or did me missing something?
> >
> > Also, we already talked about it: there's a problem that if BMC loses the
> > power before it sends out a signal to hostlogger, data in memory won't be
> > persisted.
>
> Yes, I agree that this is a problem. But there are ways to fix it without
> breaking the current functionality of Hostlogger.
> We can use rsyslog with external log server, or increase the buffer size
> in obmc-console-server, or use systemd-cat with logrotate.
> We can even add a new mode to Hostlogger that will not use the buffer, but
> as I said earlier, there are not many common parts.
>
> --
> Regards,
> Artem Senichev
> Software Engineer, YADRO.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20210526/0d800186/attachment-0001.htm>