Adding support for custom SEL records

[Ed Tanous]

On Mon, Oct 24, 2022 at 12:03 PM Brad Bishop
<bradleyb at fuzziesquirrel.com> wrote:
>
> This is helpful, thanks Ed.
>
> On Mon, Oct 24, 2022 at 10:59:51AM -0700, Ed Tanous wrote:
> >From what I recall, the requirements were:
> >- Ability to store 4000 logs in a rotating buffer (original desire was
> >10,000, but 4000 was picked as a middle-ground that could be
> >implemented).
>
> A DBus object based implementation could meet this requirement, right?
>
> >- Ability to log 100+ entries per second, including when buffers get
> >overwritten.
>
> I guess I would not be shocked if DBus objects + serialization might not
> be able to sustain this rate of incoming logs.  Maybe it depends on the
> filesystem or how the data is serialized in the filesystem.  The DBus
> approach creates many files.  Obviously the syslog approach is only
> using a couple of files.
>
> Do you think this kind of requirement is typical?  Quoting Patrick from
> another thread here:

There are cases where every sensor will cross a threshold at the same
moment (generally because a threshold was changed).  Is this a
"typical" use case, probably not, but if it's trivial to DOS the BMC
with events with a couple commands, it's not great.

>
> >don't think anyone is intending to create 10k events in the span of
> >a minute

Intending to, no.  Created in an error case, or in the case of a
failing piece of hardware, it can happen.

>
> 100/s is only 6k in a minute but that is getting pretty close...
>
> >- (abstract) Log storage should be aware of hardware limitations (SPI
> >flash cell write endurance) and allow writing N logs per minute for
> >the lifetime of the machine without hardware failure.  (I forget the
> >value of N).
>
> Do you think the rsyslog implementation does better at this?  Why?

@Jason?  I know there were a lot of worries about write expansion (N
byte sel log turning into 100XN bytes of JSON) wearing out flash
faster, and having some interactions with jffs2.

>
> >- "ipmitool sensor list" should return the results from a full sel log
> >in less than 1 second (renegotiated from 200ms, the faster the
> >better).
>
> Ok, again I would not be shocked if DBus objects weren't able to deliver
> on this.
>
> >- The logging implementation should be able to support a well-formed,
> >version controlled, Redfish MessageRegistry to the DMTF
> >specifications.
>
> Do you think a DBus object based implementation could meet this
> requirement?

Today, as defined, without duplicating the logging strings, no, but
DBus is just an IPC;  I'm fairly certain we could define a DBus based
logging mechanism that met these requirements, but the key is that the
Redfish instance (bmcweb in this case) needs a-priori knowledge of
every possible log that the system can publish, and version them with
DMTF-appropriate semver (existing messages changes rev the patch
version, new messages revs the subminor version).  The existing
rsyslog implementation places the log strings into bmcweb, so the only
thing being transferred over the IPC is the MessageId and the
specific-instance arguments, which in theory, increases the
performance quite a bit, and avoids duplicating the strings in two
places.

>
> >- The logging implementation should be able to implement a
> >well-formed, stable, and ACID compliant IPMI SEL command
> >implementation.
>
> Do you think a DBus object based implementation could meet this
> requirement?

Same answer, with today's code, no.  DBus is just an IPC, we could
potentially define one.

>
> >
> >I don't believe the current DBus implementation can meet the previous
> >requirements,
>
> The motivation of my questions above is to understand which requirements
> cannot be met by something based on DBus objects.

Cool.


>
> Thanks,
> brad