IPMI SOL performance

[Jeremy Kerr]

Hi Tom,

> I wanted to discuss about an issue related to IPMI SOL.

Cool!

Given that:

> The same issue is not hit by the ssh console on port 2200 and does not
> miss the timeout of the petitboot menu option.

- and that we don't see the same behaviour with other SoL
implementations (ie, different BMC firmware on similar hardware), it
sounds to me like this might be due to our IPMI implementation?

Have you confirmed that we're making the best use of the SoL payloads?
(ie, we're not using smaller TX buffers for any reason).

> IPMI uses UDP protocol and the maximum size of the  SOL payload per
> packet according to the specification is 255 bytes. Also the ack for
> each of the IPMI packet requires a command lookup
> and execution. Compared with IPMI SOL, ssh supports 64kb payloads and
> ack does not need an application lookup, so the round-trip timing is
> faster. IPMI specification for SOL is written with UART speed as
> reference, but the VUART pushes the data at a much faster rate.

The console multiplexer will only push data as fast as *all* consumers
can consume it. As a bit of background: obmc-console-server will read
from the VUART into a global 16kB ringbuffer, and write to each of the
consumers (ie, the sockets that ssh and IPMI SoL daemons connect to).

If those writes block (for any one of those consumers), the ringbuffer
tail is not advanced. Only once the slowest consumer has unblocked and
consumed data, we advance the tail.

Once the ringbuffer is full though, we stop reading from the VUART until
there is more space available (ie, the slowest consumer has caught up).

This means that we have 16kB of space for bursty data, after which the
daemon will slow down reads from the VUART device.

So it sounds like our IPMI SoL implementation is slower than other BMC
implementations, as we don't see the delay on other platforms. Can we
address that?

As others have mentioned, there's currently an issue with the VUART
hardware or driver dropping serial data on high load, but that's
somewhat separate from IPMI performance. I'm currently working on that
now :)

Regards,


Jeremy