Fwd: libflash questions mostly relating to P9 bringup

[Brendan Higgins]

Sorry forgot to include the OpenBMC mail list.


---------- Forwarded message ----------
From: Brendan Higgins <brendanhiggins at google.com>
Date: Wed, Oct 19, 2016 at 1:54 PM
Subject: libflash questions mostly relating to P9 bringup
To: cyrilbur at gmail.com, joel at jms.id.au, xow at google.com, benh at kernel.crashing.org
Cc: Brendan Higgins <brendanhiggins at google.com>


> I dont know if that was mentioned earlier in the thread but we are
> working on a mailbox based protocol for flash access to avoid accessing
> the flash controller registers from any other entity than the BMC, in
> order to be able to cut off the iLPC->AHB backdoor completely.
>
> This should support having the flash either access directly for reads
> or via an in-memory image in the BMC (aka memboot) and writes will be
> done to an in-memory window on the LPC and "committed" to flash via
> mbox commands.
>
> We can get you details when we have a bit more sorted out on our side,
> hopefully in a few days.

We are also working on a flash access protocol except ours is via IPMI; this is
was a motivating reason for the 0-N response per request extension discussed
here: https://lists.ozlabs.org/pipermail/openbmc/2016-September/004523.html

The main desire for using IPMI for this is that IPMI is the closest thing we
have to a common communication protocol between the BMC and the primary CPU
across platforms. Some platforms, for example, may not have support for LPC or
eSPI.

Our plan is to implement this as an OEM/Group extension with the payload as a
Protocol Buffer encoded (see
https://developers.google.com/protocol-buffers/docs/encoding) header:

message PacketHeader {
  optional uint32 packet_idx = 1;
  optional uint32 packet_count = 2;
}

followed by a chunk of data with whatever space remains. So the actual message
would look something like:

struct Packet {
  PacketHeader header;
  char data[X];
};

where X is IPMI payload size - PacketHeader size. This is not a valid struct
definition as the fields have variable offsets and size, and is only meant for
illustrative purposes.

The motivation behind the protobuf encoding is that it can be extended in a
backwards compatible way and that unneeded fields (for example, we may only want
to report the packet_count, the number of expected packets, in the first
message) only take up space if they are sent; similarly, numeric fields use
varint encoding so that the value takes up less space if it is small.

The actual flash access message would be reassembled from multiple packets and
could contain different messages, for example:

message RawFlashRequest {
  enum RequestType {
    Read = 0;
    Write = 1;
    Info = 2;
    Checksum = 3;
  }

  optional RequestType type = 1;
  optional bytes partition_guid = 2;
  optional uint32 offset = 3;
  optional uint32 size = 4;
  repeated bytes payload = 5;
}

could be one message type.

This is not intended to be a complete specification, but an overview of what we
are working on and what our needs are. I think it would be preferential if we
could have a common implementation.

Cheers.

---------- Forwarded message ----------
From: Benjamin Herrenschmidt <benh at kernel.crashing.org>
Date: Wed, Oct 19, 2016 at 5:15 PM
Subject: Re: libflash questions mostly relating to P9 bringup
To: Brendan Higgins <brendanhiggins at google.com>, cyrilbur at gmail.com,
joel at jms.id.au, xow at google.com


On Wed, 2016-10-19 at 13:54 -0700, Brendan Higgins wrote:

> We are also working on a flash access protocol except ours is via IPMI; this is
> was a motivating reason for the 0-N response per request extension discussed
> here: https://lists.ozlabs.org/pipermail/openbmc/2016-September/004523.html
>
> The main desire for using IPMI for this is that IPMI is the closest thing we
> have to a common communication protocol between the BMC and the primary CPU
> across platforms. Some platforms, for example, may not have support for LPC or
> eSPI.

 .../...

We chose do to something a bit different for several reasons.

The main one is probably speed. IPMI is fairly slow and we don't really want
to transport the actual flash data over it.

So the idea is to read/write through the LPC FW space, by mapping it to a
region of BMC memory rather than flash, and send commands via a separate
IO channel (which could be IPMI, though in our current prototype it's not)
to "commit" the modified portions.

It's a bit more convoluted as we plan to support a sliding write window
that might be distinct from the read window as to not require using as
much BMC memory as the size of the flash, but we haven't implemented all
of the "options" yet.

It will also give us memboot for free, ie, the ability to boot from an
image of the flash coming from the network rather than flash which is useful
for development, and using memory rather than flash seems to speed things
up a bit as well.

It might not be portable to different BMCs, so having the IPMI method as
some kind of fallback might still be an options for us though, we should
probably look into it.

The other reason, which granted isn't a terribly good one, is that HostBoot
has their flash layer in their "kernel" while their IPMI stack is in their
"User space", and they are reluctant on changing that.

That being said, we could use the second BT interface along with a massively
stripped down IPMI stack in their kernel to keep the flash path separate from
the main IPMI path if we wanted.

For now, we use the mbox instead which is slightly easier to use than BT and
avoids all "conflicts" with the IPMI traffic.

Antoher advantage in segregating these is that we open the possibility of
assigning the main iBT interface directly to linux instead of OPAL, taking
OPAL out of the path of normal IPMI messages. OPAL <-> BMC would then use
our mbox sideband (but could use the secondary BT if we wanted to, it's just
inconvenient as it doesn't support interrupts in the Aspeed hardware).

Cheers,
Ben.


---------- Forwarded message ----------
From: Brendan Higgins <brendanhiggins at google.com>
Date: Thu, Oct 20, 2016 at 11:41 AM
Subject: Re: libflash questions mostly relating to P9 bringup
To: Benjamin Herrenschmidt <benh at kernel.crashing.org>, Cyril Bur
<cyrilbur at gmail.com>, Joel Stanley <joel at jms.id.au>, Xo Wang
<xow at google.com>, Abhishek Pandit <abhishekpandit at google.com>


> The main one is probably speed. IPMI is fairly slow and we don't really want
>
>  to transport the actual flash data over it.


I do not dispute that; I think we accept that as a consequence, again,
as LPC does not exist on many platforms.

> It's a bit more convoluted as we plan to support a sliding write window
> that might be distinct from the read window as to not require using as
> much BMC memory as the size of the flash, but we haven't implemented all
> of the "options" yet.


I only mentioned the most simple method that we wanted to implement,
RawFlashRequest, which would provide direct flash access, but we were
interested in supporting other methods as well, so if you would be
willing to support other access methods, I see this as being totally
compatible with what we want to do.

> It might not be portable to different BMCs, so having the IPMI method as
> some kind of fallback might still be an options for us though, we should
> probably look into it.


That sounds reasonable to me. As long as we have a common way to do
flash access on all systems we are happy; if we happen to support some
other faster access methods (as long as they share a common API), I
think that would be even better :-)

> For now, we use the mbox instead which is slightly easier to use than BT and
> avoids all "conflicts" with the IPMI traffic.


Yep, that is what we are trying to address with our OEM/Group
extension; it provides an interface that sends arbitrary data over
IPMI. Admittedly, it will make all other IPMI traffic slower; however,
this is unavoidable on platforms that only have one hardware interface
between the BMC and the host CPU.

>
> Antoher advantage in segregating these is that we open the possibility of
> assigning the main iBT interface directly to linux instead of OPAL, taking
> OPAL out of the path of normal IPMI messages. OPAL <-> BMC would then use
> our mbox sideband (but could use the secondary BT if we wanted to, it's just
> inconvenient as it doesn't support interrupts in the Aspeed hardware).


Also true, but still unavoidable if we only have a single hardware interface.

Sounds to me like we do not have any incompatible goals; although it
is true that on our side we are more concerned about a common
implementation that will work on any platform and you guys seem to be
more concerned about performance, I think that just means that we
should provide both transport layers: IPMI and LPC mailbox. I imagine
that a good deal of the actual flash access logic will be the same for
the both of us.

Cheers