Generic Flash Access
Patrick Venture
venture at google.com
Fri Sep 15 03:27:26 AEST 2017
On Mon, Aug 21, 2017 at 2:13 PM, Brendan Higgins <brendanhiggins at google.com>
wrote:
> Hello all,
>
> Nancy has informed me that there is some renewed interest in our Generic
> Flash
> Access mechanism and I know people have been asking for it for a while now,
> here it finally is!
>
> Our design has evolved quite a bit since it was originally shared (original
> proposal can be found here
> https://lists.ozlabs.org/pipermail/openbmc/2016-October/005131.html for
> anyone
> interested), so I think it makes sense to start from scratch.
>
> I decided to add Corey Minyard, the Linux IPMI maintainer on this email as
> well
> because I think he will have some useful insights and this will also shed
> some
> light on the types of things we are trying to get our BMC to do as well as
> our
> conceptual approach.
>
> Also, before anyone says anything, I am aware that Generic Flash Access and
> Generic Transport Layer are not great names, but we have not thought of
> anything
> better yet. If anyone has any suggestions of better names, or any
> suggestions of
> any sort, please feel free to share.
>
> The primary motivation for Generic Flash Access is that there is no generic
> flash access mechanism that is available to all BMC based systems. LPC
> based
> mailbox is not available on all platforms and thus cannot be the sole
> basis for
> a generic mechanism. Furthermore, we have many types of operations that we
> would
> like to do with flash that do not fit well into the mailbox model.
>
Definitely the case. Any approach should be something that can be
implemented per system. Such that the data and the command & control can
go over separate channels. And that they don't imply direct flash access.
> Generic Flash Access is composed of two parts: the Generic Transport
> Layer, and
> the Generic Flash Access Protocol.
>
> The Generic Transport Layer (GTL) attempts to provide an abstract stream
> oriented interface between the BMC and the host CPU. It accomplishes this
> by
> piggy-backing on another pre-existing interface, currently IPMI, however,
> others
> are possible.
>
> Conceptually, GTL sort of looks like a net device. A user would open a
> "socket";
> in this case, would request a transaction id, and then would be able to
> write
> arbitrary data which would be sent as a request. The request would be
> routed to
> a handler on the opposite side which would parse the request and send a
> response, which could be then read by the requester. A request may be
> initialized by either the BMC or the host.
>
> GTL is essentially a packetization specification for arbitrary data, with
> bindings on other protocols. Given an arbitrary piece of data, GTL breaks
> the
> data up into packets:
>
> ```
> struct packet {
> uint8 transaction_id;
> uint8 control;
> uint32 seq;
> uint8 checksum;
> uint8 payload[];
> };
> ```
>
This packet doesn't have a payload size. Also is an 8-bit checksum
worthwhile instead of a 16-bit checksum? If the data is sufficiently
large, the 8-bit checksum loses a lot of value.
I'm less worried about the 32-bit for the sequence number as that'll work
for quite some time (famous last words). I don't think it's worth adding a
timestamp field as was done to address sequence overflow in TCP.
>
> where `transaction_id` allows packets to be associated with a given
> transaction
> so that multiple transactions can be in progress all at once. The most
> significant bit identifies whether the transaction was originated by the
> host or
> the BMC; if it is unset it is from the host, if it is set, it is from the
> BMC.
> In this way, each endpoint only has to keep track of allocations for its
> transaction IDs.
>
> `control` is a bit field:
>
> * 7:3 reserved for future use
> * 2:0 an enum:
> - 000 final packet: indicates that it is the last packet in a message.
> - 001 not final packet: indicates that there are subsequent packets in
> the
> message.
> - 010 retransmit request: requests that all packets after the specified
> packet, using the seq field, should be resent.
> - 011 ack: indicates that the last packet for a message was received.
> - 100 invalid state: indicates that a response packet was received for
> a
> unknown request.
> - 1xx reserved for future use: packets using invalid (reserved) control
> codes must be silently dropped.
>
> `seq` is a sequence number that is incremented for each packet, for a retry
> packet the sequence number is the last packet it received correctly, or 0
> if it
> has not received the first packet.
>
> `checksum` is a CRC 8 checksum that includes all of the data of the packet
> with
> the exception of itself. If the checksum in a received packet is invalid,
> the
> packet is dropped and a retransmit request with the previous sequence
> number is
> sent.
>
> GTL itself is divided into two layers: the transaction layer and the link
> layer.
>
> The transaction layer is common across all GTL implementations and is
> responsible for translating a message (a request or a response) into a
> series
> of packets and then translating packets back into messages. When the
> transaction
> layer receives a packet, it finds the rx queue associated with the packet's
> transaction ID and then checks to see if the packet's sequence number
> immediately follows the previous; if it does, it is added to the queue; if
> not,
> it is dropped and a retransmit request is sent with the last sequence
> number it
> correctly received or zero if no packets have been received for that
> transaction. If the packet is marked "no more data available", the message
> is
> assembled and passed to the user and an ack is sent. If there is no active
> transaction for the packet, meaning a response packet was received and
> there is
> no associated request, an invalid state packet is sent with the same
> transaction
> ID, and sequence number.
>
Just to verify, if you're sending a 32MB message, I don't think it's
prudent to rebuild the entire message in memory before passing it to the
receiving end of the "socket."
In the prototype implementation design, are you planning on having
something effectively call "recv" on some library implementing this, with
similar semantics to Berkeley sockets?
>
> Sending a packet must follow the above rules. It must retain all packets
> associated with a message until an ack is sent. If it receives an invalid
> state
> response, it must cancel the corresponding request, if one exists; it may,
> but
> is not required to take additional action. NB: the most likely explanation
> for
> an invalid state packet is that the endpoint being communicated with was
> reset.
> Other methods of detecting endpoint resets are encouraged and may be used
> to
> cancel requests.
>
> Although not required for correct functionality, each message will be
> selected
> according to a round robin strategy from which to send a packet; however,
> retransmit, ack, and invalid state packets will always be sent before other
> packets.
>
> The link layer is specific to the underlying protocol and is onlg
> responsible
> for exchanging packets between the BMC and the host. For IPMI, the host
> will
> continually send GTL IPMI OEM requests to the BMC; if there are any
> packets to
> send, the body of the IPMI request will contain the next packet to send as
> determined by the transaction layer; the BMC will then complete the
> request with
> the next packet as determined by its transaction layer. If either side has
> nothing to send, the corresponding body will simply be left empty. When
> the link
> layer is sending the final packet in a message; the next packet that it
> receives
> should be an ACK because ACK packets have higher priority than all other
> packets; if an ACK was not received, an error occurred and the link layer
> should
> retransmit the final packet unless requested to retransmit an earlier
> packet as
> directed by a retransmit request.
>
This made sense for me until the messages portion that's living on top of,
but really next to, or both. Can you elaborate a use-case more fully with
pseudo-code?
>
> The host's link layer will issue a GTL IPMI OEM request whenever it has
> packets
> available to send. It must also issue a GTL IPMI OEM request if the
> response to
> the previous request contained a packet. If neither of these conditions are
> true, the host will periodically send empty requests to see if the BMC has
> anything to send. We could also maybe take advantage of `SMS_ATN`, but I
> do not
> think that is important now.
>
> As I mentioned, the Generic Flash Access Protocol is built on top of GTL:
> if GTL
> is analogous to TCP, GFA is analogous to a REST interface. All it is is a
> set of
> messages mapped to a particular transaction id. GFA is just a set of
> protocol
> buffer messages:
>
This comparison is confusing to me. REST is a layer that is directly tied
to data with actions-driven endpoints. In the use-case you're describing,
-- well, I'm confused. I plan to upload the image over mechanism in big
data packets (64KB minus headers), and handle all C2 (command & control)
over IPMI OEM.
In the use-case I'm designing, there is a setup step that precedes being
able to even send the data. Is that all handled by the receiver of the
messages or should there be "ready to send" type messages?
>
> ```
> message RawFlashRequest {
> message Read {
> uint64 offset = 1;
> uint64 size = 2;
> }
>
> message Write {
> uint64 offset = 1;
> bytes data = 2;
> }
>
> Per the security of our environment, we are disallowing direct write to
the flash chip itself. We'll be staging the image and cryptographically
verifying it before flashing it. Presumably whatever mechanism is parsing
and handling the protobufs will handle that implementation.
To send the cryptographic signature, should a separate mechanism be used,
possibly also over GTL, or would this protobuf set be extended?
> message Info {
> }
>
> message Checksum {
> uint64 offset = 1;
> uint64 size = 2;
> }
>
I'm not familiar with protobufs but I don't a corresponding response to the
Checksum request type.
>
> bytes partition_guid = 1;
>
> oneof request {
> Read read = 2;
> Write write = 3;
> Info info = 4;
> Checksum checksum = 5;
> }
> }
>
> message RawFlashResponse {
> message Info {
> bytes guid = 1;
> uint64 block_size = 2;
> uint64 num_blocks = 3;
> bool read_only = 4;
> }
>
> oneof response { // Empty for Write
> Info info = 1; // from Info
> bytes data = 2; // from Read
> bytes checksum = 3; // from Checksum
> }
> }
>
> message PblogRequest {
> message GetBootnum {
> }
>
> message WriteEvent {
> bytes payload = 1;
> }
>
> message GetRawRegions {
> }
>
> oneof request {
> GetBootnum get_bootnum = 1;
> WriteEvent write_event = 2;
> GetRawRegions get_raw_regions = 3;
> }
> }
>
> message PblogResponse {
> oneof response { // Empty for write_event
> uint32 bootnum = 1; // from GetBootnum
> bytes raw_regions = 2; // from GetRawRegions
> }
> }
>
> message FirmwareUpdateRequest {
> bytes checksum = 1;
> bytes payload = 2;
> }
>
> message FirmwareUpdateResponse {
> }
>
> message GfaRequest {
> oneof request {
> RawFlashRequest raw_request = 1;
> PblogRequest pblog_request = 2;
> FirmwareUpdateRequest firmware_update_request = 3;
> }
> }
>
> message GfaResponse {
> bool success = 1;
> string error = 2; // Set when success == false
>
> oneof response {
> RawFlashResponse raw_response = 3;
> PblogResponse pblog_response = 4;
> FirmwareUpdateResponse firmware_update_response = 5;
> }
> }
> ```
>
Unlike GTL, which is a strictly defined protocol, the above GFA messages are
> more of a suggestion of operations that would be provided. The reason we
> chose
> protobufs is that protobufs, are an efficient encoding, and more
> importantly
> that they are extensable.
Are protobufs sufficiently widely accepted that using them here is
worthwhile? I don't see anything in the structures that can't be equally
compactly represented via normal data structures, and are just as
extensible.
> The reason that this is so important is that a flash
> chip is likely to have multiple partitions that each have its own security
> policy; for example, a CPU flash chip is likely to have a firmware image,
> which
> should only be updated with cryptographically signed updates, but it will
> also
> have an NVRAM variable section, which depending on the BIOS implementation
> could
> have variables that could effect the security of the BIOS and they may
> not; we
> also have our own system event log called PBLog; most would probably not be
> interested in this, but it is a good example of a message type that the
> host
> should be able to send us at anytime and the only policy we would likely
> have is
> rate limiting.
>
> In addition to allowing easy modification to meet the needs of different
> systems, using a mechanism that readily allows modification means that the
> protocol can be updated and can evolve over time.
>
> In summary, the only required messages are GfaRequest and GfaResponse,
> which can
> be modified to support arbitrary functionality.
>
> On a side note, all of the mechanisms provided above can be used to
> implement
> other arbitrary protocols; for example, we have implemented I2C passthrough
> using raw IPMI OEM messages, which have some short comings due to the
> length
> restrictions of IPMI messages; using the above mechanisms, GTL with
> protobufs,
> implementing I2C passthrough would be very easy. The possibility of
> implementing
> other protocols beyond GFA is why I decided to support multiple concurrent
> transactions.
>
> We are planning on implementing GTL within the kernel as its own
> framework. The
> IPMI based link layer implementation is intended to be implemented on top
> of the
> BMC side IPMI framework I have been discussing with Corey. I think my
> decision
> to put it here is intuitive since it is really supposed to be a hardware
> abstraction of sorts, but we can discuss it if it is not obvious.
>
> Currently, GFA has a userland tool to be used from the host and a BMC side
> daemon. The daemon is mostly complete; the tool is basically a wrapper
> around
> the protobufs listed above, so there is not much to do.
>
> Cheers!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20170914/32a36598/attachment-0001.html>
More information about the openbmc
mailing list