Initial MCTP design proposal

[Supreeth Venkatesh]

On Mon, 2018-12-10 at 11:44 +0530, Deepak Kodihalli wrote:
> On 07/12/18 10:39 PM, Supreeth Venkatesh wrote:
> > On Fri, 2018-12-07 at 10:43 +0530, Deepak Kodihalli wrote:
> > > On 07/12/18 8:11 AM, Jeremy Kerr wrote:
> > > > Hi OpenBMCers!
> > > > 
> > > > In an earlier thread, I promised to sketch out a design for a
> > > > MCTP
> > > > implementation in OpenBMC, and I've included it below.
> > > 
> > > 
> > > Thanks Jeremy for sending this out. This looks good (have just
> > > one
> > > comment below).
> > > 
> > > Question for everyone : do you have plans to employ PLDM over
> > > MCTP?
> > 
> > Yes Deepak. we do eventually.
> 
> 
> Thanks for letting me know Supreeth!
My pleasure.

> 
> > > 
> > > We are interested in PLDM for various "inside the box"
> > > communications
> > > (at the moment for the Host <-> BMC communication). I'd like to
> > > propose
> > > a design for a PLDM stack on OpenBMC, and would send a design
> > > template
> > > for review on the mailing list in some amount of time (I've just
> > > started
> > > with some initial sketches). I'd like to also know if others have
> > > embarked on a similar activity, so that we can collaborate
> > > earlier
> > > and
> > > avoid duplicate work.
> > 
> > Yes. Interested to collaborate.
> > Which portion of PLDM are you working on, other than base?
> > Platform Monitoring and Control?
> > Firmware Update?
> > BIOS Control andConfiguration?
> > SMBIOS Transfer?
> > FRU Data?
> > Redfish Device Enablement?
> > 
> > We are currently interested in Platform Monitoring and Control.
> 
> 
> We're interested in each of these profiles for the BMC host 
> communications. Are you interested in Platform monitoring and
> control 
> for communications involving the BMC and the host firmware, or the
> BMC 
> and other devices?
BMC and the host firmware initially.

> 
> Also, I have been thinking about the usefulness/feasibility of a
> common 
> PLDM library (just the protocol piece - encoding and decoding PLDM 
> messages), so as to be able to share code between BMC and host
> firmware. 
> This of course sets expectations on the library based on OpenBMC and 
> various host firmware stacks. Do you have an opinion on this?
Glad that we are on the same page.
My thinking at this point is to come up with a generic standalone "C"
library which processes PLDM messages without regard to whether this
message contains payload for Sensors, firmware update, etc., so that it
can be ported to Host firmware if needed.


> 
> > > 
> > > > This is roughly in the OpenBMC design document format (thanks
> > > > for
> > > > the
> > > > reminder Andrew), but I've sent it to the list for initial
> > > > review
> > > > before
> > > > proposing to gerrit - mainly because there were a lot of folks
> > > > who
> > > > expressed interest on the list. I suggest we move to gerrit
> > > > once we
> > > > get
> > > > specific feedback coming in. Let me know if you have general
> > > > comments
> > > > whenever you like though.
> > > > 
> > > > In parallel, I've been developing a prototype for the MCTP
> > > > library
> > > > mentioned below, including a serial transport binding. I'll
> > > > push to
> > > > github soon and post a link, once I have it in a
> > > > slightly-more-consumable form.
> > > > 
> > > > Cheers,
> > > > 
> > > > 
> > > > Jeremy
> > > > 
> > > > --------------------------------------------------------
> > > > 
> > > > # Host/BMC communication channel: MCTP & PLDM
> > > > 
> > > > Author: Jeremy Kerr <jk at ozlabs.org> <jk>
> > > > 
> > > > ## Problem Description
> > > > 
> > > > Currently, we have a few different methods of communication
> > > > between
> > > > host
> > > > and BMC. This is primarily IPMI-based, but also includes a few
> > > > hardware-specific side-channels, like hiomap. On OpenPOWER
> > > > hardware
> > > > at
> > > > least, we've definitely started to hit some of the limitations
> > > > of
> > > > IPMI
> > > > (for example, we have need for >255 sensors), as well as the
> > > > hardware
> > > > channels that IPMI typically uses.
> > > > 
> > > > This design aims to use the Management Component Transport
> > > > Protocol
> > > > (MCTP) to provide a common transport layer over the multiple
> > > > channels
> > > > that OpenBMC platforms provide. Then, on top of MCTP, we have
> > > > the
> > > > opportunity to move to newer host/BMC messaging protocols to
> > > > overcome
> > > > some of the limitations we've encountered with IPMI.
> > > > 
> > > > ## Background and References
> > > > 
> > > > Separating the "transport" and "messaging protocol" parts of
> > > > the
> > > > current
> > > > stack allows us to design these parts separately. Currently,
> > > > IPMI
> > > > defines both of these; we currently have BT and KCS (both
> > > > defined
> > > > as
> > > > part of the IPMI 2.0 standard) as the transports, and IPMI
> > > > itself
> > > > as the
> > > > messaging protocol.
> > > > 
> > > > Some efforts of improving the hardware transport mechanism of
> > > > IPMI
> > > > have
> > > > been attempted, but not in a cross-implementation manner so
> > > > far.
> > > > This
> > > > does not address some of the limitations of the IPMI data
> > > > model.
> > > > 
> > > > MCTP defines a standard transport protocol, plus a number of
> > > > separate
> > > > hardware bindings for the actual transport of MCTP packets.
> > > > These
> > > > are
> > > > defined by the DMTF's Platform Management Working group;
> > > > standards
> > > > are
> > > > available at:
> > > > 
> > > >     https://www.dmtf.org/standards/pmci
> > > > 
> > > > I have included a small diagram of how these standards may fit
> > > > together
> > > > in an OpenBMC system. The DSP numbers there are references to
> > > > DMTF
> > > > standards.
> > > > 
> > > > One of the key concepts here is that separation of transport
> > > > protocol
> > > > from the hardware bindings; this means that an MCTP "stack" may
> > > > be
> > > > using
> > > > either a I2C, PCI, Serial or custom hardware channel, without
> > > > the
> > > > higher
> > > > layers of that stack needing to be aware of the hardware
> > > > implementation.
> > > > These higher levels only need to be aware that they are
> > > > communicating
> > > > with a certain entity, defined by an Entity ID (MCTP EID).
> > > > 
> > > > I've mainly focussed on the "transport" part of the design
> > > > here.
> > > > While
> > > > this does enable new messaging protocols (mainly PLDM), I
> > > > haven't
> > > > covered that much; we will propose those details for a separate
> > > > design
> > > > effort.
> > > > 
> > > > As part of the design, I have referred to MCTP "messages" and
> > > > "packets";
> > > > this is intentional, to match the definitions in the MCTP
> > > > standard.
> > > > MCTP
> > > > messages are the higher-level data transferred between MCTP
> > > > endpoints,
> > > > which packets are typically smaller, and are what is sent over
> > > > the
> > > > hardware. Messages that are larger than the hardware MTU are
> > > > split
> > > > into
> > > > individual packets by the transmit implementation, and
> > > > reassembled
> > > > at
> > > > the receive implementation.
> > > > 
> > > > A final important point is that this design is for the host <
> > > > -->
> > > > BMC
> > > > channel *only*. Even if we do replace IPMI for the host
> > > > interface,
> > > > we
> > > > will certainly need an IPMI interface available for external
> > > > system
> > > > management.
> > > > 
> > > > ## Requirements
> > > > 
> > > > Any channel between host and BMC should:
> > > > 
> > > >    - Have a simple serialisation and deserialisation format, to
> > > > enable
> > > >      implementations in host firmware, which have widely
> > > > varying
> > > > runtime
> > > >      capabilities
> > > > 
> > > >    - Allow different hardware channels, as we have a wide
> > > > variety of
> > > >      target platforms for OpenBMC
> > > > 
> > > >    - Be usable over simple hardware implementations, but have a
> > > > facility
> > > >      for higher bandwidth messaging on platforms that require
> > > > it.
> > > > 
> > > >    - Ideally, integrate with newer messaging protocols
> > > > 
> > > > ## Proposed Design
> > > > 
> > > > The MCTP core specification just provides the packetisation,
> > > > routing and
> > > > addressing mechanisms. The actual transmit/receive of those
> > > > packets
> > > > is
> > > > up to the hardware binding of the MCTP transport.
> > > > 
> > > > For OpenBMC, we would introduce an MCTP daemon, which
> > > > implements
> > > > the
> > > > transport over a configurable hardware channel (eg., Serial
> > > > UART,
> > > > I2C or
> > > > PCI). This daemon is responsible for the packetisation and
> > > > routing
> > > > of
> > > > MCTP messages to and from host firmware.
> > > > 
> > > > I see two options for the "inbound" or "application" interface
> > > > of
> > > > the
> > > > MCTP daemon:
> > > > 
> > > >    - it could handle upper parts of the stack (eg PLDM)
> > > > directly,
> > > > through
> > > >      in-process handlers that register for certain MCTP message
> > > > types; or
> > > 
> > > We'd like to somehow ensure (at least via documentation) that the
> > > handlers don't block the MCTP daemon from processing incoming
> > > traffic.
> > > The handlers might anyway end up making IPC calls (via D-Bus) to
> > > other
> > > processes. The second approach below seems to alleviate this
> > > problem.
> > > 
> > > >    - it could channel raw MCTP messages (reassembled from MCTP
> > > > packets) to
> > > >      DBUS messages (similar to the current IPMI host daemons),
> > > > where
> > > > the
> > > >      upper layers receive and act on those DBUS events.
> > > > 
> > > > I have a preference for the former, but I would be interested
> > > > to
> > > > hear
> > > > from the IPMI folks about how the latter structure has worked
> > > > in
> > > > the
> > > > past.
> > > > 
> > > > The proposed implementation here is to produce an MCTP
> > > > "library"
> > > > which
> > > > provides the packetisation and routing functions, between:
> > > > 
> > > >    - an "upper" messaging transmit/receive interface, for tx/rx
> > > > of a
> > > > full
> > > >      message to a specific endpoint
> > > > 
> > > >    - a "lower" hardware binding for transmit/receive of
> > > > individual
> > > >      packets, providing a method for the core to tx/rx each
> > > > packet
> > > > to
> > > >      hardware
> > > > 
> > > > The lower interface would be plugged in to one of a number of
> > > > hardware-specific binding implementations (most of which would
> > > > be
> > > > included in the library source tree, but others can be plugged-
> > > > in
> > > > too)
> > > > 
> > > > The reason for a library is to allow the same MCTP
> > > > implementation
> > > > to be
> > > > used in both OpenBMC and host firmware; the library should be
> > > > bidirectional. To allow this, the library would be written in
> > > > portable C
> > > > (structured in a way that can be compiled as "extern C" in C++
> > > > codebases), and be able to be configured to suit those runtime
> > > > environments (for example, POSIX IO may not be available on all
> > > > platforms; we should be able to compile the library to suit).
> > > > The
> > > > licence for the library should also allow this re-use; I'd
> > > > suggest
> > > > a
> > > > dual Apache & GPL licence.
> > > > 
> > > > As for the hardware bindings, we would want to implement a
> > > > serial
> > > > transport binding first, to allow easy prototyping in
> > > > simulation.
> > > > For
> > > > OpenPOWER, we'd want to implement a "raw LPC" binding for
> > > > better
> > > > performance, and later PCIe for large transfers. I imagine that
> > > > there is
> > > > a need for an I2C binding implementation for other hardware
> > > > platforms
> > > > too.
> > > > 
> > > > Lastly, I don't want to exclude any currently-used interfaces
> > > > by
> > > > implementing MCTP - this should be an optional component of
> > > > OpenBMC, and
> > > > not require platforms to implement it.
> > > > 
> > > > ## Alternatives Considered
> > > > 
> > > > There have been two main alternatives to this approach:
> > > > 
> > > > Continue using IPMI, but start making more use of OEM
> > > > extensions to
> > > > suit the requirements of new platforms. However, given that the
> > > > IPMI
> > > > standard is no longer under active development, we would likely
> > > > end
> > > > up
> > > > with a large amount of platform-specific customisations. This
> > > > also
> > > > does
> > > > not solve the hardware channel issues in a standard manner.
> > > > 
> > > > Redfish between host and BMC. This would mean that host
> > > > firmware
> > > > needs a
> > > > HTTP client, a TCP/IP stack, a JSON (de)serialiser, and support
> > > > for
> > > > Redfish schema. This is not feasible for all host firmware
> > > > implementations; certainly not for OpenPOWER. It's possible
> > > > that we
> > > > could run a simplified Redfish stack - indeed, MCTP has a
> > > > proposal
> > > > for a
> > > > Redfish-over-MCTP protocol, which uses simplified serialisation
> > > > and
> > > > no
> > > > requirement on HTTP. However, this still introduces a large
> > > > amount
> > > > of
> > > > complexity in host firmware.
> > > > 
> > > > ## Impacts
> > > > 
> > > > Development would be required to implement the MCTP transport,
> > > > plus
> > > > any
> > > > new users of the MCTP messaging (eg, a PLDM implementation).
> > > > These
> > > > would
> > > > somewhat duplicate the work we have in IPMI handlers.
> > > > 
> > > > We'd want to keep IPMI running in parallel, so the "upgrade"
> > > > path
> > > > should
> > > > be fairly straightforward.
> > > > 
> > > > Design and development needs to involve potential host firmware
> > > > implementations.
> > > > 
> > > > ## Testing
> > > > 
> > > > For the core MCTP library, we are able to run tests there in
> > > > complete
> > > > isolation (I have already been able to run a prototype MCTP
> > > > stack
> > > > through the afl fuzzer) to ensure that the core transport
> > > > protocol
> > > > works.
> > > > 
> > > > For MCTP hardware bindings, we would develop channel-specific
> > > > tests
> > > > that
> > > > would be run in CI on both host and BMC.
> > > > 
> > > > For the OpenBMC MCTP daemon implementation, testing models
> > > > would
> > > > depend
> > > > on the structure we adopt in the design section.
> > > > 
> > > 
> > > Regards,
> > > Deepak
> > > 
> 
>