Initial MCTP design proposal

[Supreeth Venkatesh]

On Fri, 2018-12-07 at 10:43 +0530, Deepak Kodihalli wrote:
> On 07/12/18 8:11 AM, Jeremy Kerr wrote:
> > Hi OpenBMCers!
> > 
> > In an earlier thread, I promised to sketch out a design for a MCTP
> > implementation in OpenBMC, and I've included it below.
> 
> 
> Thanks Jeremy for sending this out. This looks good (have just one 
> comment below).
> 
> Question for everyone : do you have plans to employ PLDM over MCTP?
Yes Deepak. we do eventually.

> 
> We are interested in PLDM for various "inside the box"
> communications 
> (at the moment for the Host <-> BMC communication). I'd like to
> propose 
> a design for a PLDM stack on OpenBMC, and would send a design
> template 
> for review on the mailing list in some amount of time (I've just
> started 
> with some initial sketches). I'd like to also know if others have 
> embarked on a similar activity, so that we can collaborate earlier
> and 
> avoid duplicate work.
Yes. Interested to collaborate.
Which portion of PLDM are you working on, other than base?
Platform Monitoring and Control?
Firmware Update?
BIOS Control andConfiguration?
SMBIOS Transfer?
FRU Data?
Redfish Device Enablement?

We are currently interested in Platform Monitoring and Control.


> 
> > This is roughly in the OpenBMC design document format (thanks for
> > the
> > reminder Andrew), but I've sent it to the list for initial review
> > before
> > proposing to gerrit - mainly because there were a lot of folks who
> > expressed interest on the list. I suggest we move to gerrit once we
> > get
> > specific feedback coming in. Let me know if you have general
> > comments
> > whenever you like though.
> > 
> > In parallel, I've been developing a prototype for the MCTP library
> > mentioned below, including a serial transport binding. I'll push to
> > github soon and post a link, once I have it in a
> > slightly-more-consumable form.
> > 
> > Cheers,
> > 
> > 
> > Jeremy
> > 
> > --------------------------------------------------------
> > 
> > # Host/BMC communication channel: MCTP & PLDM
> > 
> > Author: Jeremy Kerr <jk at ozlabs.org> <jk>
> > 
> > ## Problem Description
> > 
> > Currently, we have a few different methods of communication between
> > host
> > and BMC. This is primarily IPMI-based, but also includes a few
> > hardware-specific side-channels, like hiomap. On OpenPOWER hardware
> > at
> > least, we've definitely started to hit some of the limitations of
> > IPMI
> > (for example, we have need for >255 sensors), as well as the
> > hardware
> > channels that IPMI typically uses.
> > 
> > This design aims to use the Management Component Transport Protocol
> > (MCTP) to provide a common transport layer over the multiple
> > channels
> > that OpenBMC platforms provide. Then, on top of MCTP, we have the
> > opportunity to move to newer host/BMC messaging protocols to
> > overcome
> > some of the limitations we've encountered with IPMI.
> > 
> > ## Background and References
> > 
> > Separating the "transport" and "messaging protocol" parts of the
> > current
> > stack allows us to design these parts separately. Currently, IPMI
> > defines both of these; we currently have BT and KCS (both defined
> > as
> > part of the IPMI 2.0 standard) as the transports, and IPMI itself
> > as the
> > messaging protocol.
> > 
> > Some efforts of improving the hardware transport mechanism of IPMI
> > have
> > been attempted, but not in a cross-implementation manner so far.
> > This
> > does not address some of the limitations of the IPMI data model.
> > 
> > MCTP defines a standard transport protocol, plus a number of
> > separate
> > hardware bindings for the actual transport of MCTP packets. These
> > are
> > defined by the DMTF's Platform Management Working group; standards
> > are
> > available at:
> > 
> >    https://www.dmtf.org/standards/pmci
> > 
> > I have included a small diagram of how these standards may fit
> > together
> > in an OpenBMC system. The DSP numbers there are references to DMTF
> > standards.
> > 
> > One of the key concepts here is that separation of transport
> > protocol
> > from the hardware bindings; this means that an MCTP "stack" may be
> > using
> > either a I2C, PCI, Serial or custom hardware channel, without the
> > higher
> > layers of that stack needing to be aware of the hardware
> > implementation.
> > These higher levels only need to be aware that they are
> > communicating
> > with a certain entity, defined by an Entity ID (MCTP EID).
> > 
> > I've mainly focussed on the "transport" part of the design here.
> > While
> > this does enable new messaging protocols (mainly PLDM), I haven't
> > covered that much; we will propose those details for a separate
> > design
> > effort.
> > 
> > As part of the design, I have referred to MCTP "messages" and
> > "packets";
> > this is intentional, to match the definitions in the MCTP standard.
> > MCTP
> > messages are the higher-level data transferred between MCTP
> > endpoints,
> > which packets are typically smaller, and are what is sent over the
> > hardware. Messages that are larger than the hardware MTU are split
> > into
> > individual packets by the transmit implementation, and reassembled
> > at
> > the receive implementation.
> > 
> > A final important point is that this design is for the host <-->
> > BMC
> > channel *only*. Even if we do replace IPMI for the host interface,
> > we
> > will certainly need an IPMI interface available for external system
> > management.
> > 
> > ## Requirements
> > 
> > Any channel between host and BMC should:
> > 
> >   - Have a simple serialisation and deserialisation format, to
> > enable
> >     implementations in host firmware, which have widely varying
> > runtime
> >     capabilities
> > 
> >   - Allow different hardware channels, as we have a wide variety of
> >     target platforms for OpenBMC
> > 
> >   - Be usable over simple hardware implementations, but have a
> > facility
> >     for higher bandwidth messaging on platforms that require it.
> > 
> >   - Ideally, integrate with newer messaging protocols
> > 
> > ## Proposed Design
> > 
> > The MCTP core specification just provides the packetisation,
> > routing and
> > addressing mechanisms. The actual transmit/receive of those packets
> > is
> > up to the hardware binding of the MCTP transport.
> > 
> > For OpenBMC, we would introduce an MCTP daemon, which implements
> > the
> > transport over a configurable hardware channel (eg., Serial UART,
> > I2C or
> > PCI). This daemon is responsible for the packetisation and routing
> > of
> > MCTP messages to and from host firmware.
> > 
> > I see two options for the "inbound" or "application" interface of
> > the
> > MCTP daemon:
> > 
> >   - it could handle upper parts of the stack (eg PLDM) directly,
> > through
> >     in-process handlers that register for certain MCTP message
> > types; or
> 
> We'd like to somehow ensure (at least via documentation) that the 
> handlers don't block the MCTP daemon from processing incoming
> traffic. 
> The handlers might anyway end up making IPC calls (via D-Bus) to
> other 
> processes. The second approach below seems to alleviate this problem.
> 
> >   - it could channel raw MCTP messages (reassembled from MCTP
> > packets) to
> >     DBUS messages (similar to the current IPMI host daemons), where
> > the
> >     upper layers receive and act on those DBUS events.
> > 
> > I have a preference for the former, but I would be interested to
> > hear
> > from the IPMI folks about how the latter structure has worked in
> > the
> > past.
> > 
> > The proposed implementation here is to produce an MCTP "library"
> > which
> > provides the packetisation and routing functions, between:
> > 
> >   - an "upper" messaging transmit/receive interface, for tx/rx of a
> > full
> >     message to a specific endpoint
> > 
> >   - a "lower" hardware binding for transmit/receive of individual
> >     packets, providing a method for the core to tx/rx each packet
> > to
> >     hardware
> > 
> > The lower interface would be plugged in to one of a number of
> > hardware-specific binding implementations (most of which would be
> > included in the library source tree, but others can be plugged-in
> > too)
> > 
> > The reason for a library is to allow the same MCTP implementation
> > to be
> > used in both OpenBMC and host firmware; the library should be
> > bidirectional. To allow this, the library would be written in
> > portable C
> > (structured in a way that can be compiled as "extern C" in C++
> > codebases), and be able to be configured to suit those runtime
> > environments (for example, POSIX IO may not be available on all
> > platforms; we should be able to compile the library to suit). The
> > licence for the library should also allow this re-use; I'd suggest
> > a
> > dual Apache & GPL licence.
> > 
> > As for the hardware bindings, we would want to implement a serial
> > transport binding first, to allow easy prototyping in simulation.
> > For
> > OpenPOWER, we'd want to implement a "raw LPC" binding for better
> > performance, and later PCIe for large transfers. I imagine that
> > there is
> > a need for an I2C binding implementation for other hardware
> > platforms
> > too.
> > 
> > Lastly, I don't want to exclude any currently-used interfaces by
> > implementing MCTP - this should be an optional component of
> > OpenBMC, and
> > not require platforms to implement it.
> > 
> > ## Alternatives Considered
> > 
> > There have been two main alternatives to this approach:
> > 
> > Continue using IPMI, but start making more use of OEM extensions to
> > suit the requirements of new platforms. However, given that the
> > IPMI
> > standard is no longer under active development, we would likely end
> > up
> > with a large amount of platform-specific customisations. This also
> > does
> > not solve the hardware channel issues in a standard manner.
> > 
> > Redfish between host and BMC. This would mean that host firmware
> > needs a
> > HTTP client, a TCP/IP stack, a JSON (de)serialiser, and support for
> > Redfish schema. This is not feasible for all host firmware
> > implementations; certainly not for OpenPOWER. It's possible that we
> > could run a simplified Redfish stack - indeed, MCTP has a proposal
> > for a
> > Redfish-over-MCTP protocol, which uses simplified serialisation and
> > no
> > requirement on HTTP. However, this still introduces a large amount
> > of
> > complexity in host firmware.
> > 
> > ## Impacts
> > 
> > Development would be required to implement the MCTP transport, plus
> > any
> > new users of the MCTP messaging (eg, a PLDM implementation). These
> > would
> > somewhat duplicate the work we have in IPMI handlers.
> > 
> > We'd want to keep IPMI running in parallel, so the "upgrade" path
> > should
> > be fairly straightforward.
> > 
> > Design and development needs to involve potential host firmware
> > implementations.
> > 
> > ## Testing
> > 
> > For the core MCTP library, we are able to run tests there in
> > complete
> > isolation (I have already been able to run a prototype MCTP stack
> > through the afl fuzzer) to ensure that the core transport protocol
> > works.
> > 
> > For MCTP hardware bindings, we would develop channel-specific tests
> > that
> > would be run in CI on both host and BMC.
> > 
> > For the OpenBMC MCTP daemon implementation, testing models would
> > depend
> > on the structure we adopt in the design section.
> > 
> 
> Regards,
> Deepak
>