Multiple BMCs in a system: IPMB? Redfish? MCTP?

[Vijay Khemka]

From: openbmc <openbmc-bounces+vijaykhemka=fb.com at lists.ozlabs.org> on behalf of Nancy Yuen <yuenn at google.com>
Date: Thursday, May 7, 2020 at 12:47 PM
To: Andrew Jeffery <andrew at aj.id.au>
Cc: OpenBMC Maillist <openbmc at lists.ozlabs.org>
Subject: Re: Multiple BMCs in a system: IPMB? Redfish? MCTP?


On Mon, May 4, 2020 at 3:31 AM Andrew Jeffery <andrew at aj.id.au<mailto:andrew at aj.id.au>> wrote:


On Thu, 30 Apr 2020, at 17:08, Deepak Kodihalli wrote:
> On 30/04/20 4:21 am, Nancy Yuen wrote:
> > I've talked with some people a while back (long while back) about
> > multiple BMCs in a system.  Either for redundancy or managing separate
> > parts of a system.   I'm wondering what other people are thinking in
> > this area if at all.
> >
> > We are considering similar designs and I'm looking into options for
> > BMC-BMC communications.  Some BMCs may not be externally
> > accessible. Here are some options that we've looked at:
> >
> >  1. i2c/IPMB
> >  2. usbnet/Redfish
> >  3. i2c/MCTP/PLDM or something else?
> >  4. internal network via switch chip/Redfish or MCTP
> >
> >   I'd like to reduce our use of IPMI so I want to avoid (1).
> >
> > ----------
> > Nancy
>
> Hi Nancy,
>
> I think it depends on whether the BMCs need to talk to each other for
> platform management, or if they manage their own hosts and we need one
> of the BMCs to broadcast out of band requests and aggregate responses.
>
> For the former I think PLDM over MCTP could be a good fit. This is more
> of an "inband" use-case in my opinion so I'm not sure if Redfish is well
> suited. For the latter, a Redfish based aggregation is a good option.

I think this looks like the right approach. Certainly some thought would be
required in designing the MCTP networks given the (intentionally) limited
number of endpoint IDs.

Andrew

Hi Andrew, Deepak,

Sorry for the delayed response!

For the purposes of this discussion I have BMCs managing their own host: BMC3<--->BMC2<--->BMC1/controller<--> external.  I wrote BMC1 but there maybe some other controller there that isn't strictly a BMC.  BMC2 would be "passing through" information between BMC1 and BMC3.  But I don't necessarily want BMC2 to do any form of aggregation from information from BMC3.  For BMC1/Controller and BMC 2 we are thinking Redfish.

In my case I'm interested in what ways people are thinking BMC2 and BMC3 should be connected and speaking to each other. Is MCTP/PLDM where everyone is leaning?

Nancy,
What is connectivity between BMC1, BMC2 and BMC3. Are these exposing any out of band connections. For security reasons, aggregator should use inband connectivity between BMCs and as Deepak said MCTP is good option for inband.



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20200508/c5072c8b/attachment.htm>