BMC redundancy

[Deepak Kodihalli]

On 29/01/18 9:22 pm, Brad Bishop wrote:
> I know we have a lot of work to do with the basics before tackling something
> like supporting multiple BMCs in a single system, but its never too early to
> brainstorm.
> 
> Quick community poll:  Please share any thoughts you may have around supporting
> systems with multiple BMCs.  Does your organization care?  Thoughts on how it
> could/should be done?  System designs that are a non-starter for OpenBMC?
> 
> Would love to see some brain-dumps here.
> 
> thx - brad
> 

I had some thoughts/questions on messaging protocols that multiple BMCs 
within the same system (potentially managing their own sleds) may employ 
to talk to each other. I'm not getting into how the BMCs discover and 
address each other. I see two broad use-cases for a peer BMC 
communication in such a system :

- A group operation : where an external agent talks to a single BMC (for 
the want of not having to query each BMC), but wants an aggregate of 
information. For eg the agent wants to collect error logs, it talks to a 
single BMC but expects that BMC would give back an aggregation of errors 
across all peer BMCs.

- There may be reasons for a BMC to itself initiate communication with 
one or more peers.


I was interested in brainstorming what protocol should the BMCs use for 
peer communication. What I had in mind though was, can this be done by 
leveraging the existing D-Bus/REST API, as opposed to employing a new 
protocol (I've looked at some such as Protobufs/gRPC, MQTT, etc).


So several of the existing OpenBMC apps implement specific D-Bus 
services. What does it take to make remote D-Bus calls to such apps?
- It doesn't look like the D-Bus spec or libdbus officially has anything 
for D-Bus across computers. There are some good notes at 
https://www.freedesktop.org/wiki/Software/DBusRemote/.
- There are ways to achieve this via Qt D-Bus, but it would involve some 
amount tweaking with the D-Bus configs.
- I'm not aware of any open/active project implementing remote D-Bus.
- Thoughts on doing remote D-Bus over WebSockets? For instance one could 
write a serialized D-Bus message over a WebSocket that has been 
established between two BMCs. Since WebSockets are full-duplex, they 
could also be used to receive events/D-Bus signals from peer BMCs 
asynchronously.


How about using REST for the peer to peer communication? Although this 
would mean each of the peer BMCs in the system would need a REST server 
and client implementation. The 'group operation' I've mentioned above 
seems a logical fit to do this, presuming an external agent would be 
using a REST API to talk to one of the BMCs, and the REST server 
implementation on that BMC has to propagate/multicast that API to all 
its peers, and then aggregate the responses.
If it's a D-Bus app wanting to talk to a counterpart app on a peer BMC, 
then it would have to use the REST client to send an https request.


Thanks,
Deepak