MCTP bridging with one physical port per demux daemon instance

[Andrew Jeffery]

Hi all,

First up, I've pushed a series to gerrit implementing bridging and routing in 
libmctp. If you're interested, please provide feedback:

https://gerrit.openbmc-project.xyz/q/project:openbmc/libmctp+topic:%2522routing%2522+(status:open)

The final patch in the series implements the provisional EID concept that we've 
been discussing on the list.

Back to the subject:

I've had a short think about bridging in the configuration where we have one 
mctp-demux-daemon instance per port. I'm not sure what plans you had with 
endpoint ID assignment, so assuming I have free reign with it, one proposal 
looks like below:


          demux A                             demux B
      +------------+                       +------------+
      |-----+      |                       |      +-----|
      +--+  |   +--+      demux socket     +--+   |  +--+
+-----+P0|10|   |P1+-----------------------+P1|   |11|P0+-----+
      +--+  |   +--+                       +--+   |  +--+
      |-----+      |                       |      +-----|
      +------------+                       +------------+

'demux A' and 'demux B' are our two separate processes. Figure 9 of DSP0236 
(v1.3.1) shows the different arrangements of endpoint assignments that are 
supported by MCTP, and the key element is that ports in a bridge are not 
required to be assigned endpoint IDs. In the diagram above I'm describing two 
ports associated with each process; the physical port P0 which we're 
associating with an mctp-demux-daemon instance, and P1 which we'll have as a 
port for a vendor-defined binding. By configuration, one of either 'demux A' or 
'demux B' connects to the other and establishes a "virtual bus" via P1. An 
endpoint ID is assigned to P0 in both instances (10 and 11), but neither 
instance assigns an EID to P1. Using null EID messages each side can send 'Get 
Endpoint ID' over the virtual bus linking the two daemons to bootstrap the 
(independent) route tables. From there, each side uses 'Get Routing Table 
Entries' to pull the route table state from the other, and update their own 
tables with the endpoints reachable through the remote.

In this configuration there's no change necessary to the generic libmctp code 
to handle forwarding packets between the different instances, it takes 
advantage of bridging and routing to do the job for us in concert with a 
vendor-defined binding to handle the socket.

A problem arises with how applications should interact with the separate 
daemons: Each application must connect to all the mctp-demux-daemon instances 
to receive their locally delivered packets.

To resolve this it might be possible to take advantage of the existing "bridge" 
implementation in libmctp which really functions as an adapter that glues 
together two ports of possibly different binding types. By inserting 
appropriate route entries packets arriving on either interface are simply 
forwarded to the other as there is no endpoint ID defined for the instance. 
With this approach you could do the bridging in a third process:

       adapter A                    demux               adapter B
      +------------+             +------------+            +------------+
      |            |             |------------|            |            |
      +--+      +--+             +--+  ||  +--+            +--+      +--+
+-----+P0|      |P1+-------------+P0|10||11|P1+------------+P1|      |P0+-----+
      +--+      +--+             +--+  ||  +--+            +--+      +--+
      |            |             |------------|            |            |
      +------------+             +------------+            +------------+

We'd probably have to do some tinkering to make sure broadcast and null EID 
messages are passed on. However, applications would then connect to just the 
bridge's domain socket to receive messages for both EID 10 and 11.

That's probably enough to spark some discussion.

Let me know what you think.

Andrew