MCTP bridging with one physical port per demux daemon instance

[Thomaiyar, Richard Marian]

Hi Andrew,

Reviewed the routing series and provided comments. Overall don't find 
any major issues, and we are fine with having 2 separate demux with each 
handling the tx/rx or single demux. Each having it's own advantage and 
disadvantage as with single demux handling of multiple physical 
interface packet is going to be delayed (i.e, when one Tx is in effect 
another one is going to wait), as we are not starting multiple process 
to handle as per the interface as of today.

With multiple demux we will come up with the issue of client depending 
on multiple daemons for the same.

Regards,

Richard

On 7/2/2020 10:28 AM, Andrew Jeffery wrote:
> Hi all,
>
> First up, I've pushed a series to gerrit implementing bridging and routing in
> libmctp. If you're interested, please provide feedback:
>
> https://gerrit.openbmc-project.xyz/q/project:openbmc/libmctp+topic:%2522routing%2522+(status:open)
>
> The final patch in the series implements the provisional EID concept that we've
> been discussing on the list.
>
> Back to the subject:
>
> I've had a short think about bridging in the configuration where we have one
> mctp-demux-daemon instance per port. I'm not sure what plans you had with
> endpoint ID assignment, so assuming I have free reign with it, one proposal
> looks like below:
>
>
>            demux A                             demux B
>        +------------+                       +------------+
>        |-----+      |                       |      +-----|
>        +--+  |   +--+      demux socket     +--+   |  +--+
> +-----+P0|10|   |P1+-----------------------+P1|   |11|P0+-----+
>        +--+  |   +--+                       +--+   |  +--+
>        |-----+      |                       |      +-----|
>        +------------+                       +------------+
>
> 'demux A' and 'demux B' are our two separate processes. Figure 9 of DSP0236
> (v1.3.1) shows the different arrangements of endpoint assignments that are
> supported by MCTP, and the key element is that ports in a bridge are not
> required to be assigned endpoint IDs. In the diagram above I'm describing two
> ports associated with each process; the physical port P0 which we're
> associating with an mctp-demux-daemon instance, and P1 which we'll have as a
> port for a vendor-defined binding. By configuration, one of either 'demux A' or
> 'demux B' connects to the other and establishes a "virtual bus" via P1. An
> endpoint ID is assigned to P0 in both instances (10 and 11), but neither
> instance assigns an EID to P1. Using null EID messages each side can send 'Get
> Endpoint ID' over the virtual bus linking the two daemons to bootstrap the
> (independent) route tables.
[Richard]: Yes, connecting ports must not have EID. Also, phyiscal 
interface facing port can or can not have EID as per the MCTP 
specification. But in either case, we must not advertise about it's 
presence to the other side of the bus. Say in this diagram, devices 
behind P0 EID 10 of demux A, must not indicate about EID 11 in the 
routing table, but will indicate about all the EID's behind P0 EID 11 of 
demux B.
>   From there, each side uses 'Get Routing Table
> Entries' to pull the route table state from the other, and update their own
> tables with the endpoints reachable through the remote.
>
> In this configuration there's no change necessary to the generic libmctp code
> to handle forwarding packets between the different instances, it takes
> advantage of bridging and routing to do the job for us in concert with a
> vendor-defined binding to handle the socket.
>
> A problem arises with how applications should interact with the separate
> daemons: Each application must connect to all the mctp-demux-daemon instances
> to receive their locally delivered packets.
>
> To resolve this it might be possible to take advantage of the existing "bridge"
> implementation in libmctp which really functions as an adapter that glues
> together two ports of possibly different binding types. By inserting
> appropriate route entries packets arriving on either interface are simply
> forwarded to the other as there is no endpoint ID defined for the instance.
> With this approach you could do the bridging in a third process:
>
>         adapter A                    demux               adapter B
>        +------------+             +------------+            +------------+
>        |            |             |------------|            |            |
>        +--+      +--+             +--+  ||  +--+            +--+      +--+
> +-----+P0|      |P1+-------------+P0|10||11|P1+------------+P1|      |P0+-----+
>        +--+      +--+             +--+  ||  +--+            +--+      +--+
>        |            |             |------------|            |            |
>        +------------+             +------------+            +------------+
>
> We'd probably have to do some tinkering to make sure broadcast and null EID
> messages are passed on. However, applications would then connect to just the
> bridge's domain socket to receive messages for both EID 10 and 11.
[Richard]: Special EID's must not be forwarded or routed. i.e. Both NULL 
EID and Broadcast EID's packets arising from P0 of adapter A must not be 
routed / forwarded to P0 of Adapter B
> That's probably enough to spark some discussion.
>
> Let me know what you think.
>
> Andrew