[Cbe-oss-dev] [RFC 4/9] AXON - Ethernet over PCI-E driver

[Benjamin Herrenschmidt]

> It is a bit more complex than that. I can explain here how it works. For each 
> SKB to transfer there is a MBX dialog between the 2 sides of the PCI-E link.
> 
> It goes as follow:
> 
> /*
>  * Ethernet frame exchanged Protocol
>  *
>  *    Emitter                                         Receiver
>  * 1 -  Linux ask for transmitted a skb
>  * 2 -  Emitter build a message with the
>  *      SKB PLB addr and size
>  *      AXON_NIC_SMS_SKB_AVAIL  ---------->  3 - Receiver allocate a sk_buff
>  *                                               of the requested size, create
>  *                                               a DMA read req to xfer the 
>  *                                               data with a message notifying
>  *                                               the completion of the xfer. 
> It
>  *                                               also ask to be notified when
>  *                                               the transfer is complete.
>  *
>  * 4 - The Emitter free up the  <---------      AXON_NIC_SMS_SKB_XFERD  
>  *     sk_buff                       |
>  *                                   ---->  5 - The receiver propagate the 
>  *                                              sk_buff up to the stack 
>  *
>  * If something does wrong in step 3, the receiver send a cancel message.
>  *                                                  
>  */
> 
> So for each SKB the receiver gets 2 interrupts (+ payload) and the emitter get 
> one. It might not sound like the most efficient protocol but we do need some 
> messagery to synchronize resource usage and SKB management.

It's in fact extremely inefficient :-)

You should really implement that differently, with 2 rings of
descriptors, like normal hardware does, and proper interrupt mitigation
so you can set a threshold on interrupt emission on each side.

Ben.