Ethernet over PCIe driver for Inter-Processor Communication

[Ira W. Snyder]

On Thu, Aug 22, 2013 at 02:43:38PM -0700, David Hawkins wrote:
> Hi S.Saravanan,
> 
> > I have a custom board  with four MPC8640 nodes connected over
> > a transparent PCI express switch . In this configuration one node is
> > configured as host(Root Complex) and others as agents(End Point). Thus
> > the legacy PCI software works fine . However the mainline kernel lacks
> > any standard support for Inter-processor communication over PCI. I am
> > in the process of developing an Ethernet over  PCI driver for the same
> > on the lines of rionet . However I am facing the following problems.
> >
> > a)   I can generate MSI interrupts from End Point to Root Complex over
> > PCI . But the vice-versa is not possible . However i need a method to
> > interrupt the End Point from the Root Complex to complete my driver.
> 
> Root complex's would normally interrupt a device via a PCIe write
> to a register in a BAR on the end-point (or in extended configuration
> space registers depending on the hardware implementation).
> 
> > Only previous references I can find are this post
> > http://www.mail-archive.com/linuxppc-dev@lists.ozlabs.org/msg25765.html
> > However this uses doorbells and I think may not be possible in MPC8640.
> 
> PCIe drivers need some way to interrupt the processor, so there must
> be an option somewhere ... for example, what are the message register
> interrupts intended for? See p479
> 
> http://cache.freescale.com/files/32bit/doc/ref_manual/MPC8641DRM.pdf
> 
> (Ira and myself have not used the MPC8640 so are not familiar with
> its user manual).
> 
> > Any pointers on this issue and guidance on this driver development would
> > be helpful .
> 
> We use the Ethernet-over-PCI driver that Ira developed. Our next boards
> will use an MPC8308, but we don't currently have any in a PCIe device
> form-factor (just the MPC8038RDB), so he has not ported it to PCIe.
> 
> Feel free to discuss your ideas for your PCIe driver (eg., why start
> with rionet rather than Ira's driver), either on-list, or email Ira
> and myself directly.
> 

One further note. You might want to look at rproc/rpmsg and their virtio
driver support. That seems to be where the Linux world is moving for
inter-processor communications. See for example the ARM CPUs interfacing
with DSPs.

Ira