Ethernet over PCIe driver for Inter-Processor Communication

[Saravanan S]

Hi All,
         First of all thank you  all for taking your time out to reply



On Fri, Aug 23, 2013 at 3:59 AM, Ira W. Snyder <iws at ovro.caltech.edu> wrote:

> On Thu, Aug 22, 2013 at 02:43:38PM -0700, David Hawkins wrote:
> > Hi S.Saravanan,
> >
> > > I have a custom board  with four MPC8640 nodes connected over
> > > a transparent PCI express switch . In this configuration one node is
> > > configured as host(Root Complex) and others as agents(End Point). Thus
> > > the legacy PCI software works fine . However the mainline kernel lacks
> > > any standard support for Inter-processor communication over PCI. I am
> > > in the process of developing an Ethernet over  PCI driver for the same
> > > on the lines of rionet . However I am facing the following problems.
> > >
> > > a)   I can generate MSI interrupts from End Point to Root Complex over
> > > PCI . But the vice-versa is not possible . However i need a method to
> > > interrupt the End Point from the Root Complex to complete my driver.
> >
> > Root complex's would normally interrupt a device via a PCIe write
> > to a register in a BAR on the end-point (or in extended configuration
> > space registers depending on the hardware implementation).
>
> MPC8640 End point implements only the Type 0 header (Page 1116) . The
header implements five BARs (Page 1165).


> > > Only previous references I can find are this post
> > >
> http://www.mail-archive.com/linuxppc-dev@lists.ozlabs.org/msg25765.html
> > > However this uses doorbells and I think may not be possible in MPC8640.
> >
> > PCIe drivers need some way to interrupt the processor, so there must
> > be an option somewhere ... for example, what are the message register
> > interrupts intended for? See p479
> >
> > http://cache.freescale.com/files/32bit/doc/ref_manual/MPC8641DRM.pdf
> >
> > (Ira and myself have not used the MPC8640 so are not familiar with
> > its user manual).
>

   Message registers  are for interrupting the processor . A write to them
sends an interrupt to the processor .  Actually message registers are used
by the RC to enable interrupts to the processor when an EP sends an MSI
transaction to RC.In RC driver  i register separately for the msi
interrupts from all three EPs.

 To access them in the EP from the RC  i will have to set an inbound window
mapping the PIC register space in the EP  to the PCI mem space  assigned to
it . An inbound window maps a PCI address on the bus received by the PCIe
controller to a platform address. I will try that and let u know .

> >
> > > Any pointers on this issue and guidance on this driver development
> would
> > > be helpful .
> >
> > We use the Ethernet-over-PCI driver that Ira developed. Our next boards
> > will use an MPC8308, but we don't currently have any in a PCIe device
> > form-factor (just the MPC8038RDB), so he has not ported it to PCIe.
> >
> > Feel free to discuss your ideas for your PCIe driver (eg., why start
> > with rionet rather than Ira's driver), either on-list, or email Ira
> > and myself directly
>

To be frank with you there was no particular reason in starting with
rionet. Maybe because our board also had SRIO interface and we are using
rionet driver successfully. I had looked at Ira's driver later. I will
study that also and try   to come back with a skeleton for my driver.


>
> One further note. You might want to look at rproc/rpmsg and their virtio
> driver support. That seems to be where the Linux world is moving for
> inter-processor communications. See for example the ARM CPUs interfacing
> with DSPs.
>
> Ira
>

I will study that as i am not familiar with virtio .

Regards,

S.Saravanan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20130825/13447d1c/attachment-0001.html>