<div dir="ltr">Hi All ,<br><div class="gmail_extra"> <br><br><div class="gmail_quote">On Mon, Aug 26, 2013 at 4:08 AM, David Hawkins <span dir="ltr"><<a href="mailto:dwh@ovro.caltech.edu" target="_blank">dwh@ovro.caltech.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hi S.Saravanan,<div><br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Root complex's would normally interrupt a device via a PCIe write<br>
to a register in a BAR on the end-point (or in extended configuration<br>
space registers depending on the hardware implementation).<br>
</blockquote>
<br>
MPC8640 End point implements only the Type 0 header (Page 1116) . The<br>
header implements five BARs (Page 1165).<br>
</blockquote>
<br></div>
One of those BARs typically provides access to the PowerPC memory<br>
mapped registers (or at least a 1MB window onto those registers).<br>
This is how your root complex can write to some form of messaging<br>
register.<br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
PCIe drivers need some way to interrupt the processor, so there must<br>
be an option somewhere ... for example, what are the message register<br>
interrupts intended for? See p479<br>
<br>
<a href="http://cache.freescale.com/files/32bit/doc/ref_manual/MPC8641DRM.pdf" target="_blank">http://cache.freescale.com/files/32bit/doc/ref_manual/MPC8641DRM.pdf</a><br>
<br>
(Ira and myself have not used the MPC8640 so are not familiar with<br>
its user manual).<br>
</blockquote>
<br></div>
Message registers are for interrupting the processor. A write to<br>
them sends an interrupt to the processor. Actually message registers<div><br>
are used by the RC to enable interrupts to the processor when an EP<br></div>
sends an MSI transaction to RC. In RC driver i register separately for<div><br>
the msi interrupts from all three EPs.<br>
</div></blockquote>
<br>
This is pretty much what you are looking for then right? </blockquote><div><br><br><div>I successfully mapped the Programmable Interrupt Controller registers in the EP to the PCI space
. Thus now I can write the shared message interrupt registers in the EP from
the RC over PCI . But I am facing the following problems now .<br><br></div><div>1) In my driver at EP, to register
for this interrupt I need to know the hardware irq number but I can't find any interrupt number assigned by the PIC for the messages interrupt sources(Page 451 , MPC8641DRM manual).<br></div><div>2) Otherwise i need to get the virtual irq number assigned by kernel corresponding to
the message interrupt . I am unable to find a method to get this also. <br><br>In the RC side driver i get the virtual irq number after calling pci_enable_msi() which is straightforward. <br></div>I studied the RC code which sets up shared message interrupts (Page 481, MPC manual) for PCI MSI interrupts . When msi is enabled the "arch_setup_msi_irqs()" is called leading to the fsl_setup_msi_irqs() (<a href="http://lxr.free-electrons.com/source/arch/powerpc/sysdev/fsl_msi.c?v=3.7#L151" target="_blank">http://lxr.free-electrons.com/source/arch/powerpc/sysdev/fsl_msi.c?v=3.7#L151</a>) . In this function the virtual irq no is obtained as below:<br>
</div><div><br>
<p style="margin-bottom:0.0001pt;line-height:normal"><i><span style="font-size:12pt;font-family:"Times New Roman","serif"">virq
= irq_create_mapping(msi_data->irqhost, hwirq);</span></i></p>
<p><i><span style="line-height:115%;font-size:12pt;font-family:"Times New Roman","serif""> </span></i></p>In the above function the hardware irq number is same as the value written into the Shared Message Signaled Interrupt Index Register (Page 482) which is strange. Further these functions are called in the RC during pci_probe at boot time or when pci_enable_msi() is called . Thus there is a always a PCI slave device context to it. However I require to do it in the EP which has no pci probing nor any pci device reference whatsoever as it a slave. Is this approach right ?<pre>
</pre></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
The end-points interrrupt the root-complex using PCIe MSI interrupts,<br>
whereas the root-complex interrupts an end-point by writing directly<br>
to its MSI interrupt.<div><br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
To access them in the EP from the RC i will have to set an inbound<br>
window mapping the PIC register space in the EP to the PCI mem space<br>
assigned to it . An inbound window maps a PCI address on the bus<br>
received by the PCIe controller to a platform address. I will try that<br>
and let u know .<br>
</blockquote>
<br></div>
Right, as I comment above, one of the BARs typically exposes the PowerPC<br>
internal registers.<div><br>
<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Feel free to discuss your ideas for your PCIe driver (eg., why start<br>
with rionet rather than Ira's driver), either on-list, or email Ira<br>
and myself directly<br>
</blockquote>
<br>
To be frank with you there was no particular reason in starting with<br>
rionet. Maybe because our board also had SRIO interface and we are using<br>
rionet driver successfully. I had looked at Ira's driver later. I will<br>
study that also and try to come back with a skeleton for my driver.<br>
</blockquote>
<br></div>
Its always a good idea to discuss different options, and to stub out<br>
drivers or create minimal (but functional) drivers. That way you'll<br>
be able to see how similar your new driver is to other drivers, and<br>
you'll quickly discover if there is a hardware feature in the<br>
existing driver that you cannot emulate (eg., some SRIO feature<br>
used by the rionet driver).<br></blockquote><div><br></div><div>Right now I am trying a very primitive driver just to check the feasibility of bi-directional communication between the RC and the EP. Once this is established I will be in a better position to get inputs on making it a more effective one. <br>
</div><br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
One further note. You might want to look at rproc/rpmsg and their virtio<br>
driver support. That seems to be where the Linux world is moving for<br>
inter-processor communications. See for example the ARM CPUs interfacing<br>
with DSPs.<br>
</blockquote>
<br></div><div>
I will study that as i am not familiar with virtio .<br>
</div></blockquote>
<br>
Follow Ira's advice. Talk to the guys working on virtio, tell them what<br>
you are trying to do. They'll likely have good advice for you.<br>
<br>
Good luck!<br>
<br>
Cheers,<br>
Dave<br>
<br>
<br>
</blockquote></div><br></div><div class="gmail_extra">Warm Regards,<br><br></div><div class="gmail_extra">S.Saravanan<br></div><div class="gmail_extra"><br></div></div>