PCIe Access - achieve bursts without DMA

Gabriel Paubert paubert at iram.es
Fri Jan 31 23:31:57 EST 2014

On Thu, Jan 30, 2014 at 12:20:21PM +0000, Moese, Michael wrote:
> Hello PPC-developers,
> I'm currently trying to benchmark access speeds to our PCIe-connected IP-cores
> located inside our FPGA. On x86-based systems I was able to achieve bursts for
> both read and write access. On PPC32, using an e500v2, I had no success at all 
> so far. 
> I tried using ioremap_wc(), like I did on x86, for writing, and it only results in my
> writes just being single requests, one after another.

I believe that on PPC, write-combine is directly mapped to nocache. I can't remember
if there is a writethrough option for ioremap (but adding it would probably be
relaively easy).

> For reads, I noticed I could not ioremap_cache() on PPC, so I used simple ioremap()
> here. 

You might be able to use ioremap_cache and using direct cache control instruction
(dcbf/dcbi) to achieve your goals. This becomes similar to handling machines with 
no hardware cache coherency. You have to know the hardware cache line size to make
this work.

This said, it might be better to mark the memory as guarded and non-coherent 
(WIMG=0000), I don't know what ioremap_cache does for the MG bits and don't
have the time to look it up right now.

> I used several ways to read from the device, from simple readl(),memcpy_from_io(), 
> memcpy()  to cacheable_memcpy() - with no improvements.  Even when just issuing
> a batch of prefetch()-calls for all the memory to read did not result in read bursts.

If the device data you want to read is supposed to be cacheable (which means basically
that the data does not change unexpectedly under you, i.e., is not as volatile as
a typical device I/O register), you don't want to use readl() which adds some
synchronization to the read.

Prefetch only works on writeback memory, maybe writethrough, expecting it to work on
cache-inhibited memory is contradictory.


More information about the Linuxppc-dev mailing list