Inbound PCI and Memory Corruption

[Peter LaDow]

On Wed, Jul 10, 2013 at 2:40 PM, Benjamin Herrenschmidt
<benh at kernel.crashing.org> wrote:
> Well, it should work, ....

I tried forcing NET_IP_ALIGN to 0, and I did see the DMA accesses
align on 32-bit boundaries with all the byte enables set.  However,
the memory corruption still occurred.

> but it's possible that there is some subtle bug on this specific Freescale SoC....

I looked through the Freescale errata
(http://www.freescale.com/files/32bit/doc/errata/MPC8349ECE.pdf) and
only 2 seem relevant:  PCI19 and DMA2 (the rest are either fixed in
our core version 3.1).

PCI19:  When using a dual-address cycle for inbound write accesses
with then IOS is full, the PCI overwrites the address for the IOS with
the new address from the bus.

DMA2: There can be corruption of the DMA data.  Examples are DAHTS is
8 bytes and the source port is a 32-bit PCI bus or the source memory
space i son the PCI bus and is not prefetchable.

I don't think PCI19 applies since no dual-address cycles are
generated.  From what I've seen, all the DMA addresses in the RX ring
descriptors are in the lower 32-bit address space.

I don't think DMA2 applies because it is for the DMA controller
specific to the 8349.  And since these transactions are not setup or
managed by the DMA controller...  At least I don't think they are
(unless dma_alloc and dma_map_single do something related to this).
My understanding is that in this case the PCI inbound registers are
configured and the DMA controller is not used.

> ...Did you correlate the corruption with one such packet ?
>
> Did you get any traces that show the flow that happens around a case of
> corruption ?

Not yet.  I'm having a difficult time syncing the PCI trace with the
kernel debug output.  And since the corruption may be detected well
after the actual corruption occurs, determine which DMA transfer
caused a corruption is difficult.

I'm still trying to gather more information.

Thanks,
Pete