Inbound PCI and Memory Corruption

[David Laight]

> On Fri, Jul 19, 2013 at 6:46 AM, Gerhard Sittig <gsi at denx.de> wrote:
> > So:  No, not having to fiddle with DMA stuff when doing PCI need
> > not be a problem, it's actually expected.  But since a DMA engine
> > might be involved (that's just not under your command), the
> > accompanying problems may arise.  You may need to flush CPU
> > provided data upon write before telling an external entity to
> > access it, and may need to invalidate caches (to have data
> > re-fetched) before the CPU accesses what an external entity did
> > manipulate.  And this applies to both payload data as well as
> > management data (descriptors) if the latter apply to the former.
> 
> This is something I've been exploring today.  But what is unclear is
> _how_ to flush/invalidate the caches'.  I was going to tweak the
> driver to setup the descriptors, flush the cache, then enable the
> hardware (and when taking the device down, disable the hardware, flush
> the cache, then deallocate the descriptors).  But this is in the
> network code and it isn't obvious how to make this happen.

FWIW it is almost impossible to code for non-coherent descriptors
(even ignoring problems with speculative cache line reads).
You don't even want to try to do it except for hardware where you
can no choice.

The problem is that you have no control over the device writes
into the descriptors. In order not to lose the device writes
the cpu must not write to any cache lines that contain active
descriptors.

For the receive side this can be arranged by initialising cache
line sized blocks of descriptors (if the cache line write isn't
atomic you still have problems).

The send side is much more tricky: you either have to setup a
full cache line of descriptors or wait until the transmit is idle.

	David