scatter/gather DMA and cache coherency

[Mark A. Greer]

On Thu, Feb 16, 2006 at 05:51:20PM +1030, Phil Nitschke wrote:

> The problem is, that sometimes the data is corrupt (usually on the first
> transfer).  We've concluded that the problem is related to cache
> coherency.  The Artesyn 2.6.10 reference kernel (branched from the
> kernel at penguinppc.org) must be built with CONFIG_NOT_COHERENT_CACHE=y,
> as Artesyn have never successfully verified operation with hardware
> coherency enabled.
> My understanding is that their Marvel system controller (MV64460)
> supports cache snooping, but their Linux kernel support hasn't caught up
> yet.

It would have been useful if you had given the actual hardware you're
using.  It sure sounds like you're using a katana or a very similar
board.  Coherency can't work on the katana b/c there is a hw
erratum of the bridge that is not implemented on that board so
"CONFIG_NOT_COHERENT_CACHE=y" is the only option.  Fix the hardware
and the kernel will work with coherency enabled with a flip of a
switch (on the latest kernel).

For the record, don't assume that this is Artesyn's fault.  Artesyn says
that the erratum workaround is impractical and they may be right.
I don't know, I just write software...

> So if I understand my situation correctly, the device driver must use
> software-enforced coherency to avoid data corruption.  Is this correct?

It looks like Eugene is guiding you on this.  Listen to him.  I will add
that you should align your buffers on cacheline boundaries and make the
allocation sizes multiples of the cacheline size otherwise you could
have other data sharing the first and/or last cacheline of your buffers
and mess up your software cache mgmt.

Mark