Cache coherency question

[Martin, Tim]

> 
> Tim> I'm using an MVME6100 with Linux 2.6.14 and experiencing what I 
> Tim> think is a cache coherency problem.  An external PCIX master 
> Tim> performs a DMA transfer of a well known data pattern into SDRAM, 
> Tim> but when the device driver goes to look at the data it is mostly 
> Tim> okay, but occasionally has garbage.
> 
> If the hardware is reliable, I think you are right. The cache 
> coherency problem.

The hardware is reliable.

> 
> Tim> 1) Should the processor bus cache snooping actually work on an 
> Tim> MVME6100?
> Tim> E.g. is it correct that CONFIG_NOT_COHERENT_CACHE is undefined?
> 
> Yes, you can undefine the CONFIG_NOT_COHERENT_CACHE, but you 
> must make sure the host bridge did the snooping setup for PCI 
> inbound transaction.
> If you setup the snooping window, I think the host bridge 
> will assert /GLB signal to processor. The processor will 
> snoop the 60x/MPX bus to keep cache coherence.
> 

Ah.  This must be the problem.  I have a few PCI devices, and on one of
them it looked like snooping was working.  I just assumed the other
device was setup correctly.

> Also, you can define the CONFIG_NOT_COHERENT_CACHE, then you 
> are assuming The system has not hardware coherency. You need 
> use the software to keep the cache coherency.
> 

I tried this, and got compiler errors.

> 
> Tim> 2) If this really is a cache coherency problem, are there other 
> Tim> cache management functions available in ppc linux besides the 
> Tim> dma_cache_inv, dma_cache_wback, dma_cache_wback_inv 
> macros I should 
> Tim> be looking at?
> 
> I think it is a cache coherency problem, if you define the 
> CONFIG_NOT_COHERENT_CACHE You can get these functions, dma_map_single
> 

I added some inline assembly dcbi/dcbf (invalidate/flush) instructions
to the particular code in question, and the problems went away.  So
definitely a cache problem.  As I said above, defining
CONFIG_NOT_COHERENT_CACHE causes compiler errors, so I'm going to look
into this more.  I suppose whatever file implements the
include/linux/dma-mapping.h stuff isn't BSP specific, so its probably
just not being compiled in?  Will look into it.

Thanks for you help,
Tim