AW: PowerPC PCI DMA issues (prefetch/coherency?)

[Adam Zilkie]

Hi Prodyut,

Yes, I am using the 440EPx (same as the sequoia board). 
Our ideDriver is DMA'ing blocks of 192-byte data over the PCI bus (using
the Sil0680A PCI-IDE bridge). Most of the DMA's (depending on timing)
end up being partially corrupted when we try to parse the data in the
virtual page. We have confirmed the data is good before the PCI-IDE
bridge. We are creating two 8K pages and map them to physical DMA memory
using single-entry scatter/gather structs. When a DMA block is
corrupted, we see a random portion of it (always a multiple of 16byte
cache lines) is overwritten with old data from the last time the buffer
was used. 

Regards,
Adam

On Thu, 2009-09-03 at 13:27 -0700, Prodyut Hazarika wrote:
> Hi Adam,
> 
> > Are you sure there is L2 cache on the 440?
> 
> It depends on the SoC you are using. SoC like 460EX (Canyonlands board)
> have L2Cache.
> It seems you are using a Sequoia board, which has a 440EPx SoC. 440EPx
> has a 440 cpu core, but no L2Cache.
> Could you please tell me which SoC you are using?
> You can also refer to the appropriate dts file to see if there is L2C.
> For example, in canyonlands.dts (460EX based board), we have the L2C
> entry.
>         L2C0: l2c {
>               ...
>         }
> 
> >I am seeing this problem with our custom IDE driver which is based on 
> >pretty old code. Our driver uses pci_alloc_consistent() to allocate the
> 
> >physical DMA memory and alloc_pages() to allocate a virtual page. It 
> >then uses pci_map_sg() to map to a scatter/gather buffer. Perhaps I 
> >should convert these to the DMA API calls as you suggest.
> 
> Could you give more details on the consistency problem? It is a good
> idea to change to the new DMA APIs, but pci_alloc_consistent() should
> work too
> 
> Thanks
> Prodyut	
> 
> On Thu, 2009-09-03 at 19:57 +1000, Benjamin Herrenschmidt wrote:
> > On Thu, 2009-09-03 at 09:05 +0100, Chris Pringle wrote:
> > > Hi Adam,
> > > 
> > > If you have a look in include/asm-ppc/pgtable.h for the following
> section:
> > > #ifdef CONFIG_44x
> > > #define _PAGE_BASE    (_PAGE_PRESENT | _PAGE_ACCESSED |
> _PAGE_GUARDED)
> > > #else
> > > #define _PAGE_BASE    (_PAGE_PRESENT | _PAGE_ACCESSED)
> > > #endif
> > > 
> > > Try adding _PAGE_COHERENT to the appropriate line above and see if
> that 
> > > fixes your issue - this causes the 'M' bit to be set on the page
> which 
> > > sure enforce cache coherency. If it doesn't, you'll need to check
> the 
> > > 'M' bit isn't being masked out in head_44x.S (it was originally
> masked 
> > > out on arch/powerpc, but was fixed in later kernels when the cache 
> > > coherency issues with non-SMP systems were resolved).
> > 
> > I have some doubts about the usefulness of doing that for 4xx. AFAIK,
> > the 440 core just ignores M.
> > 
> > The problem lies probably elsewhere. Maybe the L2 cache coherency
> isn't
> > enabled or not working ?
> > 
> > The L1 cache on 440 is simply not coherent, so drivers have to make
> sure
> > they use the appropriate DMA APIs which will do cache flushing when
> > needed.
> > 
> > Adam, what driver is causing you that sort of problems ?
> > 
> > Cheers,
> > Ben.
> > 
> > 
-- 
Adam Zilkie
Software Designer,
International Datacasting Corp.

This message and the documents attached hereto are intended only for the addressee and may contain privileged or confidential information. Any unauthorized disclosure is strictly prohibited. If you have received this message in error, please notify us immediately so that we may correct our internal records. Please then delete the original message. Thank you.