Problems with DMA from user space on MPC834x (Cache coherency?)

[Manish Joshi]

Yeah it looks like a cache coherency problem from the first look.
You may try enabling snooping on the DMA channel if MPC824X supports it. Check in Mode register setting.
Else you may want to use flush_dcache_range(start, last) functions and see if it helps.
 
----- Original Message ----
From: Lauri Ehrenpreis <lauri.ehrenpreis at liewenthal.ee>
To: linuxppc-embedded at ozlabs.org
Sent: Wednesday, October 11, 2006 7:20:03 AM
Subject: Problems with DMA from user space on MPC834x (Cache coherency?)


Hi!

I have a problem with DMA from user space on a platform powered by MPC834x
processor (which has powerpc e300 core inside). Our linux kernel version is
2.6.17.

My user space program does something like this:

file_fd = open("/disk/file", O_RDONLY);
result = read(file_fd, page_aligned_buf, len);
call_driver_ioctl_which_performs_dma();

while /disk is mounted to a partition residing on USB memory stick or SD
card.
page_aligned_buf starts from page boundary and contains enough full pages
for
the data (so I can allways map full page with dma_map_page in kernel).

The buffer address and data length will then be passed to a driver, which
calls
get_user_pages, then maps each page with dma_map_page and tells a PCI
device
to start reading from that page:

...
down_read(&current->mm->mmap_sem);
result = get_user_pages(current, current->mm, page_aligned_buf, nr_pages,
0, 0, pages, NULL);
up_read(&current->mm->mmap_sem);

for (i = 0; i < nr_pages; i++) {
    find_data_checksum(pages[i]);
    dma_addr = dma_map_page(&fpga.pci_dev->dev, pages[i], 0, PAGE_SIZE,
DMA_TO_DEVICE);

    start_dma();

    wait_until_dma_ready();

    dma_unmap_page(&fpga.pci_dev->dev, dma_addr, dma_len, DMA_TO_DEVICE);

    if(!checksum_ok_in_fpga())
        print_page_data();
}

for (i = 0; i < nr_pages; i++)
    page_cache_release(pages[i]);

The problem is that sometimes the PCI device receives wrong bytes at random
locations. I find the data checksum inside the driver and inside the FPGA.
Inside the driver I use kmap(page) and kunmap(page) to access data on the
user page. Sometimes the checksums do not match and I can see with the FPGA
debugging tool, that the FPGA receives different data than the driver is
printing out.

I have noticed 3 things regarding this error:
1) When I copy the data I read from the block device to another buffer in
userspace and pass this new buffer to the driver, then this error will
never occur:

file_fd = open("/disk/file", O_RDONLY);
result = read(file_fd, page_aligned_buf, len);
memcpy(new_page_aligned_buf, page_aligned_buf, len);
call_driver_ioctl_which_performs_dma(new_page_aligned_buf, len);

2) this error does not occur when I use copy_from_user instead of
get_user_pages
and dma_map_page inside driver.

3) this error does not occur on our previuos device, which has x86
platform.

I am currently out of ideas what to do next.. It seems to me like a cache
coherency problem. Can anyone suggest what might be wrong?

--
_______________________________________________
Linuxppc-embedded mailing list
Linuxppc-embedded at ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-embedded

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://ozlabs.org/pipermail/linuxppc-embedded/attachments/20061012/b4b11193/attachment.htm