User allocated memory for DMA

[Amit K Tiwari]

I am writing a driver for a PCIX board to run on Y-HPC (64-bit Yellow
Dog Linux pre-installed on Apple XServe). The application is to
process the data acquired from the board and send the results out. The
data needs to be acquired by DMA into a user allocated buffer. The
amount of data is huge and I need about 1.5 GB per DMA operation.

Here is what I am doing:
1. malloc and mlock the memory in user mode.
2. get the user pages (get_user_pages) in the driver. 
3. Find out how much physically contiguous memory did the application
get (Pass through all the pages and see if the physical addresses got
from page_to_phys are contiguous).
4. Prepare a scatter-gather list of contiguous regions. 
5. pci_map_sg the sg list to get the DMA addresses for each of the entries.

My problem is that at step 3 I do not get any contiguous region.
Surprisingly, some of the physical addresses are such that the low
order page (say page 151) has high order physical address (say,
1ef234000) while the next high order page (say page 152) has the next
lower physical address(say 1ef233000) and this trend continues for
some 10 pages after which the addresses totally far apart.

As a result of this, my scatter gather list has as many entries 
as there are pages allocated. 

Am I doing something bad here? 
I know there is limit on memory that can be allocated through
pci_alloc_*. Is there any other way in which I can allocate memory for
DMA?

Thanks, 
Amit K T