Memory mapping PCI memory region to user space

[David Hawkins]

Hi Kumar,

>> When I was testing the Yosemite board as the host, I found
>> that I could set the endian flag on the mmapped page, which
>> then made the PCI device registers read as 32-bit quantities
>> read back with the same layout under both x86 and PPC
>> hosts.
> 
> Hmm, I guess I would handle this like how the reset of the kernel  
> handle is with the io routines handling the swapping.  Not sure if  
> there is any advantage to using the endian flag.  I guess if you have  
> something you are treating as just memory there would be.

I haven't used the feature, I just tested it to see what it did.

The application case I thought of was this; the PCI boards I built
(that I am revising, and replacing the DSP with a PPC) have an
8MB PCI region that I can mmap from the host. I have a test
suite that runs from the host that manipulates registers on the boards
to download FPGAs etc. When the boards are used in a real system,
the onboard DSP is generally used, and the host just talks to
the DSP.

However, for the test suite, if I have a header with definitions
like:

#define CONTROL_FPGA_ENABLE   (1 << 0)
#define CONTROL_FPGA_DONE_BIT (1 << 1)

that correspond to bits in a 32-bit PCI mmapped register. Then
code in the user-space test suite that did something like

   pci_addr[CONTROL_OFFSET] |= CONTROL_FPGA_ENABLE;

would instead need to be re-written, eg.,

   write_le32(&pci_addr[CONTROL_OFFSET], CONTROL_FPGA_ENABLE);

to be portable.

I definitely agree that this is how kernel-level code should be
written, but user-space code ... well, if I want to reuse code
already written, setting the page endian flag and reusing the
code would seem like the way to go. (This isn't what I need
to do, since my host will still be an x86, the PPC will
be a target device, but I still need to think about the
endian issues).

Now of course that I have seen the consequences of my coding,
I'll be more careful to deal with endianness more appropriately.

Its a tricky trade-off though. I could define control ioctl's
that hide all the endianness issues ... but then the driver just
gets bigger. I think the appropriate solution for the user-space
test code would be to use CPU-to-little-endian routines, and
wrap the lot in a re-usable library that the test suite
links against.

> There isn't a sysfs flag for the endianness page attribute since  thats 
> a PPC book-e specific feature.  We could possible expand things  to 
> support it but, I've been trying to actively avoid using the 'E' bit.

Ok, I haven't received the 8349E board that I am waiting
on, so I hadn't spotted that the PAGE_ENDIAN flag was
Book E specific.

Thanks for your insight.

Cheers
Dave