MMIO and gcc re-ordering (Was: [PATCH] [POWERPC] Improve (in|out)_beXX() asm code)

[Scott Wood]

Benjamin Herrenschmidt wrote:
> On Fri, 2008-05-23 at 08:36 -0400, Benjamin Herrenschmidt wrote:
>>         - mandate some kind of dma_sync_for_device/cpu on consistent memory.
>> Almost no driver do that currently tho. They only do that for non
>> consistent memory mapped with dma_map_*.
>>
>>         - mandate the use of wmb,rmb,mb barriers for use between memory
>> accesses and MMIOs for ordering them. (ie. fix drivers that don't do
>> it). Advantage for powerpc is that I can remove (after some auditing of
>> course) the added heavy barriers in the MMIO accessors themselves.
> 
> Note that the above is my preferred approach, and a lot of drivers
> happen to already do this.

As Trent pointed out, if you change to eieio in the accessors, that'd 
require drivers to also use mmiowb() before spin_unlock(), which fewer 
drivers currently do.

>>         - stick a full memory clobber in all MMIO (and PIO) accessors on all
>> archs.

I like this, combined with introducing raw variants of the non-PCI 
accessors (in_be32 and such).  It's slower and safe by default (i.e. no 
auditing drivers), but performance-critical paths can be optimized to 
use raw accessors combined with explicit barriers.

-Scott