MMIO and gcc re-ordering issue

[Benjamin Herrenschmidt]

On Mon, 2008-05-26 at 18:40 -0700, David Miller wrote:
> 
> > Quick summary: gcc is happily re-ordering readl/writel vs. surrounding
> > memory accesses (and thus accesses to DMA coherent memory) which is
> > obviously a _BAD_THING_.
> > 
> > This is on all archs. Quick fix is to stick a "memory" clobber in all arch
> > implementations of readl/writel/... (ie, making them a barrier()).
> > 
> > However, I'm using that as an excuse to bring back my pet subject, which
> > is basically, should we instead just finally mandate the use of explicit
> > rmb/wmb/mb's (which boils down to barrier() on x86) to drivers who want
> > to order memory consistent accesses vs. MMIO ?
> 
> This is basically what drivers are effectively doing.

Some of them. USB comes to mind. I'd be happy to make it "the rule" and
document that MMIO vs. coherent access aren't implicitely ordered. I
would still keep them ordered on powerpc for a little while tho until
I'm happy enough with driver auditing.

But heh, it's you who was telling me that it would be a bad engineering
decision and we had to make everybody look like x86 & fully ordered :-)
I decided to agree back then and stuck all those nasty heavy barriers
in the powerpc variants of readl/writel/...

Now, however, that x86 -is- also affected by the problem to some extent
(ie. compiler re-ordering, not CPU but similar), this  is why I'm asking
what's people opinion is.

I'm happy to do patches updating memory-barriers.txt and others, and
do some driver auditing (though I won't do all of them), if the general
opinion is that it's the right direction to go.

If not, then shouldn't we remove the existing spurrious wmb/rmb/mb from
drivers and slap a "memory" clobber on all archs readl/writel/... ?

Cheers,
Ben.