MMIO and gcc re-ordering issue

[Jes Sorensen]

Jesse Barnes wrote:
> On Thursday, May 29, 2008 2:40 pm Benjamin Herrenschmidt wrote:
>> On Thu, 2008-05-29 at 10:47 -0400, Jes Sorensen wrote:
>>> The only way to guarantee ordering in the above setup, is to either
>>> make writel() fully ordered or adding the mmiowb()'s inbetween the two
>>> writel's. On Altix you have to go and read from the PCI brige to
>>> ensure all writes to it have been flushed, which is also what mmiowb()
>>> is doing. If writel() was to guarantee this ordering, it would make
>>> every writel() call extremely expensive :-(
>> Interesting. I've always been taught by ia64 people that mmiowb() was
>> intended to be used solely between writel() and spin_unlock().
> 
> Well, that *was* true, afaik, but maybe these days multipath isn't just for 
> fail-over.  If that's true, then yeah making every single writeX ordered 
> would be the only way to go...

I could be getting bits wrong, but multi-path here is in the NUMA
routing, not at the device level.

>> If this is a performance problem, then provide relaxed variants and
>> use them in selected drivers.
> 
> Sounds reasonable.  That way drivers "just work" and important drivers can be 
> optimized.

That would kill all levels of performance in all drivers, resulting in
attempts to try and modify a fair bit of drivers to get the performance
back.

In reality this problem really only exists for devices where ordering of
consecutive writel's is a big issue. In my experience it really isn't
the case very frequently - and the number of mmiowb's that have put in
shows that too :-)

Cheers,
Jes