MMIO and gcc re-ordering issue
rdreier at cisco.com
Wed May 28 07:29:33 EST 2008
> > Writes are posted yes, but not reordered arbitrarily. If I have code like:
> > spin_lock(&mmio_lock);
> > writel(val1, reg1);
> > writel(val2, reg2);
> > spin_unlock(&mmio_lock);
> > then I have a reasonable expectation that if two CPUs run this at the
> > same time, their writes to reg1/reg2 won't be interleaved with each
> > other (because the whole section is inside a spinlock). And Altix
> > violates that expectation.
> Does that necessarily follow?
> If you've got a large system with multiple pci bridges, could you end
> up with posted writes coming from different cpus taking a different
> amount of time to propagate to a device and thus colliding?
Not on x86. And a given PCI device can only be reached from a single
host bridge, so I don't see how it can happen. But on SGI altix
systems, there is a routed fabric between the CPU and the PCI bus, so
the reordering can happen there. Hence mmiowb() and the endless supply
of driver bugs that it causes.
More information about the Linuxppc-dev