eieio in atomics

[David Engebretsen]

Anton Blanchard wrote:
>
> Hi Dave,
>
> > I see that you added an eieio in atomic operations in the 2.5 tree.
> > Can you shed some light on what was discovered here?  I thought these
> > were not required to imply a memory barrier.  Was an actual problem
> > hit and do we need a fix put into the 2.4 code base?
>
> I started looking through the memory barriers in the kernel and
> was worried about places where atomics were used as reference counts:
>
>         foo->value = 1;
>         if (!atomic_dec_and_test(&foo->refcnt))
>                 kfree(foo);
>
> If we do this on two cpus, we need to make sure the first instruction
> does not end up moving down past the atomic operation. The window
> would be small, the isync at the end of the atomic would prevent
> it from moving any further down, so the window would be the few
> instructions between the successful store conditional and the isync.
>
> Having said that I havent convinced myself 100% that the race is real.
>

The window you are worried about is this, if I follow your thinking:

CPU 0		CPU1
-------		------
Store A
Store B
isync
Free A
		Alloc A
		Store A <-- this gets stomped by prior Store A

For this case, the isync does not provide any value (now I am wondering why we
have it at all in these atomic functions ...).  It does not cause the Store A to
be visible, only that the store instruction has executed to completion.  An
isync in a spinlock acquisition is only an import barrier (loads following the
lock do not execute until the lock is taken).  Spinlocks rely on the spin-unlock
code to do a lwsync or eieio in order to provide correctness in the context of
locks.

Having said all this, for _this_ case, there I do not think there is an actual
problem as the kfree path does a lwsync as there is a spinlock taken & released
in free_block on that path.

Now there are probabily cases where there is actually an exposure.  I had hoped
Rusty's statement was sufficient that this is not the case, but perhaps we need
to look more carefully.  Trick will be to find them.  It would be in a place
where the atomics are used as a spinlock to gate access to a data structure.

Do you have some other concrete examples in had we can look at more closely?

Dave.

** Sent via the linuxppc64-dev mail list. See http://lists.linuxppc.org/