[PATCH 16/17] powerpc/qspinlock: allow indefinite spinning on a preempted owner

[Nicholas Piggin]

On Fri Sep 23, 2022 at 1:02 AM AEST, Laurent Dufour wrote:
> On 28/07/2022 08:31:19, Nicholas Piggin wrote:
> > Provide an option that holds off queueing indefinitely while the lock
> > owner is preempted. This could reduce queueing latencies for very
> > overcommitted vcpu situations.
> > 
> > This is disabled by default.
>
> Hi Nick,
>
> I should have missed something here.
>
> If this option is turned on, CPU trying to lock when there is a preempted
> owner will spin checking the lock->val and yielding the lock owner CPU.
> Am I right?

Yes.

> If yes, why not being queued and spin checking its own value, yielding
> against the lock owner CPU?

I guess the idea is that when we start getting vCPU preemption, queueing
behaviour causes this "train wreck" behaviour where lock waiters being
preempted can halt lock transfers to other waiters (whereas with simple
spinlocks only owner vCPU preemption matters). So the heuristics for
paravirt qspinlock basically come down to avoiding queueing and making
waiters behave more like a simple spinlock when it matters. That's the
case for upstream and this rewrite.

> This will generate less cache bouncing, which
> is what the queued spinlock is trying to address, isn't it?

It could. When the owner is preempted it's not going to be modifying
the lock word and probably not surrounding data in the same cache
line, and there won't be a lot of other try-lock operations come in
(because they'll mostly queue up here as well). So cacheline bouncing
shouldn't be the worst problem we face here. But it possibly is a
concern.

I didn't yet meausre any real improvement from this option, and it
possibly has some starvation potential, so it's disabled by default for
now.

Thanks,
Nick