[Fwd: [Linuxppc-dev] Re: [Linuxppc-cvs64] CVS: linuxppc64_2_4/arch/ppc64/kernel head.S,1.32.6.1,1.32.6.2]

[Anton Blanchard]

Hi Dave,

> Fundamentally, the problem is that the OS does not have any control over
> the ERAT and it may be invalidated at any point in time.  What we found
> is that when castouts from the SLB start to occur and we get into this
> code path with some frequency, we end up in a state where the SLB is
> full, and there are additional translations in the ERAT for kernel
> address which are not in the SLB.  This set up a situation where we were
> preparing to rfi (do_signal_ret) and were in the crital section where
> SRRO/1 were loaded when the ERAT entry for the stack gets invalidated
> from under us (presumably via a tlbie from another processor).  We then
> take an SLB fault, where the handler proceeds to clobber SRR0/1, and
> things fall apart.

Thanks for the explanation. I actually have a comment in my tree warning
about SLB castouts during the exception exit path but hadnt got around
to fixing it :)

I think there are more problems than just this one, what if the
exception took an SLB miss and chose to cast out the kernel segment
mapping the stack? My understanding is that we should be clearing the MSR
RI bit during exception exit and then replaying the final parts of the
exception exit code if necessary.

I cannot take credit for this idea, Paul has implemented it in the
32 bit port :)

> Given that this should be a generally low use path (SLB castout), I
> don't expect there would be any performance impact.

I think the SLB castout path can be quite important on large random
access workloads. We were seeing it trip on 32 bit JVM benchmarks,
imagine how bad it would be with 200GB of pagecache or process memory.
An unnecessary slbie could have a noticeable effect.

Anton

** Sent via the linuxppc64-dev mail list. See http://lists.linuxppc.org/