[1/4] RFC: SLB rewrite (core rewrite)

[David Gibson]

On Sun, Jul 11, 2004 at 11:05:22AM +1000, Anton Blanchard wrote:
>
>
> > He did do some timings, the difference was certainly tiny, although it
> > might have been (just) measurable.
>
> Yeah it was tiny, although David's slbtest showed it to be on average
> just under 1 cycle faster (probably due to averaging the extra
> misprediction per 60 iterations).
>
> > Of course, I do also have a plan to get rid of the slbmfee and that
> > loop by locking the kernel stack into a fixed SLB slot.  I have a
> > patch to do it even, but I need to figure out how to get it to work on
> > iSeries.
>
> There are a few more things that would be interesting to look at:
>
> - Are there any more segments we should bolt in? So far we have
>   PAGE_OFFSET, VMALLOC_BASE and the kernel stack. It may make sense to
>   bolt in a userspace entry or two. The userspace stack is the best
>   candidate.  Obviously the more segments we bolt in, the less that are
>   available for general use and bolting in more userspace segments will
>   mean mostly kernel bound workloads will suffer (eg SFS). It may make
>   sense to bolt in an IO region too.

Indeed.  My gut instinct would be that we want to keep the number of
bolted segments down, so as much of the SLB can be used as flexibly as
possible.

That said, I've had a couple of ideas on how to generate some
semblence of LRU data in ways that might be sufficiently low overhead
to be practical.  For example, what if we kept a variable containing
the ESID of the last segment cast out from the SLB.  Whenever we take
an SLB miss, we check the EA against that stored segment.  If they
match - i.e. this segment is so heavily used that we want it again
almost as soon as we've thrown it out - we put a flag indicating to
skip over this slot for some number of future misses.

Or rather than just having permanently bolted and non-bolted slots, we
could divide the SLB into a "fast cycle" and a "slow cycle" section
with separate RR pointers.  Either we could "semibolt" certain
segments - putting them in the slow reload section.  Or we could use
some generalization of the scheme above to dynamically promote
frequently used segments to the slow cycle area.

> - We currently preload a few userspace segments (stack, pc and task
>   unmapped base). Does it make sense to restore the entire SLB state on
>   context switch? One problem we have is that we usually enter the
>   kernel via libc which is in a different segment to our program text.
>   This means we always take an SLB miss when we return out of libc after
>   a context switch. (ie we are guaranteed to take one SLB miss per
>   context switch)
>
> It would be interesting to get a log of SLB misses for a few
> workloads.

It certainly would.  We really want to run some simulations to see how
bolting various segments, or schemes like those above would affect the
SLB miss rate.

--
David Gibson			| For every complex problem there is a
david AT gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.
http://www.ozlabs.org/people/dgibson

** Sent via the linuxppc64-dev mail list. See http://lists.linuxppc.org/
** This list is shutting down 7/24/2004.