[26-devel] v2.6 performance slowdown on MPC8xx: Measuring TLB cache misses

[Marcelo Tosatti]

On Sat, May 07, 2005 at 08:10:38PM +0200, Joakim Tjernlund wrote:
> > Hi Dan, Joakim,
> > 
> > On Sat, Apr 23, 2005 at 08:00:39PM -0400, Dan Malek wrote:
> > > 
> > > On Apr 23, 2005, at 7:51 PM, Joakim Tjernlund wrote:
> > > 
> > > >hmm, I have more than 24MB of memory and I can run CONFIG_PIN_TLB just
> > > >fine with modules off in kernel 2.4. Havn't tried 2.6 yet.
> > > 
> > > Doh.  Oh, I see.  We only do the optimization for the instruction 
> > > misses.
> > > I'll have to take a closer look at Marcelo's 2.6 tests.
> > 
> > The PIN TLB entry option does not make much difference in my tests, 
> > never did.
> > 
> > Who wrote the code? Are there results which indicate a performance gain
> > from TLB pinning on 8xx? If so, where are such results? 
> > 
> > One problem that I've noted is that initial_mmu sets {I,D}TLB index
> > to be 27 (11100). 
> > 
> > MI_RSV4I protects TLB's 27...31.
> > 
> > Given that both {I,D}TLB INDEX's are _decreased_ on each update, it seems
> > to me that initial_mmu should set {I,D}TLB INDEX to 31, which will then 
> > decrease down to 27 after 4 TLB's are created.  
> > 
> > Another question that comes to mind is why initial_mmu does create 
> > additional 8Meg TLB entries for D-cache but not for I-cache: 
> > 
> > #ifdef CONFIG_PIN_TLB
> >         /* Map two more 8M kernel data pages.
> >         */
> > 	...
> > #endif
> 
> Not completly sure that this is correct. There are a few:
> 	addi	r10, r10, 0x0100
>  	mtspr	SPRN_MD_CTR, r10
> later on which will "overflow" 0x1f00 into 0x2000 etc.

Yep. This is not correct at all: the TLB index is increased 
at each miss, not decreased as the manual says. 

I have confirmed it with the BDI... 

>  Jocke
> > 
> > I'll do some more CONFIG_PIN_TLB tests this week...
> > 
> > --- head_8xx.S.orig2	2005-04-24 17:55:59.000000000 -0300
> > +++ head_8xx.S	2005-04-24 17:57:44.000000000 -0300
> > @@ -697,7 +697,7 @@
> >  	tlbia			/* Invalidate all TLB entries */
> >  #ifdef CONFIG_PIN_TLB
> >  	lis	r8, MI_RSV4I at h
> > -	ori	r8, r8, 0x1c00
> > +	ori	r8, r8, 0x1f00
> >  #else
> >  	li	r8, 0
> >  #endif
> > @@ -705,7 +705,7 @@
> >  
> >  #ifdef CONFIG_PIN_TLB
> >  	lis	r10, (MD_RSV4I | MD_RESETVAL)@h
> > -	ori	r10, r10, 0x1c00
> > +	ori	r10, r10, 0x1f00
> >  	mr	r8, r10
> >  #else
> >  	lis	r10, MD_RESETVAL at h