Disabling L1 D-cache and side effects

[Benjamin Herrenschmidt]

On Tue, 2008-09-30 at 15:26 -0700, Tirumala Reddy Marri wrote:
> Ben,
>     I got to bring up Linux on one of the 440 processors with out L1
> dcache to do some bench marking  and compare with  L1 d-cache enabled.  
> 
> I am avoiding any references to dcbz ,dcbt and dcbst .   Also the TLB's
> are created with cache inhibited. I looked at lwarx/stwcx description,
> there seem to be no dependency on L1 cache.

Ok. Well, they are generally implemented at the L2 level but maybe not
on 440, architecturally, they must be used on cacheable memory but it's
possible that 440 being not SMP coherent, the actual implementation of
those is too dumb to care.

> I don't see any critical exceptions or traps. All I  see is /init/bin
> failing to execute because data is corrupted. 

Have you properly replaced dcbz with multiple stores ? I did some bring
up work internally on some stuff where dcbz wasn't quite there yet and
one pitfall to be careful is that if you force-enable the alternate
CONFIG_8xx implementation in the various copy & memset routines in
arch/powerpc/lib, you also need to fix those implementations to copy
or clear 32 bytes instead of just 16, as 8xx has 16 byte cache lines.

Typically failing to do so causes things like memset to fail to properly
clear things such as page tables and thus random crap occurs.

Cheers,
Ben.

> Thanks,
> Marri

> 
> -----Original Message-----
> From: Benjamin Herrenschmidt [mailto:benh at kernel.crashing.org] 
> Sent: Tuesday, September 30, 2008 2:31 PM
> To: Tirumala Reddy Marri
> Cc: Olof Johansson; linuxppc-dev at ozlabs.org
> Subject: RE: Disabling L1 D-cache and side effects
> 
> On Tue, 2008-09-30 at 09:57 -0700, Tirumala Reddy Marri wrote:
> > Ben,
> >   Thanks for the response. I am wondering how user space would get 
> > affected by absence of L1 Dcache.
> 
> You didn't answer my question :-)
> 
> Well, as I said, things like lwarx/stwcx not working, dcbz taking
> alignment exceptions, etc...
> 
> Ben.
> 
> > Thanks,
> > Marri
> > 
> > -----Original Message-----
> > From: Benjamin Herrenschmidt [mailto:benh at kernel.crashing.org]
> > Sent: Tuesday, September 30, 2008 12:16 AM
> > To: Tirumala Reddy Marri
> > Cc: Olof Johansson; linuxppc-dev at ozlabs.org
> > Subject: RE: Disabling L1 D-cache and side effects
> > 
> > On Mon, 2008-09-29 at 14:38 -0700, Tirumala Reddy Marri wrote:
> > > Could you please point me to the which does the Critical error 
> > > (Machine
> > > Check) recovery. BTW I am successful booting the Linux until rootfs 
> > > is
> > 
> > > being mounted. It fails to mount the Linux saying that blocks are 
> > > corrupted in file system. I had to modify lots of initial bring up 
> > > code to disable D-cache and make sure all TLB's are cache inhibited.
> > > Ando also made sure none of the misc_32.S , entry_32.S and head.S 
> > > makes any references to d-cache.
> > 
> > Why the heck are you doing that btw ? AFAIK, as Olof says, things like
> 
> > atomic operations will not work, dcbz neither etc... it's likely that 
> > even if you manage to plaster around all of this in the kernel, 
> > whatever userspace code you'll try to run in userspace will blow up
> too...
> > 
> > Cheers,
> > Ben.