Floating point in the kernel

[Sean MacLennan]

On Fri, 11 Dec 2009 07:19:39 +1100
Benjamin Herrenschmidt <benh at kernel.crashing.org> wrote:

> I'm not sure that will work in all cases, you are playing a bit with
> fire :-) I suppose I could think it through after breakfast but my
> first thought is "don't do that !". Among other things you may not
> have a pt_regs to save the registers to.

Actually, we usually do have pt_regs, or we are stealing some ;)

> > We also hit another problem under high RTP load... and this is the
> > patch that fixes it:
> > 
> > diff --git a/arch/powerpc/kernel/fpu.S b/arch/powerpc/kernel/fpu.S
> > index fc8f5b1..051a02c 100644
> > --- a/arch/powerpc/kernel/fpu.S
> > +++ b/arch/powerpc/kernel/fpu.S
> > @@ -83,6 +83,11 @@ END_FTR_SECTION_IFSET(CPU_FTR_VSX)
> >         stfd    fr0,THREAD_FPSCR(r4)
> >         PPC_LL  r5,PT_REGS(r4)
> >         toreal(r5)
> > +
> > +       /* Under heavy RTP load the hsp thread can have a NULL
> > pt_regs. */
> > +       PPC_LCMPI       0,r5,0
> > +       beq     1f
> > +  
> 
> Right and that means you just lost the content of your FP registers.

This only happens once in a blue moon, even under heavy RTP load. But I
agree, this could be a real problem.

> >         PPC_LL  r4,_MSR-STACK_FRAME_OVERHEAD(r5)
> >         li      r10,MSR_FP|MSR_FE0|MSR_FE1
> >         andc    r4,r4,r10               /* disable FP for previous
> > task */
> > 
> > So, if you are still reading this far, I am just looking for any
> > suggestions. Are there better ways of handling this? Have I
> > missed something? Anybody know why pt_regs might be NULL?  
> 
> Just don't schedule when you enable_kernel_fp() or move your workload
> to userspace :-)

To be honest, I can't find *why* we are scheduling :( They only way we
give up the CPU is with locking... and none of the locks where hit
during the problem. We also never get near our timeslice... the longest
run I saw when the problem happened was 670us. 

Is there a way to disable scheduling? We currently do not have preempt
enabled... but may in the future.

Cheers,
   Sean