<html><body>
<p><tt>Ankita Garg <ankita@in.ibm.com> wrote on 08/19/2010 10:58:24 AM:<br>
> Subject</tt><br>
<tt>> Re: [PATCH][RFC] preempt_count corruption across H_CEDE call with <br>
> CONFIG_PREEMPT on pseries</tt><br>
<tt>> <br>
> Hi Darren,<br>
> <br>
> On Thu, Jul 22, 2010 at 11:24:13AM -0700, Darren Hart wrote:<br>
> > <br>
> > With some instrumentation we were able to determine that the<br>
> > preempt_count() appears to change across the extended_cede_processor()<br>
> > call. Specifically across the plpar_hcall_norets(H_CEDE) call. On<br>
> > PREEMPT_RT we call this with preempt_count=1 and return with<br>
> > preempt_count=0xffffffff. On mainline with CONFIG_PREEMPT=y, the value<br>
> > is different (0x65) but is still incorrect.<br>
> <br>
> I was trying to reproduce this issue on a 2.6.33.7-rt29 kernel. I could<br>
> easily reproduce this on the RT kernel and not the non-RT kernel.<br>
> However, I hit it every single time I did a cpu online operation. I also<br>
> noticed that the issue persists even when I disable H_CEDE by passing<br>
> the "cede_offline=0" kernel commandline parameter. Could you pl confirm<br>
> if you observe the same in your setup ? </tt><br>
<br>
<tt>If you see it every time, double-check that you have </tt><br>
<tt><a href="http://patchwork.ozlabs.org/patch/60922/">http://patchwork.ozlabs.org/patch/60922/</a> or an equivalent in your tree. (The</tt><br>
<tt>patch moves the preempt_enable_no_resched() call up above a call to cpu_die(). An</tt><br>
<tt>earlier variation called a preempt_enable before calling start_secondary_resume()).</tt><br>
<br>
<tt>Darren and I have been seeing different problems (different dedicated-processor LPARS</tt><br>
<tt>on the same physical system). He's seeing scrambled preempt_count values, I'm tending</tt><br>
<tt>to see a system hang/death [with processor backtraces showing .pseries_mach_cpu_die or</tt><br>
<tt>.pseries_dedicated_idle_sleep as expected, but no processes running] .</tt><br>
<br>
<tt>I'll be curious which you end up seeing. With 2.6.33.7-rt27 as of a few minutes ago, </tt><br>
<tt>I'm still seeing a system hang/death.</tt><br>
<br>
<br>
<tt><br>
> <br>
> However, the issue still remains. Will spend few cycles looking into<br>
> this issue.<br>
> <br>
> > <br>
> > Also of interest is that this path<br>
> > cpu_idle()->cpu_die()->pseries_mach_cpu_die() to start_secondary()<br>
> > enters with a preempt_count=1 if it wasn't corrupted across the hcall.<br>
> > The early boot path from _start however appears to call<br>
> > start_secondary() with a preempt_count of 0.<br>
> > <br>
> > The following patch is most certainly not correct, but it does eliminate<br>
> > the situation on mainline 100% of the time (there is still a 25%<br>
> > reproduction rate on PREEMPT_RT). Can someone comment on:<br>
> > <br>
> > 1) How can the preempt_count() get mangled across the H_CEDE hcall?<br>
> > 2) Should we call preempt_enable() in cpu_idle() prior to cpu_die() ?<br>
> > <br>
> > Hacked-up-by: Darren Hart <dvhltc@us.ibm.com><br>
> > <br>
> > Index: linux-2.6.33.6/arch/powerpc/platforms/pseries/hotplug-cpu.c<br>
> > ===================================================================<br>
> > --- linux-2.6.33.6.orig/arch/powerpc/platforms/pseries/hotplug-cpu.c<br>
> > +++ linux-2.6.33.6/arch/powerpc/platforms/pseries/hotplug-cpu.c<br>
> > @@ -138,6 +138,7 @@ static void pseries_mach_cpu_die(void)<br>
> > * Kernel stack will be reset and start_secondary()<br>
> > * will be called to continue the online operation.<br>
> > */<br>
> > + preempt_count() = 0;<br>
> > start_secondary_resume();<br>
> > }<br>
> > }<br>
> > <br>
> > <br>
> <br>
> -- <br>
> Regards,<br>
> Ankita Garg (ankita@in.ibm.com)<br>
> Linux Technology Center<br>
> IBM India Systems & Technology Labs,<br>
> Bangalore, India<br>
</tt></body></html>