3.5.0-rc5: BUG: soft lockup - CPU#0 stuck for 22s

[Christian Kujau]

On Mon, 2 Jul 2012 at 09:45, Paul E. McKenney wrote:
> On Sun, Jul 01, 2012 at 11:30:40PM -0700, Christian Kujau wrote:
> > On Mon, 2 Jul 2012 at 14:50, Benjamin Herrenschmidt wrote:
> > > Interesting... I observed something roughly similar on a dual G4
> > > the other day associated with a 30s to 1mn pause during boot. RCU
> > > was complaining loudly.
> > > 
> > > In my case, it did continue booting normally, is that the case for you ?
> > 
> > No, in my case it stopped booting, though there was no "panic" message.
> > 
> > > Also if I compile the kernel without CONFIG_SMP, it did go away as well,
> > > do you observe that too ?
> > 
> > This PoweBook G4 is UP anyway and I'm always building w/o CONFIG_SMP.
> > 
> > > I don't have a spare cycle to investigate this problem this week I'm
> > > afraid, it might help if you could try a bisection though.
> > 
> > Yes, I've started a bisection already and will report back.
> > 
> > FYI, I've put a netconsole-log below from another boot (in the 
> > middle of a bisection), the instruction dump is slightly more complete.
> 
> Li Zhong posted a patch to fix this (async_synchronize_full() below):
> 
> 	https://lkml.org/lkml/2012/7/2/32
> 
> This gets rid of this problem on my UP testing.  If it works for you,
> please give Li Zhong a Tested-by.

Great! When applied to -rc5 Li's patch fixes it for my powerpc UP system 
as well.

  Tested-by: Christian Kujau <lists at nerdbynature.de>

Many thanks!
Christian.

> > [   40.345973] BUG: soft lockup - CPU#0 stuck for 22s! [modprobe:691]
> > [   40.347737] Modules linked in: sd_mod arc4 firewire_sbp2 scsi_mod b43 mac80211 cfg80211
> > [   40.349443] irq event stamp: 65779272
> > [   40.351156] hardirqs last  enabled at (65779271): [<c052fae8>] _raw_spin_unlock_irqrestore+0x40/0x9c
> > [   40.352911] hardirqs last disabled at (65779272): [<c0011648>] reenable_mmu+0x40/0x98
> > [   40.354649] softirqs last  enabled at (65769864): [<c000f3c0>] call_do_softirq+0x14/0x24
> > [   40.356386] softirqs last disabled at (65769857): [<c000f3c0>] call_do_softirq+0x14/0x24
> > [   40.358098] NIP: c005ee64 LR: c005ee54 CTR: c009c578
> > [   40.359788] REGS: eece3d70 TRAP: 0901   Not tainted  (3.4.0-05609-gc80ddb5)
> > [   40.361517] MSR: 00009032 <EE,ME,IR,DR,RI>  CR: 24042482  XER: 00000000
> > [   40.363261] TASK = eec30b80[691] 'modprobe' THREAD: eece2000#012
> > [   40.363261] GPR00: c005ee54 eece3e20 eec30b80 00000000 00000002 c005edd0 00000000 00000000 #012
> > [   40.363261] GPR08: 00000000 eece2000 00000000 00000001 24042488 
> > [   40.368310] NIP [c005ee64] async_synchronize_cookie_domain+0x70/0x1f4
> > [   40.369978] LR [c005ee54] async_synchronize_cookie_domain+0x60/0x1f4
> > [   40.371614] Call Trace:
> > [   40.373226] [eece3e20] [c005ee54] async_synchronize_cookie_domain+0x60/0x1f4 (unreliable)
> > [   40.374884] [eece3e70] [c005f034] async_synchronize_full+0x3c/0x74
> > [   40.376524] [eece3e90] [c0082790] sys_init_module+0x178/0x1164
> > [   40.378151] [eece3f40] [c00117a8] ret_from_syscall+0x0/0x38
> > [   40.379783] --- Exception: c01 at 0xff5d694#012[   40.379783]     LR = 0x100041c0
> > [   40.382928] Instruction dump:
> > [   40.384489] 93a10044 419e0014 3d20c06f 8009d020 2f800000 419e017c 7fc3f378 4bfffed9 
> > [   40.386081] 7f9f1840 419d0060 7f9f1800 419e0050 <881bf008> 2f800000 419e0014 3d20c06f

-- 
BOFH excuse #280:

Traceroute says that there is a routing problem in the backbone.  It's not our problem.