[powerpc][next-20210625] Kernel warning(arch/powerpc/kernel/interrupt.c:518) during boot

[Nicholas Piggin]

Excerpts from Nicholas Piggin's message of June 27, 2021 4:57 am:
> Excerpts from Sachin Sant's message of June 26, 2021 11:52 pm:
>> Following kernel warning is seen while booting 5.13.0-rc7-next-20210625
>> on POWER9 LPAR.
>> 
>> [   40.573592] ------------[ cut here ]------------
>> [   40.573604] WARNING: CPU: 6 PID: 4743 at arch/powerpc/kernel/interrupt.c:518 interrupt_exit_kernel_prepare+0x280/0x2a0
>> [   40.573614] Modules linked in: dm_mod bonding nft_ct nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set rfkill nf_tables libcrc32c nfnetlink sunrpc pseries_rng xts uio_pdrv_genirq uio vmx_crypto sch_fq_codel ip_tables ext4 mbcache jbd2 sd_mod t10_pi sg ibmvscsi ibmveth scsi_transport_srp fuse
>> [   40.573649] CPU: 6 PID: 4743 Comm: dracut-install Not tainted 5.13.0-rc7-next-20210625 #1
>> [   40.573655] NIP:  c000000000032990 LR: c00000000000c958 CTR: 000000000048dd1c
>> [   40.573660] REGS: c0000000414db640 TRAP: 0700   Not tainted  (5.13.0-rc7-next-20210625)
>> [   40.573664] MSR:  8000000000021033 <SF,ME,IR,DR,RI,LE>  CR: 28044288  XER: 00000000
>> [   40.573674] CFAR: c0000000000327a4 IRQMASK: 1 
>>                GPR00: c00000000000c958 c0000000414db8e0 c0000000029bbd00 c0000000414db9a0 
>>                GPR04: 8000000000001033 0000000000000093 0000000000000048 ffffffffffffffbf 
>>                GPR08: 0000000000000008 0000000000000000 0000000000000003 0000000000000010 
>>                GPR12: 0000000000004000 c000000005587a00 0000000101dc15a8 0000000101dc1590 
>>                GPR16: 0000000101dc05a8 00007fffc7abe353 00007fffb7926740 0000000000000000 
>>                GPR20: 00007fffc7ab7ae0 fffffffffffff000 0000000000000006 c000000043cbbc00 
>>                GPR24: 0000000000000000 000001003da495d0 0000000000000000 0000000000000000 
>>                GPR28: 0000000000000000 fcffffffffffffff 0000000000000000 c0000000414db9a0 
>> [   40.573725] NIP [c000000000032990] interrupt_exit_kernel_prepare+0x280/0x2a0
>> [   40.573730] LR [c00000000000c958] interrupt_return_srr_user_restart+0x34/0x118
> 
> BTW this isn't a restart but a kernel exit. I'll have to update labels 
> to make this clear.
> 
>> [   40.573736] Call Trace:
>> [   40.573738] [c0000000414db8e0] [c000000043cbbc00] 0xc000000043cbbc00 (unreliable)
>> [   40.573744] [c0000000414db930] [c00000000000c958] interrupt_return_srr_user_restart+0x34/0x118
>> [   40.573751] --- interrupt: 300 at strnlen_user+0x74/0x240
>> [   40.573756] NIP:  c00000000070ccf4 LR: c00000000048a460 CTR: 000000000003fffe
>> [   40.573760] REGS: c0000000414db9a0 TRAP: 0300   Not tainted  (5.13.0-rc7-next-20210625)
>> [   40.573764] MSR:  8000000000001033 <SF,ME,IR,DR,RI,LE>  CR: 48044228  XER: 20040000
>> [   40.573774] CFAR: c00000000048a45c DAR: 000001003da495d0 DSISR: 40000000 IRQMASK: 0 
>>                GPR00: c00000000048a44c c0000000414dbc40 c0000000029bbd00 0000000000000000 
>>                GPR04: 0000000000200000 0000000000000030 c000000043cbbc00 000001003da495d0 
>>                GPR08: a8aaaaaaaaaaaaaa bcffffffffffffff 000001003da495d0 0000000000000000 
>>                GPR12: 0000000000004000 c000000005587a00 0000000101dc15a8 0000000101dc1590 
>>                GPR16: 0000000101dc05a8 00007fffc7abe353 00007fffb7926740 0000000000000000 
>>                GPR20: 00007fffc7ab7ae0 fffffffffffff000 0000000000000006 c000000043cbbc00 
>>                GPR24: 0000000000000000 000001003da495d0 0000000000000000 0000000000000000 
>>                GPR28: 0000000000000000 c000000043b6a000 c000000043cbbc00 0000000000000000 
>> [   40.573826] NIP [c00000000070ccf4] strnlen_user+0x74/0x240
>> [   40.573830] LR [c00000000048a460] copy_strings.isra.42+0xb0/0x350
> 
> So there's definitely IRQMASK=0 and no MSR[EE]=0 in this frame, which is 
> what the warning was.
> 
> I'd say either something hasn't set PACA_IRQ_HARD_DIS properly, so EE 
> doesn't get enabled when irqs are restored, or maybe the  change to
> arch_local_irq_restore(). Less likely that the stack got messed up.
> 
> Can you try run with CONFIG_PPC_IRQ_SOFT_MASK_DEBUG=y ?

Nevermind, I think I've found the problem. Some code runs in the
implicit soft-mask region without expecting to be masked. Working
on a fix...

Thanks,
Nick