[5.6.0-rc2-next-20200218/powerpc] Boot failure on POWER9

[Michal Hocko]

[Cc ppc maintainers]
On Thu 27-02-20 17:16:41, Vlastimil Babka wrote:
> On 2/27/20 5:00 PM, Sachin Sant wrote:
> > 
> > 
> >> On 27-Feb-2020, at 5:42 PM, Michal Hocko <mhocko at kernel.org> wrote:
> >> 
> >> A very good hint indeed. I would do this
> >> diff --git a/include/linux/topology.h b/include/linux/topology.h
> >> index eb2fe6edd73c..d9f1b6737e4d 100644
> >> --- a/include/linux/topology.h
> >> +++ b/include/linux/topology.h
> >> @@ -137,6 +137,8 @@ static inline void set_numa_mem(int node)
> >> {
> >> 	this_cpu_write(_numa_mem_, node);
> >> 	_node_numa_mem_[numa_node_id()] = node;
> >> +	pr_info("%s %d -> %d\n", __FUNCTION__, numa_node_id(), node);
> >> +	dump_stack();
> >> }
> >> #endif
> >> 
> >> Btw. it would be also helpful to get
> >> `faddr2line ___slab_alloc+0x334' from your kernel Sachin.
> > 
> > [linux-next]# ./scripts/faddr2line ./vmlinux ___slab_alloc+0x334 
> > ___slab_alloc+0x334/0x760:
> > new_slab_objects at mm/slub.c:2478
> > (inlined by) ___slab_alloc at mm/slub.c:2628
> > [linux-next]# 
> 
> Hmm that doesn't look relevant, but that address was marked as unreliable, no?
> Don't we actually need this one?
> 
> [    8.768727] NIP [c0000000003d55f4] ___slab_alloc+0x1f4/0x760
> 
> > I have also attached boot log with a kernel that include about change.
> > I see the following o/p during boot:
> > 
> > [    0.005269] set_numa_mem 1 -> 1
> 
> So there's no "set_numa_mem 0 -> X", specifically not
> "set_numa_mem 0 -> 1" which I would have expected. That seems to confirm my
> suspicion that the arch code doesn't set up the memoryless node 0 properly.

Please have a look at http://lkml.kernel.org/r/52EF4673-7292-4C4C-B459-AF583951BA48@linux.vnet.ibm.com
for the boot log with the debugging patch which tracks set_numa_mem.
This seems to lead to a crash in the slab allocator bebcause
node_to_mem_node(0) for memory less node resolves to the memory less
node http://lkml.kernel.org/r/dd450314-d428-6776-af07-f92c04c7b967@suse.cz.
The original report is http://lkml.kernel.org/r/3381CD91-AB3D-4773-BA04-E7A072A63968@linux.vnet.ibm.com

> 
> > [    0.005270] CPU: 12 PID: 0 Comm: swapper/12 Not tainted 5.6.0-rc3-next-20200227-autotest+ #6
> > [    0.005271] Call Trace:
> > [    0.005272] [c0000008b37dfe80] [c000000000b5d948] dump_stack+0xbc/0x104 (unreliable)
> > [    0.005274] [c0000008b37dfec0] [c000000000059320] start_secondary+0x600/0x6e0
> > [    0.005277] [c0000008b37dff90] [c00000000000ac54] start_secondary_prolog+0x10/0x14
> > 
> > Thanks
> > -Sachin
> > 

-- 
Michal Hocko
SUSE Labs