Nodes with no memory

[Dave Hansen]

On Fri, 2008-11-21 at 18:49 -0600, Nathan Lynch wrote:
> Dave Hansen wrote:
> > I was handed off a bug report about a blade not booting with a, um
> > "newer" kernel.
> 
> If you're unable to provide basic information such as the kernel
> version then perhaps this isn't the best forum for discussing this.  :)

Let's just say a derivative of 2.6.27.5.  I will, of course be trying to reproduce on mainline.  I'm just going with the kernel closest to the bug report as I can get for now. 

> > I'm thinking that we need to at least fix careful_allocation() to oops
> > and not return NULL, or check to make sure all it callers check its
> > return code.
> 
> Well, careful_allocation() in current mainline tries pretty hard to
> panic if it can't satisfy the request.  Why isn't that happening?

I added some random debugging to careful_alloc() to find out.

careful_allocation(1, 7680, 80, 0)
careful_allocation() ret1: 00000001dffe4100
careful_allocation() ret2: 00000001dffe4100
careful_allocation() ret3: 00000001dffe4100
careful_allocation() ret4: c000000000000000
careful_allocation() ret5: 0000000000000000

It looks to me like it is hitting 'the memory came from a previously
allocated node' check.  So, the __lmb_alloc_base() appears to get
something worthwhile, but that gets overwritten later.

I'm still not quite sure what this comment means.  Are we just trying to
get node locality from the allocation?

I also need to go look at how __alloc_bootmem_node() ends up returning
c000000000000000.  It should be returning NULL, and panic'ing, in
careful_alloc().  This probably has to do with the fact that NODE_DATA()
isn't set up, yet, but I'll double check.

        /*
         * If the memory came from a previously allocated node, we must
         * retry with the bootmem allocator.
         */
        new_nid = early_pfn_to_nid(ret >> PAGE_SHIFT);
        if (new_nid < nid) {
                ret = (unsigned long)__alloc_bootmem_node(NODE_DATA(new_nid),
                                size, align, 0);
                dbg("careful_allocation() ret4: %016lx\n", ret);

                if (!ret)
                        panic("numa.c: cannot allocate %lu bytes on node %d",
                              size, new_nid);

                ret = __pa(ret);
                dbg("careful_allocation() ret5: %016lx\n", ret);

                dbg("alloc_bootmem %lx %lx\n", ret, size);
        }



-- Dave