[Cbe-oss-dev] Bug: early_pfn_in_nid() called when not early

[Arnd Bergmann]

After a lot of debugging in spufs, I found that a crash that we encountered
on Cell actually was caused by a change in the memory management.

The patch that caused it is archived in http://lkml.org/lkml/2006/11/1/43,
and this one has been discussed back and forth, but I fear that the current
version may be broken for all setups that do memory hotplug with sparsemen
and NUMA, at least on powerpc.

What happens exactly is that the spufs code tries to register the memory
area owned by the SPU as hotplug memory in order to get page structs (we
probably shouldn't do it that way, but that's a separate discussion).

memmap_init_zone now calls early_pfn_valid() and early_pfn_in_nid()
in order to determine if the page struct should be initialized. This
is wrong for two reasons:

- early_pfn_in_nid checks the early_node_map variable to determine
  to which node the hot plugged memory belongs. However, the new
  memory never was part of the early_node_map to start with, so
  it incorrectly returns node zero, and then fails to initialize
  the page struct if we were trying to add it to a nonzero node.
  This is probably not a problem for pseries, but it is for cell.

- both early_pfn_{in,to}_nid and early_node_map are in the __init
  section and may already have been freed at the time we are calling
  memmap_init_zone().

The patch below is not a suggested fix that I want to get into mainline
(checking slab_is_available is the wrong here), but it is a quick fix
that you should apply if you want to run a recent (post-2.6.18) kernel
on the IBM QS20 blade. I'm sorry for not having reported this earlier,
but we were always trying to find the problem in my own code...

	Arnd <><

--- linux-2.6.orig/mm/page_alloc.c
+++ linux-2.6/mm/page_alloc.c
@@ -1962,7 +1962,8 @@ void __meminit memmap_init_zone(unsigned
 	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
 		if (!early_pfn_valid(pfn))
 			continue;
-		if (!early_pfn_in_nid(pfn, nid))
+		if (!slab_is_available() &&
+		    !early_pfn_in_nid(pfn, nid))
 			continue;
 		page = pfn_to_page(pfn);
 		set_page_links(page, zone, nid, pfn);