[Skiboot] [PATCH] Adjust skiboot_cpu_stacks region size according to real max PIR

[Stewart Smith]

In skiboot, CPU stacks are indexed by PIR.

During boot, we have two ideas about what the actual maximum PIR is:
1) detect CPU type (P7 or P8): we know max PIR is max for that proc
   (e.g. 1024, 8192)
2) start all CPUs (go through device tree for CPUs that exist).
   We now know the *actual* CPUs we have and the max PIR.
   e.g 1, 64, 3319 or whatever

Each CPU stack is 16KB.
So max CPU stacks size for P7 is 16MB, for P8 is 128MB.
The *actual* max for the machine we're booting on is based on max PIR
we detect during boot. I have found the following:
Mambo: 16kb max (one CPU)
P7: 64, meaning 64*16k = 1MB
P8: 3320, meaning 3320*16k = 51MB

So, currently, we were not reseting the size of the skiboot_cpu_stacks
memory region correctly before boot (we construct that part of the device
tree as the very last thing before booting the payload), even though the
comment in mem_region.c would suggest we were, we weren't. Because code
comments are evil and are nothing but filty, filthy lies.

With this patch, we now properly adjust the CPU stacks memory region
size after we've detected CPU type and after we've found the real
max PIR.

This saves between about 77MB and 128MB-16kB of memory from being in a
reserved region and it'll now be available to the OS to use for things
such as cat pictures rather than being firmware stack space waiting for
a CPU that will never appear.

You can see the difference in skiboot log, "Reserved regions:":

Before:
ALL:   0x000031a00000..0000399fffff : ibm,firmware-stacks

AFTER:
Mambo: 0x000031a00000..000031a1ffff : ibm,firmware-stacks
P7:    0x000031a00000..000031afffff : ibm,firmware-stacks.
P8:    0x000031a00000..000034ddffff : ibm,firmware-stacks

Signed-off-by: Stewart Smith <stewart at linux.vnet.ibm.com>
---
 core/init.c          |    2 ++
 core/mem_region.c    |   12 ++++++++++--
 include/mem_region.h |    2 +-
 3 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/core/init.c b/core/init.c
index 445272a..0e91a9d 100644
--- a/core/init.c
+++ b/core/init.c
@@ -634,6 +634,8 @@ void __noreturn main_cpu_entry(const void *fdt, u32 master_cpu)
 
 	/* Initialize the rest of the cpu thread structs */
 	init_all_cpus();
+	/* We now know real max PIR, so adjust mem region appropriately */
+	adjust_cpu_stacks_len_to_max_pir();
 
 	/* Allocate our split trace buffers now. Depends add_opal_node() */
 	init_trace_buffers();
diff --git a/core/mem_region.c b/core/mem_region.c
index 5a496aa..569f8fd 100644
--- a/core/mem_region.c
+++ b/core/mem_region.c
@@ -725,6 +725,15 @@ struct mem_region *find_mem_region(const char *name)
 	return NULL;
 }
 
+void adjust_cpu_stacks_len_to_max_pir(void)
+{
+	/* CPU stacks start at 0, then when we know max possible PIR,
+	 * we adjust, then when we bring all CPUs online we know the
+	 * runtime max PIR, so we adjust this a few times during boot.
+	 */
+	skiboot_cpu_stacks.len = (cpu_max_pir + 1) * STACK_SIZE;
+}
+
 /* Trawl through device tree, create memory regions from nodes. */
 void mem_region_init(void)
 {
@@ -773,8 +782,7 @@ void mem_region_init(void)
 		unlock(&mem_region_lock);
 	}
 
-	/* Now we know how many CPU stacks we have, fix that up. */
-	skiboot_cpu_stacks.len = (cpu_max_pir + 1) * STACK_SIZE;
+	adjust_cpu_stacks_len_to_max_pir();
 
 	lock(&mem_region_lock);
 
diff --git a/include/mem_region.h b/include/mem_region.h
index 23ee18c..96dfddc 100644
--- a/include/mem_region.h
+++ b/include/mem_region.h
@@ -60,7 +60,7 @@ void mem_region_release_unused(void);
 extern struct mem_region skiboot_heap;
 
 void mem_region_init(void);
-
+void adjust_cpu_stacks_len_to_max_pir(void);
 void mem_region_add_dt_reserved(void);
 
 /* Mark memory as reserved */
-- 
1.7.10.4