[Skiboot] [PATCH] asm/head.S: fix hang in multi-threaded mambo

[Oliver O'Halloran]

In very early boot, skiboot places secondary threads in holding loop
until the primary thread has finished relocating skiboot. The secondary
threads wait until a flag at INITAL_OFFSET + offsetof(boot_flag) is set
to one before continuing. After relocating the skiboot image and
applying ELF relocations the sets this flag and continues booting.

To avoid having to copy skiboot at runtime within the simulation e605691e
introduced a change to have the skiboot.tcl mambo script load skiboot at
it's preferred location of 0x30000000 rather than 0x00000000.
Unfortunately, the relocation code path is also responsible for saving the
inital offset pointer (into r15). When not taken this results in the main
thread writing to r15 + offsetof(boot_flag) instead, which leaves the
secondary threads stuck waiting for boot_flag and the main thread stuck
when trying to call-in the secondary threads during CPU initialisation.

This patch fixes the bug by saving the inital offset before checking if
relocation is necessary and adds a few comments to better document the
relocation process.

Signed-off-by: Oliver O'Halloran <oohall at gmail.com>
---
 asm/head.S | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/asm/head.S b/asm/head.S
index b514b22..66d405f 100644
--- a/asm/head.S
+++ b/asm/head.S
@@ -310,6 +310,12 @@ boot_entry:
 	/* Initialize thread SPRs */
 	bl init_replicated_sprs
 
+	/* Save the initial offset. The secondary threads will spin on boot_flag
+	 * before relocation so we need to keep track of its location to wake
+	 * them up.
+	 */
+	mr	%r15,%r30
+
 	/* Check if we need to copy ourselves up and update %r30 to
 	 * be our new offset
 	 */
@@ -319,17 +325,19 @@ boot_entry:
 	srdi	%r3,%r3,3
 	mtctr	%r3
 	mr	%r4,%r30
-	mr	%r15,%r30
 	mr	%r30,%r29
+	/* copy the skiboot image to the new offset */
 1:	ld	%r0,0(%r4)
 	std	%r0,0(%r29)
 	addi	%r29,%r29,8
 	addi	%r4,%r4,8
 	bdnz	1b
+	/* flush caches, etc */
 	sync
 	icbi	0,%r29
 	sync
 	isync
+	/* branch to the new image location and continue */
 	LOAD_IMM32(%r3, 2f - __head)
 	add	%r3,%r3,%r30
 	mtctr	%r3
-- 
2.5.0