Re: [Help] Microwatt (Zynqwatt) — Kernel halts after Radix MMU init on booting Linux on Zynq version of Microwatt
Mohammad Amin Nili
manili.devteam at gmail.com
Sat Nov 22 22:32:46 AEDT 2025
Hello Oliver,
> That's very odd. rcu_reclaim_tiny() is probably being folded into
> rcu_process_callbacks() by the compiler and the crash occurs when
> branching to the callback function from the rcu_head
> (https://elixir.bootlin.com/linux/v6.18-rc5/source/kernel/rcu/tiny.c#L95 <https://elixir.bootlin.com/linux/v6.18-rc5/source/kernel/rcu/tiny.c#L95>).
> That said, the "callback" address it branched to (0xc00c000000000000)
> is actually the base of the vmemmap (i.e. the struct page array) so I
> doubt that's actually the callback address stored in the rcu_head. You
> can use xmon to dump the registers and examine memory to confirm this.
> It's hard to say why this is happening, but it's pretty likely to
> either be the compiler optimizing away code you'd prefer to keep or a
> bug in the core itself.
>
> I'd compare the disasm of rcu_process_callbacks() with and without
> your wait loop added and see how the emitted code changes. If adding
> the loop changes nothing then it might be a logic bug in microwatt
> itself or some other timing induced problem.
I just replaced the wait-loop with the following (and it also prevents the
kernel panic):
> volatile uint32_t dumb = 0xFEDC0000;
> dumb++;
- Here is the disasm of the `rcu_process_callbacks` with the injected code:
c00000000008c5e8 <rcu_process_callbacks>:
c00000000008c5e8: 32 00 4c 3c addis r2,r12,50
c00000000008c5ec: 18 ca 42 38 addi r2,r2,-13800
c00000000008c5f0: a6 02 08 7c mflr r0
c00000000008c5f4: a1 c3 23 48 bl c0000000002c8994 <_savegpr0_27>
c00000000008c5f8: a1 ff 21 f8 stdu r1,-96(r1)
c00000000008c5fc: fa 00 6d 88 lbz r3,250(r13)
c00000000008c600: 01 00 69 60 ori r9,r3,1
c00000000008c604: fa 00 2d 99 stb r9,250(r13)
c00000000008c608: 2b 01 42 3d addis r10,r2,299
c00000000008c60c: 2b 01 22 3d addis r9,r2,299
c00000000008c610: d0 3c 4a e9 ld r10,15568(r10)
c00000000008c614: c8 3c 29 39 addi r9,r9,15560
c00000000008c618: 00 48 2a 7c cmpd r10,r9
c00000000008c61c: 14 00 82 40 bne c00000000008c630 <rcu_process_callbacks+0x48>
c00000000008c620: 6d d6 f8 4b bl c000000000019c8c <arch_local_irq_restore+0x8>
c00000000008c624: 00 00 00 60 nop
c00000000008c628: 60 00 21 38 addi r1,r1,96
c00000000008c62c: b8 c3 23 48 b c0000000002c89e4 <_restgpr0_27>
c00000000008c630: 00 00 0a e9 ld r8,0(r10)
c00000000008c634: 2b 01 e2 3c addis r7,r2,299
c00000000008c638: c8 3c e7 eb ld r31,15560(r7)
c00000000008c63c: c8 3c 07 f9 std r8,15560(r7)
c00000000008c640: 00 00 00 39 li r8,0
c00000000008c644: 00 00 0a f9 std r8,0(r10)
c00000000008c648: 2b 01 42 3d addis r10,r2,299
c00000000008c64c: d8 3c 0a e9 ld r8,15576(r10)
c00000000008c650: 2b 01 42 3d addis r10,r2,299
c00000000008c654: d0 3c 4a e9 ld r10,15568(r10)
c00000000008c658: 00 50 28 7c cmpd r8,r10
c00000000008c65c: 08 00 82 40 bne c00000000008c664 <rcu_process_callbacks+0x7c>
c00000000008c660: 10 00 29 f9 std r9,16(r9)
c00000000008c664: 08 00 29 f9 std r9,8(r9)
c00000000008c668: dc fe 80 3f lis r28,-292 <<<<====== "THIS IS INJECTED CODE"
c00000000008c66c: 21 d6 f8 4b bl c000000000019c8c <arch_local_irq_restore+0x8>
c00000000008c670: 00 00 00 60 nop
c00000000008c674: 00 00 60 3b li r27,0
c00000000008c678: 00 00 3f 2c cmpdi r31,0
c00000000008c67c: ac ff 82 41 beq c00000000008c628 <rcu_process_callbacks+0x40>
c00000000008c680: 00 00 df eb ld r30,0(r31)
c00000000008c684: 00 00 3e 2c cmpdi r30,0
c00000000008c688: 08 00 82 41 beq c00000000008c690 <rcu_process_callbacks+0xa8>
c00000000008c68c: 2c f2 00 7c dcbtct 0,r30
c00000000008c690: 2c 00 81 93 stw r28,44(r1)
c00000000008c694: 08 00 bf eb ld r29,8(r31)
c00000000008c698: 00 00 3d 2c cmpdi r29,0
c00000000008c69c: 2c 00 21 81 lwz r9,44(r1)
c00000000008c6a0: 01 00 29 39 addi r9,r9,1
c00000000008c6a4: 2c 00 21 91 stw r9,44(r1)
c00000000008c6a8: 10 00 82 40 bne c00000000008c6b8 <rcu_process_callbacks+0xd0>
c00000000008c6ac: 78 fb e3 7f mr r3,r31
c00000000008c6b0: 15 f8 03 48 bl c0000000000cbec4 <kmem_dump_obj+0x8>
c00000000008c6b4: 00 00 00 60 nop
c00000000008c6b8: 08 00 7f fb std r27,8(r31)
c00000000008c6bc: 78 fb e3 7f mr r3,r31
c00000000008c6c0: a6 03 a9 7f mtctr r29
c00000000008c6c4: 78 eb ac 7f mr r12,r29
c00000000008c6c8: 18 00 41 f8 std r2,24(r1)
c00000000008c6cc: 78 f3 df 7f mr r31,r30
c00000000008c6d0: 21 04 80 4e bctrl
c00000000008c6d4: 18 00 41 e8 ld r2,24(r1)
c00000000008c6d8: a0 ff ff 4b b c00000000008c678 <rcu_process_callbacks+0x90>
- And here is the disasm without the injected code:
c00000000008c5e8 <rcu_process_callbacks>:
c00000000008c5e8: 32 00 4c 3c addis r2,r12,50
c00000000008c5ec: 18 ca 42 38 addi r2,r2,-13800
c00000000008c5f0: a6 02 08 7c mflr r0
c00000000008c5f4: 95 c3 23 48 bl c0000000002c8988 <_savegpr0_28>
c00000000008c5f8: c1 ff 21 f8 stdu r1,-64(r1)
c00000000008c5fc: fa 00 6d 88 lbz r3,250(r13)
c00000000008c600: 01 00 69 60 ori r9,r3,1
c00000000008c604: fa 00 2d 99 stb r9,250(r13)
c00000000008c608: 2b 01 42 3d addis r10,r2,299
c00000000008c60c: 2b 01 22 3d addis r9,r2,299
c00000000008c610: d0 3c 4a e9 ld r10,15568(r10)
c00000000008c614: c8 3c 29 39 addi r9,r9,15560
c00000000008c618: 00 48 2a 7c cmpd r10,r9
c00000000008c61c: 14 00 82 40 bne c00000000008c630 <rcu_process_callbacks+0x48>
c00000000008c620: 6d d6 f8 4b bl c000000000019c8c <arch_local_irq_restore+0x8>
c00000000008c624: 00 00 00 60 nop
c00000000008c628: 40 00 21 38 addi r1,r1,64
c00000000008c62c: ac c3 23 48 b c0000000002c89d8 <_restgpr0_28>
c00000000008c630: 00 00 0a e9 ld r8,0(r10)
c00000000008c634: 2b 01 e2 3c addis r7,r2,299
c00000000008c638: c8 3c e7 eb ld r31,15560(r7)
c00000000008c63c: c8 3c 07 f9 std r8,15560(r7)
c00000000008c640: 00 00 00 39 li r8,0
c00000000008c644: 00 00 0a f9 std r8,0(r10)
c00000000008c648: 2b 01 42 3d addis r10,r2,299
c00000000008c64c: d8 3c 0a e9 ld r8,15576(r10)
c00000000008c650: 2b 01 42 3d addis r10,r2,299
c00000000008c654: d0 3c 4a e9 ld r10,15568(r10)
c00000000008c658: 00 50 28 7c cmpd r8,r10
c00000000008c65c: 08 00 82 40 bne c00000000008c664 <rcu_process_callbacks+0x7c>
c00000000008c660: 10 00 29 f9 std r9,16(r9)
c00000000008c664: 08 00 29 f9 std r9,8(r9)
c00000000008c668: 00 00 80 3b li r28,0
c00000000008c66c: 21 d6 f8 4b bl c000000000019c8c <arch_local_irq_restore+0x8>
c00000000008c670: 00 00 00 60 nop
c00000000008c674: 00 00 3f 2c cmpdi r31,0
c00000000008c678: b0 ff 82 41 beq c00000000008c628 <rcu_process_callbacks+0x40>
c00000000008c67c: 00 00 df eb ld r30,0(r31)
c00000000008c680: 00 00 3e 2c cmpdi r30,0
c00000000008c684: 08 00 82 41 beq c00000000008c68c <rcu_process_callbacks+0xa4>
c00000000008c688: 2c f2 00 7c dcbtct 0,r30
c00000000008c68c: 08 00 bf eb ld r29,8(r31)
c00000000008c690: 00 00 3d 2c cmpdi r29,0
c00000000008c694: 10 00 82 40 bne c00000000008c6a4 <rcu_process_callbacks+0xbc>
c00000000008c698: 78 fb e3 7f mr r3,r31
c00000000008c69c: 19 f8 03 48 bl c0000000000cbeb4 <kmem_dump_obj+0x8>
c00000000008c6a0: 00 00 00 60 nop
c00000000008c6a4: 08 00 9f fb std r28,8(r31)
c00000000008c6a8: 78 fb e3 7f mr r3,r31
c00000000008c6ac: a6 03 a9 7f mtctr r29
c00000000008c6b0: 78 eb ac 7f mr r12,r29
c00000000008c6b4: 18 00 41 f8 std r2,24(r1)
c00000000008c6b8: 78 f3 df 7f mr r31,r30
c00000000008c6bc: 21 04 80 4e bctrl
c00000000008c6c0: 18 00 41 e8 ld r2,24(r1)
c00000000008c6c4: b0 ff ff 4b b c00000000008c674 <rcu_process_callbacks+0x8c>
Sounds like some sorta compiler optimizations, right?
Bests,
Manili
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20251122/d0cd40b3/attachment.htm>
More information about the Linuxppc-dev
mailing list