[Help] Microwatt (Zynqwatt) — Kernel halts after Radix MMU init on booting Linux on Zynq version of Microwatt
Mohammad Amin Nili
manili.devteam at gmail.com
Thu Nov 13 01:45:22 AEDT 2025
Dear PPC64 Kernel Developers,
Hi all,
I'm trying to redo the Microwatt project on my own ZCU104 evaluation board
(Zynq US+). The main goal is to expose PS-side peripherals (specifically
the UART0 and LPDDR4 memory) to the PL-side Microwatt via the AXI4 bus and
MMIO. For simplicity, I'll call this whole project 'Zynqwatt'.
I successfully managed to boot and execute both `Hello World` and
`MicroPython` on my Zynqwatt. Here's what I've done so far:
1. I removed almost every peripheral except the `XICS` from the `SoC`
module.
2. I routed all addresses to a `WB2AXI` bridge so that the Zynq's PS-side
memory/MMIO space is completely accessible by the PL-side `Microwatt`.
3. I booted up my PS (Arm Cortex-A53) and let the Arm's `FSBL` initialize
almost all of the PS peripherals (e.g. UART, LPDDR4, etc.).
4. I copied my program from the SD card (`Hello World` or `MicroPython`)
into a fixed location of the PS DRAM using the Arm.
5. I released the `Microwatt` reset signal.
6. I parked the Arm processor.
7. The system worked successfully.
However, booting Linux seems to be a whole different story! Here's what
I've done so far to boot Linux on my Zynqwatt platform:
1. I’ve written a simple **bootloader** that runs on the Zynq’s Arm core.
Its job is to read the kernel image — `dtbImage.microwatt.elf` — from
the SD card and relocate it to the proper PS-side memory according to
the ELF header information.
2. The bootloader **releases the Microwatt reset signal**, and Microwatt
starts executing a super simple “hello world” bootloader. This minimal
program just prints a `Welcome` message and jumps to the kernel start
address (usually at `0x01700000`). Here is the log of this stage:
```
Zynq MP First Stage Boot Loader
Release 2025.1 Nov 9 2025 - 19:22:42
PMU-FW is not running, certain applications may not be supported.
Downloading bootloader to the DRAM...
Successfully downloaded bootloader to the DRAM at 0x20000000!
Downloading Linux ELF file to the DRAM...
Starting ELF read from SD card...
Initializing SDPS driver...
SDPS driver and card initialized successfully.
Reading 7340032 bytes from sector offset 0 to address 0x30000000
ELF file read from SD card successfully.
Successfully downloaded ELF file to the DRAM at 0x30000000!
Extracting Linux ELF file to the DRAM...
Successfully extracted ELF file to the DRAM!
Configuring Microwatt for booting...
Successfully configured Microwatt!
Booting up Microwatt from bootloader at 0x20000000...
--------------------------------------------------
.oOOo.
." ".
; .mw. ; Microwatt, it works.
. ' ' .
\ || /
;..;
;..;
`ww'
Function <my_printf> is located at 0x00001354.
Executing: *(0x01700000) --> 0x480000d0.
Press any key to continue...
```
3. The **kernel wrapper** tries to read the DTB and decompress the gzipped
kernel into `0x00000000`:
```
zImage starting: loaded at 0x0000000001700000 (sp: 0x0000000001b1beb0)
Allocating 0x16c5144 bytes for kernel...
Decompressing (0x0000000000000000 <- 0x0000000001714000:0x0000000001b19fa2)...
Done! Decompressed 0x16c5144 bytes
Linux/PowerPC load: earlycon earlyprintk debug loglevel=9
Finalizing device tree... flat tree at 0x1b1cc80
```
4. The kernel successfully decompresses and hands control over to
`arch/powerpc/kernel/head_64.S`. This is exactly where I’m stuck. Here’s
the relevant log output:
```
Linux/PowerPC load: earlycon earlyprintk debug loglevel=9
Finalizing device tree... flat tree at 0x1b1cc80
[ 0.000000] printk: legacy bootconsole [udbg0] enabled
[ 0.000000] ioremap() called early from of_setup_earlycon+0xd0/0x2d4. Use early_ioremap() instead
[ 0.000000] earlycon: earlycon_map: Couldn't map 0x00000000ff000000
[ 0.000000] earlycon: cdns0 at MMIO 0x00000000ff000000 (options '')
[ 0.000000] printk: legacy bootconsole [cdns0] enabled
[ 0.000000] dt-cpu-ftrs: setup for ISA 3100
[ 0.000000] dt-cpu-ftrs: final cpu/mmu features = 0x00040083800bb181 0x20005040
[ 0.000000] radix-mmu: Page sizes from device-tree:
[ 0.000000] radix-mmu: Page size shift = 12 AP=0x0
[ 0.000000] radix-mmu: Page size shift = 16 AP=0x5
[ 0.000000] radix-mmu: Page size shift = 21 AP=0x1
[ 0.000000] radix-mmu: Page size shift = 30 AP=0x2
[ 0.000000] radix-mmu: Mapped 0x0000000000000000-0x0000000001800000 with 2.00 MiB pages (exec)
[ 0.000000] radix-mmu: Mapped 0x0000000001800000-0x0000000010000000 with 2.00 MiB pages
[ 0.000000] radix-mmu: Initializing Radix MMU
```
NOTE: As you know, the `earlycon` driver depends on calling `ioremap`
in `drivers/tty/serial/earlycon.c:L47` to get the virtual address of its
`mapbase/membase`, which is later used by the main console driver (e.g.
`cdns,uart-r1p12`). During the very early boot phase, `ioremap` returns
`0x00000000` because it is called before `early_init_mmu` and
`early_ioremap_setup` in the `early_setup` of
`arch/powerpc/kernel/setup_64.S`.
For debugging purposes, I hardcoded it to return the **physical address**
`0xFF000000` — which corresponds to the Zynq PS-side UART0.
5. From what I can tell, **the system crashes or loses console output
immediately after the MMU is activated** — specifically right after the
call to `RFI_TO_KERNEL` in `arch/powerpc/kernel/head_64.S:L1019`.
6. To test my assumption, I tried **disabling the MMU** by inserting the
following lines around `head_64.S:L1019`:
```
LOAD_REG_ADDR(r3, start_here_common)
ld r4,PACAKMSR(r13)
//*****************************************************
li r5, -49
and r4, r4, r5
//*****************************************************
mtspr SPRN_SRR0,r3
mtspr SPRN_SRR1,r4
RFI_TO_KERNEL
b . /* prevent speculative execution */
```
The result was:
```
[ 0.000000] printk: legacy bootconsole [udbg0] enabled
[ 0.000000] ioremap() called early from earlycon_map+0x20/0x6c. Use early_ioremap() instead
[ 0.000000] earlycon: earlycon_map: Couldn't map 0x00000000ff000000
[ 0.000000] earlycon: Temporarily mapped it to 0x00000000ff000000
[ 0.000000] earlycon: cdns0 at MMIO 0x00000000ff000000 (options '')
[ 0.000000] printk: legacy bootconsole [cdns0] enabled
[ 0.000000] dt-cpu-ftrs: setup for ISA 3100
[ 0.000000] dt-cpu-ftrs: final cpu/mmu features = 0x00040083800bb181 0x20005040
[ 0.000000] radix-mmu: Page sizes from device-tree:
[ 0.000000] radix-mmu: Page size shift = 12 AP=0x0
[ 0.000000] radix-mmu: Page size shift = 16 AP=0x5
[ 0.000000] radix-mmu: Page size shift = 21 AP=0x1
[ 0.000000] radix-mmu: Page size shift = 30 AP=0x2
[ 0.000000] radix-mmu: Mapped 0x0000000000000000-0x0000000001800000 with 2.00 MiB pages (exec)
[ 0.000000] radix-mmu: Mapped 0x0000000001800000-0x0000000010000000 with 2.00 MiB pages
[ 0.000000] radix-mmu: Initializing Radix MMU
[ 0.000000] Linux version 6.18.0-rc3-00007-gfd57572253bc-dirty (manili at manili) (powerpc64le-linux-gcc.br_real (Buildroot 2021.11-18033-g83947c7bb6) 14.3.0, GNU ld (GNU Binutils) 2.43.1) #124 Sun Nov 9 23:46:11 EST 2025
[ 0.000000] Hardware name: microwatt Microwatt 0x630000 microwatt
[ 0.000000] -----------------------------------------------------
[ 0.000000] phys_mem_size = 0x10000000
[ 0.000000] dcache_bsize = 0x40
[ 0.000000] icache_bsize = 0x40
[ 0.000000] cpu_features = 0x00040083800bb181
[ 0.000000] possible = 0x003ffbebcb5fb185
[ 0.000000] always = 0x0000000380008181
[ 0.000000] cpu_user_features = 0xcc002102 0x8c940000
[ 0.000000] mmu_features = 0x20005040
[ 0.000000] firmware_features = 0x0000000000000000
[ 0.000000] vmalloc start = 0xc008000000000000
[ 0.000000] IO start = 0xc00a000000000000
[ 0.000000] vmemmap start = 0xc00c000000000000
[ 0.000000] -----------------------------------------------------
[ 0.000000] barrier-nospec: using ORI speculation barrier
[ 0.000000] barrier-nospec: patched 100 locations
[ 0.000000] Top of RAM: 0x10000000, Total RAM: 0x10000000
[ 0.000000] Memory hole size: 0MB
[ 0.000000] Zone ranges:
[ 0.000000] Normal [mem 0x0000000000000000-0x000000000fffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000000000-0x000000000fffffff]
[ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000000fffffff]
```
So unlike the previous logs, it looks like the kernel continues the boot
process after `radix-mmu: Initializing Radix MMU`, but this time halts at
`Initmem setup...` because the MMU is off (I guess).
7. Here’s my current DTS configuration. I’ve kept it as **simple and close
to Microwatt’s default** as possible:
```
/dts-v1/;
#include <dt-bindings/gpio/gpio.h>
/ {
#size-cells = <0x02>;
#address-cells = <0x02>;
model = "microwatt";
compatible = "microwatt-soc";
aliases {
serial0 = &UART0;
};
reserved-memory {
#size-cells = <0x02>;
#address-cells = <0x02>;
ranges;
};
memory at 0 {
device_type = "memory";
reg = <0x00000000 0x00000000 0x00000000 0x10000000>;
};
clocks {
sys_clk: litex_sys_clk {
#clock-cells = <0>;
compatible = "fixed-clock";
clock-frequency = <100000000>;
};
};
cpus {
#size-cells = <0x00>;
#address-cells = <0x01>;
ibm,powerpc-cpu-features {
display-name = "Microwatt";
isa = <3100>;
device_type = "cpu-features";
compatible = "ibm,powerpc-cpu-features";
mmu-radix {
isa = <3000>;
usable-privilege = <6>;
os-support = <0>;
};
little-endian {
isa = <0>;
usable-privilege = <7>;
os-support = <0>;
hwcap-bit-nr = <1>;
};
cache-inhibited-large-page {
isa = <0>;
usable-privilege = <6>;
os-support = <0>;
};
fixed-point-v3 {
isa = <3000>;
usable-privilege = <7>;
};
no-execute {
isa = <0x00>;
usable-privilege = <2>;
os-support = <0>;
};
floating-point {
hfscr-bit-nr = <0>;
hwcap-bit-nr = <27>;
isa = <0>;
usable-privilege = <7>;
hv-support = <1>;
os-support = <0>;
};
prefixed-instructions {
hfscr-bit-nr = <13>;
fscr-bit-nr = <13>;
isa = <3010>;
usable-privilege = <7>;
os-support = <1>;
hv-support = <1>;
};
tar {
hfscr-bit-nr = <8>;
fscr-bit-nr = <8>;
isa = <2070>;
usable-privilege = <7>;
os-support = <1>;
hv-support = <1>;
hwcap-bit-nr = <58>;
};
control-register {
isa = <0>;
usable-privilege = <7>;
};
system-call-vectored {
isa = <3000>;
usable-privilege = <7>;
os-support = <1>;
fscr-bit-nr = <12>;
hwcap-bit-nr = <52>;
};
};
PowerPC,Microwatt at 0 {
i-cache-sets = <2>;
ibm,dec-bits = <64>;
reservation-granule-size = <64>;
clock-frequency = <100000000>;
timebase-frequency = <100000000>;
i-tlb-sets = <1>;
ibm,ppc-interrupt-server#s = <0>;
i-cache-block-size = <64>;
d-cache-block-size = <64>;
d-cache-sets = <2>;
i-tlb-size = <64>;
cpu-version = <0x990000>;
status = "okay";
i-cache-size = <0x1000>;
ibm,processor-radix-AP-encodings = <0x0c 0xa0000010 0x20000015 0x4000001e>;
tlb-size = <0>;
tlb-sets = <0>;
device_type = "cpu";
d-tlb-size = <128>;
d-tlb-sets = <2>;
reg = <0>;
general-purpose;
64-bit;
d-cache-size = <0x1000>;
ibm,chip-id = <0>;
ibm,mmu-lpid-bits = <12>;
ibm,mmu-pid-bits = <20>;
};
};
soc {
compatible = "simple-bus";
#address-cells = <0x02>;
#size-cells = <0x02>;
ranges;
interrupt-controller at c0004000 {
compatible = "openpower,xics-presentation", "ibm,ppc-xicp";
ibm,interrupt-server-ranges = <0x0 0x1>;
reg = <0x0 0xc0004000 0x0 0x10>;
};
ICS: interrupt-controller at c0005000 {
compatible = "openpower,xics-sources";
interrupt-controller;
interrupt-ranges = <0x10 0x10>;
reg = <0x0 0xc0005000 0x0 0x100>;
#address-cells = <0>;
#size-cells = <0>;
#interrupt-cells = <2>;
};
UART0: serial at ff000000 {
device_type = "serial";
compatible = "cdns,uart-r1p12";
reg = <0x0 0xff000000 0x0 0x1000>;
clock-frequency = <100000000>;
current-speed = <115200>;
};
};
chosen {
bootargs = "earlycon earlyprintk debug loglevel=9";
ibm,architecture-vec-5 = [19 00 10 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 40 00 40];
stdout-path = &UART0;
};
};
```
So, why is the MMU preventing the kernel from booting properly — or at
least stopping the UART driver from accessing its mapped address at
`0xFF000000`? Could this be an issue specific to the **Cadence/Xilinx UART
driver**? Or am I missing something fundamental about how the MMU should
be configured in this setup?
Let me know if you need any other sources or data to share. I’ve tried
everything I can think of but still cannot figure out what’s wrong with my
kernel porting process. Any help would be much appreciated.
Bests,
Manili
More information about the Linuxppc-dev
mailing list