Kernel panic when net booting OpenBMC

Thu Oct 15 22:08:13 AEDT 2020

Hi Nirenjan,

On Thu, Oct 15, 2020 at 5:13 AM Nirenjan Krishnan <nkrishnan at nvidia.com>
wrote:

> Hi,
>
> I’m trying to netboot an OpenBMC image on a custom target board that we
> are building. The board boots fine when we boot from flash, but when
> booting from the network, I run into the following kernel panic:
>
> [    1.195106] /dev/root: Can't open blockdev
> [    1.195425] VFS: Cannot open root device "nfs" or unknown-block(0,255):
> error -6
> [    1.195736] Please append a correct "root=" boot option; here are the
> available partitions:
> [    1.196391] 1f00           32768 mtdblock0
> [    1.196438]  (driver?)
> [    1.196769] 1f01             384 mtdblock1
> [    1.196775]  (driver?)
> [    1.197074] 1f02             128 mtdblock2
> [    1.197079]  (driver?)
> [    1.197357] 1f03            4352 mtdblock3
> [    1.197362]  (driver?)
> [    1.197647] 1f04           23808 mtdblock4
> [    1.197652]  (driver?)
> [    1.197949] 1f05            4096 mtdblock5
> [    1.197955]  (driver?)
> [    1.198246] 1f06           32768 mtdblock6
> [    1.198252]  (driver?)
> [    1.198713] Kernel panic - not syncing: VFS: Unable to mount root fs on
> unknown-block(0,255)
> [    1.199310] CPU: 0 PID: 1 Comm: swapper Not tainted
> 5.8.0-671550e-dirty-3ae2c18 #1
> [    1.199860] Hardware name: Generic DT based system
> [    1.200228] Backtrace:
> [    1.200671] [<80106f1c>] (dump_backtrace) from [<80107170>]
> (show_stack+0x20/0x24)
> [    1.201155]  r7:00008000 r6:808ec4b4 r5:00000000 r4:808fe258
> [    1.201512] [<80107150>] (show_stack) from [<807a5414>]
> (dump_stack+0x28/0x30)
> [    1.201882] [<807a53ec>] (dump_stack) from [<80116160>]
> (panic+0xf8/0x320)
> [    1.202229]  r5:00000000 r4:80b84290
> [    1.202499] [<80116068>] (panic) from [<80a01aa0>]
> (mount_block_root+0x2e4/0x398)
> [    1.202881]  r3:80b03008 r2:00000000 r1:8e4a1ec4 r0:808ec4b4
> [    1.203168]  r7:00008000
> [    1.203316] [<80a017bc>] (mount_block_root) from [<80a01bcc>]
> (mount_root+0x78/0x94)
> [    1.203718]  r10:8098cf04 r9:80a2983c r8:80a2985c r7:00000008
> r6:8e4966e0 r5:80a29875
> [    1.204102]  r4:000000ff
> [    1.204242] [<80a01b54>] (mount_root) from [<80a01d24>]
> (prepare_namespace+0x13c/0x194)
> [    1.204640]  r5:80a29875 r4:80b84028
> [    1.204840] [<80a01be8>] (prepare_namespace) from [<80a0154c>]
> (kernel_init_freeable+0x1b0/0x1f8)
> [    1.205282]  r5:80b84000 r4:00000096
> [    1.205495] [<80a0139c>] (kernel_init_freeable) from [<807bedc4>]
> (kernel_init+0x18/0x120)
> [    1.205913]  r10:00000000 r9:00000000 r8:00000000 r7:00000000
> r6:00000000 r5:807bedac
> [    1.206305]  r4:00000000
> [    1.206454] [<807bedac>] (kernel_init) from [<80100128>]
> (ret_from_fork+0x14/0x2c)
> [    1.206898] Exception stack(0x8e4a1fb0 to 0x8e4a1ff8)
> [    1.207303] 1fa0:                                     00000000 00000000
> 00000000 00000000
> [    1.207831] 1fc0: 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000 00000000
> [    1.208353] 1fe0: 00000000 00000000 00000000 00000000 00000013 00000000
> [    1.208739]  r5:807bedac r4:00000000
> [    1.209505] ---[ end Kernel panic - not syncing: VFS: Unable to mount
> root fs on unknown-block(0,255) ]---
>
> I enabled CONFIG_NETWORK_FILESYSTEMS, CONFIG_NFS_FS and CONFIG_ROOT_NFS in
> the kernel configuration, rebuilt the kernel and verified that the flash
> boot works fine.
>
> These are the commands that I ran to netboot
>
> ast# tftp 83000000 uImage
> ast# tftp 84000000 aspeed.dtb
> ast# setenv serverip 10.x.x.x
> ast# setenv bootargs root=/dev/nfs rw
> nfsroot=${serverip}:/data/nfs/test,nolock,tcp console=ttyS4,115200n8
> mem.devmem=1 ip=${ipaddr}:${serverip}:${serverip}:255.0.0.0::eth0
> nfsrootdebug
> ast# bootm 83000000 - 84000000
>
> I also used the fitImage directly instead of a separate uImage and DTB,
> and ran into the same error. I tried enabling CPIO image builds by adding
> the following line in local.conf:
>
> IMAGE_FSTYPES += "cpio.${INITRAMFS_CTYPE}.u-boot"
>

It seems like the rootfs can't be found. The fitImage based approach
usually works well for me. Can you elaborate what you did with the fitImage
based approach? I typically do this (on an AST2500) :

1) Add IMAGE_FSTYPES += "cpio.${INITRAMFS_CTYPE}.u-boot" to the conf
2) Copy fitImage and obmc-phosphor-image-<platform>.cpio.lzma.u-boot to the
tftp server.
3) Setup the tftp server and the BMC network
4) tftp 0x83000000 fitImage
5) tftp 0x93000000 initrd (in step 2 I had named the cpio image as initrd
on the tftp server). At this stage 'iminfo' should show what images have
been loaded.
6) bootm 0x83000000 0x93000000

If you want to instead boot from NFS, then I see Yu Lei has some
suggestions/questions for you.

Regards,
Deepak

> However, if I use the generated CPIO image directly as the initramfs, then
> all I see is "Starting kernel ..." and no further output from the system.
>
> Could you please suggest how I could go about debugging this? I've
> followed all the netboot suggestions from these threads on this mailing
> list:\
> - https://lists.ozlabs.org/pipermail/openbmc/2018-June/012054.html
> - https://lists.ozlabs.org/pipermail/openbmc/2015-October/000010.html
>
> Thanks and regards
> Nirenjan Krishnan
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20201015/902a9780/attachment.htm>