AST2500 u-boot breakage with CONFIG_RAM=y

[Zev Weiss]

Hello u-boot/Aspeed folks,

I recently set about getting e3c246d4i switched over to 
u-boot-aspeed-sdk from the old u-boot branch, but after building and 
installing it I ran into some odd misbehavior.

It's not entirely consistent -- sometimes the kernel hand-off goes 
alright but then the kernel's boot sequence hangs a few seconds in, 
sometimes it hangs before I get any kernel console output at all, and 
sometimes I end up with the following error message from u-boot as it 
loads the kernel FIT image:

   fdt_find_or_add_subnode: chosen: FDT_ERR_BADSTRUCTURE
   ERROR: /chosen node create failed
    - must RESET the board to recover.
   
   FDT creation failed! hanging...### ERROR ### Please RESET the board ###

Additionally, if I try to boot a FIT image loaded via TFTP I 
consistently get checksum-mismatch errors on the kernel subimage portion 
(though the exact same FIT image loaded from flash passes that check).  

I wasn't able to reproduce any misbehavior in qemu, unfortunately.

I had previously tested the 2019.04 u-boot branch on this platform and 
not hit any problems like this, so I started bisecting between the 
version that had worked before and the current version, which landed me 
on Joel's recent patch adding CONFIG_RAM=y to the evb-ast2500 defconfig 
(commit 0545d7325ff0cb1a77dc6f8007f74f415840fd90).  I confirmed that 
setting CONFIG_RAM=n gets things working normally again.

Looking into the code that CONFIG_RAM=y enables, I added some debug 
prints to arch/arm/mach-aspeed/ast2500/board_common.c and verified that 
the dram_init() routine was setting gd->ram_size to the same value in 
both cases.  However, I noticed there's also one instruction in 
platform.S that's included when CONFIG_RAM is enabled [1].  My 
recollection of ARM assembly is fairly rusty, but I believe that's just 
an early return short-circuiting the rest of the initialization code in 
that routine, perhaps with the intent that that work will get taken care 
of by C code in the Aspeed RAM driver instead?  In any case, I 
experimented with leaving CONFIG_RAM=y but removing just that 
instruction, and it seems to resolve the problems I was seeing -- so if 
my interpretation does match the actual intent, it seems like there's 
some discrepancy between the initialization done in the C code and the 
assembly code, though I'm not sure what it might be.  For what it's 
worth, it did seem like the CONFIG_RAM=n build ran noticeably faster.

Does anyone have any thoughts as to what might be going on, or tips on 
how to go about debugging further?

Thanks,
Zev

[1] https://github.com/openbmc/u-boot/blob/8e834983d0a6b9265cee1674564b016565630883/arch/arm/mach-aspeed/ast2500/platform.S#L663-L665