OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions

Wed May 22 03:05:35 AEST 2024

Hi Andrew,

thanks so much for your kind response! I've come across your project (culvert)
on my little quest to liberate my board's BMC already, and I'm eager to try
working with it as soon as I manage to build it (which will leventually
happen)!

But first, I need to share the good news: Yesterday, shortly after I had sent
my initial post to this list, my friend Paul on IRC supplied me with ideas and
instructions to get the OpenBMC stuff to fly on the Gigabyte-supplied AST2500,
and we had smashing success after a rather short while! We patched OpenBMC's
u-boot to assume a CONFIG_SYS_TEXT_BASE of 0x83000000, tftp'd the resulting
binary to that address, issued

`go 0x83000000`

in the AMI u-boot monitor, and... had successfully chainloaded the OpenBMC
u-boot on the Gigabyte BMC!

From there, I could load the OpenBMC FIT and went to a initrd rescue shell
rather quickly. The sequence of commands was, for posterity (assuming a TFTP
server on 192.168.2.123 can supply the mentioned artifacts after successful
DHCPv4 autoconfig):

```
### original/stock AMI uboot monitor:
AST2500EVB> dhcp
AST2500EVB> tftp 0x83000000 192.168.2.123:hacked-uboot
AST2500EVB> go 0x83000000

### chainloaded OpenBMC u-boot:
ast# setenv ethact ethernet at 1e680000
ast# dhcp
ast# tftpboot 0x84000000 192.168.2.123:openbmc-fit-image
ast# setenv bootargs console=ttyS4,115200n8 root=/dev/ram rw enable-initrd-debug-sh debug-init-sh
ast# bootm 0x84000000
```

I have provided a copy of the `hacked-uboot` file contents at [0]. Its
sha256sum hexdigest reads
  2937963c78f577355dc3bd15b8b8e57dcacf72fd74cd82854d1136bb26356e68

From there, I could reconfigure the BMC's Ethernet interface for my LAN,
shuttle in the OpenBMC squashfs-xz image via tftp onto tmpfs, loop-mount that,
and chroot into the resulting directory tree - the userland seems to work,
too! :)

What I am struggling with (as other plans leave me little time today
unfortunately) is how to boot into this environment after manually massaging
the rescue environment into the half-working state. If you have any advice on
how to streamline my boot process, I'd be eager to read about it!

On Tue, May 21, 2024 at 10:35:34AM +0930, Andrew Jeffery wrote:
> [...]
> > The BMC implementation seems lean very much onto the AST2500 Evaluation Board
> > (the unchanged name pops up in a number of places in the stock firmware), so I
> > would guess that the OpenBMC evb-ast2500 machine should be able to get
> > somewhere (and even if it's just to see OpenBMC's instead of AMI's Linux start,
> > and then crash hard due to some subtle incompatibility :)) at least.
> 
> Perhaps. While the SoC itself is obviously supported, how it's
> integrated into the rest of the platform will likely be very different
> to how things are wired up on the EVB.

I think I've already discovered some of the rough edges that are still coming
for me and this attempt at liberating the Gigabyte BMC, as the host does not
seem to want to power on without the (stock) BMC running. That, and all the
other obstacles that are sure to come, I will try to tackle in time :)

> > My understanding is that there are two paths forward:
> > 
> > 1.) To make OpenBMC's build artifacts into FMH-style BLOBs, and find a way to
> > feed them to the stock fw's u-boot, which would ideally result in OpenBMC being
> > able to boot this way.
> > 
> > 2.) To replace the stock fw's u-boot release with an OpenBMC u-boot that can
> > load the evb-ast2500 artifacts natively, and have the BMC boot OpenBMC that
> > way.
> > 
> > Can you please confirm that this assessment is sane, or correct me if it isn't?
> 
> Yeah, that's sane. I'm not familiar with FMH so can't really comment on
> how feasible that approach might be, but my intuition is it's probably
> a bunch of pain.

Because of the successfull attempt at chainloading a hacked u-boot, I think we
can safely discard trying to travel down path #1 and skip learning more about
FMH than absolutely necessary :)

> > If so, I would presume option 2.) to be the easier road to travel, but I am
> > somewhat stumped as to how I can get OpenBMC's u-boot onto the BMC, *ideally*
> > (but not necessarily) in a non-destructive way (as I do not know how to or have
> > a way to revert to the original state without access to the bootloader or even
> > stock firmware running).
> 
> Something to consider is using qemu to boot the proprietary firmware,
[...]
> ast_eth0, ast_eth1
> DRAM ECC enabled
> Hit any key to stop autoboot:  0 
> Image to be booted is 1
> EMMC and EXT4 is not enabled - Cannot locate kernel file in Root
> Initing KCS...done
> Uboot waiting for firmware update to start...
> Uboot waiting for fwupdate to start timed out
> Disabling Watchdog 2 Timer
> AST2500EVB>
> ```

Thanks for this; while I've had success in booting the evb-2500ast artifacts
in QEMU (as documented by OpenBMC and QEMU), I had not successfully tried with
the AMI-supplied artifacts yet. This saved me a ton of time trying to figure
that out!

However, what I do not really understand about the situation in the emulator
is that `fmh` will not list any FMH-style blocks/partitions/area in that case,
whilst it will on the physical BMC. As a consequence, I cannot actually boot
the stock firmware in QEMU passt the u-boot stage, as that relies on `bootfmh`
and the FMH areas to be properly recognized.

Maybe the munging of the image size with `truncate` changes something that the
FMH format can't cope with? I am unsure, and haven't yet looked into the
public FMH bits to confirm this theory's merit.

> If you want to do something more destructive and overwrite the firmware
> on the flash, then you should probably have a flash programmer handy,
> and/or become familiar with something like culvert (which I maintain):
> 
> https://github.com/amboar/culvert

I'll be sure to try my luck with culvert as soon as I am able, and I would
have tried working with it earlier more eagerly if I had not made such
unexpectedly quick progress with the other ways tried ;)

> Note that gigabyte do ship "gigaflash" in that zip file above, which is
> a closed-source application similar to culvert, that derives from
> Aspeed's own "socflash" tool.

I noticed that (but did not know about its lineage), but cannot get it to
actually work for me on my Linux-based host system (grml 2024.02 for amd64 at
the moment), neither for updating nor for dumping the BMC ROM.

Since the binary is not stripped (and seems to have some understanding of FMH
and whatever funny signature business that is going on in the `rom.ima_enc`
parts of the BMC FW ZIP archives), it might still be useful in other ways
after all...

> > [...]
> 
> u-boot is linked at 0 and uses XIP from the SPI-NOR (which is memory-
> mapped at 0), so I suspect it has read out the vendor copy again.

That's what Paul and me guessed on IRC yesterday, and the first attempt at
working around that seems to have been successful, as stated above! :)

> [...]
> A potential collaborator on Gigabyte stuff is bielids. I don't have
> their contact details, but you can find some of our past discussion
> here:
> 
> https://github.com/amboar/culvert/issues/51

Thanks, I've had this issue bookmarked since the start of last weekend
already, and your patient and helpful approach there actually was one of the
main motivating factors for me to actually embark on this journey :) It's kind
people like you who make FOSS a great world to engage in - so let me say thank
you for your service to the general public! :)

Have a nice day!
- Johannes

[0]: https://johannes.truschnigg.info/assets/AMI_ast2500_hacked_u-boot_0x8300000

-- 
with best regards:
- Johannes Truschnigg ( johannes at truschnigg.info )

www:   https://johannes.truschnigg.info/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.ozlabs.org/pipermail/openbmc/attachments/20240521/07d1cf83/attachment.sig>