[SLOF] [PATCH 0/5] Fix stack overflow problem when using many USB devices

[Thomas Huth]

Current version of SLOF has a stack overflow problem that
can be triggered by using a lot of USB devices, for example with:

qemu-system-ppc64 -nographic -vga none -device pci-ohci,id=ohci0 \
    `for ((i=0;i<38;i++)); do echo -n "-device usb-mouse " ; done`

When using so many USB devices, QEMU builds a tree of USB hubs to
connect them. When the USB scan walks this tree, there is a
recursion via the engine() function (in slof/paflof.c):
Forth code from dev-hub.fs calls usb_hub_init(), which calls
setup_new_device(), which then calls slof_usb_handle() to execute
Forth code for the next device via engine(). If that next device
is a hub, we end up recursively in dev-hub.fs again.

That's theoretically fine, but currently the engine() function
has a very huge stack frame (more than 3000 bytes if I measured
it right), so we quickly overflow the system stack (which is
currently defined as 8k bytes in entry.S).

There are two major culprits for this heavy stack usage: libnvram
and the FAST_RFILL macro, which are both using large arrays
in the Forth-to-C glue code (that gets included into the engine()
function). This patch series fixes these problems by moving the
code with the arrays to separate C functions instead, so that the
arrays only temporarily occupy the stack instead of blocking the
longer-lasting stack frames of the engine() function.

Beside these two major fixes, the first patch also introduces
a simple check for stack overlow in engine(), so that such
problems are easier to detect and debug in the future.
The second patch saves some additional bytes in libusb, which
is neglectable compared to libnvram and FAST_RFILL, but I
noticed this first, before finding the other two, so I've
included this patch, too.

All in all, stack usage in paflof is now down from more than
16kB to ca. 6kB when the USB bus is fully populated.

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1261288

Thomas Huth (5):
  Check for stack overflow in paflof engine
  Stack optimization in libusb: split up setup_new_device()
  Improve stack usage in libnvram
  Rework wrapper for new_nvram_partition() and fix possible bug in there
  Move the code for rfill into a separate function

 include/ppcp7/cache.h      | 13 ++------
 lib/libhvcall/Makefile     |  2 +-
 lib/libhvcall/rfill.c      | 38 +++++++++++++++++++++++
 lib/libnvram/envvar.c      | 43 ++++++++++++++++++++++++++
 lib/libnvram/libnvram.code | 75 +++++++++++++++++++---------------------------
 lib/libnvram/nvram.c       | 26 ++++++++++++++++
 lib/libnvram/nvram.h       |  6 ++++
 lib/libusb/usb-core.c      | 35 ++--------------------
 lib/libusb/usb-core.h      |  7 +++--
 lib/libusb/usb-ehci.c      |  4 ++-
 lib/libusb/usb-hub.c       |  4 ++-
 lib/libusb/usb-ohci.c      |  4 ++-
 lib/libusb/usb-slof.c      | 34 ++++++++++++++++++++-
 lib/libusb/usb-xhci.c      |  4 ++-
 slof/entry.S               |  2 +-
 slof/paflof.c              |  9 ++++++
 16 files changed, 209 insertions(+), 97 deletions(-)
 create mode 100644 lib/libhvcall/rfill.c

-- 
Stack Wars - The Forth awakens... Coming soon to a theater near you!