serial hang in qemu-system-ppc64 -M pseries

[Fabiano Rosas]

Rob Landley <rob at landley.net> writes:

> On 4/27/22 10:27, Thomas Huth wrote:
>> On 26/04/2022 12.26, Rob Landley wrote:
>>> When I cut and paste 80-ish characters of text into the Linux serial console, it
>>> reads 16 characters and stops. When I hit space, it reads another 16 characters,
>>> and if I keep at it will eventually catch up without losing data. If I type,
>>> every character shows up immediately.
>> 
>> That "16" certainly comes from VTERM_BUFSIZE in hw/char/spapr_vty.c in the 
>> QEMU sources, I think.
>> 
>>> (On other qemu targets and kernels I can cut and paste an entire uuencoded
>>> binary and it goes through just fine in one go, but this target hangs with big
>>> pastes until I hit keys.)
>>> 
>>> Is this a qemu-side bug, or a kernel-side bug?
>>> 
>>> Kernel config attached (linux 5.18-rc3 or thereabouts), qemu invocation is:
>>> 
>>> qemu-system-ppc64 -M pseries -vga none -nographic -no-reboot -m 256 -kernel
>>> vmlinux -initrd powerpc64leroot.cpio.gz -append "panic=1 HOST=powerpc64le
>>> console=hvc0"
>> 
>> Which version of QEMU are you using?
>
> $ qemu-system-ppc64 --version
> QEMU emulator version 6.2.92 (v6.2.0-rc2)
> Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
>
> From november. I can pull and rebuild but it'll take a bit. (Hopefully
> rebuilding would fix the need to echo -e '\e[?7h' afterwards to undo the "bash
> command line history marches up the screen because qemu's x86 bios disabled line
> wrap and then left it that way" issue...)
>
>> Which frontend (GTK or terminal?) ... 
>
> The above command line has -nographic, forcing terminal. Running ldd on the
> binary doesn't pull up anything gtk. (It pulls up libncursesw though.)
>
> If you want to reproduce my test locally:
>
> wget https://landley.net/toybox/downloads/binaries/mkroot/0.8.5/powerpc64le.tgz
> tar xvzf powerpc64le.tgz
> cd powerpc64le
> ./qemu-*.sh
>
> Then paste something longer than 16 characters at the eventual command prompt
> once the kernel finishes booting.

I suspect this is due to how the tty driver (n_tty.c) interacts with
the console (hvc_console.c) driver's buffer size.

This is the stack:

#0 hvc_push          <-- calls into KVM/QEMU to write up to 16 bytes
#1 hvc_write
#2 tty_put_char
#3 do_output_char
#4 __process_echoes          <-- writes up to tty_write_room() chars
#5 flush_echoes              <-- returns early if ~ECHO && ~ECHONL
#6 n_tty_receive_buf_common  <-- buffers more than 16 bytes
#7 tty_ldisc_receive_buf
#8 tty_port_default_receive_buf
#9 receive_buf
#10 process_one_work

In my busybox instance which does not have this issue I can see that
termios.c_lflag = 0x447, so in the stack above #4 is not called (ECHO
is 0x8, ECHONL is 0x10).

In the bug scenario: termios.c_lflag = 0x5cf, so we go into #4 which
is supposed to write (say) 17 bytes, but ends up writing only 16
because that is what tty_write_room() returns.

What I think is causing this issue is that the hvc_vio.c driver is
configured with hp->outbuf_size = 16. That number comes from the
H_PUT_TERM_CHAR hypercall spec which reads two registers at a
time. However, the hvc_write function supports writes larger than 16
bytes so it seems we're needlessly propagating the 16 byte limitation
to the upper layer.

The driver is also not buffering the write, so if it gets called to
write one char at a time (like __process_echoes does) there should be no
limit to how much it can write.

I think if we increase hp->outbuf_size to a value that is larger than
N_TTY_BUF_SIZE=4096 the echo buffer would drain before reaching this new
hvc driver limit.

I tested that and it seems to work, but I'm not sure if it's the right
fix, there are some things I couldn't figure out:

   i) if a driver actually runs out of buffer space, what
      __process_echoes should do about it;

  ii) why the bug sometimes happens only at the 32 characters boundary
      (or other multiple of 16);

 iii) why the ECHO flag differs from the working case.

> If you want to reproduce it all from source:
>
> git clone https://github.com/landley/toybox
> cd toybox && mkdir ccc && cd ccc
> wget
> https://landley.net/toybox/downloads/binaries/toolchains/latest/powerpc64le-linux-musl-cross.tar.xz
> -O - | tar xv
> cd ..
> CROSS=powerpc64le LINUX=~/linux scripts/mkroot.sh
> cd root/powerpc64le
> ./qemu-*.sh
>
> This assumes your linux kernel source directory is in ~/linux of course, and
> that qemu-system-ppc64 is in the $PATH...
>
>> this rings a distant bell, but I thought we had fixed these issues long ago 
>> in the past... e.g.:
>>
>> https://yhbt.net/lore/all/1380113886-16845-10-git-send-email-mdroth@linux.vnet.ibm.com/
>> 
>> https://git.qemu.org/?p=qemu.git;a=commitdiff;h=8a273cbe53221d28
>
> The qemu I'm running is newer than 2016. :)
>
> Most targets are fine with this: I cut and paste entire uuencoded binaries into
> the serial console as an easy way to insert a file into an initramfs. It can
> usually take multiple megabytes without dropping a character, so you just
> "uudecode" enter, and then paste.
>
> Even my 32 bit powerpc target works fine with this. (Although -M g3beige is a

> very different machine from -M pseries...)
>
> Alas this target (and sh4 -m r2d) stop at 16 chars. (On sh4 the extra is
> discarded, not delivered progressively as more interrupts are generated.)
>
>> ... but maybe my memory also just fails and this has never been properly fixed.
>> 
>>   Thomas
>
> Rob