CONFIG_PPC_VAS depends on 64k pages...?

[Bulent Abali]

I don't know anything about VAS page size requirements in the kernel.  I 
checked the user compression library and saw that we do a sysconf to get 
the page size; so the library should be immune to page size by design.
But it wouldn't surprise me if a 64KB constant is inadvertently hardcoded 
somewhere else in the library.  Giving heads up to Tulio and Raphael who 
are owners of the github repo.

https://github.com/libnxz/power-gzip/blob/master/lib/nx_zlib.c#L922

If we got this wrong in the library it might manifest itself as an error 
message of the sort "excessive page faults".  The library must touch pages 
ahead to make them present in the memory; occasional page faults is 
acceptable. It will retry.


Bulent




From:   "Sukadev Bhattiprolu" <sukadev at linux.ibm.com>
To:     "Christophe Leroy" <christophe.leroy at csgroup.eu>
Cc:     "Will Springer" <skirmisher at protonmail.com>, 
linuxppc-dev at lists.ozlabs.org, daniel at octaforge.org, Bulent 
Abali/Watson/IBM at IBM, haren at linux.ibm.com
Date:   12/01/2020 12:53 AM
Subject:        Re: CONFIG_PPC_VAS depends on 64k pages...?




Christophe Leroy [christophe.leroy at csgroup.eu] wrote:
> Hi,
> 
> Le 19/11/2020 à 11:58, Will Springer a écrit :
> > I learned about the POWER9 gzip accelerator a few months ago when the
> > support hit upstream Linux 5.8. However, for some reason the Kconfig
> > dictates that VAS depends on a 64k page size, which is problematic as 
I
> > run Void Linux, which uses a 4k-page kernel.
> > 
> > Some early poking by others indicated there wasn't an obvious page 
size
> > dependency in the code, and suggested I try modifying the config to 
switch
> > it on. I did so, but was stopped by a minor complaint of an 
"unexpected DT
> > configuration" by the VAS code. I wasn't equipped to figure out 
exactly what
> > this meant, even after finding the offending condition, so after 
writing a
> > very drawn-out forum post asking for help, I dropped the subject.
> > 
> > Fast forward to today, when I was reminded of the whole thing again, 
and
> > decided to debug a bit further. Apparently the VAS platform device
> > (derived from the DT node) has 5 resources on my 4k kernel, instead of 
4
> > (which evidently works for others who have had success on 64k 
kernels). I
> > have no idea what this means in practice (I don't know how to 
introspect
> > it), but after making a tiny patch[1], everything came up smoothly and 
I
> > was doing blazing-fast gzip (de)compression in no time.
> > 
> > Everything seems to work fine on 4k pages. So, what's up? Are there
> > pitfalls lurking around that I've yet to stumble over? More 
reasonably,
> > I'm curious as to why the feature supposedly depends on 64k pages, or 
if
> > there's anything else I should be concerned about.

Will,

The reason I put in that config check is because we were only able to
test 64K pages at that point.

It is interesting that it is working for you. Following code in skiboot
https://github.com/open-power/skiboot/blob/master/hw/vas.c should restrict
it to 64K pages. IIRC there is also a corresponding change in some NX 
registers that should also be configured to allow 4K pages.


                 static int init_north_ctl(struct proc_chip *chip)
                 {
                                 uint64_t val = 0ULL;

                                 val = SETFIELD(VAS_64K_MODE_MASK, val, 
true);
                                 val = SETFIELD(VAS_ACCEPT_PASTE_MASK, 
val, true);
                                 val = SETFIELD(VAS_ENABLE_WC_MMIO_BAR, 
val, true);
                                 val = SETFIELD(VAS_ENABLE_UWC_MMIO_BAR, 
val, true);
                                 val = SETFIELD(VAS_ENABLE_RMA_MMIO_BAR, 
val, true);

                                 return vas_scom_write(chip, 
VAS_MISC_N_CTL, val);
                 }

I am copying Bulent Albali and Haren Myneni who have been working with
VAS/NX for their thoughts/experience.

> > 
> 
> Maybe ask Sukadev who did the implementation and is maintaining it ?
> 
> > I do have to say I'm quite satisfied with the results of the NX
> > accelerator, though. Being able to shuffle data to a RaptorCS box over 
gigE
> > and get compressed data back faster than most software gzip could ever
> > hope to achieve is no small feat, let alone the instantaneous results 
locally.
> > :)
> > 
> > Cheers,
> > Will Springer [she/her]
> > 
> > [1]: 
https://github.com/Skirmisher/void-packages/blob/vas-4k-pages/srcpkgs/linux5.9/patches/ppc-vas-on-4k.patch

> > 
> 
> 
> Christophe




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20201201/7f813355/attachment-0001.htm>