CONFIG_PPC_VAS depends on 64k pages...?

Sat Jan 16 00:54:27 AEDT 2021

Hi all, any news on this matter?

Can a patch be submitted for evaluation?

Thanks,
Carlos

On Wed, Dec 2, 2020 at 4:19 AM Will Springer <skirmisher at protonmail.com>
wrote:

> On Tuesday, December 1, 2020 5:16:51 AM PST Bulent Abali wrote:
> > I don't know anything about VAS page size requirements in the kernel.  I
> > checked the user compression library and saw that we do a sysconf to
> > get the page size; so the library should be immune to page size by
> > design. But it wouldn't surprise me if a 64KB constant is inadvertently
> > hardcoded somewhere else in the library.  Giving heads up to Tulio and
> > Raphael who are owners of the github repo.
> >
> > https://github.com/libnxz/power-gzip/blob/master/lib/nx_zlib.c#L922
> >
> > If we got this wrong in the library it might manifest itself as an error
> > message of the sort "excessive page faults".  The library must touch
> > pages ahead to make them present in the memory; occasional page faults
> > is acceptable. It will retry.
>
> Hm, good to know. As I said I haven't noticed any problems so far, over a
> few different days of testing. My change is now in the Void Linux kernel
> package, and is working for others as well (including the Void maintainer
> Daniel/q66 who I CC'd initially).
>
> >
> > Bulent
> >
> >
> >
> >
> > From:        "Sukadev Bhattiprolu" <sukadev at linux.ibm.com>
> > To:        "Christophe Leroy" <christophe.leroy at csgroup.eu>
> > Cc:        "Will Springer" <skirmisher at protonmail.com>,
> > linuxppc-dev at lists.ozlabs.org, daniel at octaforge.org, Bulent
> > Abali/Watson/IBM at IBM, haren at linux.ibm.com Date:        12/01/2020 12:53
> > AM
> > Subject:        Re: CONFIG_PPC_VAS depends on 64k pages...?
> >
> > Christophe Leroy [christophe.leroy at csgroup.eu] wrote:
> > > Hi,
> > >
> > > Le 19/11/2020 à 11:58, Will Springer a écrit :
> > > > I learned about the POWER9 gzip accelerator a few months ago when
> > > > the
> > > > support hit upstream Linux 5.8. However, for some reason the Kconfig
> > > > dictates that VAS depends on a 64k page size, which is problematic
> > > > as I
> > > > run Void Linux, which uses a 4k-page kernel.
> > > >
> > > > Some early poking by others indicated there wasn't an obvious page
> > > > size
> > > > dependency in the code, and suggested I try modifying the config to
> > > > switch it on. I did so, but was stopped by a minor complaint of an
> > > > "unexpected DT configuration" by the VAS code. I wasn't equipped to
> > > > figure out exactly what this meant, even after finding the
> > > > offending condition, so after writing a very drawn-out forum post
> > > > asking for help, I dropped the subject.
> > > >
> > > > Fast forward to today, when I was reminded of the whole thing again,
> > > > and decided to debug a bit further. Apparently the VAS platform
> > > > device (derived from the DT node) has 5 resources on my 4k kernel,
> > > > instead of 4 (which evidently works for others who have had success
> > > > on 64k kernels). I have no idea what this means in practice (I
> > > > don't know how to introspect it), but after making a tiny patch[1],
> > > > everything came up smoothly and I was doing blazing-fast gzip
> > > > (de)compression in no time.
> > > >
> > > > Everything seems to work fine on 4k pages. So, what's up? Are there
> > > > pitfalls lurking around that I've yet to stumble over? More
> > > > reasonably,
> > > > I'm curious as to why the feature supposedly depends on 64k pages,
> > > > or if there's anything else I should be concerned about.
> >
> > Will,
> >
> > The reason I put in that config check is because we were only able to
> > test 64K pages at that point.
> >
> > It is interesting that it is working for you. Following code in skiboot
> > https://github.com/open-power/skiboot/blob/master/hw/vas.cshould
> > restrict it to 64K pages. IIRC there is also a corresponding change in
> > some NX registers that should also be configured to allow 4K pages.
>
> Huh, that is interesting indeed. As far as the kernel code, the only thing
> specific to 64k pages I could find was in [1], where
> VAS_XLATE_LPCR_PAGE_SIZE is set. There is also NX_PAGE_SIZE in drivers/
> crypto/nx/nx.h, which is set to 4096, but I don't know if that's related
> to
> kernel page size at all. Without a better idea of the code base, I didn't
> examine more thoroughly.
>
> [1]:
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tree/arch/powerpc/platforms/powernv/vas-window.c#n293
>
> >                 static int init_north_ctl(struct proc_chip *chip)
> >                 {
> >                                  uint64_t val = 0ULL;
> >
> >                                  val = SETFIELD(VAS_64K_MODE_MASK, val,
> > true); val = SETFIELD(VAS_ACCEPT_PASTE_MASK, val, true); val =
> > SETFIELD(VAS_ENABLE_WC_MMIO_BAR, val, true); val =
> > SETFIELD(VAS_ENABLE_UWC_MMIO_BAR, val, true); val =
> > SETFIELD(VAS_ENABLE_RMA_MMIO_BAR, val, true);
> >
> >                                  return vas_scom_write(chip,
> > VAS_MISC_N_CTL, val); }
> >
> > I am copying Bulent Albali and Haren Myneni who have been working with
> > VAS/NX for their thoughts/experience.
>
> Thanks for this and for your input, by the way.
>
> >
> > > Maybe ask Sukadev who did the implementation and is maintaining it ?
> > >
> > > > I do have to say I'm quite satisfied with the results of the NX
> > > > accelerator, though. Being able to shuffle data to a RaptorCS box
> > > > over gigE and get compressed data back faster than most software
> > > > gzip could ever hope to achieve is no small feat, let alone the
> > > > instantaneous results locally.> >
> > > > :)
> > > >
> > > > Cheers,
> > > > Will Springer [she/her]
> > > >
> > > > [1]:
> > > >
> https://github.com/Skirmisher/void-packages/blob/vas-4k-pages/srcpkgs/linux5.9/patches/ppc-vas-on-4k.patch
> > > Christophe
>
> Will [she/her]
>
>
>
>
>

-- 
________________________________________
Carlos Eduardo de Paula
me at carlosedp.com
http://carlosedp.com
https://twitter.com/carlosedp
https://www.linkedin.com/in/carlosedp/
________________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20210115/db864aaf/attachment-0001.htm>