CONFIG_PPC_VAS depends on 64k pages...?

[Sukadev Bhattiprolu]

Christophe Leroy [christophe.leroy at csgroup.eu] wrote:
> Hi,
> 
> Le 19/11/2020 à 11:58, Will Springer a écrit :
> > I learned about the POWER9 gzip accelerator a few months ago when the
> > support hit upstream Linux 5.8. However, for some reason the Kconfig
> > dictates that VAS depends on a 64k page size, which is problematic as I
> > run Void Linux, which uses a 4k-page kernel.
> > 
> > Some early poking by others indicated there wasn't an obvious page size
> > dependency in the code, and suggested I try modifying the config to switch
> > it on. I did so, but was stopped by a minor complaint of an "unexpected DT
> > configuration" by the VAS code. I wasn't equipped to figure out exactly what
> > this meant, even after finding the offending condition, so after writing a
> > very drawn-out forum post asking for help, I dropped the subject.
> > 
> > Fast forward to today, when I was reminded of the whole thing again, and
> > decided to debug a bit further. Apparently the VAS platform device
> > (derived from the DT node) has 5 resources on my 4k kernel, instead of 4
> > (which evidently works for others who have had success on 64k kernels). I
> > have no idea what this means in practice (I don't know how to introspect
> > it), but after making a tiny patch[1], everything came up smoothly and I
> > was doing blazing-fast gzip (de)compression in no time.
> > 
> > Everything seems to work fine on 4k pages. So, what's up? Are there
> > pitfalls lurking around that I've yet to stumble over? More reasonably,
> > I'm curious as to why the feature supposedly depends on 64k pages, or if
> > there's anything else I should be concerned about.

Will,

The reason I put in that config check is because we were only able to
test 64K pages at that point.

It is interesting that it is working for you. Following code in skiboot
https://github.com/open-power/skiboot/blob/master/hw/vas.c should restrict
it to 64K pages. IIRC there is also a corresponding change in some NX 
registers that should also be configured to allow 4K pages.


	static int init_north_ctl(struct proc_chip *chip)
	{
		uint64_t val = 0ULL;

		val = SETFIELD(VAS_64K_MODE_MASK, val, true);
		val = SETFIELD(VAS_ACCEPT_PASTE_MASK, val, true);
		val = SETFIELD(VAS_ENABLE_WC_MMIO_BAR, val, true);
		val = SETFIELD(VAS_ENABLE_UWC_MMIO_BAR, val, true);
		val = SETFIELD(VAS_ENABLE_RMA_MMIO_BAR, val, true);

		return vas_scom_write(chip, VAS_MISC_N_CTL, val);
	}

I am copying Bulent Albali and Haren Myneni who have been working with
VAS/NX for their thoughts/experience.

> > 
> 
> Maybe ask Sukadev who did the implementation and is maintaining it ?
> 
> > I do have to say I'm quite satisfied with the results of the NX
> > accelerator, though. Being able to shuffle data to a RaptorCS box over gigE
> > and get compressed data back faster than most software gzip could ever
> > hope to achieve is no small feat, let alone the instantaneous results locally.
> > :)
> > 
> > Cheers,
> > Will Springer [she/her]
> > 
> > [1]: https://github.com/Skirmisher/void-packages/blob/vas-4k-pages/srcpkgs/linux5.9/patches/ppc-vas-on-4k.patch
> > 
> 
> 
> Christophe