[RFC] powerpc/numa: Use VPHN based node ID information on shared processor LPARs

[Michael Ellerman]

On Wed, 2015-10-14 at 15:43 +0530, Anshuman Khandual wrote:
> On 10/14/2015 02:49 PM, Michael Ellerman wrote:
> > On Wed, 2015-10-14 at 14:32 +0530, Anshuman Khandual wrote:
> >> On shared processor LPARs, H_HOME_NODE_ASSOCIATIVITY hcall provides the
> >> dynamic virtual-physical mapping for any given processor. Currently we
> >> use VPHN node ID information only after getting either a PRRN or a VPHN
> >> event. But during boot time inside the function numa_setup_cpu, we still
> >> query the OF device tree for the node ID value which might be different
> >> than what can be fetched from the H_HOME_NODE_ASSOCIATIVITY hcall. In a
> >> scenario where there are no PRRN or VPHN event after boot, all node-cpu
> >> mapping will remain incorrect there after.
> >>
> >> With this proposed change, numa_setup_cpu will try to override the OF
> >> device tree fetched node ID information with H_HOME_NODE_ASSOCIATIVITY
> >> hcall fetched node ID value. Right now shared processor property of the
> >> LPAR cannot be queried as VPA inializaion happens after numa_setup_cpu
> >> during boot time. So initmem_init function has been moved after ppc_md.
> >> setup_arch inside setup_arch during boot.
> > 
> > I would be *very* reluctant to change the order of initmem_init() vs
> > setup_arch().
> > 
> > At a minimum you'd need to go through every setup_arch() implementation and
> > carefully determine if the ordering of what it does matters vs initmem_init().
> > And then you'd need to test on every affected platform.
> > 
> > So I suggest you think of a different way to do it if at all possible.
> 
> vpa_init() is being called inside pSeries_setup_arch which is ppc_md
> .setup_arch for the platform. Its called directly for the boot cpu
> and through smp_init_pseries_xics for other cpus on the system. Not
> sure what is the reason behind calling vpa_init() from XICS init
> though.
> 
> If we can move all these vpa_init() calls from pSeries_setup_arch
> to initmem_init just before calling numa_setup_cpu, the VPA area
> would be initialized when we need it during boot. Will look in
> this direction.

Back up a bit. The dependency on vpa_init() is only because you want to call
lppaca_shared_proc() right?

But do you really need to? What happens if you call VPHN on a non-shared proc
machine? Does it 1) give you something sane or 2) give you an error or 3) give
you a junk value?

If it's either of 1 or 2 then you should be OK to just call it. You either use
the value it returned which is sane or you see the error and just fall back to
the device tree nid.

cheers