[Skiboot] [RFC 1/1] OPAL : Support spr save-restore using new opal call

[Nicholas Piggin]

On Tue, 24 Jul 2018 15:58:58 +1000
Benjamin Herrenschmidt <benh at kernel.crashing.org> wrote:

> On Tue, 2018-07-24 at 15:36 +1000, Nicholas Piggin wrote:
> > On Tue, 24 Jul 2018 10:06:12 +1000
> > Benjamin Herrenschmidt <benh at kernel.crashing.org> wrote:
> >   
> > > On Mon, 2018-07-23 at 23:41 +1000, Nicholas Piggin wrote:  
> > > > Right, so we should make OPAL do the whole thing (except catch the
> > > > SRESET wakeup which Linux has to do and has well architected SRR1
> > > > bits).
> > > >     
> > > > > 
> > > > > We could have a separate OPAL entry point for the wakeup case but
> > > > > putting the entirety of the enter/exit into OPAL is complicated.
> > > > > 
> > > > > We have to arbitrate between multiple threads entering different
> > > > > states. We don't know until we exit how deep the core actually went and
> > > > > what actually needs to be restored. It's a function of the SRR1 bits on
> > > > > return *and* the state that was requested.
> > > > > 
> > > > > There's also the need to synchronize/rendez-vous threads on wakeup in
> > > > > some cases such as TB resync, and dealing with big vs. small core.
> > > > > 
> > > > > Doing it all in OPAL would require *all* stop states to go to OPAL, I
> > > > > don't think that's a great idea for performance.    
> > > > 
> > > > Oh you want Linux native and OPAL drivers to be used simultaneously?
> > > > Is that the right thing to optimise for?    
> > > 
> > > We want Linux native for states that are known to work (hopefully
> > > STOP0/1) and OPAL for things that might need additional resources saved
> > > or restored and/or workarounds.
> > > 
> > > At least that's how I thought of it so far... you prefer having an "all
> > > or nothing" approach ?  
> > 
> > As an option, yes. And once you have that option I don't think there is
> > much point requiring kernel to know anything about deep states or even
> > the STOP instruction or PSSCR register.
> > 
> > I would not object to an OPAL API that requires all deep states to be
> > entered via OPAL but allows shallow states (GPR loss only) to be used
> > natively by the kernel concurrently. The shallow states are pretty well
> > architected and the deep states not nearly so performance critical.  
> 
> Where do you put in between states like 2 or 4/5 ? They are somwhat
> performance critical, enough that you don't want the round trip to
> OPAL, do you ?

I think an OPAL call for 4/5 would be acceptable. They get very
expensive when power turns off.

For 2 I'm not so sure. It's still 20000ns hardware cost.

> 
> Or do we have to go there to resync TB anyway ?

POWER9 is better about keeping TB, so it's not required I think.

> I wish the HW could handle the TB resync and stave save/restore using
> the CMEs and completely remove the problem...

Yeah, there's lots we could improve there...

Thanks,
Nick