[Skiboot] [PATCH] core/cpu.c: Add OPAL call to setup Nest MMU

[Balbir Singh]

On Wed, 2016-08-03 at 03:17 +1000, Alistair Popple wrote:
> On Tue, 2 Aug 2016 11:15:43 Balbir Singh wrote:
> > 
> > On Tue, 2016-08-02 at 06:50 +1000, Alistair Popple wrote:
> > > 
> > > POWER9 has an off core MMU called the Nest MMU which allows other
> > > units within a chip to perform address translations. The context and
> > > setup for translations is handled by the requesting agents, however
> > > the Nest MMU does need to know where in system memory the page tables
> > > are located.
> > > 
> > > This patch adds a call to setup the Nest MMU page table pointer on a
> > > per-chip basis.
> > > 
> > > Signed-off-by: Alistair Popple <alistair at popple.id.au>
> > > ---
> > >  core/cpu.c                              | 25 ++++++++++++++++++++++++-
> > >  doc/opal-api/opal-nmmu-set-ptcr-127.txt | 19 +++++++++++++++++++
> > >  include/opal-api.h                      |  3 ++-
> > >  3 files changed, 45 insertions(+), 2 deletions(-)
> > >  create mode 100644 doc/opal-api/opal-nmmu-set-ptcr-127.txt
> > > 
> > > diff --git a/core/cpu.c b/core/cpu.c
> > > index f33ac48..072f6ce 100644
> > > --- a/core/cpu.c
> > > +++ b/core/cpu.c
> > > @@ -30,6 +30,7 @@
> > >  #include <timebase.h>
> > >  #include <ccan/str/str.h>
> > >  #include <ccan/container_of/container_of.h>
> > > +#include <xscom.h>
> > >  
> > >  /* The cpu_threads array is static and indexed by PIR in
> > >   * order to speed up lookup from asm entry points
> > > @@ -667,7 +668,7 @@ void cpu_bringup(void)
> > >  
> > >  	op_display(OP_LOG, OP_MOD_CPU, 0x0000);
> > >  
> > > -	/* Tell everybody to chime in ! */	
> > > +	/* Tell everybody to chime in ! */
> > Whitespace cleanups -- split into another patch?
> Doh. Missed that.
> 
> > 
> > > 
> > >  	prlog(PR_INFO, "CPU: Calling in all processors...\n");
> > >  	cpu_secondary_start = 1;
> > >  	sync();
> > > @@ -894,3 +895,25 @@ static int64_t opal_reinit_cpus(uint64_t flags)
> > >  	return rc;
> > >  }
> > >  opal_call(OPAL_REINIT_CPUS, opal_reinit_cpus, 1);
> > > +
> > > +/*
> > > + * Setup the the Nest MMU PTCR register for the specified chip id.
> > -1ULL is special, so the comment needs to mention it?
> Right, I meant to add that to the documentation but forgot. I will do that.
> 
> > 
> > > 
> > > + */
> > > +#define NMMU_CFG_XLAT_CTL_PTCR 0x5012c4b
> > > +static int64_t opal_nmmu_set_ptcr(uint64_t chip_id, uint64_t ptcr)
> > Can this API be used to clear the PTCR as well? If a PTCR of 0 is sent?
> > 
> > Do we want to check if the intention is to clear/set and do additional
> > error checks? Overwriting a set PTCR? I presume a valid write sequence
> > would be
> > 
> > 1. set PTCR
> > 2. clear PTCR
> > 3. set..
> > 4. clear..
> > ...
> > 
> > Or can we have
> > 
> > 1. set PTCR
> > 2. set PTCR
> > ..
> We can do either and I don't think it is a problem, but it has reminded me of 
> something I needed to look into - if there is some way to quiesce the NMMU 
> before changing PTCR as PTCR will need to change on kexec.  In theory any 
> agents should also be quiesced prior to a kexec as they shouldn't be issuing 
> DMA requests, etc. but I haven't found anything in the NMMU to enforce this.
> 

Yes and it needs to be co-ordinated. We need to ensure that we quiesce the
NMMU first (set PTCR to 0?) and then clear up the actual PTCR


> As far as I can tell from the docs you can just change PTCR, however I'm not 
> at all convinced there isn't state hanging round in there that would need to 
> be cleared as well.
>

Good point, I presume by the time we get to reboot, all drivers/tasks are
hopefully killed/unloaded. But there might be some additional work required
as well

Balbir Singh