[Skiboot] [PATCH] core/cpu.c: Add OPAL call to setup Nest MMU

[Alistair Popple]

On Tue, 2 Aug 2016 11:15:43 Balbir Singh wrote:
> On Tue, 2016-08-02 at 06:50 +1000, Alistair Popple wrote:
> > POWER9 has an off core MMU called the Nest MMU which allows other
> > units within a chip to perform address translations. The context and
> > setup for translations is handled by the requesting agents, however
> > the Nest MMU does need to know where in system memory the page tables
> > are located.
> > 
> > This patch adds a call to setup the Nest MMU page table pointer on a
> > per-chip basis.
> > 
> > Signed-off-by: Alistair Popple <alistair at popple.id.au>
> > ---
> >  core/cpu.c                              | 25 ++++++++++++++++++++++++-
> >  doc/opal-api/opal-nmmu-set-ptcr-127.txt | 19 +++++++++++++++++++
> >  include/opal-api.h                      |  3 ++-
> >  3 files changed, 45 insertions(+), 2 deletions(-)
> >  create mode 100644 doc/opal-api/opal-nmmu-set-ptcr-127.txt
> > 
> > diff --git a/core/cpu.c b/core/cpu.c
> > index f33ac48..072f6ce 100644
> > --- a/core/cpu.c
> > +++ b/core/cpu.c
> > @@ -30,6 +30,7 @@
> >  #include <timebase.h>
> >  #include <ccan/str/str.h>
> >  #include <ccan/container_of/container_of.h>
> > +#include <xscom.h>
> >  
> >  /* The cpu_threads array is static and indexed by PIR in
> >   * order to speed up lookup from asm entry points
> > @@ -667,7 +668,7 @@ void cpu_bringup(void)
> >  
> >  	op_display(OP_LOG, OP_MOD_CPU, 0x0000);
> >  
> > -	/* Tell everybody to chime in ! */	
> > +	/* Tell everybody to chime in ! */
> 
> Whitespace cleanups -- split into another patch?

Doh. Missed that.

> >  	prlog(PR_INFO, "CPU: Calling in all processors...\n");
> >  	cpu_secondary_start = 1;
> >  	sync();
> > @@ -894,3 +895,25 @@ static int64_t opal_reinit_cpus(uint64_t flags)
> >  	return rc;
> >  }
> >  opal_call(OPAL_REINIT_CPUS, opal_reinit_cpus, 1);
> > +
> > +/*
> > + * Setup the the Nest MMU PTCR register for the specified chip id.
> 
> -1ULL is special, so the comment needs to mention it?

Right, I meant to add that to the documentation but forgot. I will do that.

> > + */
> > +#define NMMU_CFG_XLAT_CTL_PTCR 0x5012c4b
> > +static int64_t opal_nmmu_set_ptcr(uint64_t chip_id, uint64_t ptcr)
> 
> Can this API be used to clear the PTCR as well? If a PTCR of 0 is sent?
> 
> Do we want to check if the intention is to clear/set and do additional
> error checks? Overwriting a set PTCR? I presume a valid write sequence
> would be
> 
> 1. set PTCR
> 2. clear PTCR
> 3. set..
> 4. clear..
> ...
> 
> Or can we have
> 
> 1. set PTCR
> 2. set PTCR
> ..

We can do either and I don't think it is a problem, but it has reminded me of 
something I needed to look into - if there is some way to quiesce the NMMU 
before changing PTCR as PTCR will need to change on kexec.  In theory any 
agents should also be quiesced prior to a kexec as they shouldn't be issuing 
DMA requests, etc. but I haven't found anything in the NMMU to enforce this.

As far as I can tell from the docs you can just change PTCR, however I'm not 
at all convinced there isn't state hanging round in there that would need to 
be cleared as well.

> > +{
> > +	struct proc_chip *chip;
> > +
> > +	if (chip_id == -1ULL)
> > +		for_each_chip(chip)
> > +			xscom_write(chip->id, NMMU_CFG_XLAT_CTL_PTCR, ptcr);
> 
> Check the return value?

Sure.

> > +	else {
> > +		if (!(chip = get_chip(chip_id)))
> > +			return OPAL_PARAMETER;
> > +
> > +		xscom_write(chip->id, NMMU_CFG_XLAT_CTL_PTCR, ptcr);
> Return value check?

Sure.

> > +	}
> > +
> > +	return OPAL_SUCCESS;
> > +}
> > +opal_call(OPAL_NMMU_SET_PTCR, opal_nmmu_set_ptcr, 2);
> > diff --git a/doc/opal-api/opal-nmmu-set-ptcr-127.txt b/doc/opal-api/opal-
nmmu-set-ptcr-127.txt
> > new file mode 100644
> > index 0000000..7c30772
> > --- /dev/null
> > +++ b/doc/opal-api/opal-nmmu-set-ptcr-127.txt
> > @@ -0,0 +1,19 @@
> > +OPAL_NMMU_SET_PTCR
> > +-------------------
> > +
> > +#define OPAL_NMMU_SET_PTCR			127
> > +
> > +int64 opal_nmmu_set_ptcr(uint64 chip_id, uint64_t ptcr)
> > +
> > +Arguments:
> > +
> > +  uint64 chip_id
> > +  uint64 ptcr
> > +    ptcr value pointing to either the radix tables or hash tables.
> > +
> > +This OPAL call sets up the Nest MMU by pointing it at the radix page
> > +table base or the hash page table base (HTABORG).
> > +
> > +Return Values:
> > +OPAL_SUCCESS: the power down was updated successful
> > +OPAL_PARAMETER: a parameter was incorrect
> > diff --git a/include/opal-api.h b/include/opal-api.h
> > index c86244b..e82210c 100644
> > --- a/include/opal-api.h
> > +++ b/include/opal-api.h
> > @@ -172,7 +172,8 @@
> >  #define OPAL_INT_EOI				124
> >  #define OPAL_INT_SET_MFRR			125
> >  #define OPAL_PCI_TCE_KILL			126
> > -#define OPAL_LAST				126
> > +#define OPAL_NMMU_SET_PTCR			127
> > +#define OPAL_LAST				127
> >  
> >  /* Device tree flags */
> >