[PATCH v5 4/7] powerpc/mm: Use UV_WRITE_PATE ucall to register a PATE

[Sukadev Bhattiprolu]

Michael Ellerman [mpe at ellerman.id.au] wrote:

> Hi Claudio,
> 
> Claudio Carvalho <cclaudio at linux.ibm.com> writes:
> > From: Michael Anderson <andmike at linux.ibm.com>
> >
> > In ultravisor enabled systems, the ultravisor creates and maintains the
> > partition table in secure memory where the hypervisor cannot access, and
>                                    ^
>                                    which?
> 
> > therefore, the hypervisor have to do the UV_WRITE_PATE ucall whenever it
>                             ^          ^
>                             has        a
> > wants to set a partition table entry (PATE).
> >
> > This patch adds the UV_WRITE_PATE ucall and uses it to set a PATE if
> > ultravisor is enabled. Additionally, this also also keeps a copy of the
> > partition table because the nestMMU does not have access to secure
> > memory. Such copy has entries for nonsecure and hypervisor partition.
> 
> I'm having trouble parsing the last sentence there.
> 
> Or at least it doesn't seem to match the code, or I don't understand
> either the code or the comment. More below.

Yes, good catch. We could drop the last sentence. Or maybe change the
last para to:

	This patch adds the UV_WRITE_PATE ucall which is used to update
	the partition table entry (PATE) for a VM (both normal and secure).

	When UV is enabled, the partition table is stored in secure memory
	and can only be accessed via the UV. The HV however maintains a
	copy of the partition table in normal memory to allow NMMU
	translations to occur (for normal VMs). The HV copy includes PATEs
	for secure VMs which would currently be unused (NMMU translations
	cannot access secure memory) but they would be needed as we add
	functionality.

Basically, with UV, PTCR is controlled by the UV and address translations
occur based on the UV's copy of the partition table. (See also:
try_set_ptcr() in "PATCH 5/7 powerpc/mm: Write to PTCR only if ultravisor
disabled")

> 
> > diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/pgtable.c
> > index 85bc81abd286..033731f5dbaa 100644
> > --- a/arch/powerpc/mm/book3s64/pgtable.c
> > +++ b/arch/powerpc/mm/book3s64/pgtable.c
> > @@ -213,34 +223,50 @@ void __init mmu_partition_table_init(void)
> >  	powernv_set_nmmu_ptcr(ptcr);
> >  }
> >  
> > -void mmu_partition_table_set_entry(unsigned int lpid, unsigned long dw0,
> > -				   unsigned long dw1)
> > +/*
> > + * Global flush of TLBs and partition table caches for this lpid. The type of
> > + * flush (hash or radix) depends on what the previous use of this partition ID
> > + * was, not the new use.
> > + */
> > +static void flush_partition(unsigned int lpid, unsigned long old_patb0)
> 
> A nicer API would be for the 2nd param to be a "bool radix", and have
> the caller worry about the fact that it comes from (patb0 & PATB_HR).

Agree

> 
> >  {
> > -	unsigned long old = be64_to_cpu(partition_tb[lpid].patb0);
> > -
> > -	partition_tb[lpid].patb0 = cpu_to_be64(dw0);
> > -	partition_tb[lpid].patb1 = cpu_to_be64(dw1);
> > -
> > -	/*
> > -	 * Global flush of TLBs and partition table caches for this lpid.
> > -	 * The type of flush (hash or radix) depends on what the previous
> > -	 * use of this partition ID was, not the new use.
> > -	 */
> >  	asm volatile("ptesync" : : : "memory");
> > -	if (old & PATB_HR) {
> > -		asm volatile(PPC_TLBIE_5(%0,%1,2,0,1) : :
> > +	if (old_patb0 & PATB_HR) {
> > +		asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 1) : :
> >  			     "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid));
> > -		asm volatile(PPC_TLBIE_5(%0,%1,2,1,1) : :
> > +		asm volatile(PPC_TLBIE_5(%0, %1, 2, 1, 1) : :
> 
> That looks like an unrelated whitespace change.
> 
> >  			     "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid));
> >  		trace_tlbie(lpid, 0, TLBIEL_INVAL_SET_LPID, lpid, 2, 0, 1);
> >  	} else {
> > -		asm volatile(PPC_TLBIE_5(%0,%1,2,0,0) : :
> > +		asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 0) : :
> 
> Ditto.
> 
> >  			     "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid));
> >  		trace_tlbie(lpid, 0, TLBIEL_INVAL_SET_LPID, lpid, 2, 0, 0);
> >  	}
> >  	/* do we need fixup here ?*/
> >  	asm volatile("eieio; tlbsync; ptesync" : : : "memory");
> >  }
> > +
> > +void mmu_partition_table_set_entry(unsigned int lpid, unsigned long dw0,
> > +				  unsigned long dw1)
> > +{
> > +	unsigned long old = be64_to_cpu(partition_tb[lpid].patb0);
> > +
> > +	partition_tb[lpid].patb0 = cpu_to_be64(dw0);
> > +	partition_tb[lpid].patb1 = cpu_to_be64(dw1);
> 
> ie. here we always update the copy of the partition table, regardless of
> whether we're running under an ultravisor or not. So the copy is a
> complete copy isn't it?

Yes.
> 
> > +	/*
> > +	 * In ultravisor enabled systems, the ultravisor maintains the partition
> > +	 * table in secure memory where we don't have access, therefore, we have
> > +	 * to do a ucall to set an entry.
> > +	 */
> > +	if (firmware_has_feature(FW_FEATURE_ULTRAVISOR)) {
> > +		uv_register_pate(lpid, dw0, dw1);
> > +		pr_info("PATE registered by ultravisor: dw0 = 0x%lx, dw1 = 0x%lx\n",
> > +			dw0, dw1);
> > +	} else {
> > +		flush_partition(lpid, old);
> > +	}
> 
> What is different is whether we flush or not.

only differences are where the partition table used by hardware is stored
(secure memory) and updated (in UV, with higher privilege).

> 
> And don't we still need to do the flush for the nestMMU? I assume we're
> saying the ultravisor will broadcast a flush for us, which will also
> handle the nestMMU case?

The same sequence of instructions (as HV) are used in uv_register_pate()
to flush partition and process scoped entries (so nest MMU would also be
covered when NMMU sees the tlbie?)

Thanks,

Sukadev