[Skiboot] [PATCH v2 4/4] npu3: Register virtual PHBs with static IDs

[Andrew Donnellan]

On 9/8/19 3:05 pm, Frederic Barrat wrote:
> Assigning opal IDs to virtual PHBs dynamically may lead to userland
> seeing the PCI domain ID for an adapter vary when adding or removing
> another adapter (GPU or opencapi).
> 
> This patch switches to using static opal IDs for virtual PHBs, based
> on their ibm,phb-index property, which was made static by a previous
> patch.
> 
> Note that the PCI domain IDs will increase on the second chip (or
> more, if we had more) because we now reserve 16 IDs per chip for PHBs.

This affects phb4 PHBs on Axone as well? I didn't figure that out 
immediately when I read the commit message.

> 
> This affects Axone only. We don't change anything on P9 and npu2, to
> avoid altering how domain IDs have been shown on already GA'd
> platforms.
> 
> Signed-off-by: Frederic Barrat <fbarrat at linux.ibm.com>
> ---
> Changelog:
> v2: new patch: use static opal IDs for virtual PHBs on axone
> 
> 
>   hw/npu3-nvlink.c | 3 ++-
>   include/npu3.h   | 6 ++++++
>   include/phb4.h   | 7 ++++++-
>   3 files changed, 14 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/npu3-nvlink.c b/hw/npu3-nvlink.c
> index 99608368..af2dda9f 100644
> --- a/hw/npu3-nvlink.c
> +++ b/hw/npu3-nvlink.c
> @@ -904,7 +904,8 @@ static void npu3_create_phb(struct npu3 *npu)
>   	assert(phb->dt_node);
>   
>   	list_head_init(&phb->virt_devices);
> -	pci_register_phb(phb, OPAL_DYNAMIC_PHB_ID);
> +	pci_register_phb(phb, npu3_get_opal_id(npu->chip_id,
> +					       npu3_get_phb_index(npu->index)));
>   	npu3_create_phb_slot(npu);
>   	npu3_ioda_reset(phb, true);
>   }
> diff --git a/include/npu3.h b/include/npu3.h
> index 0fdad4df..dda60ae1 100644
> --- a/include/npu3.h
> +++ b/include/npu3.h
> @@ -20,6 +20,7 @@
>   #include <phys-map.h>
>   #include <pci.h>
>   #include <npu3-regs.h>
> +#include <phb4.h>
>   
>   enum npu3_dev_type {
>   	NPU3_DEV_TYPE_UNKNOWN = 0,
> @@ -183,4 +184,9 @@ static inline int npu3_get_phb_index(unsigned int npu_index)
>   	return NPU3_PHB_INDEX_BASE + npu_index;
>   }
>   
> +static inline int npu3_get_opal_id(unsigned int chip_id, unsigned int index)
> +{
> +	return phb4_get_opal_id(chip_id, index);
> +}
> +
>   #endif /* __NPU3_H */
> diff --git a/include/phb4.h b/include/phb4.h
> index af2e3eda..adc19f84 100644
> --- a/include/phb4.h
> +++ b/include/phb4.h
> @@ -245,10 +245,15 @@ static inline void phb4_set_err_pending(struct phb4 *p, bool pending)
>   }
>   
>   #define PHB4_PER_CHIP                        6 /* Max 6 PHBs per chip on p9 */
> +#define PHB4_MAX_PHBS_PER_CHIP_P9            PHB4_PER_CHIP
> +#define PHB4_MAX_PHBS_PER_CHIP_P9P           0x10 /* extra for virt PHBs */
>   
>   static inline int phb4_get_opal_id(unsigned int chip_id, unsigned int index)
>   {
> -	return chip_id * PHB4_PER_CHIP + index;
> +	if (PVR_TYPE(mfspr(SPR_PVR)) == PVR_TYPE_P9)
> +		return chip_id * PHB4_MAX_PHBS_PER_CHIP_P9 + index;
> +	else
> +		return chip_id * PHB4_MAX_PHBS_PER_CHIP_P9P + index;
>   }
>   
>   #endif /* __PHB4_H */
> 

-- 
Andrew Donnellan              OzLabs, ADL Canberra
ajd at linux.ibm.com             IBM Australia Limited