[Skiboot] [PATCH 3/4] hw/npu2-opencapi: FIR masking for mixed setups

[Frederic Barrat]

Le 18/10/2018 à 03:16, Andrew Donnellan a écrit :
> When setting up an NPU with an OpenCAPI device, we need to mask the FIR
> bits for NDL Stall/NoStall, which are used for a different purpose on
> OpenCAPI vs NVLink.
> 
> Currently, we just mask the bits for all links/DLs. When we support mixed
> setups of OpenCAPI + NVLink on the same NPU, we don't want to mask all the
> bits.
> 
> Only mask the FIR bits for the specific links which are OpenCAPI.
> 
> Signed-off-by: Andrew Donnellan <andrew.donnellan at au1.ibm.com>
> ---
>   hw/npu2-opencapi.c | 30 +++++++++++++++++++++---------
>   1 file changed, 21 insertions(+), 9 deletions(-)
> 
> diff --git a/hw/npu2-opencapi.c b/hw/npu2-opencapi.c
> index 76b81d679046..fdad7c5c1bbe 100644
> --- a/hw/npu2-opencapi.c
> +++ b/hw/npu2-opencapi.c
> @@ -1318,7 +1318,8 @@ static void npu2_opencapi_final_fixup(struct phb *phb)
> 
>   static void mask_nvlink_fir(struct npu2 *p)
>   {
> -	uint64_t reg;
> +	uint64_t reg, mask = 0ull;
> +	int link_num;
> 
>   	/*
>   	 * From section 13.1.3.10 of the NPU workbook: "the NV-Link
> @@ -1327,35 +1328,46 @@ static void mask_nvlink_fir(struct npu2 *p)
>   	 * OpenCAPI. Therefore, the corresponding bits in NPU FIR
>   	 * Register 1 must be masked and configured to NOT cause the
>   	 * NPU to go into Freeze or Fence mode or send an Interrupt."
> -	 *
> -	 * FIXME: will need to revisit when mixing nvlink with
> -	 * opencapi. Assumes an opencapi-only setup on both PHYs for
> -	 * now.
>   	 */
> 
> +	for (int i = 0; i < p-> total_devices; i++) {
> +		struct npu2_dev *dev = &p->devices[i];
> +		/* Only mask OpenCAPI links */
> +		if (dev->type != NPU2_DEV_TYPE_OPENCAPI)
> +			continue;
> +
> +		if (dev->brick_index == 2 || dev->brick_index == 3) {
> +			link_num = dev->brick_index - 2;
> +		} else {
> +			link_num = dev->brick_index;
> +		}
> +		mask = SETFIELD(PPC_BITMASK(link_num * 2,
> +					    link_num * 2 + 1),
> +				mask, 0b11);
> +	}
>   	/* Mask FIRs */
>   	xscom_read(p->chip_id, p->xscom_base + NPU2_MISC_FIR_MASK1, &reg);
> -	reg = SETFIELD(PPC_BITMASK(0, 11), reg, 0xFFF);
> +	reg |= mask;
>   	xscom_write(p->chip_id, p->xscom_base + NPU2_MISC_FIR_MASK1, reg);
> 
>   	/* freeze disable */
>   	reg = npu2_scom_read(p->chip_id, p->xscom_base,
>   			NPU2_MISC_FREEZE_ENABLE1, NPU2_MISC_DA_LEN_8B);
> -	reg = SETFIELD(PPC_BITMASK(0, 11), reg, 0);
> +	reg &= ~mask;
>   	npu2_scom_write(p->chip_id, p->xscom_base,
>   			NPU2_MISC_FREEZE_ENABLE1, NPU2_MISC_DA_LEN_8B, reg);
> 

For link0|1, we need to mask some extra bits (per chat with John and 
table in section 13.1.3.10). Or did you figure out something?

   Fred


>   	/* fence disable */
>   	reg = npu2_scom_read(p->chip_id, p->xscom_base,
>   			NPU2_MISC_FENCE_ENABLE1, NPU2_MISC_DA_LEN_8B);
> -	reg = SETFIELD(PPC_BITMASK(0, 11), reg, 0);
> +	reg &= ~mask;
>   	npu2_scom_write(p->chip_id, p->xscom_base,
>   			NPU2_MISC_FENCE_ENABLE1, NPU2_MISC_DA_LEN_8B, reg);
> 
>   	/* irq disable */
>   	reg = npu2_scom_read(p->chip_id, p->xscom_base,
>   			NPU2_MISC_IRQ_ENABLE1, NPU2_MISC_DA_LEN_8B);
> -	reg = SETFIELD(PPC_BITMASK(0, 11), reg, 0);
> +	reg &= ~mask;
>   	npu2_scom_write(p->chip_id, p->xscom_base,
>   			NPU2_MISC_IRQ_ENABLE1, NPU2_MISC_DA_LEN_8B, reg);
>   }
>