[Skiboot] [PATCH v4] phb4: Enable PCI peer-to-peer

[Russell Currey]

On Tue, 2017-08-01 at 14:36 +0200, Frederic Barrat wrote:
> P9 supports PCI peer-to-peer: a PCI device can write directly to the
> mmio space of another PCI device. It completely by-passes the CPU.
> 
> It requires some configuration on the PHBs involved:
> 
> 1. on the initiating side, the address for the read/write operation is
> in the mmio space of the target, i.e. well outside the range normally
> allowed. So we disable range-checking on the TVT entry in bypass mode.
> 
> 2. on the target side, we need to explicitly enable p2p by setting a
> bit in a configuration register. It has the side-effect of reserving
> an outbound (as seen from the CPU) store queue for p2p. Therefore we
> only enable p2p on the PHBs using it, as we don't want to waste the
> resource if we don't have to.
> 
> P9 supports p2p mmio writes. Reads are currently only supported if the
> two devices are under the same PHB but that is expected to change in
> the future, and it raises questions about intermediate switches
> configuration, so we report an error for the time being.
> 
> The patch adds a new OPAL call to allow the OS to declare a p2p
> (initiator, target) pair.
> 
> Signed-off-by: Frederic Barrat <fbarrat at linux.vnet.ibm.com>
> ---

Just a tiny nitpick below.

Reviewed-by: Russell Currey <ruscur at russell.cc>

> +static void phb4_p2p_set_initiator(struct phb4 *p, uint16_t pe_number)
> +{
> +	uint64_t tve;
> +	uint16_t window_id = (pe_number << 1) + 1;
> +
> +	/*
> +	 * Initiator needs access to the MMIO space of the target,
> +	 * which is well beyond the 'normal' memory area. Set its TVE
> +	 * with no range checking.
> +	 */
> +	PHBDBG(p, "Setting TVE#1 for peer-to-peer for pe %d\n", pe_number);
> +	tve = PPC_BIT(51);

Would be nicer to set tve to bit 51 in the declaration.

> +	phb4_ioda_sel(p, IODA3_TBL_TVT, window_id, false);
> +	out_be64(p->regs + PHB_IODA_DATA0, tve);
> +	p->tve_cache[window_id] = tve;
> +}