[Skiboot] [PATCH 1/2] phb4: Reallocate PEC2 DMA-Read engines to improve GPU-Direct bandwidth

[Stewart Smith]

Vaibhav Jain <vaibhav at linux.ibm.com> writes:
> We reallocate additional 16/8 DMA-Read engines allocated to stack0/1
> on PEC2 respectively. This is needed to improve bandwidth available to
> the Mellanox CX5 adapter when trying to read GPU memory (GPU-Direct).
>
> If kernel cxl driver indicates a request to allocate maximum possible
> DMA read engines when calling enable_capi_mode() and card is attached
> to PEC2/stack0 slot then we assume its a Mellanox CX5 adapter. We then
> allocate additional 16/8 extra DMA read engines to stack0 and stack1
> respectively on PEC2. This is done by populating the
> XPEC_PCI_PRDSTKOVR and XPEC_NEST_READ_STACK_OVERRIDE as suggested by
> the h/w team.
>
> Signed-off-by: Christophe Lombard <clombard at linux.vnet.ibm.com>
> Signed-off-by: Vaibhav Jain <vaibhav at linux.ibm.com>
> ---
>  hw/phb4.c           | 40 +++++++++++++++++++++++++++++++++++++---
>  include/phb4-regs.h |  2 ++
>  2 files changed, 39 insertions(+), 3 deletions(-)

thanks, and thanks for addressing things Andrew and I brought up in that
random internal Slack thread to get this ready to go in so quickly.

Merged to master as of 5690c5a8980faf9e528df65dd95535e21c2c868f
and to 6.0.x as of b50653f4c4707f3406269573bc421df3cffce950

I fixed the obvious nitpicks in the doc with merging... although I
didn't do the "nor on P8 for set_mode_pcie" as I'm not 100% sure of that
and wanted to check (I thought we did that... or maybe we just *think*
we can and reset things appropriately in fast-reboot - or I'm just plain
wrong and shouldn't be allowed near computers).

-- 
Stewart Smith
OPAL Architect, IBM.