[PATCH kernel] powerpc/pseries/ddw: Extend upper limit for huge DMA window for persistent memory

[Alexey Kardashevskiy]

Hi Wen,

Can you please try this? It contains 3 patches from Christoph Hellwig
plus my patch on top, this should improve performance when DMA-ing not
to/from persistent memory:

https://github.com/aik/linux/commits/dma-bypass.3

I am looking for any benchmarks not related to persistent memory. Thanks,



On 03/04/2020 09:09, Wen Xiong wrote:
> I applied the patch on top of the latest upstream kernel. I ran HTX over
> pmem nodes for several hours and it works.
>  
> Tested-by: Wen Xiong<wenxiong at linux.vnet.ibm.com>
>  
> Thanks,
> Wendy
> 
>     ----- Original message -----
>     From: Alexey Kardashevskiy <aik at ozlabs.ru>
>     To: linuxppc-dev at lists.ozlabs.org
>     Cc: Alexey Kardashevskiy <aik at ozlabs.ru>, David Gibson
>     <david at gibson.dropbear.id.au>, Michael Ellerman
>     <mpe at ellerman.id.au>, Oliver O'Halloran <oohall at gmail.com>, "Aneesh
>     Kumar K . V" <aneesh.kumar at linux.ibm.com>, Wen Xiong
>     <wenxiong at us.ibm.com>, Brian J King <bjking1 at us.ibm.com>
>     Subject: [EXTERNAL] [PATCH kernel] powerpc/pseries/ddw: Extend upper
>     limit for huge DMA window for persistent memory
>     Date: Mon, Mar 30, 2020 8:23 PM
>      
>     Unlike normal memory ("memory" compatible type in the FDT),
>     the persistent memory ("ibm,pmemory" in the FDT) can be mapped anywhere
>     in the guest physical space and it can be used for DMA.
> 
>     In order to maintain 1:1 mapping via the huge DMA window, we need to
>     know the maximum physical address at the time of the window setup.
>     So far we've been looking at "memory" nodes but "ibm,pmemory" does not
>     have fixed addresses and the persistent memory may be mapped afterwards.
> 
>     Since the persistent memory is still backed with page structs,
>     use MAX_PHYSMEM_BITS as the upper limit.
> 
>     This effectively disables huge DMA window in LPAR under pHyp if
>     persistent memory is present but this is the best we can do.
> 
>     Signed-off-by: Alexey Kardashevskiy <aik at ozlabs.ru>
>     ---
>      arch/powerpc/platforms/pseries/iommu.c | 9 +++++++++
>      1 file changed, 9 insertions(+)
> 
>     diff --git a/arch/powerpc/platforms/pseries/iommu.c
>     b/arch/powerpc/platforms/pseries/iommu.c
>     index 2e0a8eab5588..6d47b4a3ce39 100644
>     --- a/arch/powerpc/platforms/pseries/iommu.c
>     +++ b/arch/powerpc/platforms/pseries/iommu.c
>     @@ -945,6 +945,15 @@ static phys_addr_t ddw_memory_hotplug_max(void)
>       phys_addr_t max_addr = memory_hotplug_max();
>       struct device_node *memory;
>      
>     + /*
>     + * The "ibm,pmemory" can appear anywhere in the address space.
>     + * Assuming it is still backed by page structs, set the upper limit
>     + * for the huge DMA window as MAX_PHYSMEM_BITS.
>     + */
>     + if (of_find_node_by_type(NULL, "ibm,pmemory"))
>     + return (sizeof(phys_addr_t) * 8 <= MAX_PHYSMEM_BITS) ?
>     + (phys_addr_t) -1 : (1ULL << MAX_PHYSMEM_BITS);
>     +
>       for_each_node_by_type(memory, "memory") {
>       unsigned long start, size;
>       int n_mem_addr_cells, n_mem_size_cells, len;
>     --
>     2.17.1
>      
> 
>  
> 

-- 
Alexey