[kernel] powerpc/powernv/ioda2: Update iommu table base on ownership change

[Michael Ellerman]

On Tue, 2017-02-21 at 02:41:31 UTC, Alexey Kardashevskiy wrote:
> On POWERNV platform, in order to do DMA via IOMMU (i.e. 32bit DMA in
> our case), a device needs an iommu_table pointer set via
> set_iommu_table_base().
> 
> The codeflow is:
> - pnv_pci_ioda2_setup_dma_pe()
> 	- pnv_pci_ioda2_setup_default_config()
> 	- pnv_ioda_setup_bus_dma() [1]
> 
> pnv_pci_ioda2_setup_dma_pe() creates IOMMU groups,
> pnv_pci_ioda2_setup_default_config() does default DMA setup,
> pnv_ioda_setup_bus_dma() takes a bus PE (on IODA2, all physical function
> PEs as bus PEs except NPU), walks through all underlying buses and
> devices, adds all devices to an IOMMU group and sets iommu_table.
> 
> On IODA2, when VFIO is used, it takes ownership over a PE which means it
> removes all tables and creates new ones (with a possibility of sharing
> them among PEs). So when the ownership is returned from VFIO to
> the kernel, the iommu_table pointer written to a device at [1] is
> stale and needs an update.
> 
> This adds an "add_to_group" parameter to pnv_ioda_setup_bus_dma()
> (in fact re-adds as it used to be there a while ago for different
> reasons) to tell the helper if a device needs to be added to
> an IOMMU group with an iommu_table update or just the latter.
> 
> This calls pnv_ioda_setup_bus_dma(..., false) from
> pnv_ioda2_release_ownership() so when the ownership is restored,
> 32bit DMA can work again for a device. This does the same thing
> on obtaining ownership as the iommu_table point is stale at this point
> anyway and it is safer to have NULL there.
> 
> We did not hit this earlier as all tested devices in recent years were
> only using 64bit DMA; the rare exception for this is MPT3 SAS adapter
> which uses both 32bit and 64bit DMA access and it has not been tested
> with VFIO much.
> 
> Cc: Gavin Shan <gwshan at linux.vnet.ibm.com>
> Signed-off-by: Alexey Kardashevskiy <aik at ozlabs.ru>
> Acked-by: Gavin Shan <gwshan at linux.vnet.ibm.com>
> Reviewed-by: David Gibson <david at gibson.dropbear.id.au>

Applied to powerpc fixes, thanks.

https://git.kernel.org/powerpc/c/db08e1d53034a54fe177ced70476fd

cheers