[PATCH v2 00/32] powerpc: Add MSI IRQ domains to PCI drivers

[Cédric Le Goater]

Hello,

This series adds support for MSI IRQ domains on top of the XICS (P8)
and XIVE (P9/P10) IRQ domains for the PowerNV (baremetal) and pSeries
(VM) platforms. It should simplify and improve IRQ affinity of PCI
MSIs under these PowerPC platforms, specially for drivers distributing
multiple RX/TX queues on the different CPUs of the system.

Data locality can still be improved with an interrupt controller node
per chip but this requires FW changes. It could be done under OPAL.

The patchset has a large impact but it is well contained under the MSI
support. Initial tests were done on the P8, P9 and P10 PowerNV and
pSeries platforms, under the KVM and PowerVM hypervisor. PCI passthrough
was tested on P8/KVM, P9/KVM and P9/pVM with both interrupt modes.

P8 passthrough has some optimization to EOI MSIs when under real mode :

 e3c13e56a471 ("KVM: PPC: Book3S HV: Handle passthrough interrupts in guest")
 5d375199ea96 ("KVM: PPC: Book3S HV: Set server for passed-through interrupts")

They give us a ~10% bandwidth improvement on some 100G adapters
(Thanks Alexey), so it's good to keep but they require access to the
low level IRQ domain of the machine. It should be possible to rework
the code and use the MSI IRQ domains instead but for now, it's simpler
to keep the bypass. That can come later.

The P8/CAPI driver is also impacted. Tests were done on a Firestone
system with a memory AFU.

Thanks,

C.

Changes since v2 :

 - Included some CONFIG_IRQ_DOMAIN_HIERARCHY ifdefs
 - Microwatt fixes for ICS native
 - Removed irqd_is_started() check when setting the affinity

Cédric Le Goater (32):
  powerpc/pseries/pci: Introduce __find_pe_total_msi()
  powerpc/pseries/pci: Introduce rtas_prepare_msi_irqs()
  powerpc/xive: Add support for IRQ domain hierarchy
  powerpc/xive: Ease debugging of xive_irq_set_affinity()
  powerpc/pseries/pci: Add MSI domains
  powerpc/xive: Drop unmask of MSIs at startup
  powerpc/xive: Remove irqd_is_started() check when setting the affinity
  powerpc/pseries/pci: Add a domain_free_irqs() handler
  powerpc/pseries/pci: Add a msi_free() handler to clear XIVE data
  powerpc/pseries/pci: Add support of MSI domains to PHB hotplug
  powerpc/powernv/pci: Introduce __pnv_pci_ioda_msi_setup()
  powerpc/powernv/pci: Add MSI domains
  KVM: PPC: Book3S HV: Use the new IRQ chip to detect passthrough
    interrupts
  KVM: PPC: Book3S HV: XIVE: Change interface of passthrough interrupt
    routines
  KVM: PPC: Book3S HV: XIVE: Fix mapping of passthrough interrupts
  powerpc/xics: Remove ICS list
  powerpc/xics: Rename the map handler in a check handler
  powerpc/xics: Give a name to the default XICS IRQ domain
  powerpc/xics: Add debug logging to the set_irq_affinity handlers
  powerpc/xics: Add support for IRQ domain hierarchy
  powerpc/powernv/pci: Customize the MSI EOI handler to support PHB3
  powerpc/pci: Drop XIVE restriction on MSI domains
  powerpc/xics: Drop unmask of MSIs at startup
  powerpc/pseries/pci: Drop unused MSI code
  powerpc/powernv/pci: Drop unused MSI code
  powerpc/powernv/pci: Adapt is_pnv_opal_msi() to detect passthrough
    interrupt
  powerpc/xics: Fix IRQ migration
  powerpc/powernv/pci: Set the IRQ chip data for P8/CXL devices
  powerpc/powernv/pci: Rework pnv_opal_pci_msi_eoi()
  KVM: PPC: Book3S HV: XICS: Fix mapping of passthrough interrupts
  powerpc/xive: Use XIVE domain under xmon and debugfs
  genirq: Improve "hwirq" output in /proc and /sys/

 arch/powerpc/include/asm/kvm_ppc.h         |   4 +-
 arch/powerpc/include/asm/pci-bridge.h      |   5 +
 arch/powerpc/include/asm/pnv-pci.h         |   2 +-
 arch/powerpc/include/asm/xics.h            |   3 +-
 arch/powerpc/include/asm/xive.h            |   1 +
 arch/powerpc/platforms/powernv/pci.h       |   6 -
 arch/powerpc/platforms/pseries/pseries.h   |   2 +
 arch/powerpc/kernel/pci-common.c           |   6 +
 arch/powerpc/kvm/book3s_hv.c               |  18 +-
 arch/powerpc/kvm/book3s_hv_rm_xics.c       |   8 +-
 arch/powerpc/kvm/book3s_xive.c             |  18 +-
 arch/powerpc/platforms/powernv/pci-ioda.c  | 256 ++++++++++++++++--
 arch/powerpc/platforms/powernv/pci.c       |  67 -----
 arch/powerpc/platforms/pseries/msi.c       | 296 ++++++++++++++++-----
 arch/powerpc/platforms/pseries/pci_dlpar.c |   4 +
 arch/powerpc/platforms/pseries/setup.c     |   2 +
 arch/powerpc/sysdev/xics/ics-native.c      |  13 +-
 arch/powerpc/sysdev/xics/ics-opal.c        |  40 +--
 arch/powerpc/sysdev/xics/ics-rtas.c        |  40 +--
 arch/powerpc/sysdev/xics/xics-common.c     | 129 ++++++---
 arch/powerpc/sysdev/xive/common.c          |  98 +++++--
 kernel/irq/irqdesc.c                       |   2 +-
 kernel/irq/irqdomain.c                     |   1 +
 kernel/irq/proc.c                          |   2 +-
 24 files changed, 710 insertions(+), 313 deletions(-)

-- 
2.31.1