[PATCH 00/14] powerpc/64: fast interrupt exits

[Nicholas Piggin]

This applies to powerpc next-test (particularly Christophe's ppc32
interrupt conversion) plus the 64e interrupt conversion patches I
recently posted.

This series attempts to improve the speed of interrupts and system calls
in three major ways.

Firstly, the SRR/HSRR registers do not need to be reloaded if they were
not used or clobbered fur the duration of the interrupt. 64e does not
implement this, but it could.

Secondly, an alternate return location facility is added for soft-masked
asynchronous interrupts and then that's used to set everything up for
return without having to disable MSR RI or EE.

Thirdly, mtmsrd and mtspr are reduced by various means. This is mostly
specific to 64s.

After this series, the entire system call / interrupt handler fast path
executes no mtsprs and one mtmsrd to enable interrupts initially, and
the system call vectored path doesn't even need to do that. This gives a
decent performance benefit. On POWER9 with a powernv_defconfig without
VIRT_CPU_ACCOUNTING_NATIVE, no meltdown workarounds, gettid sc system
call goes from 481 -> 344 cycles, gettid scv 345->299 cycles, and page
fault 1225->1064 cycles.

Since RFC, this no longer breaks 64e, several techniques for reducing
MSR/SPR updates become possible or tidier with interrupt wrappers, and
security fallback flushes aren't broken, usual bug fixes.

Thanks,
Nick

Nicholas Piggin (14):
  powerpc: remove interrupt exit helpers unused argument
  powerpc/64s: security fallback improvement
  powerpc/64s: introduce different functions to return from SRR vs HSRR
    interrupts
  powerpc/64s: avoid reloading (H)SRR registers if they are still valid
  powerpc/64: move interrupt return asm to interrupt_64.S
  powerpc/64s: save one more register in the masked interrupt handler
  powerpc/64: allow alternate return locations for soft-masked
    interrupts
  powerpc/64: interrupt soft-enable race fix
  powerpc/64: treat low kernel text as irqs soft-masked
  powerpc/64: use interrupt restart table to speed up return from
    interrupt
  powerpc/64e: Remove PPR from pt_regs
  powerpc/64s: system call avoid setting MSR[RI] until we set MSR[EE]
  powerpc/64: handle MSR EE and RI in interrupt entry wrapper
  powerpc/64s: use the same default PPR for user and kernel

 arch/powerpc/Kconfig.debug                 |   5 +
 arch/powerpc/include/asm/asm-prototypes.h  |   4 +-
 arch/powerpc/include/asm/exception-64e.h   |   6 +
 arch/powerpc/include/asm/exception-64s.h   |  52 +-
 arch/powerpc/include/asm/feature-fixups.h  |  18 +
 arch/powerpc/include/asm/head-64.h         |   2 +-
 arch/powerpc/include/asm/interrupt.h       |  41 +-
 arch/powerpc/include/asm/paca.h            |   9 +-
 arch/powerpc/include/asm/ppc_asm.h         |   8 +
 arch/powerpc/include/asm/processor.h       |   4 +-
 arch/powerpc/include/asm/ptrace.h          |  65 +-
 arch/powerpc/kernel/asm-offsets.c          |   7 +-
 arch/powerpc/kernel/entry_64.S             | 516 --------------
 arch/powerpc/kernel/exceptions-64e.S       |  53 +-
 arch/powerpc/kernel/exceptions-64s.S       | 384 +++++------
 arch/powerpc/kernel/fpu.S                  |   2 +
 arch/powerpc/kernel/head_64.S              |   5 +-
 arch/powerpc/kernel/interrupt.c            | 319 +++++----
 arch/powerpc/kernel/interrupt_64.S         | 738 +++++++++++++++++++++
 arch/powerpc/kernel/irq.c                  |  81 ++-
 arch/powerpc/kernel/kgdb.c                 |   2 +-
 arch/powerpc/kernel/kprobes-ftrace.c       |   2 +-
 arch/powerpc/kernel/kprobes.c              |  10 +-
 arch/powerpc/kernel/process.c              |  20 +-
 arch/powerpc/kernel/rtas.c                 |  13 +-
 arch/powerpc/kernel/signal.c               |   2 +-
 arch/powerpc/kernel/signal_64.c            |  14 +
 arch/powerpc/kernel/syscalls.c             |   2 +
 arch/powerpc/kernel/traps.c                |  18 +-
 arch/powerpc/kernel/vector.S               |   6 +-
 arch/powerpc/kernel/vmlinux.lds.S          |  24 +
 arch/powerpc/kvm/book3s_hv_rmhandlers.S    |   4 +
 arch/powerpc/lib/Makefile                  |   2 +-
 arch/powerpc/lib/feature-fixups.c          | 241 ++++---
 arch/powerpc/lib/restart_table.c           |  29 +
 arch/powerpc/lib/sstep.c                   |   5 +-
 arch/powerpc/math-emu/math.c               |   2 +-
 arch/powerpc/platforms/powernv/opal-call.c |   3 +
 arch/powerpc/sysdev/fsl_pci.c              |   2 +-
 39 files changed, 1631 insertions(+), 1089 deletions(-)
 create mode 100644 arch/powerpc/kernel/interrupt_64.S
 create mode 100644 arch/powerpc/lib/restart_table.c

-- 
2.23.0