[PATCH 00/10] powerpc/pseries/vas: NXGZIP support with DLPAR
Haren Myneni
haren at linux.ibm.com
Tue Nov 30 04:44:53 AEDT 2021
PowerPC provides HW compression with NX coprocessor. This feature
is available on both PowerNV and PowerVM and included in Linux.
Since each powerpc chip has one NX coprocessor, the VAS introduces
the concept of windows / credits to manage access to this hardware
resource. On powerVM, these limited resources should be available
across all LPARs. So the hypervisor assigns the specific credits
to each LPAR based on processor entitlement so that one LPAR does
not overload NX. The hypervisor can reject the window open request
to a partition if exceeds its credit limit (1 credit per window).
So the total number of target credits in a partition can be changed
if the core configuration is modified. The hypervisor expects the
partition to modify its window usage depends on new target
credits. For example, if the partition uses more credits than the
new target credits, it should close the excessive windows so that
the NX resource will be available to other partitions.
This patch series enables OS to support this dynamic credit
management with DLPAR core removal/add.
Core removal operation:
- Get new VAS capabilities from the hypervisor when the DLPAR
notifier is received. This capabilities provides the new target
credits based on new processor entitlement. In the case of QoS
credit changes, the notification will be issued by updating
the target_creds via sysfs.
- If the partition is already used more than the new target credits,
the kernel selects windows, unmap the current paste address and
close them in the hypervisor, It uses LIFO to identify these
windows - last windows that are opened are the first ones to be
closed.
- When the user space issue requests on these windows, NX generates
page fault on the unmap paste address. The kernel handles the
fault by returning the paste instruction failure if the window is
not active (means unmap paste). Then up to the library / user
space to fall back to SW compression or manage with the current
windows.
Core add operation:
- The kernel can see increased target credits from the new VAS
capabilities.
- Scans the window list for the closed windows in the hypervisor
due to lost credit before and selects windows based on same LIFO.
- Make these corresponding windows active and create remap with
the same VMA on the new paste address in the fault handler.
- Then the user space should expect paste successful later.
Patch 1: Define common names for sysfs target/used/avail_creds so
that same sysfs entries can be used even on PowerNV later.
Patch 2: Add VAS notifier for DLPAR core add / removal
Patch 3: Save LPID in the vas window struct during initial window
open and use it when reopen later.
Patch 4: When credits are available, reopen windows that are closed
before with core removal.
Patch 5: Close windows in the hypervisor when the partition exceeds
its usage than the new target credits.
Patch 6: If the window is closed in the hypervisor before the user
space issue the initial mmap(), return -EACCES failure.
Patch 7: Add new mmap fault handler which handles the page fault
from NX on paste address.
Patch 8: Return the paste instruction failure if the window is not
active.
Patch 9 & 10: The user space determines the credit usage with sysfs
target/avail/used_creds interfaces. drmgr uses target_creds
to notify OS for QoS credit changes.
Thanks to Nicholas Piggin and Aneesh Kumar for the valuable suggestions
on the NXGZIP design to support DLPAR operations.
Haren Myneni (10):
powerpc/pseries/vas: Use common names in VAS capability structure
powerpc/pseries/vas: Add notifier for DLPAR core removal/add
powerpc/pseries/vas: Save partition PID in pseries_vas_window struct
powerpc/pseries/vas: Reopen windows with DLPAR core add
powerpc/pseries/vas: Close windows with DLPAR core removal
powerpc/vas: Map paste address only if window is active
powerpc/vas: Add paste address mmap fault handler
powerpc/vas: Return paste instruction failure if window is not active
powerpc/pseries/vas: sysfs interface to export capabilities
powerpc/pseries/vas: Write 'target_creds' for QoS credits change
arch/powerpc/include/asm/ppc-opcode.h | 2 +
arch/powerpc/include/asm/vas.h | 16 ++
arch/powerpc/platforms/book3s/vas-api.c | 129 ++++++++-
arch/powerpc/platforms/pseries/Makefile | 2 +-
arch/powerpc/platforms/pseries/vas-sysfs.c | 246 +++++++++++++++++
arch/powerpc/platforms/pseries/vas.c | 293 ++++++++++++++++++++-
arch/powerpc/platforms/pseries/vas.h | 16 +-
7 files changed, 691 insertions(+), 13 deletions(-)
create mode 100644 arch/powerpc/platforms/pseries/vas-sysfs.c
--
2.27.0
More information about the Linuxppc-dev
mailing list