[RFC PATCH 00/11] New TM Model

[Michael Neuling]

On Wed, 2018-09-12 at 16:40 -0300, Breno Leitao wrote:
> This patchset for the hardware transactional memory (TM) subsystem aims to
> avoid spending a lot of time on TM suspended mode in kernel space. It
> basically
> changes where the reclaim/recheckpoint will be executed.
> 
> Once a CPU enters in transactional state it uses a footprint area to track
> down the load/stores performed in transaction so it can be verified later
> to decide if a conflict happened due to some change done in that state. If
> a transaction is active in userspace and there is an exception that takes
> the CPU to the kernel space the CPU moves the transaction to suspended
> state but does not discard the footprint area.

In this description, you should differente between memory and register
(GPR/VSX/SPR) footprints.

In suspend, the CPU can disregard the memory footprint at any point, but it has
to keep the register footprint.  

In the above paragraph you are talking about register footprint but not memory
footprint. 

> 
> POWER9 has a known problem[1][2] and does not have enough room in
> footprint area for several transactions to be suspended at the same time
> on concurrent CPUs leading to CPU stalls.
> 
> This patchset aims to reclaim the checkpointed footprint as soon as the
> kernel is invoked, in the beginning of the exception handlers, thus freeing
> room to other CPUs enter in suspended mode, avoiding too many CPUs in
> suspended
> state that can cause the CPUs to stall. The same mechanism is done on kernel
> exit, doing a recheckpoint as late as possible (which will reload the
> checkpointed state into CPU's room) at the exception return.

OK, but we are still potentially in suspend in userspace, so that doesn't help
us on the lockup issue.

We need fake suspend in userspace to prevent lockups.

> The way to achieve this goal is creating a macro (TM_KERNEL_ENTRY) which
> will check if userspace was in an active transaction just after getting
> into kernel and reclaiming if that's the case. Thus all exception handlers
> will call this macro as soon as possible.
> 
> All exceptions should reclaim (if necessary) at this stage and only
> recheckpoint if the task is tagged as TIF_RESTORE_TM (i.e. was in
> transactional state before being interrupted), which will be done at
> ret_from_except_lite().
> 
> Ideally all reclaims will happen at the exception entrance, however during
> the recheckpoint process another exception can hit the CPU which might
> cause the current thread to be rescheduled, thus there is another reclaim
> point to be considered at __switch_to().

Can we do the recheckpoint() later so that it's when we have interrupts off and
can't be rescheduled?

> Hence, by allowing the CPU to be in suspended state for only a brief period
> it's possible to cope with the TM hardware limitations like the current
> problem on the new POWER9.

As mentioned, since we're still running userspace with real suspend, we still
have an issue.

> This patchset was tested in different scenarios using different test
> suites, as the kernel selftests and htm-torture[3], in the following
> configuration:
> 
>  * POWER8/pseries LE and BE
>  * POWER8/powernv LE
>  * POWER9/powernv LE hosting KVM guests running TM tests
> 
> This patchset is based on initial work done by Cyril Bur:
>     https://patchwork.ozlabs.org/cover/875341/

Adding Cyril to CC.

Mikey