[Skiboot] [PATCH] SLW : Increase stop4-5 residency by 5x

[Abhishek]

On 11/25/2019 06:39 PM, Klaus Heinrich Kiwi wrote:
>
> On 11/21/2019 9:10 AM, Abhishek Goel wrote:
>> Using ppdcf2d run, we observed that there was a drop of about 50%
>> throughput with stop4/5. In this benchmark the GPU waits on the cpu to
>> wakeup and provide the subsequent data block to compute. CPUs serving
>> GPUs workloads are idle enough for the current residency values for the
>> governor to pick up stop4/5 states for those CPUs. The higher wakeup
>> latency of these deep states accumulates over the run and shows up as a
>> performance drop.
>>
>> We observe no perfromance drop with increased residency values.
>
> Do we know or can we estimate what are the power-saving drawbacks from 
> such a change? Why did we choose 10ms to begin with? If this is a 
> corner case, wouldn't it be better if that was a runtime choice, or in 
> the extreme case disable powersave altogether?
>
> -Klaus
>
I observe <5% regression at max. No regression for very high or very low 
utilisation.

Originally residency was 1ms, which was 10x of the measured wakeup 
latency. But,
last year there was a similar issue with a GPU workload and a patch was 
rolled out to
increase the residency for deep stop states to 10x, so the new residency 
became
10ms from the earlier value of 1ms.

The problem has been observed only with the deep states, i.e. stop4 and 
stop5,
so we do not want to disable all the other states altogether. All these 
issues go away if
disable deep states. But, shallow states don't give us the WOF benefits 
so we need these
deep states for that purpose.

After discussion with Michael Ellerman and Oliver in a separate channel, 
we have decided
to go with a kernel approach for this and will be dropping this approach 
of bumping up the
residency for deep states.

-Abhishek