[PATCH v2] powerpc: A new cache geometry aux vectors

Sat Mar 4 00:51:11 AEDT 2017

On 03/02/2017 10:01 PM, Michael Ellerman wrote:
> Paul Clarke <pc at us.ibm.com> writes:
>> On 03/02/2017 12:33 AM, Michael Ellerman wrote:
>>> Paul Clarke <pc at us.ibm.com> writes:
>>>> On 02/02/2017 12:22 AM, Benjamin Herrenschmidt wrote:
>>>>> This adds AUX vectors for the L1I,D, L2 and L3 cache levels
>>>>> providing for each cache level the size of the cache in bytes
>>>>> and the geometry (line size and number of ways).
>>>>>
>>>>> We chose to not use the existing alpha/sh definition which
>>>>> packs all the information in a single entry per cache level as
>>>>> it is too restricted to represent some of the geometries used
>>>>> on POWER.
>>>>>
>>>>> Signed-off-by: Benjamin Herrenschmidt <benh at kernel.crashing.org>
>>>>> ---
>>>>>
>>>>> v2. Remove some leftovers and fix 32-bit build
>>>>>
>>>>> Paul, can you test & ack this please ? It replaces the last patch of
>>>>> my previous series (the rest is identical).
>>>>
>>>> Sorry for the delay on this, I totally missed this e-mail until this week.
>>>>
>>>> I tested this with the Power Functional Simulator (a.k.a. Mambo), with
>>>> a fairly recent 4.9 kernel from the powerpc git tree.
>>>
>>> You mean 4.9 plus this patch?
>>>
>>>> Using Mambo-P8, I see what I expect for cache line size, but associativity is 0:
>>>> AT_L1I_CACHESIZE: 32768 (0x8000)
>>>> AT_L1I_CACHEGEOMETRY: associativity 0; line size 128
>>>> AT_L1D_CACHESIZE: 32768 (0x8000)
>>>> AT_L1D_CACHEGEOMETRY: associativity 0; line size 128
>>>> AT_L2_CACHESIZE: 524288 (0x80000)
>>>> AT_L2_CACHEGEOMETRY: associativity 0; line size 128
>>>> AT_L3_CACHESIZE: 8388608 (0x800000)
>>>> AT_L3_CACHEGEOMETRY: associativity 0; line size 128
>>>>
>>>> ...shouldn't associativity be 8 for all levels?
>>>
>>> Yes.
>>>
>>> I fixed it in v3: http://patchwork.ozlabs.org/patch/723576/
>>>
>>> Which is now merged in Linus tree, so please test that.
>>
>> It's not working for me, but it's also very likely that I could be
>> doing something stupid. Just pulling from
>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git, it
>> built (without the "systemsim" patches"), but wouldn't boot in Mambo
>> (not because it couldn't find root...something else not fully
>> explored).
>
> Hmm OK, it's booting for me, but might depend on your config/toolchain
> etc. Linus' tree just before rc1 is a bit of a dangerous place to be.
>
>> I checked out tag "v4.10", and that built (with the
>> "systemsim" patches, Ben's 1-8 and your 9v3) and booted.
>
> I think you need my entire v3, ignore Ben's series.
>
>> On Mambo-P8,
>> the device tree looks correct (to me), but the associativity is still
>> 0.
>
> OK, I think if you use the full v3 the associativity will be fixed.
>
> Though it might still be wrong, because the kernel code hard codes the
> values on P8 bare metal, because on real machines (running skiboot) the
> values provided in the device tree are wrong.
>
> Hopefully those hard coded values are also correct on p8 mambo, but we
> should probably check.
>
> See: https://git.kernel.org/torvalds/c/608b42140e96

OK, thanks for your help.  All is working as expected now.  mambo-p8 looks fine:
--
AT_L1I_CACHESIZE: 32768 (0x8000)
getauxval(AT_L1I_CACHEGEOMETRY) = 0x0000000000080080
AT_L1I_CACHEGEOMETRY: associativity 8; line size 128
AT_L1D_CACHESIZE: 32768 (0x8000)
getauxval(AT_L1D_CACHEGEOMETRY) = 0x0000000000080080
AT_L1D_CACHEGEOMETRY: associativity 8; line size 128
AT_L2_CACHESIZE: 524288 (0x80000)
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000080080
AT_L2_CACHEGEOMETRY: associativity 8; line size 128
AT_L3_CACHESIZE: 8388608 (0x800000)
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000080080
AT_L3_CACHEGEOMETRY: associativity 8; line size 128
--

>> On P9, the device tree isn't correct, so I'm not sure what to expect.
>
> Junk :)
>
> On P9 it all comes from the device tree so if that's wrong the AUX
> vectors will definitely be wrong.

mambo-p9 is still falling victim to the device-tree, but "working":
--
getauxval(AT_L1I_CACHEGEOMETRY) = 0x0000000000000080
AT_L1I_CACHEGEOMETRY: associativity 0; line size 128
getauxval(AT_L1D_CACHEGEOMETRY) = 0x0000000000000080
AT_L1D_CACHEGEOMETRY: associativity 0; line size 128
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000000000
getauxval(AT_L2_CACHEGEOMETRY) = 0x0000000000000000
--

>>> Can you try the test program I wrote (which may also have bugs):
>>>
>>>   http://patchwork.ozlabs.org/patch/724418/
>>
>> I couldn't get this working, either, so I'm probably doing something stupid.
>
> Did you apply the first patch of the series also?

I did now, and it works, with expected results:
P8:
--
# ./cache_shape
test: cache_shape
tags: git_version:v4.10-0-gc470abd-dirty
L1I  cache size:     0x8000      32768B         32K
L1I  line size:        0x80       8-way associative
L1D  cache size:     0x8000      32768B         32K
L1D  line size:        0x80       8-way associative
L2   cache size:    0x80000     524288B        512K
L2   line size:        0x80       8-way associative
L3   cache size:   0x800000    8388608B       8192K
L3   line size:        0x80       8-way associative
success: cache_shape
--

P9 (with bad device-tree):
--
# ./cache_shape
test: cache_shape
tags: git_version:v4.10-0-gc470abd-dirty
L1I  cache size:          0          0B          0K
L1I  line size:        0x80       fully associative
L1D  cache size:          0          0B          0K
L1D  line size:        0x80       fully associative
L2   cache size:          0          0B          0K
L2   line size:           0       fully associative
L3   cache size:          0          0B          0K
L3   line size:           0       fully associative
success: cache_shape
--

Do you want me to Ack your v3 patch set in that thread?

PC