[PATCH 0/3] Reintroduce cpu_core_mask
Daniel Henrique Barboza
danielhb413 at gmail.com
Thu Apr 15 22:17:50 AEST 2021
Hi,
Using a QEMU pseries guest with this follwing SMP topology, with a
single NUMA node:
(...) -smp 32,threads=4,cores=4,sockets=2, (...)
This is the output of lscpu with a guest running v5.12-rc5:
[root at localhost ~]# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Thread(s) per core: 4
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Model: 2.2 (pvr 004e 1202)
Model name: POWER9 (architected), altivec supported
Hypervisor vendor: KVM
Virtualization type: para
L1d cache: 32K
L1i cache: 32K
NUMA node0 CPU(s): 0-31
[root at localhost ~]#
The changes with cpu_core_mask made the topology sockets matching NUMA nodes.
In this case, given that we have a single NUMA node, the SMP topology got
adjusted to have 8 cores instead of 4 so we can have a single socket as well.
Although sockets equal to NUMA nodes is true for Power hardware, QEMU doesn't
have this constraint and users expect sockets and NUMA nodes to be kind of
independent, regardless of how unpractical that would be with real hardware.
The same guest running a kernel with this series applied:
[root at localhost ~]# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Thread(s) per core: 4
Core(s) per socket: 4
Socket(s): 2
NUMA node(s): 1
Model: 2.2 (pvr 004e 1202)
Model name: POWER9 (architected), altivec supported
Hypervisor vendor: KVM
Virtualization type: para
L1d cache: 32K
L1i cache: 32K
NUMA node0 CPU(s): 0-31
The sockets and NUMA nodes are being represented separately, as intended via
the QEMU command line.
Thanks for the looking this up, Srikar. For all patches:
Tested-by: Daniel Henrique Barboza <danielhb413 at gmail.com>
On 4/15/21 9:09 AM, Srikar Dronamraju wrote:
> Daniel had reported that
> QEMU is now unable to see requested topologies in a multi socket single
> NUMA node configurations.
> -smp 8,maxcpus=8,cores=2,threads=2,sockets=2
>
> This patchset reintroduces cpu_core_mask so that users can see requested
> topologies while still maintaining the boot time of very large system
> configurations.
>
> It includes caching the chip_id as suggested by Michael Ellermann
>
> 4 Threads/Core; 4 cores/Socket; 4 Sockets/Node, 2 Nodes in System
> -numa node,nodeid=0,memdev=m0 \
> -numa node,nodeid=1,memdev=m1 \
> -smp 128,sockets=8,threads=4,maxcpus=128 \
>
> 5.12.0-rc5 (or any kernel with commit 4ca234a9cbd7)
> ---------------------------------------------------
> srikar at cloudy:~$ lscpu
> Architecture: ppc64le
> Byte Order: Little Endian
> CPU(s): 128
> On-line CPU(s) list: 0-127
> Thread(s) per core: 4
> Core(s) per socket: 16
> Socket(s): 2 <<<<<-----
> NUMA node(s): 2
> Model: 2.3 (pvr 004e 1203)
> Model name: POWER9 (architected), altivec supported
> Hypervisor vendor: KVM
> Virtualization type: para
> L1d cache: 1 MiB
> L1i cache: 1 MiB
> NUMA node0 CPU(s): 0-15,32-47,64-79,96-111
> NUMA node1 CPU(s): 16-31,48-63,80-95,112-127
> --
> srikar at cloudy:~$ dmesg |grep smp
> [ 0.010658] smp: Bringing up secondary CPUs ...
> [ 0.424681] smp: Brought up 2 nodes, 128 CPUs
> --
>
> 5.12.0-rc5 + 3 patches
> ----------------------
> srikar at cloudy:~$ lscpu
> Architecture: ppc64le
> Byte Order: Little Endian
> CPU(s): 128
> On-line CPU(s) list: 0-127
> Thread(s) per core: 4
> Core(s) per socket: 4
> Socket(s): 8 <<<<-----
> NUMA node(s): 2
> Model: 2.3 (pvr 004e 1203)
> Model name: POWER9 (architected), altivec supported
> Hypervisor vendor: KVM
> Virtualization type: para
> L1d cache: 1 MiB
> L1i cache: 1 MiB
> NUMA node0 CPU(s): 0-15,32-47,64-79,96-111
> NUMA node1 CPU(s): 16-31,48-63,80-95,112-127
> --
> srikar at cloudy:~$ dmesg |grep smp
> [ 0.010372] smp: Bringing up secondary CPUs ...
> [ 0.417892] smp: Brought up 2 nodes, 128 CPUs
>
> 5.12.0-rc5
> ----------
> srikar at cloudy:~$ lscpu
> Architecture: ppc64le
> Byte Order: Little Endian
> CPU(s): 1024
> On-line CPU(s) list: 0-1023
> Thread(s) per core: 8
> Core(s) per socket: 128
> Socket(s): 1
> NUMA node(s): 1
> Model: 2.3 (pvr 004e 1203)
> Model name: POWER9 (architected), altivec supported
> Hypervisor vendor: KVM
> Virtualization type: para
> L1d cache: 4 MiB
> L1i cache: 4 MiB
> NUMA node0 CPU(s): 0-1023
> srikar at cloudy:~$ dmesg | grep smp
> [ 0.027753 ] smp: Bringing up secondary CPUs ...
> [ 2.315193 ] smp: Brought up 1 node, 1024 CPUs
>
> 5.12.0-rc5 + 3 patches
> ----------------------
> srikar at cloudy:~$ dmesg | grep smp
> [ 0.027659 ] smp: Bringing up secondary CPUs ...
> [ 2.532739 ] smp: Brought up 1 node, 1024 CPUs
>
> I also have booted and tested the kernels on PowerVM and PowerNV and
> even there I see a very negligible increase in the bringing up time of
> secondary CPUs
>
> Srikar Dronamraju (3):
> powerpc/smp: Reintroduce cpu_core_mask
> Revert "powerpc/topology: Update topology_core_cpumask"
> powerpc/smp: Cache CPU to chip lookup
>
> arch/powerpc/include/asm/smp.h | 6 ++++
> arch/powerpc/include/asm/topology.h | 2 +-
> arch/powerpc/kernel/prom.c | 19 +++++++---
> arch/powerpc/kernel/smp.c | 56 +++++++++++++++++++++++++----
> 4 files changed, 71 insertions(+), 12 deletions(-)
>
More information about the Linuxppc-dev
mailing list