ppc64 NUMA code needs an IQ injection

[Anton Blanchard]

Hi,

I was shown a box containing 128GB RAM of which only 32GB was visible
inside Linux. The dmesg shows:

memory region 0 to 10000000 maps to domain 0
memory region 10000000 to 20000000 maps to domain 0
memory region 20000000 to 30000000 maps to domain 1
memory region 30000000 to 40000000 maps to domain 2
Hole in node, disabling region start 40000000 length 10000000
...
Hole in node, disabling region start 1800000000 length 10000000
memory region 1810000000 to 1820000000 maps to domain 3

The device tree highlights the strange behaviour. The NUMA domain is the
last number:

./memory at 0/ibm,associativity
                 00000003 00000000 00000000 00000000
./memory at 10000000/ibm,associativity
                 00000003 00000000 00000000 00000000
./memory at 20000000/ibm,associativity
                 00000003 00000000 00000000 00000001
./memory at 30000000/ibm,associativity
                 00000003 00000000 00000000 00000002
./memory at 40000000/ibm,associativity
                 00000003 00000000 00000000 00000000
./memory at 50000000/ibm,associativity
                 00000003 00000000 00000000 00000000

Firmware thinks there are 2 256MB regions on node 1 and node 2. This
turned out to be a firmware bug, however its a bit rude to disable 3/4
of the machines memory because of it.

Our numa code expects each node to be contiguous and to appear in order,
and gets upset with the above layout.

Anton