libnuma interleaving oddness

[Nishanth Aravamudan]

On 31.08.2006 [09:47:30 +0200], Andi Kleen wrote:
> On Thursday 31 August 2006 08:00, Nishanth Aravamudan wrote:
> > On 30.08.2006 [14:04:40 -0700], Christoph Lameter wrote:
> > > > I took out the mlock() call, and I get the same results, FWIW.
> > > 
> > > What zones are available on your box? Any with HIGHMEM?
> > 
> > How do I tell the available zones from userspace? This is ppc64 with
> > about 64GB of memory total, it looks like. So, none of the nodes
> > (according to /sys/devices/system/node/*/meminfo) have highmem.
> 
> The zones are listed at the beginning of dmesg
> 
> "On node X total pages ...
>       DMA zone ...
>       ..." 

Page orders: linear mapping = 24, others = 12
<snip>
[boot]0100 MM Init
[boot]0100 MM Init Done
Linux version 2.6.16.21-0.8-ppc64 (geeko at buildhost) (gcc version 4.1.0 (SUSE Linux)) #1 SMP Mon Jul 3 18:25:39 UTC 2006
[boot]0012 Setup Arch
Node 0 Memory: 0x0-0x1b0000000
Node 1 Memory: 0x1b0000000-0x3b0000000
Node 2 Memory: 0x3b0000000-0x5b0000000
Node 3 Memory: 0x5b0000000-0x7b0000000
Node 4 Memory: 0x7b0000000-0x9a0000000
Node 5 Memory: 0x9a0000000-0xba0000000
Node 6 Memory: 0xba0000000-0xda0000000
Node 7 Memory: 0xda0000000-0xf90000000
EEH: PCI Enhanced I/O Error Handling Enabled
PPC64 nvram contains 7168 bytes
Using dedicated idle loop
On node 0 totalpages: 1769472
  DMA zone: 1769472 pages, LIFO batch:31
  DMA32 zone: 0 pages, LIFO batch:0
  Normal zone: 0 pages, LIFO batch:0
  HighMem zone: 0 pages, LIFO batch:0
On node 1 totalpages: 2097152
  DMA zone: 2097152 pages, LIFO batch:31
  DMA32 zone: 0 pages, LIFO batch:0
  Normal zone: 0 pages, LIFO batch:0
  HighMem zone: 0 pages, LIFO batch:0
On node 2 totalpages: 2097152
  DMA zone: 2097152 pages, LIFO batch:31
  DMA32 zone: 0 pages, LIFO batch:0
  Normal zone: 0 pages, LIFO batch:0
  HighMem zone: 0 pages, LIFO batch:0
On node 3 totalpages: 2097152
  DMA zone: 2097152 pages, LIFO batch:31
  DMA32 zone: 0 pages, LIFO batch:0
  Normal zone: 0 pages, LIFO batch:0
  HighMem zone: 0 pages, LIFO batch:0
On node 4 totalpages: 2031616
  DMA zone: 2031616 pages, LIFO batch:31
  DMA32 zone: 0 pages, LIFO batch:0
  Normal zone: 0 pages, LIFO batch:0
  HighMem zone: 0 pages, LIFO batch:0
On node 5 totalpages: 2097152
  DMA zone: 2097152 pages, LIFO batch:31
  DMA32 zone: 0 pages, LIFO batch:0
  Normal zone: 0 pages, LIFO batch:0
  HighMem zone: 0 pages, LIFO batch:0
On node 6 totalpages: 2097152
  DMA zone: 2097152 pages, LIFO batch:31
  DMA32 zone: 0 pages, LIFO batch:0
  Normal zone: 0 pages, LIFO batch:0
  HighMem zone: 0 pages, LIFO batch:0
On node 7 totalpages: 2031616
  DMA zone: 2031616 pages, LIFO batch:31
  DMA32 zone: 0 pages, LIFO batch:0
  Normal zone: 0 pages, LIFO batch:0
  HighMem zone: 0 pages, LIFO batch:0
[boot]0015 Setup Done
Built 8 zonelists

Thanks,
Nish

-- 
Nishanth Aravamudan <nacc at us.ibm.com>
IBM Linux Technology Center