[Linuxppc-users] memmap kernel option on ubuntu 16.04 power8

[Michael Ellerman]

Mel Bakhshi <melb at ca.ibm.com> writes:

> You could also consider the huge page allocation. 
>
> For example ,to have 128GB memory spread across all numa nodes, in a bare-metal
> system wth 4 numa nodes:
>
> #echo 14236 >
> /sys/devices/system/node/node0/hugepages/hugepages-16384kB/nr_hugepages 
>
> #echo 14332 >
> /sys/devices/system/node/node1/hugepages/hugepages-16384kB/nr_hugepages
>
> #echo 14316 >
> /sys/devices/system/node/node16/hugepages/hugepages-16384kB/nr_hugepages
>
> #echo 14269 >
> /sys/devices/system/node/node17/hugepages/hugepages-16384kB/nr_hugepages

Hmm, that will probably "work" in that you'll have less memory reported
as free. But AFAIK there is no allocation policy for those hugepages,
ie. they will just be allocated wherever they can on each node. So
you'll potentially end up with a lot of fragmentation.

I don't know of a NUMA-aware hotunplug program, though it would be nice.

You can get a reasonable approximation with something like this script,
which offlines half the blocks on each node, starting at the highest
address and moving down.

cheers


#!/bin/bash

cd /sys/devices/system/node || exit 1

for node in $(ls -1d node*)
do
	blocks=$(ls -1d $node/memory* | wc -l)
	blocks_to_offline=$((blocks / 2))

	for block in $(grep . $node/memory*/phys_index | \
		       sort -r -t: -k 2 | \
		       head -$blocks_to_offline | \
		       cut -d/ -f -2)
	do
		echo "Offlining $block ..."
		echo 0 > $block/online

		if [[ $? -ne 0 ]]; then
			echo "Failed to offline block $block"
		fi
	done
done