<html><body><p><tt><font size="2">"Linuxppc-users" <linuxppc-users-bounces+calvins=us.ibm.com@lists.ozlabs.org> wrote on 09/27/2017 08:56:33 PM:<br><br>> From: Michael Ellerman <mpe@ellerman.id.au></font></tt><br><tt><font size="2">> To: Brian Horton <brianh@linux.vnet.ibm.com>, linuxppc-users@lists.ozlabs.org</font></tt><br><tt><font size="2">> Date: 09/27/2017 08:59 PM</font></tt><br><tt><font size="2">> Subject: Re: [Linuxppc-users] memmap kernel option on ubuntu 16.04 power8</font></tt><br><tt><font size="2">> Sent by: "Linuxppc-users" <linuxppc-users-bounces<br>> +calvins=us.ibm.com@lists.ozlabs.org></font></tt><br><tt><font size="2">> <br>> Brian Horton <brianh@linux.vnet.ibm.com> writes:<br>> > On 09/27/2017 05:45 AM, Michael Ellerman wrote:<br>> ...<br>> >> What hardware/platform are you trying to do this on? Are you familiar<br>> >> with kexec?<br>> ><br>> > P8, Tuleta and Firestone, baremetal, ppc64le w/ Ubuntu 16.04 HWE kernel.<br>> ><br>> > No, I haven't looked into anything with kexec - what are my options with <br>> > that? I would prefer to stay away from building my own kernel code - <br>> > i'll probably just live with the testing that i'm doing if that's my <br>> > only option. i was just trying to be a little more 'realistic'.<br>> <br>> Using kexec you can modify the device tree before booting the 2nd kernel.<br>> <br>> So you could remove memory from the device tree to simulate the setup<br>> you're looking for, and then as far as the 2nd kernel is concerned that<br>> memory doesn't exist.<br>> <br>> On my Tuleta here, the normal config is 4 nodes with 32GB per node:<br>> <br>> # numactl -H<br>> available: 4 nodes (0-1,16-17)<br>> ...<br>> node 0 size: 32599 MB<br>> ...<br>> node 1 size: 32713 MB<br>> ...<br>> node 16 size: 32714 MB<br>> ...<br>> node 17 size: 32569 MB<br>> <br>> <br>> If I reboot and drop to the petiboot shell, then I can do the following:<br>> <br>> $ dtc -O dts -I fs -o devtree.dts /proc/device-tree<br>> <br>> $ vi devtree.dts ...<br>> <br>> Look for all the nodes with device_type = "memory", eg:<br>> <br>> memory@0 {<br>> device_type = "memory";<br>> ibm,associativity = <0x4 0x0 0x0 0x0 0x0>;<br>> ibm,chip-id = <0x0>;<br>> phandle = <0x86>;<br>> reg = <0x0 0x0 0x8 0x0>;<br>> };<br>> <br>> memory@1800000000 {<br>> device_type = "memory";<br>> ibm,associativity = <0x4 0x0 0x0 0x1 0x11>;<br>> ibm,chip-id = <0x11>;<br>> phandle = <0x89>;<br>> reg = <0x18 0x0 0x8 0x0>;<br>> };<br>> <br>> memory@800000000 {<br>> device_type = "memory";<br>> ibm,associativity = <0x4 0x0 0x0 0x0 0x1>;<br>> ibm,chip-id = <0x1>;<br>> phandle = <0x87>;<br>> reg = <0x8 0x0 0x8 0x0>;<br>> };<br>> <br>> memory@1000000000 {<br>> device_type = "memory";<br>> ibm,associativity = <0x4 0x0 0x0 0x1 0x10>;<br>> ibm,chip-id = <0x10>;<br>> phandle = <0x88>;<br>> reg = <0x10 0x0 0x8 0x0>;<br>> };<br>> <br>> For each one, edit the reg so that the second value is 0x4 0x0, meaning<br>> 0x400000000 or 16GB, eg:<br>> <br>> reg = <0x10 0x0 0x8 0x0>;<br>> <br>> Becomes:<br>> <br>> reg = <0x10 0x0 0x4 0x0>;<br>> <br>> And so on for each memory node.<br>> <br>> Then:<br>> <br>> $ dtc -I dts -O dtb -o devtree.dtb devtree.dts<br>> $ kexec -l -b devtree.dtb -c "loglevel=8 nosplash <br>> root=PARTUUID=9ee9301a-50b3-4cae-8cf7-d46fff4ea920" vmlinux <br>> $ kexec -e<br>> <br>> <br>> Then once it boots:<br>> <br>> $ numactl -H<br>> available: 4 nodes (0-1,16-17)<br>> node 0 size: 16244 MB<br>> ...<br>> node 1 size: 16345 MB<br>> ...<br>> node 16 size: 16346 MB<br>> ...<br>> node 17 size: 16273 MB<br>> <br>> <br>> cheers</font></tt><br><br><tt><font size="2">Thank you Michael, I was able to do it on a Briigs machine following your instructions, this is one way to handle it.</font></tt><br><br><br><tt><font size="2">Before</font></tt><br><br><tt><font size="2">numactl -H</font></tt><br><tt><font size="2">available: 2 nodes (0,8)</font></tt><br><tt><font size="2">....</font></tt><br><tt><font size="2">node 0 size: 131072 MB</font></tt><br><tt><font size="2">...</font></tt><br><tt><font size="2">node 8 size: 131072 MB</font></tt><br><br><br><tt><font size="2">After</font></tt><br><br><tt><font size="2">numactl -H</font></tt><br><tt><font size="2">available: 2 nodes (0,8)</font></tt><br><tt><font size="2">node 0 size: 65536 MB</font></tt><br><tt><font size="2">...</font></tt><br><tt><font size="2">node 8 size: 65536 MB</font></tt><br><tt><font size="2">...</font></tt><br><br><br><br><tt><font size="2"><br>> _______________________________________________<br>> Linuxppc-users mailing list<br>> Linuxppc-users@lists.ozlabs.org<br>> <a href="https://urldefense.proofpoint.com/v2/url?">https://urldefense.proofpoint.com/v2/url?</a><br>> u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-<br>> siA1ZOg&r=XEmD5LeSMcW0pq1ccjREOfGKKSZEo6Uip7M_x_R-NUY&m=dskCpMc-L-<br>> vwg-<br>> CbWnmcGz83RpAHnlLTAHA06UfuqVc&s=2QZvf0mKFlokttVbA0rfb3dtwxQDXvLDZy5D_Q-o1_I&e=<br>> <br></font></tt><BR>
</body></html>