<div class="socmaildefaultfont" dir="ltr" style="font-family:Arial, Helvetica, sans-serif;font-size:10.5pt" ><div dir="ltr" ><p style="margin: 0px; font-stretch: normal; font-size: 12px; line-height: normal; font-family: "Helvetica Neue"; color: rgb(69, 69, 69);" >To setup the system to have 128GB memory spread across all numa nodes, I use the huge page allocation .</p>
<div style="margin: 0px; font-stretch: normal; font-size: 12px; line-height: normal; font-family: "Helvetica Neue"; color: rgb(69, 69, 69);" > </div>
<p style="margin: 0px; font-stretch: normal; font-size: 12px; line-height: normal; font-family: "Helvetica Neue"; color: rgb(69, 69, 69);" >For example in a bare-metal system wth 4 numa nodes, after allocation the huge pages, I get about 32GB each left on each numa node </p>
<div style="margin: 0px; font-stretch: normal; font-size: 12px; line-height: normal; font-family: "Helvetica Neue"; color: rgb(69, 69, 69);" > </div>
<div style="margin: 0px; font-stretch: normal; font-size: 12px; line-height: normal; font-family: "Helvetica Neue"; color: rgb(69, 69, 69);" >echo xyz > /sys/devices/system/node/node17/hugepages/hugepages-16384kB/nr_hugepages</div>
<p style="margin: 0px; font-stretch: normal; font-size: 12px; line-height: normal; font-family: "Helvetica Neue"; color: rgb(69, 69, 69);" >echo xyz > /sys/devices/system/node/node16/hugepages/hugepages-16384kB/nr_hugepages</p>
<p style="margin: 0px; font-stretch: normal; font-size: 12px; line-height: normal; font-family: "Helvetica Neue"; color: rgb(69, 69, 69);" >echo xyz > /sys/devices/system/node/node1/hugepages/hugepages-16384kB/nr_hugepages</p>
<p style="margin: 0px; font-stretch: normal; font-size: 12px; line-height: normal; font-family: "Helvetica Neue"; color: rgb(69, 69, 69);" >echo xyz > /sys/devices/system/node/node0/hugepages/hugepages-16384kB/nr_hugepages</p>
<div style="margin: 0px; font-stretch: normal; font-size: 12px; line-height: normal; font-family: "Helvetica Neue"; color: rgb(69, 69, 69);" > </div>
<div style="margin: 0px; font-stretch: normal; font-size: 12px; line-height: normal; font-family: "Helvetica Neue"; color: rgb(69, 69, 69);" > </div></div>
<div dir="ltr" ><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial;font-size:10.5pt" ><div class="socmaildefaultfont" dir="ltr" style="font-family:Arial;font-size:10.5pt" ><div dir="ltr" ><div><font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size="2" ><font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size="2" >Regards</font></font></div>
<div><br><font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size="2" ><font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size="2" >Mel Bakhshi <br>Power Systems Performance <br>8200 Warden Ave</font></font></div>
<div><font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size="2" ><font face="Default Sans Serif,Verdana,Arial,Helvetica,sans-serif" size="2" >Markham, L6G 1C7<br>Phone: +1-905-413-3982 <br> </font></font></div></div></div></div></div>
<div dir="ltr" > </div>
<div dir="ltr" > </div>
<blockquote data-history-content-modified="1" dir="ltr" style="border-left:solid #aaaaaa 2px; margin-left:5px; padding-left:5px; direction:ltr; margin-right:0px" >----- Original message -----<br>From: linuxppc-users-request@lists.ozlabs.org<br>Sent by: "Linuxppc-users" <linuxppc-users-bounces+melb=ca.ibm.com@lists.ozlabs.org><br>To: linuxppc-users@lists.ozlabs.org<br>Cc:<br>Subject: Linuxppc-users Digest, Vol 4, Issue 13<br>Date: Wed, Sep 27, 2017 1:47 PM<br>
<div><font face="Default Monospace,Courier New,Courier,monospace" size="2" >Send Linuxppc-users mailing list submissions to<br>linuxppc-users@lists.ozlabs.org<br><br>To subscribe or unsubscribe via the World Wide Web, visit<br><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=vfSI789crSkT8xJzRaoouC0-Vu3Dd8aaKJVhxDEXY3o&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=vfSI789crSkT8xJzRaoouC0-Vu3Dd8aaKJVhxDEXY3o&e=</a> <br>or, via email, send a message with subject or body 'help' to<br>linuxppc-users-request@lists.ozlabs.org<br><br>You can reach the person managing the list at<br>linuxppc-users-owner@lists.ozlabs.org<br><br>When replying, please edit your Subject line so it is more specific<br>than "Re: Contents of Linuxppc-users digest..."<br><br><br>Today's Topics:<br><br> 1. Re: memmap kernel option on ubuntu 16.04 power8 (Calvin Sze)<br> 2. Re: memmap kernel option on ubuntu 16.04 power8<br> (Cédric Le Goater)<br> 3. Re: memmap kernel option on ubuntu 16.04 power8 (Calvin Sze)<br><br><br>----------------------------------------------------------------------<br><br>Message: 1<br>Date: Wed, 27 Sep 2017 10:53:14 -0500<br>From: "Calvin Sze" <calvins@us.ibm.com><br>To: "Jenifer Hopper" <jhopper@us.ibm.com><br>Cc: Brian Horton <brianh@linux.vnet.ibm.com>,<br>linuxppc-users@lists.ozlabs.org,<br>"Linuxppc-users"<linuxppc-users-bounces+calvins=us.ibm.com@lists.ozlabs.org><br><br>Subject: Re: [Linuxppc-users] memmap kernel option on ubuntu 16.04<br>power8<br>Message-ID:<br><OF60CDE08E.CC7516F2-ON002581A8.005732AA-862581A8.005745C4@notes.na.collabserv.com><br><br>Content-Type: text/plain; charset="utf-8"<br><br><br>Checked with BMC people, they said It cannot be done in BMC.<br><br><br>Calvin Sze<br>OpenSystem Enablement/Optimization<br>email: calvins@us.ibm.com<br><br><br><br><br>From: "Jenifer Hopper" <jhopper@us.ibm.com><br>To: Brian Horton <brianh@linux.vnet.ibm.com><br>Cc: linuxppc-users@lists.ozlabs.org<br>Date: 09/27/2017 10:18 AM<br>Subject: Re: [Linuxppc-users] memmap kernel option on ubuntu 16.04<br> power8<br>Sent by: "Linuxppc-users" <linuxppc-users-bounces<br> +calvins=us.ibm.com@lists.ozlabs.org><br><br><br><br>For FSP based systems you can go into the ASM and deconfigure dimms (System<br>Config -> HW Deconfig -> Mem Deconfig), but I don't know of a way to do<br>that for BMC based systems.<br><br>Thanks,<br>Jenifer Hopper<br>IBM Linux Technology Center, Linux Optimization<br>jhopper@us.ibm.com<br>(512) 286-6701<br><br>-----"Linuxppc-users" <linuxppc-users-bounces<br>+jhopper=us.ibm.com@lists.ozlabs.org> wrote: -----<br>To: Michael Ellerman <mpe@ellerman.id.au>, linuxppc-users@lists.ozlabs.org<br>From: Brian Horton<br>Sent by: "Linuxppc-users"<br>Date: 09/27/2017 10:12AM<br>Subject: Re: [Linuxppc-users] memmap kernel option on ubuntu 16.04 power8<br><br><br>On 09/27/2017 05:45 AM, Michael Ellerman wrote:<br>> Hi Brian,<br>><br>> Brian Horton <brianh@linux.vnet.ibm.com> writes:<br>>> Greetings all.<br>>><br>>> I'm trying to run some tests with less memory than what's installed. the<br>>> 'mem=128G' option works, but it ends up putting all of the memory on the<br>>> first numa node:<br>><br>> It doesn't put memory anywhere, it just clamps the total memory, and it<br>> happens that the first (lowest address) 128G of RAM is all on node 0.<br><br>right, 'put' was a bad word choice. but yes, i see that the affect is<br>that the first 128G of RAM is used.<br><br><br>><br>>> someone said that it worked on RHEL, so maybe it's an Ubuntu-specific<br>bug..<br>><br>> OK that would be interesting, but I suspect it's not true :)<br><br>correct, on 2nd look it was mis-information. they confirmed that it<br>doesn't work on RHEL either.<br><br>>> thoughts? ideas?<br>><br>> AFAIK there's no way to achieve what you're after, without changing<br>> kernel code or creating a custom device tree.<br>><br>> The mem= option is very primitive, it just clamps the total as I said<br>> above, it's not NUMA aware.<br>><br>> What hardware/platform are you trying to do this on? Are you familiar<br>> with kexec?<br><br>P8, Tuleta and Firestone, baremetal, ppc64le w/ Ubuntu 16.04 HWE kernel.<br><br>No, I haven't looked into anything with kexec - what are my options with<br>that? I would prefer to stay away from building my own kernel code -<br>i'll probably just live with the testing that i'm doing if that's my<br>only option. i was just trying to be a little more 'realistic'.<br><br>Thanks! .bri.<br><br><br>> cheers<br>><br><br>_______________________________________________<br>Linuxppc-users mailing list<br>Linuxppc-users@lists.ozlabs.org<br><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=59cao60PiAeGQQaaFLkjnWW7CHr6qoRfN8NBRYVc9UQ&m=R9rn3U537CNoKLsBMZtzHp6VJc2mlnNdfmtsygTu4A0&s=o-r0sCpTTOrApxT7FVtnYCyWMpr9QjeO3pt0SW1yJm8&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=59cao60PiAeGQQaaFLkjnWW7CHr6qoRfN8NBRYVc9UQ&m=R9rn3U537CNoKLsBMZtzHp6VJc2mlnNdfmtsygTu4A0&s=o-r0sCpTTOrApxT7FVtnYCyWMpr9QjeO3pt0SW1yJm8&e=</a><br><br><br>_______________________________________________<br>Linuxppc-users mailing list<br>Linuxppc-users@lists.ozlabs.org<br><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=XEmD5LeSMcW0pq1ccjREOfGKKSZEo6Uip7M_x_R-NUY&m=WNMxAOec1e_w34zBe3xVYIoJbJamikId375eNQw97IA&s=DGTWVG8bGGNjZ-cfXVvDyGfk8R4CJxIGw84m3Wi56Tk&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=XEmD5LeSMcW0pq1ccjREOfGKKSZEo6Uip7M_x_R-NUY&m=WNMxAOec1e_w34zBe3xVYIoJbJamikId375eNQw97IA&s=DGTWVG8bGGNjZ-cfXVvDyGfk8R4CJxIGw84m3Wi56Tk&e=</a><br><br><br><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ozlabs.org_pipermail_linuxppc-2Dusers_attachments_20170927_7735e97d_attachment-2D0001.html&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=TlprbfjYB-TuSpRRRZNO8DwGPkLBpQTyYN8WBnqdtr0&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ozlabs.org_pipermail_linuxppc-2Dusers_attachments_20170927_7735e97d_attachment-2D0001.html&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=TlprbfjYB-TuSpRRRZNO8DwGPkLBpQTyYN8WBnqdtr0&e=</a> ><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: graycol.gif<br>Type: image/gif<br>Size: 105 bytes<br>Desc: not available<br>URL: <<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ozlabs.org_pipermail_linuxppc-2Dusers_attachments_20170927_7735e97d_attachment-2D0001.gif&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=KzHq2-P3oJsykTS20ti6K_9A2WtH0QiKPL7tDY1QBo4&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ozlabs.org_pipermail_linuxppc-2Dusers_attachments_20170927_7735e97d_attachment-2D0001.gif&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=KzHq2-P3oJsykTS20ti6K_9A2WtH0QiKPL7tDY1QBo4&e=</a> ><br><br>------------------------------<br><br>Message: 2<br>Date: Wed, 27 Sep 2017 18:19:53 +0200<br>From: Cédric Le Goater <clg@fr.ibm.com><br>To: Calvin Sze <calvins@us.ibm.com>, Jenifer Hopper<br><jhopper@us.ibm.com><br>Cc: Linuxppc-users<br><linuxppc-users-bounces+calvins=us.ibm.com@lists.ozlabs.org>,<br>linuxppc-users@lists.ozlabs.org<br>Subject: Re: [Linuxppc-users] memmap kernel option on ubuntu 16.04<br>power8<br>Message-ID: <0869bfd6-6e13-48c2-39a7-776500338867@fr.ibm.com><br>Content-Type: text/plain; charset=utf-8<br><br>Not even with a guard record in the PNOR ?<br><br><br><br>C.<br><br><br><br>==On 09/27/2017 05:53 PM, Calvin Sze wrote:<br><br>> Checked with BMC people, they said It cannot be done in BMC.<br><br>><br><br>><br><br>> Calvin Sze<br><br>> OpenSystem Enablement/Optimization<br><br>> email: calvins@us.ibm.com<br><br>><br><br>><br><br>> Inactive hide details for "Jenifer Hopper" ---09/27/2017 10:18:36 AM---For FSP based systems you can go into the ASM and deconf"Jenifer Hopper" ---09/27/2017 10:18:36 AM---For FSP based systems you can go into the ASM and deconfigure dimms (System Config -> HW Deconfig -><br><br>><br><br>> From: "Jenifer Hopper" <jhopper@us.ibm.com><br><br>> To: Brian Horton <brianh@linux.vnet.ibm.com><br><br>> Cc: linuxppc-users@lists.ozlabs.org<br><br>> Date: 09/27/2017 10:18 AM<br><br>> Subject: Re: [Linuxppc-users] memmap kernel option on ubuntu 16.04 power8<br><br>> Sent by: "Linuxppc-users" <linuxppc-users-bounces+calvins=us.ibm.com@lists.ozlabs.org><br><br>><br><br>> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------<br><br>><br><br>><br><br>><br><br>> For FSP based systems you can go into the ASM and deconfigure dimms (System Config -> HW Deconfig -> Mem Deconfig), but I don't know of a way to do that for BMC based systems.<br><br>><br><br>> Thanks,<br><br>> Jenifer Hopper<br><br>> IBM Linux Technology Center, Linux Optimization_<br><br>> __jhopper@us.ibm.com_ <<a href="mailto:jhopper@us.ibm.com" target="_blank" >mailto:jhopper@us.ibm.com</a>><br><br>> (512) 286-6701<br><br>><br><br>> -----"Linuxppc-users" <_linuxppc-users-bounces+jhopper=us.ibm.com@lists.ozlabs.org_ <<a href="mailto:linuxppc-users-bounces+jhopper=us.ibm.com@lists.ozlabs.org" target="_blank" >mailto:linuxppc-users-bounces+jhopper=us.ibm.com@lists.ozlabs.org</a>>> wrote: -----<br><br>> To: Michael Ellerman <_mpe@ellerman.id.au_ <<a href="mailto:mpe@ellerman.id.au" target="_blank" >mailto:mpe@ellerman.id.au</a>>>, _linuxppc-users@lists.ozlabs.org_ <<a href="mailto:linuxppc-users@lists.ozlabs.org" target="_blank" >mailto:linuxppc-users@lists.ozlabs.org</a>><br><br>> From: Brian Horton<br><br>> Sent by: "Linuxppc-users"<br><br>> Date: 09/27/2017 10:12AM<br><br>> Subject: Re: [Linuxppc-users] memmap kernel option on ubuntu 16.04 power8<br><br>><br><br>><br><br>> On 09/27/2017 05:45 AM, Michael Ellerman wrote:<br><br>>> Hi Brian,<br><br>>><br><br>>> Brian Horton <_brianh@linux.vnet.ibm.com_ <<a href="mailto:brianh@linux.vnet.ibm.com" target="_blank" >mailto:brianh@linux.vnet.ibm.com</a>>> writes:<br><br>>>> Greetings all.<br><br>>>><br><br>>>> I'm trying to run some tests with less memory than what's installed. the<br><br>>>> 'mem=128G' option works, but it ends up putting all of the memory on the<br><br>>>> first numa node:<br><br>>><br><br>>> It doesn't put memory anywhere, it just clamps the total memory, and it<br><br>>> happens that the first (lowest address) 128G of RAM is all on node 0.<br><br>><br><br>> right, 'put' was a bad word choice. but yes, i see that the affect is<br><br>> that the first 128G of RAM is used.<br><br>><br><br>><br><br>>><br><br>>>> someone said that it worked on RHEL, so maybe it's an Ubuntu-specific bug..<br><br>>><br><br>>> OK that would be interesting, but I suspect it's not true :)<br><br>><br><br>> correct, on 2nd look it was mis-information. they confirmed that it<br><br>> doesn't work on RHEL either.<br><br>><br><br>>>> thoughts? ideas?<br><br>>><br><br>>> AFAIK there's no way to achieve what you're after, without changing<br><br>>> kernel code or creating a custom device tree.<br><br>>><br><br>>> The mem= option is very primitive, it just clamps the total as I said<br><br>>> above, it's not NUMA aware.<br><br>>><br><br>>> What hardware/platform are you trying to do this on? Are you familiar<br><br>>> with kexec?<br><br>><br><br>> P8, Tuleta and Firestone, baremetal, ppc64le w/ Ubuntu 16.04 HWE kernel.<br><br>><br><br>> No, I haven't looked into anything with kexec - what are my options with<br><br>> that? I would prefer to stay away from building my own kernel code -<br><br>> i'll probably just live with the testing that i'm doing if that's my<br><br>> only option. i was just trying to be a little more 'realistic'.<br><br>><br><br>> Thanks! .bri.<br><br>><br><br>><br><br>>> cheers<br><br>>><br><br>><br><br>> _______________________________________________<br><br>> Linuxppc-users mailing list_<br><br>> __Linuxppc-users@lists.ozlabs.org_ <<a href="mailto:Linuxppc-users@lists.ozlabs.org" target="_blank" >mailto:Linuxppc-users@lists.ozlabs.org</a>>_<br><br>> __https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=59cao60PiAeGQQaaFLkjnWW7CHr6qoRfN8NBRYVc9UQ&m=R9rn3U537CNoKLsBMZtzHp6VJc2mlnNdfmtsygTu4A0&s=o-r0sCpTTOrApxT7FVtnYCyWMpr9QjeO3pt0SW1yJm8&e=_<br><br>><br><br>> _______________________________________________<br><br>> Linuxppc-users mailing list<br><br>> Linuxppc-users@lists.ozlabs.org<br><br>> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=XEmD5LeSMcW0pq1ccjREOfGKKSZEo6Uip7M_x_R-NUY&m=WNMxAOec1e_w34zBe3xVYIoJbJamikId375eNQw97IA&s=DGTWVG8bGGNjZ-cfXVvDyGfk8R4CJxIGw84m3Wi56Tk&e= <br><br>><br><br>><br><br>><br><br>><br><br>><br><br>> _______________________________________________<br><br>> Linuxppc-users mailing list<br><br>> Linuxppc-users@lists.ozlabs.org<br><br>> <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=XHJsZhhuWSw9713Fp0ciew&m=TmviNY9vCNXvoo8fJe9fezSUxE-oPeWagE1fD19ZpLU&s=RT48AqWNNoMza32hJQTGnEeLGicTutGj2yrpsbdoYmk&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=XHJsZhhuWSw9713Fp0ciew&m=TmviNY9vCNXvoo8fJe9fezSUxE-oPeWagE1fD19ZpLU&s=RT48AqWNNoMza32hJQTGnEeLGicTutGj2yrpsbdoYmk&e=</a> <br><br>><br><br><br><br><br>------------------------------<br><br>Message: 3<br>Date: Wed, 27 Sep 2017 12:46:05 -0500<br>From: "Calvin Sze" <calvins@us.ibm.com><br>To: Cédric Le Goater <clg@fr.ibm.com><br>Cc: "Jenifer Hopper" <jhopper@us.ibm.com>,<br>linuxppc-users@lists.ozlabs.org,<br>Linuxppc-users<linuxppc-users-bounces+calvins=us.ibm.com@lists.ozlabs.org><br><br>Subject: Re: [Linuxppc-users] memmap kernel option on ubuntu 16.04<br>power8<br>Message-ID:<br><OF4416450B.36CBA7A5-ON002581A8.006170AE-862581A8.00619A55@notes.na.collabserv.com><br><br>Content-Type: text/plain; charset="iso-8859-1"<br><br><br>Hi Cedric,<br>Thanks, Jyo is checking for me.<br><br>Calvin Sze<br>OpenSystem Enablement/Optimization<br>email: calvins@us.ibm.com<br><br><br><br><br>From: Cédric Le Goater <clg@fr.ibm.com><br>To: Calvin Sze/Austin/IBM@IBMUS, Jenifer Hopper/Austin/IBM@IBMUS<br>Cc: Linuxppc-users <linuxppc-users-bounces<br> +calvins=us.ibm.com@lists.ozlabs.org>,<br> linuxppc-users@lists.ozlabs.org<br>Date: 09/27/2017 11:20 AM<br>Subject: Re: [Linuxppc-users] memmap kernel option on ubuntu 16.04<br> power8<br><br><br><br>Not even with a guard record in the PNOR ?<br><br>C.<br><br>==On 09/27/2017 05:53 PM, Calvin Sze wrote:<br>> Checked with BMC people, they said It cannot be done in BMC.<br>><br>><br>> Calvin Sze<br>> OpenSystem Enablement/Optimization<br>> email: calvins@us.ibm.com<br>><br>><br>> Inactive hide details for "Jenifer Hopper" ---09/27/2017 10:18:36<br>AM---For FSP based systems you can go into the ASM and deconf"Jenifer<br>Hopper" ---09/27/2017 10:18:36 AM---For FSP based systems you can go into<br>the ASM and deconfigure dimms (System Config -> HW Deconfig -><br>><br>> From: "Jenifer Hopper" <jhopper@us.ibm.com><br>> To: Brian Horton <brianh@linux.vnet.ibm.com><br>> Cc: linuxppc-users@lists.ozlabs.org<br>> Date: 09/27/2017 10:18 AM<br>> Subject: Re: [Linuxppc-users] memmap kernel option on ubuntu 16.04 power8<br>> Sent by: "Linuxppc-users" <linuxppc-users-bounces<br>+calvins=us.ibm.com@lists.ozlabs.org><br>><br>><br>------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------<br><br>><br>><br>><br>> For FSP based systems you can go into the ASM and deconfigure dimms<br>(System Config -> HW Deconfig -> Mem Deconfig), but I don't know of a way<br>to do that for BMC based systems.<br>><br>> Thanks,<br>> Jenifer Hopper<br>> IBM Linux Technology Center, Linux Optimization_<br>> __jhopper@us.ibm.com_ <<a href="mailto:jhopper@us.ibm.com" target="_blank" >mailto:jhopper@us.ibm.com</a>><br>> (512) 286-6701<br>><br>> -----"Linuxppc-users" <_linuxppc-users-bounces<br>+jhopper=us.ibm.com@lists.ozlabs.org_ <<a href="mailto:linuxppc-users-bounces+jhopper=us.ibm.com@lists.ozlabs.org" target="_blank" >mailto:linuxppc-users-bounces+jhopper=us.ibm.com@lists.ozlabs.org</a>>> wrote: -----<br>> To: Michael Ellerman <_mpe@ellerman.id.au_ <<a href="mailto:mpe@ellerman.id.au" target="_blank" >mailto:mpe@ellerman.id.au</a>>>,<br>_linuxppc-users@lists.ozlabs.org_ <<a href="mailto:linuxppc-users@lists.ozlabs.org" target="_blank" >mailto:linuxppc-users@lists.ozlabs.org</a>><br>> From: Brian Horton<br>> Sent by: "Linuxppc-users"<br>> Date: 09/27/2017 10:12AM<br>> Subject: Re: [Linuxppc-users] memmap kernel option on ubuntu 16.04 power8<br>><br>><br>> On 09/27/2017 05:45 AM, Michael Ellerman wrote:<br>>> Hi Brian,<br>>><br>>> Brian Horton <_brianh@linux.vnet.ibm.com_ <<a href="mailto:brianh@linux.vnet.ibm.com" target="_blank" >mailto:brianh@linux.vnet.ibm.com</a>>> writes:<br>>>> Greetings all.<br>>>><br>>>> I'm trying to run some tests with less memory than what's installed.<br>the<br>>>> 'mem=128G' option works, but it ends up putting all of the memory on<br>the<br>>>> first numa node:<br>>><br>>> It doesn't put memory anywhere, it just clamps the total memory, and it<br>>> happens that the first (lowest address) 128G of RAM is all on node 0.<br>><br>> right, 'put' was a bad word choice. but yes, i see that the affect is<br>> that the first 128G of RAM is used.<br>><br>><br>>><br>>>> someone said that it worked on RHEL, so maybe it's an Ubuntu-specific<br>bug..<br>>><br>>> OK that would be interesting, but I suspect it's not true :)<br>><br>> correct, on 2nd look it was mis-information. they confirmed that it<br>> doesn't work on RHEL either.<br>><br>>>> thoughts? ideas?<br>>><br>>> AFAIK there's no way to achieve what you're after, without changing<br>>> kernel code or creating a custom device tree.<br>>><br>>> The mem= option is very primitive, it just clamps the total as I said<br>>> above, it's not NUMA aware.<br>>><br>>> What hardware/platform are you trying to do this on? Are you familiar<br>>> with kexec?<br>><br>> P8, Tuleta and Firestone, baremetal, ppc64le w/ Ubuntu 16.04 HWE kernel.<br>><br>> No, I haven't looked into anything with kexec - what are my options with<br>> that? I would prefer to stay away from building my own kernel code -<br>> i'll probably just live with the testing that i'm doing if that's my<br>> only option. i was just trying to be a little more 'realistic'.<br>><br>> Thanks! .bri.<br>><br>><br>>> cheers<br>>><br>><br>> _______________________________________________<br>> Linuxppc-users mailing list_<br>> __Linuxppc-users@lists.ozlabs.org_ <<a href="mailto:Linuxppc-users@lists.ozlabs.org" target="_blank" >mailto:Linuxppc-users@lists.ozlabs.org</a>>_<br>><br>__https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=59cao60PiAeGQQaaFLkjnWW7CHr6qoRfN8NBRYVc9UQ&m=R9rn3U537CNoKLsBMZtzHp6VJc2mlnNdfmtsygTu4A0&s=o-r0sCpTTOrApxT7FVtnYCyWMpr9QjeO3pt0SW1yJm8&e=_<br><br>><br>> _______________________________________________<br>> Linuxppc-users mailing list<br>> Linuxppc-users@lists.ozlabs.org<br>><br>https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=XEmD5LeSMcW0pq1ccjREOfGKKSZEo6Uip7M_x_R-NUY&m=WNMxAOec1e_w34zBe3xVYIoJbJamikId375eNQw97IA&s=DGTWVG8bGGNjZ-cfXVvDyGfk8R4CJxIGw84m3Wi56Tk&e= <br><br>><br>><br>><br>><br>><br>> _______________________________________________<br>> Linuxppc-users mailing list<br>> Linuxppc-users@lists.ozlabs.org<br>><br><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=XHJsZhhuWSw9713Fp0ciew&m=TmviNY9vCNXvoo8fJe9fezSUxE-oPeWagE1fD19ZpLU&s=RT48AqWNNoMza32hJQTGnEeLGicTutGj2yrpsbdoYmk&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=XHJsZhhuWSw9713Fp0ciew&m=TmviNY9vCNXvoo8fJe9fezSUxE-oPeWagE1fD19ZpLU&s=RT48AqWNNoMza32hJQTGnEeLGicTutGj2yrpsbdoYmk&e=</a><br><br>><br><br><br><br>-------------- next part --------------<br>An HTML attachment was scrubbed...<br>URL: <<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ozlabs.org_pipermail_linuxppc-2Dusers_attachments_20170927_26c7a322_attachment.html&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=hcvYylDss1VVWOjRbBmXAUy1FlSE_Zgq_ifMr1uYhpY&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ozlabs.org_pipermail_linuxppc-2Dusers_attachments_20170927_26c7a322_attachment.html&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=hcvYylDss1VVWOjRbBmXAUy1FlSE_Zgq_ifMr1uYhpY&e=</a> ><br>-------------- next part --------------<br>A non-text attachment was scrubbed...<br>Name: graycol.gif<br>Type: image/gif<br>Size: 105 bytes<br>Desc: not available<br>URL: <<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ozlabs.org_pipermail_linuxppc-2Dusers_attachments_20170927_26c7a322_attachment.gif&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=QnkpkA8YQttwJz_UcaXv-1dhnlDWtitlVNxjofGuMcs&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ozlabs.org_pipermail_linuxppc-2Dusers_attachments_20170927_26c7a322_attachment.gif&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=QnkpkA8YQttwJz_UcaXv-1dhnlDWtitlVNxjofGuMcs&e=</a> ><br><br>------------------------------<br><br>Subject: Digest Footer<br><br>_______________________________________________<br>Linuxppc-users mailing list<br>Linuxppc-users@lists.ozlabs.org<br><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=vfSI789crSkT8xJzRaoouC0-Vu3Dd8aaKJVhxDEXY3o&e=" target="_blank" >https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.ozlabs.org_listinfo_linuxppc-2Dusers&d=DwIGaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=k4HOp_6V5T3GUyW-vS9wFA&m=SMRku5C58cv8vRfs6o0CQXjGa3uNF7MkgqdcVkBJgN0&s=vfSI789crSkT8xJzRaoouC0-Vu3Dd8aaKJVhxDEXY3o&e=</a> <br><br><br>------------------------------<br><br>End of Linuxppc-users Digest, Vol 4, Issue 13<br>*********************************************</font><br> </div></blockquote>
<div dir="ltr" > </div></div><BR>