[PATCH v4 2/3] powerpc/mm/hugetlb: Add support for reserving gigantic huge pages via kernel command line
Christophe LEROY
christophe.leroy at c-s.fr
Wed Aug 2 18:52:44 AEST 2017
Le 02/08/2017 à 10:10, Christophe LEROY a écrit :
>
>
> Le 02/08/2017 à 09:31, Aneesh Kumar K.V a écrit :
>> Christophe LEROY <christophe.leroy at c-s.fr> writes:
>>
>>> Hi,
>>>
>>> Le 28/07/2017 à 07:01, Aneesh Kumar K.V a écrit :
>>>> With commit aa888a74977a8 ("hugetlb: support larger than MAX_ORDER")
>>>> we added
>>>> support for allocating gigantic hugepages via kernel command line.
>>>> Switch
>>>> ppc64 arch specific code to use that.
>>>>
>>>> W.r.t FSL support, we now limit our allocation range using
>>>> BOOTMEM_ALLOC_ACCESSIBLE.
>>>>
>>>> We use the kernel command line to do reservation of hugetlb pages on
>>>> powernv
>>>> platforms. On pseries hash mmu mode the supported gigantic huge page
>>>> size is
>>>> 16GB and that can only be allocated with hypervisor assist. For
>>>> pseries the
>>>> command line option doesn't do the allocation. Instead pseries does
>>>> gigantic
>>>> hugepage allocation based on hypervisor hint that is specified via
>>>> "ibm,expected#pages" property of the memory node.
>>>
>>> It looks like it doesn't work on the 8xx:
>>>
>>> root at vgoip:~# dmesg | grep -i huge
>>> [ 0.000000] Kernel command line: console=ttyCPM0,115200N8
>>> ip=172.25.231.25:172.25.231.1::255.0.0.0:vgoip:eth0:off hugepagesz=8M
>>> hugepages=4
>>> [ 0.416722] HugeTLB registered 8.00 MiB page size, pre-allocated 4
>>> pages
>>> [ 0.423184] HugeTLB registered 512 KiB page size, pre-allocated 0
>>> pages
>>> root at vgoip:~# cat /proc/meminfo
>>> MemTotal: 123388 kB
>>> MemFree: 77900 kB
>>> MemAvailable: 78412 kB
>>> Buffers: 0 kB
>>> Cached: 3964 kB
>>> SwapCached: 0 kB
>>> Active: 3788 kB
>>> Inactive: 1680 kB
>>> Active(anon): 1636 kB
>>> Inactive(anon): 20 kB
>>> Active(file): 2152 kB
>>> Inactive(file): 1660 kB
>>> Unevictable: 0 kB
>>> Mlocked: 0 kB
>>> SwapTotal: 0 kB
>>> SwapFree: 0 kB
>>> Dirty: 0 kB
>>> Writeback: 0 kB
>>> AnonPages: 1552 kB
>>> Mapped: 2404 kB
>>> Shmem: 152 kB
>>> Slab: 0 kB
>>> SReclaimable: 0 kB
>>> SUnreclaim: 0 kB
>>> KernelStack: 304 kB
>>> PageTables: 208 kB
>>> NFS_Unstable: 0 kB
>>> Bounce: 0 kB
>>> WritebackTmp: 0 kB
>>> CommitLimit: 45308 kB
>>> Committed_AS: 16664 kB
>>> VmallocTotal: 866304 kB
>>> VmallocUsed: 0 kB
>>> VmallocChunk: 0 kB
>>> HugePages_Total: 0
>>> HugePages_Free: 0
>>> HugePages_Rsvd: 0
>>> HugePages_Surp: 0
>>> Hugepagesize: 512 kB
>>
>> But you are printing above the default hugepaeg details. You haven't
>> changed that in kernel command line. What does
>> /sys/kernel/mm/hugepages/<hugepages-size>/nr_hugepages show ?
>
> It says 4, so indeed it seems to work.
>
>>
>> To change the default hugepage size you may want to use
>> default_hugepagesz=8M
>>
>
> Ah ? Documentation/admin-guide/kernel-parameters.txt says it is the same
> as hugepagesz:
>
> default_hugepagesz=
> [same as hugepagesz=] The size of the default
> HugeTLB page size. This is the size represented by
> the legacy /proc/ hugepages APIs, used for SHM, and
> default size when mounting hugetlbfs filesystems.
> Defaults to the default architecture's huge page size
> if not specified.
>
>
> You are right, with default_hugepagesz instead of hugepagesz I get the
> following in 16k page sizes mode (ie 8M is < MAX_ORDER)
>
> root at vgoip:~# dmesg | grep -i huge
> [ 0.000000] Kernel command line: console=ttyCPM0,115200N8
> ip=172.25.231.25:172.25.231.1::255.0.0.0:vgoip:eth0:off
> default_hugepagesz=8M hugepages=4
> [ 0.410451] HugeTLB registered 512 KiB page size, pre-allocated 0 pages
> [ 0.417091] HugeTLB registered 8.00 MiB page size, pre-allocated 4 pages
> root at vgoip:~# cat /proc/meminfo
> MemTotal: 123968 kB
> MemFree: 73248 kB
> MemAvailable: 75120 kB
> Buffers: 0 kB
> Cached: 5936 kB
> SwapCached: 0 kB
> Active: 6912 kB
> Inactive: 2496 kB
> Active(anon): 3664 kB
> Inactive(anon): 80 kB
> Active(file): 3248 kB
> Inactive(file): 2416 kB
> Unevictable: 0 kB
> Mlocked: 0 kB
> SwapTotal: 0 kB
> SwapFree: 0 kB
> Dirty: 0 kB
> Writeback: 0 kB
> AnonPages: 3536 kB
> Mapped: 3216 kB
> Shmem: 272 kB
> Slab: 0 kB
> SReclaimable: 0 kB
> SUnreclaim: 0 kB
> KernelStack: 304 kB
> PageTables: 672 kB
> NFS_Unstable: 0 kB
> Bounce: 0 kB
> WritebackTmp: 0 kB
> CommitLimit: 45600 kB
> Committed_AS: 18528 kB
> VmallocTotal: 866304 kB
> VmallocUsed: 0 kB
> VmallocChunk: 0 kB
> HugePages_Total: 4
> HugePages_Free: 4
> HugePages_Rsvd: 0
> HugePages_Surp: 0
> Hugepagesize: 8192 kB
>
> However, in 4K pages mode, ie (8M > MAX_ORDER), I get
>
> root at vgoip:~# dmesg | grep -i huge
> [ 0.000000] Kernel command line: console=ttyCPM0,115200N8
> ip=172.25.231.25:172.25.231.1::255.0.0.0:vgoip:eth0:off
> default_hugepagesz=8M hugepages=4
> [ 0.413274] HugeTLB registered 512 KiB page size, pre-allocated 0 pages
> [ 0.419844] HugeTLB registered 8.00 MiB page size, pre-allocated 0 pages
>
> root at vgoip:~# cat /proc/meminfo
> ...
> HugePages_Total: 0
> HugePages_Free: 0
> HugePages_Rsvd: 0
> HugePages_Surp: 0
> Hugepagesize: 8192 kB
>
> Looks like for gigantic pages, we have to specify both
> default_hugepagesz=8M and hugepagesz=8M. When we do it it works back:
>
> [ 0.000000] Kernel command line: console=ttyCPM0,115200N8
> ip=172.25.231.25:172.25.231.1::255.0.0.0:vgoip:eth0:off
> default_hugepagesz=8M hugepagesz=8M hugepages=4
> [ 0.420093] HugeTLB registered 8.00 MiB page size, pre-allocated 4 pages
> [ 0.426538] HugeTLB registered 512 KiB page size, pre-allocated 0 pages
>
Indeed it works even without your patch.
Christophe
More information about the Linuxppc-dev
mailing list