vmalloc_node_range for size 4198400 failed: Address range restricted to 0xf1000000 - 0xf5110000 (kernel 6.14-rc4, ppc32)
Erhard Furtner
erhard_f at mailbox.org
Fri Feb 28 01:50:56 AEDT 2025
On Thu, 27 Feb 2025 10:42:44 +0530
Ritesh Harjani (IBM) <ritesh.list at gmail.com> wrote:
> Since only the swapon failed, I think you might still have the console
> up right? So this is mostly a vmalloc allocation failure report?
Correct. The machine stays usable via VNC in my case, even after the vmalloc allocation failure.
> Though above are mainly the physical mem info printed, but vmalloc can
> also fail sometimes (e.g. this report), it is nice if we can print how
> much of vmalloc space is free out of vmalloc total in show_mem() here.
>
> Maybe linux-mm can tell if we should add this diff change for future?
>
> diff --git a/mm/show_mem.c b/mm/show_mem.c
> index 43afb56abbd3..b3af59fced02 100644
> --- a/mm/show_mem.c
> +++ b/mm/show_mem.c
> @@ -14,6 +14,7 @@
> #include <linux/mmzone.h>
> #include <linux/swap.h>
> #include <linux/vmstat.h>
> +#include <linux/vmalloc.h>
>
> #include "internal.h"
> #include "swap.h"
> @@ -416,6 +417,8 @@ void __show_mem(unsigned int filter, nodemask_t *nodemask, int max_zone_idx)
> printk("%lu pages RAM\n", total);
> printk("%lu pages HighMem/MovableOnly\n", highmem);
> printk("%lu pages reserved\n", reserved);
> + printk("%lu pages Vmalloc Total\n", (unsigned long)VMALLOC_TOTAL >> PAGE_SHIFT);
> + printk("%lu pages Vmalloc Used\n", vmalloc_nr_pages());
> #ifdef CONFIG_CMA
> printk("%lu pages cma reserved\n", totalcma_pages);
> #endif
I applied your patch and now get this output:
[...]
vmalloc_node_range for size 1196032 failed: Address range restricted to 0xf1000000 - 0xf5110000
modprobe: vmalloc error: size 1189956, vm_struct allocation failed, mode:0xcc0(GFP_KERNEL), nodemask=(null),cpuset=/,mems_allowed=0
CPU: 1 UID: 0 PID: 1853 Comm: modprobe Tainted: G W 6.14.0-rc4-PMacG4-dirty #9
Tainted: [W]=WARN
Hardware name: PowerMac3,6 7455 0x80010303 PowerMac
Call Trace:
[f427b850] [c14cfe28] dump_stack_lvl+0x70/0x8c (unreliable)
[f427b870] [c04fba74] warn_alloc+0x154/0x2b8
[f427b930] [c04dea08] __vmalloc_node_range_noprof+0x154/0x958
[f427ba00] [c04df2f8] __vmalloc_node_noprof+0xec/0xf4
[f427ba40] [c062da4c] kernel_read_file+0x514/0x7a0
[f427baf0] [c01ee5d4] init_module_from_file+0xb0/0x108
[f427bbd0] [c01eed04] sys_finit_module+0x390/0x7a4
[f427bca0] [c001e574] system_call_exception+0x2dc/0x420
[f427bf30] [c00291ac] ret_from_syscall+0x0/0x2c
--- interrupt: c00 at 0x321334
NIP: 00321334 LR: 005c9340 CTR: 002ad944
REGS: f427bf40 TRAP: 0c00 Tainted: G W (6.14.0-rc4-PMacG4-dirty)
MSR: 0000d032 <EE,PR,ME,IR,DR,RI> CR: 2822242c XER: 20000000
GPR00: 00000161 afb2c9e0 a7a03540 00000001 005d6a48 00000000 afb2c9b5 0000007f
GPR08: 00000000 af3da4b8 00000000 00000008 00000003 005efe10 2422242c 00000000
GPR16: 00040000 00000000 00000000 00000000 00750610 00000000 00751a30 00000000
GPR24: 00750610 00000000 005d6a48 00751330 00040000 00000000 005f7aac 00750610
NIP [00321334] 0x321334
LR [005c9340] 0x5c9340
--- interrupt: c00
Mem-Info:
active_anon:16715 inactive_anon:0 isolated_anon:0
active_file:25985 inactive_file:12645 isolated_file:0
unevictable:0 dirty:23 writeback:0
slab_reclaimable:2378 slab_unreclaimable:22874
mapped:14774 shmem:155 pagetables:355
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:388018 free_pcp:426 free_cma:0
Node 0 active_anon:66860kB inactive_anon:0kB active_file:103940kB inactive_file:50580kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:59096kB dirty:92kB writeback:0kB shmem:620kB writeback_tmp:0kB kernel_stack:3008kB pagetables:1420kB sec_pagetables:0kB all_unreclaimable? no
DMA free:566884kB boost:0kB min:3380kB low:4224kB high:5068kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:4kB inactive_file:28264kB unevictable:0kB writepending:80kB present:786432kB managed:716496kB mlocked:0kB bounce:0kB free_pcp:1704kB local_pcp:1304kB free_cma:0kB
lowmem_reserve[]: 0 0 1184 0
DMA: 43*4kB (U) 29*8kB (UM) 21*16kB (UE) 41*32kB (UME) 9*64kB (UME) 6*128kB (UE) 3*256kB (UE) 3*512kB (UM) 6*1024kB (UME) 5*2048kB (UM) 133*4096kB (M) = 566852kB
38772 total pagecache pages
0 pages in swap cache
Free swap = 8388604kB
Total swap = 8388604kB
524288 pages RAM
327680 pages HighMem/MovableOnly
42060 pages reserved
16656 pages Vmalloc Total
6771 pages Vmalloc Used
Memory allocations:
124 MiB 31813 mm/readahead.c:187 func:ractl_alloc_folio
88.6 MiB 7937 mm/slub.c:2423 func:alloc_slab_page
50.3 MiB 12883 ./include/linux/highmem.h:229 func:vma_alloc_zeroed_movable_folio
26.5 MiB 6781 mm/filemap.c:1970 func:__filemap_get_folio
14.5 MiB 3714 mm/memory.c:1063 func:folio_prealloc
12.3 MiB 3280 mm/execmem.c:44 func:execmem_vmalloc
9.22 MiB 2360 mm/kasan/shadow.c:304 func:kasan_populate_vmalloc_pte
5.44 MiB 348 lib/stackdepot.c:627 func:stack_depot_save_flags
4.40 MiB 531 mm/slub.c:2425 func:alloc_slab_page
4.01 MiB 1025 mm/swap_cgroup.c:140 func:swap_cgroup_swapon
vmalloc_node_range for size 1196032 failed: Address range restricted to 0xf1000000 - 0xf5110000
vmalloc_node_range for size 1196032 failed: Address range restricted to 0xf1000000 - 0xf5110000
[...]
Looks like there would be still Vmalloc pages free though? 6771 used vs. 16656 total.
Also it seems more clear I get this vmalloc error at the module loading stage during bootup. For my original report the rather big btrfs module got pulled in at boot so I got the vmalloc failure instantly.
Now the machine boots up ok, but as soon as some module gets loaded, e.g. nfs via mounting a remote directory I get the failure. This is the case for both INLINE_KASAN and OUTLINE_KASAN. Maybe for my original report the btrfs module was not loaded for my OUTLINE_KASAN boot so I got no hit early. Doesn't seem to be 100% deterministic what modules get loaded in what order.
> But meanwhile below data can give more details about the vmalloc area.
>
> 1. cat /proc/vmallocinfo
> 2. cat /proc/meminfo
Did that, with "no_hash_pointers" used and for both INLINE and OUTLINE KASAN as suggested by Christophe. Both after I get the vmalloc failure after mounting a remote NFS directory. See attached files.
For completeness also the OUTLINE_KASAN vmalloc failure:
[...]
vmalloc_node_range for size 839680 failed: Address range restricted to 0xf1000000 - 0xf5110000
modprobe: vmalloc error: size 832096, vm_struct allocation failed, mode:0xcc0(GFP_KERNEL), nodemask=(null),cpuset=/,mems_allowed=0
CPU: 0 UID: 0 PID: 1857 Comm: modprobe Tainted: G W 6.14.0-rc4-PMacG4-dirty #10
Tainted: [W]=WARN
Hardware name: PowerMac3,6 7455 0x80010303 PowerMac
Call Trace:
[f34db980] [c09e7f34] dump_stack_lvl+0x70/0x8c (unreliable)
[f34db9a0] [c02887a4] warn_alloc+0x154/0x208
[f34dba60] [c027a308] __vmalloc_node_range_noprof+0x144/0x530
[f34dbb10] [c027a770] __vmalloc_node_noprof+0x7c/0x84
[f34dbb50] [c032a0d4] kernel_read_file+0x314/0x440
[f34dbbf0] [c0102918] init_module_from_file+0xb0/0x108
[f34dbcd0] [c0102d7c] sys_finit_module+0x1d0/0x37c
[f34dbda0] [c00138b8] system_call_exception+0x154/0x1b8
[f34dbf30] [c001a1ac] ret_from_syscall+0x0/0x2c
--- interrupt: c00 at 0x491334
NIP: 00491334 LR: 00739340 CTR: 0041d944
REGS: f34dbf40 TRAP: 0c00 Tainted: G W (6.14.0-rc4-PMacG4-dirty)
MSR: 0000d032 <EE,PR,ME,IR,DR,RI> CR: 2822242c XER: 20000000
GPR00: 00000161 af942800 a7fae540 00000001 00746a48 00000000 af9427d5 0000007f
GPR08: 00000000 aea582d8 00000000 00000008 00000003 0075fe10 2422242c 00000000
GPR16: 00040000 00000000 00000000 00000000 00ee8610 00000000 00ee9a30 00000000
GPR24: 00ee8610 00000000 00746a48 00ee9330 00040000 00000000 00767aac 00ee8610
NIP [00491334] 0x491334
LR [00739340] 0x739340
--- interrupt: c00
Mem-Info:
active_anon:16937 inactive_anon:0 isolated_anon:0
active_file:25969 inactive_file:11637 isolated_file:0
unevictable:0 dirty:21 writeback:0
slab_reclaimable:2396 slab_unreclaimable:23326
mapped:14784 shmem:155 pagetables:353
sec_pagetables:0 bounce:0
kernel_misc_reclaimable:0
free:392772 free_pcp:777 free_cma:0
Node 0 active_anon:67748kB inactive_anon:0kB active_file:103876kB inactive_file:46548kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:59136kB dirty:84kB writeback:0kB shmem:620kB writeback_tmp:0kB kernel_stack:3216kB pagetables:1412kB sec_pagetables:0kB all_unreclaimable? no
DMA free:576744kB boost:0kB min:3412kB low:4264kB high:5116kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:4kB inactive_file:28408kB unevictable:0kB writepending:80kB present:786432kB managed:729040kB mlocked:0kB bounce:0kB free_pcp:3108kB local_pcp:1788kB free_cma:0kB
lowmem_reserve[]: 0 0 1184 0
DMA: 0*4kB 2*8kB (UM) 5*16kB (UM) 36*32kB (UM) 15*64kB (UME) 2*128kB (ME) 3*256kB (UM) 2*512kB (UM) 3*1024kB (ME) 4*2048kB (UM) 137*4096kB (M) = 576672kB
37789 total pagecache pages
0 pages in swap cache
Free swap = 8388604kB
Total swap = 8388604kB
524288 pages RAM
327680 pages HighMem/MovableOnly
38924 pages reserved
16656 pages Vmalloc Total
5814 pages Vmalloc Used
Memory allocations:
120 MiB 30794 mm/readahead.c:187 func:ractl_alloc_folio
90.4 MiB 8145 mm/slub.c:2423 func:alloc_slab_page
50.6 MiB 12941 ./include/linux/highmem.h:229 func:vma_alloc_zeroed_movable_folio
26.6 MiB 6819 mm/filemap.c:1970 func:__filemap_get_folio
15.0 MiB 3837 mm/memory.c:1063 func:folio_prealloc
8.71 MiB 2230 mm/kasan/shadow.c:304 func:kasan_populate_vmalloc_pte
8.34 MiB 2276 mm/execmem.c:44 func:execmem_vmalloc
5.14 MiB 329 lib/stackdepot.c:627 func:stack_depot_save_flags
4.42 MiB 531 mm/slub.c:2425 func:alloc_slab_page
4.01 MiB 1025 mm/swap_cgroup.c:140 func:swap_cgroup_swapon
vmalloc_node_range for size 839680 failed: Address range restricted to 0xf1000000 - 0xf5110000
vmalloc_node_range for size 839680 failed: Address range restricted to 0xf1000000 - 0xf5110000
[...]
Regards,
Erhard
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: vmallocinfo-outline_kasan.txt
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20250227/8999f4bf/attachment.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: vmallocinfo-inline_kasan.txt
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20250227/8999f4bf/attachment-0001.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: meminfo-outline_kasan.txt
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20250227/8999f4bf/attachment-0002.txt>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: meminfo-inline_kasan.txt
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20250227/8999f4bf/attachment-0003.txt>
More information about the Linuxppc-dev
mailing list