Kernel panic from malloc() on SUSE 15.1?
Carl Jacobsen
cjacobsen at storix.com
Wed Nov 4 09:09:44 AEDT 2020
The panic (on a call to malloc from static linked libcrypto) looks like
this:
Bad kernel stack pointer 7fffffffeac0 at 700
Oops: Bad kernel stack pointer, sig: 6 [#1]
SMP NR_CPUS=2048
NUMA
pSeries
Modules linked in: scsi_transport_iscsi af_packet xt_tcpudp ip6t_rpfilter
ip6t_REJECT ipt_REJECT xt_conntrack ip_set nfnetlink ebtable_nat
ebtable_broute br_netfilter bridge stp llc ip6table_nat nf_conntrack_ipv6
nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw ip6table_security
iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat
nf_conntrack libcrc32c iptable_mangle iptable_raw iptable_security
ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables
x_tables ibmveth(X) vmx_crypto gf128mul crct10dif_vpmsum rtc_generic btrfs
xor zstd_decompress zstd_compress xxhash raid6_pq sr_mod cdrom sd_mod
ibmvscsi(X) scsi_transport_srp crc32c_vpmsum sg dm_multipath dm_mod
scsi_dh_rdac scsi_dh_emc scsi_dh_alua scsi_mod autofs4
Supported: Yes, External
CPU: 0 PID: 14144 Comm: rand_test_no_pt Tainted: G
4.12.14-197.18-default #1 SLE15-SP1
task: c00000002fa23b80 task.stack: c000000032824000
NIP: 0000000000000700 LR: 0000000010004ad0 CTR: 0000000000000000
REGS: c00000001ec2fd40 TRAP: 0300 Tainted: G
(4.12.14-197.18-default)
MSR: 8000000000001000 <SF,ME>
CR: 44000844 XER: 20000000
CFAR: 00000000000010f0 DAR: ffffffffffffb27a DSISR: 40000000 SOFTE: 0
GPR00: 0000000020000000 00007fffffffeac0 00000000102af788 fffffffffffffffd
GPR04: 0000000000000020 0000000000000030 00000000102b0550 0000000000000001
GPR08: 0000000000000000 00007fffb7dacc00 00000000102b0520 800000010280f033
GPR12: 0000000000004000 00007fffb7ffa100 0000000000000000 0000000000000000
GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR24: 0000000000000000 0000000000000000 0000000000000000 00007fffb7fef4b8
GPR28: 00007fffb7ff0000 0000000000000000 0000000000000000 00007fffffffeac0
NIP [0000000000000700] 0x700
LR [0000000010004ad0] 0x10004ad0
Call Trace:
Instruction dump:
00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
00000000 00000000 00000000 00000000 7db243a6 7db142a6 f92d0080 7d20e2a6
---[ end trace cc04515f274cfbf6 ]---
Sending IPI to other CPUs
IPI complete
kexec: Starting switchover sequence.
I'm in purgatory
-> smp_release_cpus()
spinning_secondaries = 0
<- smp_release_cpus()
Kernel panic - not syncing: Out of memory and no killable processes...
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.12.14-197.18-default #1
SLE15-SP1
Call Trace:
[c000000012457210] [c000000008a20140] dump_stack+0xb0/0xf0 (unreliable)
[c000000012457250] [c000000008a1ccd4] panic+0x144/0x31c
[c0000000124572e0] [c0000000082efcc0] out_of_memory+0x3f0/0x700
[c000000012457380] [c0000000082f7ed4] __alloc_pages_nodemask+0x1004/0x10b0
[c000000012457570] [c00000000837f4d8] alloc_page_interleave+0x58/0x110
[c0000000124575b0] [c0000000083800bc] alloc_pages_current+0x16c/0x1d0
[c000000012457610] [c0000000082e8398] __page_cache_alloc+0xd8/0x150
[c000000012457650] [c0000000082e8574] pagecache_get_page+0x164/0x440
[c0000000124576b0] [c0000000082e8884] grab_cache_page_write_begin+0x34/0x70
[c0000000124576e0] [c00000000840ede8] simple_write_begin+0x48/0x190
[c000000012457720] [c0000000082e7c7c] generic_perform_write+0xec/0x270
[c0000000124577b0] [c0000000082ea2e0] __generic_file_write_iter+0x250/0x2a0
[c000000012457810] [c0000000082ea53c] generic_file_write_iter+0x20c/0x2e0
[c000000012457850] [c0000000083cc0e0] __vfs_write+0x120/0x1e0
[c0000000124578e0] [c0000000083cdfc8] vfs_write+0xd8/0x220
[c000000012457930] [c0000000083cfeec] SyS_write+0x6c/0x110
[c000000012457980] [c000000008d154c4] xwrite+0x54/0xb8
[c0000000124579c0] [c000000008d15574] do_copy+0x4c/0x17c
[c0000000124579f0] [c000000008d15140] write_buffer+0x64/0x90
[c000000012457a20] [c000000008d151d4] flush_buffer+0x68/0xf4
[c000000012457a70] [c000000008d62268] unxz+0x210/0x398
[c000000012457b10] [c000000008d15efc] unpack_to_rootfs+0x1f0/0x360
[c000000012457bc0] [c000000008d16108] populate_rootfs+0x9c/0x188
[c000000012457c40] [c00000000800f5d4] do_one_initcall+0x64/0x1d0
[c000000012457d00] [c000000008d14474] kernel_init_freeable+0x294/0x388
[c000000012457dc0] [c00000000801026c] kernel_init+0x2c/0x160
[c000000012457e30] [c00000000800b560] ret_from_kernel_thread+0x5c/0x7c
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at ../drivers/tty/vt/vt.c:3887
do_unblank_screen+0x1d0/0x270
Modules linked in:
Supported: Yes
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.12.14-197.18-default #1
SLE15-SP1
task: c000000012449680 task.stack: c000000012454000
NIP: c0000000086d1ac0 LR: c0000000086d1918 CTR: c0000000085fc390
REGS: c000000012456f60 TRAP: 0700 Not tainted (4.12.14-197.18-default)
MSR: 8000000002029033 <SF,VEC,EE,ME,IR,DR,RI,LE>
CR: 28242222 XER: 20000008
CFAR: c0000000086d1934 SOFTE: 0
GPR00: c0000000086d1918 c0000000124571e0 c000000009240000 0000000000000000
GPR04: 0000000000000000 c00000001237e00e 00000000000010b9 c000000012457170
GPR08: 000000000a610000 0000000000000000 c0000000090f38f0 c0000000122bc3d7
GPR12: 0000000028242428 c00000000f6c0000 00000000014200c2 00000000014200c2
GPR16: 00000000014200c2 0000000000000001 0000000000000000 0000000000000240
GPR20: 0000000000000001 0000000000000240 0000000000000000 c0000000140e1d10
GPR24: 0000000000000000 0000000000000000 0000000000000115 c000000009282374
GPR28: c000000009403508 c0000000094034d8 0000000000000000 0000000000000000
NIP [c0000000086d1ac0] do_unblank_screen+0x1d0/0x270
LR [c0000000086d1918] do_unblank_screen+0x28/0x270
Call Trace:
[c0000000124571e0] [c000000012457250] 0xc000000012457250 (unreliable)
[c000000012457250] [c000000008a1cd44] panic+0x1b4/0x31c
[c0000000124572e0] [c0000000082efcc0] out_of_memory+0x3f0/0x700
[c000000012457380] [c0000000082f7ed4] __alloc_pages_nodemask+0x1004/0x10b0
[c000000012457570] [c00000000837f4d8] alloc_page_interleave+0x58/0x110
[c0000000124575b0] [c0000000083800bc] alloc_pages_current+0x16c/0x1d0
[c000000012457610] [c0000000082e8398] __page_cache_alloc+0xd8/0x150
[c000000012457650] [c0000000082e8574] pagecache_get_page+0x164/0x440
[c0000000124576b0] [c0000000082e8884] grab_cache_page_write_begin+0x34/0x70
[c0000000124576e0] [c00000000840ede8] simple_write_begin+0x48/0x190
[c000000012457720] [c0000000082e7c7c] generic_perform_write+0xec/0x270
[c0000000124577b0] [c0000000082ea2e0] __generic_file_write_iter+0x250/0x2a0
[c000000012457810] [c0000000082ea53c] generic_file_write_iter+0x20c/0x2e0
[c000000012457850] [c0000000083cc0e0] __vfs_write+0x120/0x1e0
[c0000000124578e0] [c0000000083cdfc8] vfs_write+0xd8/0x220
[c000000012457930] [c0000000083cfeec] SyS_write+0x6c/0x110
[c000000012457980] [c000000008d154c4] xwrite+0x54/0xb8
[c0000000124579c0] [c000000008d15574] do_copy+0x4c/0x17c
[c0000000124579f0] [c000000008d15140] write_buffer+0x64/0x90
[c000000012457a20] [c000000008d151d4] flush_buffer+0x68/0xf4
[c000000012457a70] [c000000008d62268] unxz+0x210/0x398
[c000000012457b10] [c000000008d15efc] unpack_to_rootfs+0x1f0/0x360
[c000000012457bc0] [c000000008d16108] populate_rootfs+0x9c/0x188
[c000000012457c40] [c00000000800f5d4] do_one_initcall+0x64/0x1d0
[c000000012457d00] [c000000008d14474] kernel_init_freeable+0x294/0x388
[c000000012457dc0] [c00000000801026c] kernel_init+0x2c/0x160
[c000000012457e30] [c00000000800b560] ret_from_kernel_thread+0x5c/0x7c
Instruction dump:
3d22001c 39293920 81290000 2f890000 409cff00 ebe10068 38210070 e8010010
ebc1fff0 7c0803a6 4e800020 60000000 <0fe00000> 4bfffe74 60000000 60000000
---[ end trace ad1803c957b45442 ]---
---[ end Kernel panic - not syncing: Out of memory and no killable
processes...
On Mon, Nov 2, 2020 at 6:26 PM Michael Ellerman <mpe at ellerman.id.au> wrote:
> Carl Jacobsen <cjacobsen at storix.com> writes:
> > I've got a SUSE 15.1 install (on ppc64le) that kernel panics on a very
> > simple
> > test program, built in a slightly unusual way.
> >
> > I'm compiling on SUSE 12, using gcc 4.8.3. I'm linking to a static
> > copy of libcrypto.a (from openssl-1.1.1g), built without threads.
> > I have a 10 line C test program that compiles and runs fine on the
> > SUSE 12 system. If I compile the same program on SUSE 15.1 (with
> > gcc 7.4.1), it runs fine on SUSE 15.1.
> >
> > But, if I run the version that I compiled on SUSE 12, on the SUSE 15.1
> > system, the call to RAND_status() gets to a malloc() and then panics.
> > (And, of course, if I just compile a call to malloc(), that runs fine
> > on both systems.) Here's the test program, it's really just a call to
> > RAND_status():
> >
> > #include <stdio.h>
> > #include <openssl/rand.h>
> >
> > int main(int argc, char **argv)
> > {
> > int has_enough_data = RAND_status();
> > printf("The PRNG %s been seeded with enough data\n",
> > has_enough_data ? "HAS" : "has NOT");
> > return 0;
> > }
> >
> > openssl is configured/built with:
> > ./config no-shared no-dso no-threads -fPIC -ggdb3 -debug -static
> > make
> >
> > and the test program is compiled with:
> > gcc -ggdb3 -o rand_test rand_test.c libcrypto.a
> >
> > The kernel on SUSE 12 is: 3.12.28-4-default
> > And glibc is: 2.19
> >
> > The kernel on SUSE 15.1 is: 4.12.14-197.18-default
> > And glibc is: 2.26
> >
> > In a previous iteration it was panicking in pthread_once(), so
> > I compiled openssl without pthreads support, and now it panics
> > calling malloc().
>
> What's the panic look like?
>
> cheers
>
--
Carl Jacobsen
Storix, Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ozlabs.org/pipermail/linuxppc-dev/attachments/20201103/f3d23716/attachment-0001.htm>
More information about the Linuxppc-dev
mailing list