<div dir="ltr">The panic (on a call to malloc from static linked libcrypto) looks like this:<div><br></div><div>Bad kernel stack pointer 7fffffffeac0 at 700<br>Oops: Bad kernel stack pointer, sig: 6 [#1]<br>SMP NR_CPUS=2048 <br>NUMA <br>pSeries<br>Modules linked in: scsi_transport_iscsi af_packet xt_tcpudp ip6t_rpfilter ip6t_REJECT ipt_REJECT xt_conntrack ip_set nfnetlink ebtable_nat ebtable_broute br_netfilter bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack libcrc32c iptable_mangle iptable_raw iptable_security ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables x_tables ibmveth(X) vmx_crypto gf128mul crct10dif_vpmsum rtc_generic btrfs xor zstd_decompress zstd_compress xxhash raid6_pq sr_mod cdrom sd_mod ibmvscsi(X) scsi_transport_srp crc32c_vpmsum sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua scsi_mod autofs4<br>Supported: Yes, External<br>CPU: 0 PID: 14144 Comm: rand_test_no_pt Tainted: G                   4.12.14-197.18-default #1 SLE15-SP1<br>task: c00000002fa23b80 task.stack: c000000032824000<br>NIP: 0000000000000700 LR: 0000000010004ad0 CTR: 0000000000000000<br>REGS: c00000001ec2fd40 TRAP: 0300   Tainted: G                    (4.12.14-197.18-default)<br>MSR: 8000000000001000 <SF,ME><br>  CR: 44000844  XER: 20000000<br>CFAR: 00000000000010f0 DAR: ffffffffffffb27a DSISR: 40000000 SOFTE: 0 <br>GPR00: 0000000020000000 00007fffffffeac0 00000000102af788 fffffffffffffffd <br>GPR04: 0000000000000020 0000000000000030 00000000102b0550 0000000000000001 <br>GPR08: 0000000000000000 00007fffb7dacc00 00000000102b0520 800000010280f033 <br>GPR12: 0000000000004000 00007fffb7ffa100 0000000000000000 0000000000000000 <br>GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 <br>GPR20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 <br>GPR24: 0000000000000000 0000000000000000 0000000000000000 00007fffb7fef4b8 <br>GPR28: 00007fffb7ff0000 0000000000000000 0000000000000000 00007fffffffeac0 <br>NIP [0000000000000700] 0x700<br>LR [0000000010004ad0] 0x10004ad0<br>Call Trace:<br>Instruction dump:<br>00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 <br>00000000 00000000 00000000 00000000 7db243a6 7db142a6 f92d0080 7d20e2a6 <br>---[ end trace cc04515f274cfbf6 ]---<br><br>Sending IPI to other CPUs<br>IPI complete<br>kexec: Starting switchover sequence.<br>I'm in purgatory<br> -> smp_release_cpus()<br>spinning_secondaries = 0<br> <- smp_release_cpus()<br>Kernel panic - not syncing: Out of memory and no killable processes...<br><br>CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.12.14-197.18-default #1 SLE15-SP1<br>Call Trace:<br>[c000000012457210] [c000000008a20140] dump_stack+0xb0/0xf0 (unreliable)<br>[c000000012457250] [c000000008a1ccd4] panic+0x144/0x31c<br>[c0000000124572e0] [c0000000082efcc0] out_of_memory+0x3f0/0x700<br>[c000000012457380] [c0000000082f7ed4] __alloc_pages_nodemask+0x1004/0x10b0<br>[c000000012457570] [c00000000837f4d8] alloc_page_interleave+0x58/0x110<br>[c0000000124575b0] [c0000000083800bc] alloc_pages_current+0x16c/0x1d0<br>[c000000012457610] [c0000000082e8398] __page_cache_alloc+0xd8/0x150<br>[c000000012457650] [c0000000082e8574] pagecache_get_page+0x164/0x440<br>[c0000000124576b0] [c0000000082e8884] grab_cache_page_write_begin+0x34/0x70<br>[c0000000124576e0] [c00000000840ede8] simple_write_begin+0x48/0x190<br>[c000000012457720] [c0000000082e7c7c] generic_perform_write+0xec/0x270<br>[c0000000124577b0] [c0000000082ea2e0] __generic_file_write_iter+0x250/0x2a0<br>[c000000012457810] [c0000000082ea53c] generic_file_write_iter+0x20c/0x2e0<br>[c000000012457850] [c0000000083cc0e0] __vfs_write+0x120/0x1e0<br>[c0000000124578e0] [c0000000083cdfc8] vfs_write+0xd8/0x220<br>[c000000012457930] [c0000000083cfeec] SyS_write+0x6c/0x110<br>[c000000012457980] [c000000008d154c4] xwrite+0x54/0xb8<br>[c0000000124579c0] [c000000008d15574] do_copy+0x4c/0x17c<br>[c0000000124579f0] [c000000008d15140] write_buffer+0x64/0x90<br>[c000000012457a20] [c000000008d151d4] flush_buffer+0x68/0xf4<br>[c000000012457a70] [c000000008d62268] unxz+0x210/0x398<br>[c000000012457b10] [c000000008d15efc] unpack_to_rootfs+0x1f0/0x360<br>[c000000012457bc0] [c000000008d16108] populate_rootfs+0x9c/0x188<br>[c000000012457c40] [c00000000800f5d4] do_one_initcall+0x64/0x1d0<br>[c000000012457d00] [c000000008d14474] kernel_init_freeable+0x294/0x388<br>[c000000012457dc0] [c00000000801026c] kernel_init+0x2c/0x160<br>[c000000012457e30] [c00000000800b560] ret_from_kernel_thread+0x5c/0x7c<br>------------[ cut here ]------------<br>WARNING: CPU: 0 PID: 1 at ../drivers/tty/vt/vt.c:3887 do_unblank_screen+0x1d0/0x270<br>Modules linked in:<br>Supported: Yes<br>CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.12.14-197.18-default #1 SLE15-SP1<br>task: c000000012449680 task.stack: c000000012454000<br>NIP: c0000000086d1ac0 LR: c0000000086d1918 CTR: c0000000085fc390<br>REGS: c000000012456f60 TRAP: 0700   Not tainted  (4.12.14-197.18-default)<br>MSR: 8000000002029033 <SF,VEC,EE,ME,IR,DR,RI,LE><br>  CR: 28242222  XER: 20000008<br>CFAR: c0000000086d1934 SOFTE: 0 <br>GPR00: c0000000086d1918 c0000000124571e0 c000000009240000 0000000000000000 <br>GPR04: 0000000000000000 c00000001237e00e 00000000000010b9 c000000012457170 <br>GPR08: 000000000a610000 0000000000000000 c0000000090f38f0 c0000000122bc3d7 <br>GPR12: 0000000028242428 c00000000f6c0000 00000000014200c2 00000000014200c2 <br>GPR16: 00000000014200c2 0000000000000001 0000000000000000 0000000000000240 <br>GPR20: 0000000000000001 0000000000000240 0000000000000000 c0000000140e1d10 <br>GPR24: 0000000000000000 0000000000000000 0000000000000115 c000000009282374 <br>GPR28: c000000009403508 c0000000094034d8 0000000000000000 0000000000000000 <br>NIP [c0000000086d1ac0] do_unblank_screen+0x1d0/0x270<br>LR [c0000000086d1918] do_unblank_screen+0x28/0x270<br>Call Trace:<br>[c0000000124571e0] [c000000012457250] 0xc000000012457250 (unreliable)<br>[c000000012457250] [c000000008a1cd44] panic+0x1b4/0x31c<br>[c0000000124572e0] [c0000000082efcc0] out_of_memory+0x3f0/0x700<br>[c000000012457380] [c0000000082f7ed4] __alloc_pages_nodemask+0x1004/0x10b0<br>[c000000012457570] [c00000000837f4d8] alloc_page_interleave+0x58/0x110<br>[c0000000124575b0] [c0000000083800bc] alloc_pages_current+0x16c/0x1d0<br>[c000000012457610] [c0000000082e8398] __page_cache_alloc+0xd8/0x150<br>[c000000012457650] [c0000000082e8574] pagecache_get_page+0x164/0x440<br>[c0000000124576b0] [c0000000082e8884] grab_cache_page_write_begin+0x34/0x70<br>[c0000000124576e0] [c00000000840ede8] simple_write_begin+0x48/0x190<br>[c000000012457720] [c0000000082e7c7c] generic_perform_write+0xec/0x270<br>[c0000000124577b0] [c0000000082ea2e0] __generic_file_write_iter+0x250/0x2a0<br>[c000000012457810] [c0000000082ea53c] generic_file_write_iter+0x20c/0x2e0<br>[c000000012457850] [c0000000083cc0e0] __vfs_write+0x120/0x1e0<br>[c0000000124578e0] [c0000000083cdfc8] vfs_write+0xd8/0x220<br>[c000000012457930] [c0000000083cfeec] SyS_write+0x6c/0x110<br>[c000000012457980] [c000000008d154c4] xwrite+0x54/0xb8<br>[c0000000124579c0] [c000000008d15574] do_copy+0x4c/0x17c<br>[c0000000124579f0] [c000000008d15140] write_buffer+0x64/0x90<br>[c000000012457a20] [c000000008d151d4] flush_buffer+0x68/0xf4<br>[c000000012457a70] [c000000008d62268] unxz+0x210/0x398<br>[c000000012457b10] [c000000008d15efc] unpack_to_rootfs+0x1f0/0x360<br>[c000000012457bc0] [c000000008d16108] populate_rootfs+0x9c/0x188<br>[c000000012457c40] [c00000000800f5d4] do_one_initcall+0x64/0x1d0<br>[c000000012457d00] [c000000008d14474] kernel_init_freeable+0x294/0x388<br>[c000000012457dc0] [c00000000801026c] kernel_init+0x2c/0x160<br>[c000000012457e30] [c00000000800b560] ret_from_kernel_thread+0x5c/0x7c<br>Instruction dump:<br>3d22001c 39293920 81290000 2f890000 409cff00 ebe10068 38210070 e8010010 <br>ebc1fff0 7c0803a6 4e800020 60000000 <0fe00000> 4bfffe74 60000000 60000000 <br>---[ end trace ad1803c957b45442 ]---<br>---[ end Kernel panic - not syncing: Out of memory and no killable processes...<br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Nov 2, 2020 at 6:26 PM Michael Ellerman <<a href="mailto:mpe@ellerman.id.au">mpe@ellerman.id.au</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Carl Jacobsen <<a href="mailto:cjacobsen@storix.com" target="_blank">cjacobsen@storix.com</a>> writes:<br>
> I've got a SUSE 15.1 install (on ppc64le) that kernel panics on a very<br>
> simple<br>
> test program, built in a slightly unusual way.<br>
><br>
> I'm compiling on SUSE 12, using gcc 4.8.3. I'm linking to a static<br>
> copy of libcrypto.a (from openssl-1.1.1g), built without threads.<br>
> I have a 10 line C test program that compiles and runs fine on the<br>
> SUSE 12 system. If I compile the same program on SUSE 15.1 (with<br>
> gcc 7.4.1), it runs fine on SUSE 15.1.<br>
><br>
> But, if I run the version that I compiled on SUSE 12, on the SUSE 15.1<br>
> system, the call to RAND_status() gets to a malloc() and then panics.<br>
> (And, of course, if I just compile a call to malloc(), that runs fine<br>
> on both systems.) Here's the test program, it's really just a call to<br>
> RAND_status():<br>
><br>
>     #include <stdio.h><br>
>     #include <openssl/rand.h><br>
><br>
>     int main(int argc, char **argv)<br>
>     {<br>
>         int has_enough_data = RAND_status();<br>
>         printf("The PRNG %s been seeded with enough data\n",<br>
>                has_enough_data ? "HAS" : "has NOT");<br>
>         return 0;<br>
>     }<br>
><br>
> openssl is configured/built with:<br>
>     ./config no-shared no-dso no-threads -fPIC -ggdb3 -debug -static<br>
>     make<br>
><br>
> and the test program is compiled with:<br>
>     gcc -ggdb3 -o rand_test rand_test.c libcrypto.a<br>
><br>
> The kernel on SUSE 12 is: 3.12.28-4-default<br>
> And glibc is: 2.19<br>
><br>
> The kernel on SUSE 15.1 is: 4.12.14-197.18-default<br>
> And glibc is: 2.26<br>
><br>
> In a previous iteration it was panicking in pthread_once(), so<br>
> I compiled openssl without pthreads support, and now it panics<br>
> calling malloc().<br>
<br>
What's the panic look like?<br>
<br>
cheers<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">Carl Jacobsen<div>Storix, Inc.</div></div></div>