[v4.12-rc1 regression] mount ext4 fs results in kernel crash on PPC64le host

Balbir Singh bsingharora at gmail.com
Thu Jun 29 03:16:10 AEST 2017


On Wed, Jun 28, 2017 at 6:32 PM, Eryu Guan <eguan at redhat.com> wrote:
> Hi all,
>
> Li Wang and I are constantly seeing ppc64le hosts crashing due to bad
> page access. But it's not reproducing on every ppc64le host we've
> tested, but it usually happened in filesystem testings.
>
> [  207.403459] Unable to handle kernel paging request for unaligned access at address 0xc0000001c52c5e7f
> [  207.403470] Faulting instruction address: 0xc0000000004d470c
> [  207.403475] Oops: Kernel access of bad area, sig: 7 [#1]
> [  207.403477] SMP NR_CPUS=2048
> [  207.403478] NUMA
> [  207.403480] pSeries
> [  207.403483] Modules linked in: ext4 jbd2 mbcache sg pseries_rng ghash_generic gf128mul xts vmx_crypto nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables xfs libcrc32c sd_mod ibmveth ibmvscsi scsi_transport_srp
> [  207.403503] CPU: 0 PID: 2263 Comm: mount Not tainted 4.12.0-rc7 #26
> [  207.403506] task: c0000003ef2fde00 task.stack: c0000003de394000
> [  207.403509] NIP: c0000000004d470c LR: c00000000011cd24 CTR: c000000000130de0
> [  207.403512] REGS: c0000003de397450 TRAP: 0600   Not tainted  (4.12.0-rc7)
> [  207.403515] MSR: 800000010280b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE,TM[E]>
> [  207.403521]   CR: 28028844  XER: 00000001
> [  207.403525] CFAR: c00000000011cd20 DAR: c0000001c52c5e7f DSISR: 00000000 SOFTE: 0
> [  207.403525] GPR00: c00000000011cce8 c0000003de3976d0 c000000001049500 c0000003f2c6ec20
> [  207.403525] GPR04: c0000003f2c6ec20 c0000001c52c5e7f 0000000000000000 0000000000000001
> [  207.403525] GPR08: 000c5543cab19830 0000000198e19900 0000000000000008 0000000000000000
> [  207.403525] GPR12: c000000000130de0 c00000000fac0000 0000000000000000 c0000003f1328000
> [  207.403525] GPR16: 0000000000000000 c0000003de700400 0000000000000000 c0000003de700594
> [  207.403525] GPR20: 0000000000000002 0000000000000000 0000000000004000 c000000000cc5780
> [  207.403525] GPR24: 00000001c45ffc5f 0000000000000000 00000001c45ffc5f c00000000107dd00
> [  207.403525] GPR28: c0000003f2c6f434 0000000000000004 0000000000000800 c0000003f2c6ec00
> [  207.403567] NIP [c0000000004d470c] llist_add_batch+0xc/0x40
> [  207.403571] LR [c00000000011cd24] try_to_wake_up+0x4a4/0x5b0
> [  207.403573] Call Trace:
> [  207.403576] [c0000003de3976d0] [c00000000011cce8] try_to_wake_up+0x468/0x5b0 (unreliable)
> [  207.403581] [c0000003de397750] [c000000000102cc8] create_worker+0x148/0x250
> [  207.403585] [c0000003de3977f0] [c000000000105e7c] alloc_unbound_pwq+0x3bc/0x4c0
> [  207.403589] [c0000003de397850] [c0000000001064bc] apply_wqattrs_prepare+0x2ac/0x320
> [  207.403593] [c0000003de3978c0] [c00000000010656c] apply_workqueue_attrs_locked+0x3c/0xa0
> [  207.403597] [c0000003de3978f0] [c000000000106acc] apply_workqueue_attrs+0x4c/0x80
> [  207.403601] [c0000003de397930] [c00000000010866c] __alloc_workqueue_key+0x16c/0x4e0
> [  207.403615] [c0000003de3979f0] [d000000013de5ce0] ext4_fill_super+0x1c70/0x3390 [ext4]
> [  207.403620] [c0000003de397b30] [c00000000031739c] mount_bdev+0x21c/0x250
> [  207.403633] [c0000003de397bd0] [d000000013dddb80] ext4_mount+0x20/0x40 [ext4]
> [  207.403637] [c0000003de397bf0] [c000000000318944] mount_fs+0x74/0x210
> [  207.403641] [c0000003de397ca0] [c000000000340638] vfs_kern_mount+0x68/0x1d0
> [  207.403644] [c0000003de397d10] [c000000000345348] do_mount+0x278/0xef0
> [  207.403648] [c0000003de397de0] [c0000000003463e4] SyS_mount+0x94/0x100
> [  207.403652] [c0000003de397e30] [c00000000000af84] system_call+0x38/0xe0
> [  207.403655] Instruction dump:
> [  207.403658] 60420000 38600000 4e800020 60000000 60420000 7c832378 4e800020 60000000
> [  207.403663] 60000000 e9250000 f9240000 7c0004ac <7d4028a8> 7c2a4800 40c20010 7c6029ad
> [  207.403669] ---[ end trace 4fa94bf890f28f69 ]---
>
> Today I've finally found a host that could reliably trigger the crash by
> mounting an ext4 filesystem and I've done a git bisect. The first bad
> pointed to this commit:

Thanks for the excellent bug report, I am a little lost on the stack
trace, it shows a bad page access that we think is triggered by the
mmap changes? The patch changed the return type to integrate the call
into trace-cmd. Could you point me to the tests that can help
reproduce the crash. Could you also suggest how long to try the test
cases for?

Balbir Singh


More information about the Linuxppc-dev mailing list