Kernel Crash Dump (kdump) broken with 6.5

[Sachin Sant]

> On 19-Jul-2023, at 11:33 PM, Mahesh J Salgaonkar <mahesh at linux.ibm.com> wrote:
> 
> On 2023-07-18 23:19:23 Tue, Michael Ellerman wrote:
>> Mahesh J Salgaonkar <mahesh at linux.ibm.com> writes:
>>> On 2023-07-17 20:15:53 Mon, Sachin Sant wrote:
>>>> Kdump seems to be broken with 6.5 for ppc64le.
>>>> 
>>>> [ 14.200412] systemd[1]: Starting dracut pre-pivot and cleanup hook...
>>>> [[0;32m OK [0m] Started dracut pre-pivot and cleanup hook.
>>>> Starting Kdump Vmcore Save Service...
>>>> [ 14.231669] systemd[1]: Started dracut pre-pivot and cleanup hook.
>>>> [ 14.231801] systemd[1]: Starting Kdump Vmcore Save Service...
>>>> [ 14.341035] kdump.sh[297]: kdump: saving to /sysroot//var/crash//127.0.0.1-2023-07-14-13:32:34/
>>>> [ 14.350053] EXT4-fs (sda2): re-mounted e971a335-1ef8-4295-ab4e-3940f28e53fc r/w. Quota mode: none.
>>>> [ 14.345979] kdump.sh[297]: kdump: saving vmcore-dmesg.txt to /sysroot//var/crash//127.0.0.1-2023-07-14-13:32:34/
>>>> [ 14.348742] kdump.sh[331]: Cannot open /proc/vmcore: No such file or directory
>>>> [ 14.348845] kdump.sh[297]: kdump: saving vmcore-dmesg.txt failed
>>>> [ 14.349014] kdump.sh[297]: kdump: saving vmcore
>>>> [ 14.443422] kdump.sh[332]: open_dump_memory: Can't open the dump memory(/proc/vmcore). No such file or directory
>>>> [ 14.456413] kdump.sh[332]: makedumpfile Failed.
>>>> [ 14.456662] kdump.sh[297]: kdump: saving vmcore failed, _exitcode:1
>>>> [ 14.456822] kdump.sh[297]: kdump: saving the /run/initramfs/kexec-dmesg.log to /sysroot//var/crash//127.0.0.1-2023-07-14-13:32:34/
>>>> [ 14.487002] kdump.sh[297]: kdump: saving vmcore failed
>>>> [[0;1;31mFAILED[0m] Failed to start Kdump Vmcore Save Service.
>>> 
>>> Thanks Sachin for catching this.
>>> 
>>>> 
>>>> 6.4 was good. Git bisect points to following patch
>>>> 
>>>> commit 606787fed7268feb256957872586370b56af697a
>>>>    powerpc/64s: Remove support for ELFv1 little endian userspace
>>>> 
>>>> Reverting this patch allows a successful capture of vmcore.
>>>> 
>>>> Does this change require any corresponding change to kdump
>>>> and/or kexec tools?
>>> 
>>> Need to investigate that. It looks like vmcore_elf64_check_arch(&ehdr)
>>> check from fs/proc/vmcore.c is failing after above commit.
>>> 
>>> static int __init parse_crash_elf64_headers(void)
>>> {
>>> [...]
>>> 
>>>        /* Do some basic Verification. */
>>>        if (memcmp(ehdr.e_ident, ELFMAG, SELFMAG) != 0 ||
>>>                (ehdr.e_type != ET_CORE) ||
>>>                !vmcore_elf64_check_arch(&ehdr) ||
>>> [...]
>> 
>> Where vmcore_elf64_check_arch() calls elf_check_arch(), which was
>> modified by the commit, so that makes sense.
>> 
>>> It looks like ehdr->e_flags are not set properly while generating vmcore
>>> ELF header. I see that in kexec_file_load, ehdr->e_flags left set to 0
>>> irrespective of IS_ENABLED(CONFIG_PPC64_ELF_ABI_V2) is true or false.
>> 
>> Does initialising it in crash_prepare_elf64_headers() fix the issue?
> 
> Yes, the bellow change fixes the issue. Can't use
> IS_ENABLED(CONFIG_PPC64_ELF_ABI_V2)i check in common code. I see that
> fs/proc/kcore.c uses ELF_CORE_EFLAGS to set e_flags. Will send out
> formal patch.
> 
> From 2d12fe7dff5dfe9035a75b1fe8d7da7da3000b90 Mon Sep 17 00:00:00 2001
> From: Mahesh Salgaonkar <mahesh at linux.ibm.com>
> Date: Wed, 19 Jul 2023 20:36:37 +0530
> Subject: [PATCH] kdump fix
> 
> Signed-off-by: Mahesh Salgaonkar <mahesh at linux.ibm.com>
> ---

Thanks Mahesh for the fix. With this patch applied I am able to capture kdump.

Tested-by: Sachin Sant <sachinp at linux.ibm.com>

I also see that Andrew has posted a patch [0] to revert commit 606787fed726

If the said commit is reverted, do we still need this change?

- Sachin

[0] https://lists.ozlabs.org/pipermail/linuxppc-dev/2023-July/260928.html