[PATCH v2 1/2] fadump: reduce memory consumption for capture kernel

[Hari Bathini]

Hi Michael,


On Friday 07 April 2017 07:24 AM, Michael Ellerman wrote:
> Hari Bathini <hbathini at linux.vnet.ibm.com> writes:
>
>> In case of fadump, capture (fadump) kernel boots like a normal kernel.
>> While this has its advantages, the capture kernel would initialize all
>> the components like normal kernel, which may not necessarily be needed
>> for a typical dump capture kernel. So, fadump capture kernel ends up
>> needing more memory than a typical (read kdump) capture kernel to boot.
>>
>> This can be overcome by introducing parameters like fadump_nr_cpus=1,
>> similar to nr_cpus=1 parameter, applicable only when fadump is active.
>> But this approach needs introduction of special parameters applicable
>> only when fadump is active (capture kernel), for every parameter that
>> reduces memory/resource consumption.
>>
>> A better approach would be to pass extra parameters to fadump capture
>> kernel. As firmware leaves the memory contents intact from the time of
>> crash till the new kernel is booted up, parameters to append to capture
>> kernel can be saved in real memory region and retrieved later when the
>> capture kernel is in its early boot process for appending to command
>> line parameters.
>>
>> This patch introduces a new node /sys/kernel/fadump_cmdline_append to
>> specify the parameters to pass to fadump capture kernel, saves them in
>> real memory region and appends these parameters to capture kernel early
>> in its boot process.
> As we discussed on IRC I don't really like this.
>
> It's clever, (ab)using the fact that the first kernel's memory is left
> intact. But it's also a bit gross :)

No doubt. It is an ugly trick :)

> It also has a few real problems, like hard coding 128MB as the handover
> location. You may not have memory there, or it may be reserved.
>

Yeah, there is a chance that appending parameters is not possible
like in the scenarios you mentioned above. My intention behind this
hack is to build on this handover area later to probably pass off a
special intird which brings down the dump capture time and memory
consumption further. But to put it in your words, it would be abusing
it even more :P . So, I would take it as a road not worthing taking..

> My preference would be that the fadump kernel "just works". If it's
> using too much memory then the fadump kernel should do whatever it needs
> to use less memory, eg. shrinking nr_cpu_ids etc.

> Do we actually know *why* the fadump kernel is running out of memory?
> Obviously large numbers of CPUs is one of the main drivers (lots of
> stacks required). But other than that what is causing the memory
> pressure? I would like some data on that before we proceed.

Almost the same amount of memory in comparison with the memory
required to boot the production kernel but that is unwarranted for fadump
(dump capture) kernel. Let's say the production kernel is configured for
memory cgroups or hugepages which is not required in a dump capture kernel
but with no option to say so, we are wasting that much more memory on fadump
and eventually depriving the production kernel of that memory.

So, if parameters like cgroup_disable=memory, transparent_hugepages=never,
numa=off, nr_cpus=1, etc.. are passed to fadump (dump capture) kernel it 
would
be beneficial. Not to mention any future additions to the kernel that 
increase the
footprint of a production kernel..

> If we *must* have a way to pass command line arguments to the fadump
> kernel then I think we should just use a command line argument that
> specifies them.
>
> eg: fadump_append=nr_cpus=1,use_less_memory,some_other_obscure_parameter=100
>
>

Hmmm.. this sounds like a better interface. But I would like know your 
preference on
how to process fadump_append parameter:

1. Modify cmdline early in fadump kernel boot process (before parsing 
parameters) to change
    fadump_append="nr_cpus=1 cgroup_disable=memory" in cmdline to 
"nr_cpus=1 cgroup_disable=memory"
    so that fadump doesn't have to bother about processing this 
parameters later.
2. A parse function in fadump to parse fadump_append parameters. A 
function similar to parse_early_param()
    meant for fadump_append parameter alone..
3. fadump code processes fadump_append for each parameter passed in it.

The third one sounds like a nightmare to me as we need to make fadump 
code aware of every new parameter
we want to enforce on fadump..

Thanks
Hari