[PATCH v2 1/2] fadump: reduce memory consumption for capture kernel

[Hari Bathini]

On Friday 07 April 2017 12:54 PM, Hari Bathini wrote:
> Hi Michael,
>
>
> On Friday 07 April 2017 07:24 AM, Michael Ellerman wrote:
>> Hari Bathini <hbathini at linux.vnet.ibm.com> writes:
>>
>>> In case of fadump, capture (fadump) kernel boots like a normal kernel.
>>> While this has its advantages, the capture kernel would initialize all
>>> the components like normal kernel, which may not necessarily be needed
>>> for a typical dump capture kernel. So, fadump capture kernel ends up
>>> needing more memory than a typical (read kdump) capture kernel to boot.
>>>
>>> This can be overcome by introducing parameters like fadump_nr_cpus=1,
>>> similar to nr_cpus=1 parameter, applicable only when fadump is active.
>>> But this approach needs introduction of special parameters applicable
>>> only when fadump is active (capture kernel), for every parameter that
>>> reduces memory/resource consumption.
>>>
>>> A better approach would be to pass extra parameters to fadump capture
>>> kernel. As firmware leaves the memory contents intact from the time of
>>> crash till the new kernel is booted up, parameters to append to capture
>>> kernel can be saved in real memory region and retrieved later when the
>>> capture kernel is in its early boot process for appending to command
>>> line parameters.
>>>
>>> This patch introduces a new node /sys/kernel/fadump_cmdline_append to
>>> specify the parameters to pass to fadump capture kernel, saves them in
>>> real memory region and appends these parameters to capture kernel early
>>> in its boot process.
>> As we discussed on IRC I don't really like this.
>>
>> It's clever, (ab)using the fact that the first kernel's memory is left
>> intact. But it's also a bit gross :)
>
> No doubt. It is an ugly trick :)
>
>> It also has a few real problems, like hard coding 128MB as the handover
>> location. You may not have memory there, or it may be reserved.
>>
>
> Yeah, there is a chance that appending parameters is not possible
> like in the scenarios you mentioned above. My intention behind this
> hack is to build on this handover area later to probably pass off a
> special intird which brings down the dump capture time and memory
> consumption further. But to put it in your words, it would be abusing
> it even more :P . So, I would take it as a road not worthing taking..
>
>> My preference would be that the fadump kernel "just works". If it's
>> using too much memory then the fadump kernel should do whatever it needs
>> to use less memory, eg. shrinking nr_cpu_ids etc.
>
>> Do we actually know *why* the fadump kernel is running out of memory?
>> Obviously large numbers of CPUs is one of the main drivers (lots of
>> stacks required). But other than that what is causing the memory
>> pressure? I would like some data on that before we proceed.
>
> Almost the same amount of memory in comparison with the memory
> required to boot the production kernel but that is unwarranted for fadump
> (dump capture) kernel. Let's say the production kernel is configured for
> memory cgroups or hugepages which is not required in a dump capture 
> kernel
> but with no option to say so, we are wasting that much more memory on 
> fadump
> and eventually depriving the production kernel of that memory.
>
> So, if parameters like cgroup_disable=memory, 
> transparent_hugepages=never,
> numa=off, nr_cpus=1, etc.. are passed to fadump (dump capture) kernel 
> it would
> be beneficial. Not to mention any future additions to the kernel that 
> increase the
> footprint of a production kernel..
>
>> If we *must* have a way to pass command line arguments to the fadump
>> kernel then I think we should just use a command line argument that
>> specifies them.
>>
>> eg: 
>> fadump_append=nr_cpus=1,use_less_memory,some_other_obscure_parameter=100
>>
>>
>
> Hmmm.. this sounds like a better interface. But I would like know your 
> preference on
> how to process fadump_append parameter:
>
> 1. Modify cmdline early in fadump kernel boot process (before parsing 
> parameters) to change
>    fadump_append="nr_cpus=1 cgroup_disable=memory" in cmdline to 
> "nr_cpus=1 cgroup_disable=memory"
>    so that fadump doesn't have to bother about processing this 
> parameters later.
> 2. A parse function in fadump to parse fadump_append parameters. A 
> function similar to parse_early_param()
>    meant for fadump_append parameter alone..
> 3. fadump code processes fadump_append for each parameter passed in it.
>
> The third one sounds like a nightmare to me as we need to make fadump 
> code aware of every new parameter
> we want to enforce on fadump..
>

I prefer option 2 for it is simple and cleaner..

Thanks
Hari