[PATCH 1/8] pseries: phyp dump: Docmentation
ntl at pobox.com
Wed Jan 9 15:29:11 EST 2008
Manish Ahuja wrote:
> + Hypervisor-Assisted Dump
> + ------------------------
> + November 2007
Date is unneeded (and, uhm, dated :)
> +The goal of hypervisor-assisted dump is to enable the dump of
> +a crashed system, and to do so from a fully-reset system, and
> +to minimize the total elapsed time until the system is back
> +in production use.
Is it actually faster than kdump?
> +As compared to kdump or other strategies, hypervisor-assisted
> +dump offers several strong, practical advantages:
> +-- Unlike kdump, the system has been reset, and loaded
> + with a fresh copy of the kernel. In particular,
> + PCI and I/O devices have been reinitialized and are
> + in a clean, consistent state.
> +-- As the dump is performed, the dumped memory becomes
> + immediately available to the system for normal use.
> +-- After the dump is completed, no further reboots are
> + required; the system will be fully usable, and running
> + in it's normal, production mode on it normal kernel.
> +The above can only be accomplished by coordination with,
> +and assistance from the hypervisor. The procedure is
> +as follows:
> +-- When a system crashes, the hypervisor will save
> + the low 256MB of RAM to a previously registered
> + save region. It will also save system state, system
> + registers, and hardware PTE's.
> +-- After the low 256MB area has been saved, the
> + hypervisor will reset PCI and other hardware state.
> + It will *not* clear RAM. It will then launch the
> + bootloader, as normal.
> +-- The freshly booted kernel will notice that there
> + is a new node (ibm,dump-kernel) in the device tree,
> + indicating that there is crash data available from
> + a previous boot. It will boot into only 256MB of RAM,
> + reserving the rest of system memory.
> +-- Userspace tools will parse /sys/kernel/release_region
> + and read /proc/vmcore to obtain the contents of memory,
> + which holds the previous crashed kernel. The userspace
> + tools may copy this info to disk, or network, nas, san,
> + iscsi, etc. as desired.
> + For Example: the values in /sys/kernel/release-region
> + would look something like this (address-range pairs).
> + CPU:0x177fee000-0x10000: HPTE:0x177ffe020-0x1000: /
> + DUMP:0x177fff020-0x10000000, 0x10000000-0x16F1D370A
> +-- As the userspace tools complete saving a portion of
> + dump, they echo an offset and size to
> + /sys/kernel/release_region to release the reserved
> + memory back to general use.
> + An example of this is:
> + "echo 0x40000000 0x10000000 > /sys/kernel/release_region"
> + which will release 256MB at the 1GB boundary.
This violates the "one file, one value" rule of sysfs, but nobody
really takes that seriously, I guess. In any case, consider
documenting this in Documentation/ABI.
> +Please note that the hypervisor-assisted dump feature
> +is only available on Power6-based systems with recent
> +firmware versions.
This statement will of course become dated/incorrect so I recommend
> +Implementation details:
> +In order for this scheme to work, memory needs to be reserved
> +quite early in the boot cycle. However, access to the device
> +tree this early in the boot cycle is difficult, and device-tree
> +access is needed to determine if there is a crash data waiting.
I don't think this bit about early device tree access is correct. By
the time your code is reserving memory (from early_init_devtree(), I
think), RTAS has been instantiated and you are able to test for the
existence of /rtas/ibm,dump-kernel.
> +To work around this problem, all but 256MB of RAM is reserved
> +during early boot. A short while later in boot, a check is made
> +to determine if there is dump data waiting. If there isn't,
> +then the reserved memory is released to general kernel use.
So I think these gymnastics are unneeded -- unless I'm
misunderstanding something, you should be able to determine very early
whether to reserve that memory.
> +If there is dump data, then the /sys/kernel/release_region
> +file is created, and the reserved memory is held.
> +If there is no waiting dump data, then all but 256MB of the
> +reserved ram will be released for general kernel use. The
> +highest 256 MB of RAM will *not* be released: this region
> +will be kept permanently reserved, so that it can act as
> +a receptacle for a copy of the low 256MB in the case a crash
> +does occur. See, however, "open issues" below, as to whether
> +such a reserved region is really needed.
> +Currently the dump will be copied from /proc/vmcore to a
> +a new file upon user intervention. The starting address
> +to be read and the range for each data point in provided
> +in /sys/kernel/release_region.
> +The tools to examine the dump will be same as the ones
> +used for kdump.
> +General notes:
> +Security: please note that there are potential security issues
> +with any sort of dump mechanism. In particular, plaintext
> +(unencrypted) data, and possibly passwords, may be present in
> +the dump data. Userspace tools must take adequate precautions to
> +preserve security.
> +Open issues/ToDo:
> + o The various code paths that tell the hypervisor that a crash
> + occurred, vs. it simply being a normal reboot, should be
> + reviewed, and possibly clarified/fixed.
> + o Instead of using /sys/kernel, should there be a /sys/dump
> + instead? There is a dump_subsys being created by the s390 code,
> + perhaps the pseries code should use a similar layout as well.
Well, it seems to me that there's little reason to duplicate the s390
layout unless we can actually share code.
FWIW, I've been thinking about making a /sys/firmware/phyp hierarchy
which could contain much of the System P-specific functions (DLPAR,
lparcfg, other crud in /proc/ppc64)... seems suited to this
platform-specific dump mechanism.
More information about the Linuxppc-dev