lpar issue for ZONE_DEVICE p2pmem in 4.14-rc

[Stephen Bates]

Hi All

I am hoping someone can help shed some light on an issue I am seeing with my attempt to add p2pmem [1] to the ppc64el kernel. p2pmem is a (currently) out-of-tree patchset that allows one to add device memory with struct page backing into the Linux kernel. It leverages MEMORY_HOTPLUG and ZONE_DEVICE which were added to ppc64 in 4.14 so I thought it would be fun to try it out.

We constructed a patchset based off 4.14-rc1 [1] and build a kernel based off the pseries defconfig and ran this on upstream qemu-system-ppc64. The exact command to run QEMU was:

qemu-system-ppc64 \
    -machine pseries \
    -cpu power8 \
    -smp 1 -m 2048 \
    -kernel ~/kernel/linux-ppc64el/vmlinux \
    -append "nvme.use_cmb=24 console=hvc root=/dev/sda rootwait rw" \
    -serial mon:stdio -drive file=image-ppc64el.img,if=scsi,format=raw,index=0 \
    -nographic \
    -drive file=../images/nvme.qcow2,if=none,id=nvme1,snapshot=on \
    -device nvme,drive=nvme1,serial=nvme1 \
    -drive file=../images/nvme2.qcow2,if=none,id=nvme2,snapshot=on \
    -device nvme,drive=nvme2,serial=nvme2,cmb_size_mb=64

This resulted in the following extract from dmesg when registering the p2pmem associated with one of the NVMe SSDs.

[    3.508497] nvme 0000:00:03.0: enabling device (0100 -> 0102)
[    3.510743] nvme 0000:00:03.0: Using 64-bit direct DMA at offset 800000000000000
[    3.535706] p2pmem p2pmem0: registered
[    3.537780] lpar: Attempting to resize HPT to shift 21
[    3.539251] Unable to resize hash page table to target order 21: -1
[    3.541079] Unable to create mapping for hot added memory 0xc000210000000000..0xc000210004000000: -2
[    3.550407] p2pmem p2pmem0: unregistered

I am not that familiar with the pseries architecture so I was hoping for some guidance concerning the lpar error message. If any ppc64 coders fancy having a go at this the kernel code is at [1] and I’d be happy to provide the .config used if needed. Just let me know.

Cheers
 
Stephen

[1] https://github.com/sbates130272/linux-p2pmem