[PATCH 00/49] mm: Generalize vmemmap optimization for DAX and HugeTLB

[Frank van der Linden]

On Sun, Apr 5, 2026 at 5:53 AM Muchun Song <songmuchun at bytedance.com> wrote:
>
> Overview:
> This patch series generalizes the HugeTLB Vmemmap Optimization (HVO)
> into a generic vmemmap optimization framework that can be used by both
> HugeTLB and DAX.
>
> Background:
> Currently, the vmemmap optimization feature is highly coupled with
> HugeTLB. However, DAX also has similar requirements for optimizing vmemmap
> pages to save memory. The current implementation has separate vmemmap
> optimization paths for HugeTLB and DAX, leading to duplicated logic,
> complex initialization sequences, and architecture-specific flags.
>
> Implementation:
> This series breaks down the optimization into a generic framework:
> - Patch 1-6: Fix bugs related to sparse vmemmap initialization and DAX.
> - Patch 7-13: Refactor the existing sparse vmemmap initialization.
> - Patch 14-26: Decouple the vmemmap optimization from HugeTLB and
>   introduce generic optimization macros and functions.
> - Patch 27-39: Switch HugeTLB and DAX to use the generic framework.
> - Patch 40-49: Clean up the old HVO-specific code and simplify it.
>
> Benifit:
> - When CONFIG_DEFERRED_STRUCT_PAGE_INIT is disabled, all struct pages
>   utilizing HVO (HugeTLB Vmemmap Optimization) skip initialization in
>   memmap_init, significantly accelerating boot times.
> - All architectures supporting HVO benefit from the optimizations
>   provided by SPARSEMEM_VMEMMAP_PREINIT without requiring
>   architecture-specific adaptations.
> - Device DAX struct page savings are further improved, saving an
>   additional 4KB of struct page memory for every 2MB huge page.
> - Vmemmap tail pages used for Device DAX shared mappings are changed
>   from read-write to read-only, enhancing system security.
> - HugeTLB and Device DAX now share a unified vmemmap optimization
>   framework, reducing long-term maintenance overhead.
>
> Testing:
> - Verification: Compiled and tested on x86 architecture.
>
>
> Chengkaitao (1):
>   mm: Convert vmemmap_p?d_populate() to static functions
>
> Muchun Song (48):
>   mm/sparse: fix vmemmap accounting imbalance on memory hotplug error
>   mm/sparse: add a @pgmap argument to memory deactivation paths
>   mm/sparse: fix vmemmap page accounting for HVOed DAX
>   mm/sparse: add a @pgmap parameter to arch vmemmap_populate()
>   mm/sparse: fix missing architecture-specific page table sync for HVO
>     DAX
>   mm/mm_init: fix uninitialized pageblock migratetype for ZONE_DEVICE
>     compound pages
>   mm/mm_init: use pageblock_migratetype_init_range() in
>     deferred_free_pages()
>   mm: panic on memory allocation failure in sparse_init_nid()
>   mm: move subsection_map_init() into sparse_init()
>   mm: defer sparse_init() until after zone initialization
>   mm: make set_pageblock_order() static
>   mm: integrate sparse_vmemmap_init_nid_late() into sparse_init_nid()
>   mm/cma: validate hugetlb CMA range by zone at reserve time
>   mm/hugetlb: free cross-zone bootmem gigantic pages after allocation
>   mm/hugetlb: initialize vmemmap optimization in early stage
>   mm: remove sparse_vmemmap_init_nid_late()
>   mm/mm_init: make __init_page_from_nid() static
>   mm/sparse-vmemmap: remove the VMEMMAP_POPULATE_PAGEREF flag
>   mm: rename vmemmap optimization macros to generic names
>   mm/sparse: drop power-of-2 size requirement for struct mem_section
>   mm/sparse: introduce compound page order to mem_section
>   mm/mm_init: skip initializing shared tail pages for compound pages
>   mm/sparse-vmemmap: initialize shared tail vmemmap page upon allocation
>   mm/sparse-vmemmap: support vmemmap-optimizable compound page
>     population
>   mm/hugetlb: use generic vmemmap optimization macros
>   mm: call memblocks_present() before HugeTLB initialization
>   mm/hugetlb: switch HugeTLB to use generic vmemmap optimization
>   mm: extract pfn_to_zone() helper
>   mm/sparse-vmemmap: remove unused SPARSEMEM_VMEMMAP_PREINIT feature
>   mm/hugetlb: remove HUGE_BOOTMEM_HVO flag and simplify pre-HVO logic
>   mm/sparse-vmemmap: consolidate shared tail page allocation
>   mm: introduce CONFIG_SPARSEMEM_VMEMMAP_OPTIMIZATION
>   mm/sparse-vmemmap: switch DAX to use generic vmemmap optimization
>   mm/sparse-vmemmap: introduce section zone to struct mem_section
>   powerpc/mm: use generic vmemmap_shared_tail_page() in compound vmemmap
>   mm/sparse-vmemmap: unify DAX and HugeTLB vmemmap optimization
>   mm/sparse-vmemmap: remap the shared tail pages as read-only
>   mm/sparse-vmemmap: remove unused ptpfn argument
>   mm/hugetlb_vmemmap: remove vmemmap_wrprotect_hvo() and related code
>   mm/sparse: simplify section_vmemmap_pages()
>   mm/sparse-vmemmap: introduce section_vmemmap_page_structs()
>   powerpc/mm: rely on generic vmemmap_can_optimize() to simplify code
>   mm/sparse-vmemmap: drop ARCH_WANT_OPTIMIZE_DAX_VMEMMAP and simplify
>     checks
>   mm/sparse-vmemmap: drop @pgmap parameter from vmemmap populate APIs
>   mm/sparse: replace pgmap with order and zone in sparse_add_section()
>   mm: redefine HVO as Hugepage Vmemmap Optimization
>   Documentation/mm: restructure vmemmap_dedup.rst to reflect generalized
>     HVO
>   mm: consolidate struct page power-of-2 size checks for HVO
>
>  .../admin-guide/kernel-parameters.txt         |   2 +-
>  Documentation/admin-guide/sysctl/vm.rst       |   2 +-
>  Documentation/mm/vmemmap_dedup.rst            | 218 ++--------
>  arch/powerpc/Kconfig                          |   1 -
>  arch/powerpc/include/asm/book3s/64/radix.h    |  12 -
>  arch/powerpc/mm/book3s64/radix_pgtable.c      | 114 +----
>  arch/powerpc/mm/init_64.c                     |   1 +
>  arch/riscv/Kconfig                            |   1 -
>  arch/x86/Kconfig                              |   2 -
>  fs/Kconfig                                    |   6 +-
>  include/linux/hugetlb.h                       |   7 +-
>  include/linux/memory_hotplug.h                |   2 +-
>  include/linux/mm.h                            |  50 +--
>  include/linux/mm_types.h                      |   2 +
>  include/linux/mm_types_task.h                 |   4 +
>  include/linux/mmzone.h                        | 143 ++++---
>  include/linux/page-flags.h                    |  31 +-
>  kernel/bounds.c                               |   2 +
>  mm/Kconfig                                    |  20 +-
>  mm/bootmem_info.c                             |   5 +-
>  mm/cma.c                                      |   3 +-
>  mm/hugetlb.c                                  | 143 +++----
>  mm/hugetlb_cma.c                              |   3 +-
>  mm/hugetlb_vmemmap.c                          | 237 +----------
>  mm/hugetlb_vmemmap.h                          |  35 +-
>  mm/internal.h                                 |  26 +-
>  mm/memory_hotplug.c                           |  15 +-
>  mm/mm_init.c                                  | 138 +++---
>  mm/sparse-vmemmap.c                           | 392 +++++-------------
>  mm/sparse.c                                   |  85 ++--
>  mm/util.c                                     |   2 +-
>  scripts/gdb/linux/mm.py                       |   6 +-
>  32 files changed, 513 insertions(+), 1197 deletions(-)
>
> --
> 2.20.1

Thanks very much for doing this. This cleans up a lot of stuff. I did
want to do some of this when I did the pre-HVO code, but I ended up
not doing it to prevent perfect being the enemy of good.

I haven't looked at all the individual patches yet, but overall this
looks great to me.

- Frank