[PATCH v2] hugetlb: simplify hugetlb handling in follow_page_mask

[Christophe Leroy]

Le 02/09/2022 à 21:03, Mike Kravetz a écrit :
> During discussions of this series [1], it was suggested that hugetlb
> handling code in follow_page_mask could be simplified.  At the beginning
> of follow_page_mask, there currently is a call to follow_huge_addr which
> 'may' handle hugetlb pages.  ia64 is the only architecture which provides
> a follow_huge_addr routine that does not return error.  Instead, at each
> level of the page table a check is made for a hugetlb entry.  If a hugetlb
> entry is found, a call to a routine associated with that entry is made.
> 
> Currently, there are two checks for hugetlb entries at each page table
> level.  The first check is of the form:
>          if (p?d_huge())
>                  page = follow_huge_p?d();
> the second check is of the form:
>          if (is_hugepd())
>                  page = follow_huge_pd().
> 
> We can replace these checks, as well as the special handling routines
> such as follow_huge_p?d() and follow_huge_pd() with a single routine to
> handle hugetlb vmas.
> 
> A new routine hugetlb_follow_page_mask is called for hugetlb vmas at the
> beginning of follow_page_mask.  hugetlb_follow_page_mask will use the
> existing routine huge_pte_offset to walk page tables looking for hugetlb
> entries.  huge_pte_offset can be overwritten by architectures, and already
> handles special cases such as hugepd entries.
> 
> [1] https://lore.kernel.org/linux-mm/cover.1661240170.git.baolin.wang@linux.alibaba.com/
> 
> Suggested-by: David Hildenbrand <david at redhat.com>
> Signed-off-by: Mike Kravetz <mike.kravetz at oracle.com>
> ---
> v2 -	Added WARN_ON_ONCE() and updated comment as suggested by David
> 	Fixed build issue found by kernel test robot
> 	Added vma (pmd sharing) locking to hugetlb_follow_page_mask
> 	ReBased on Baolin's patch to fix issues with CONT_* entries
> 
>   arch/ia64/mm/hugetlbpage.c    |  15 ---
>   arch/powerpc/mm/hugetlbpage.c |  37 -------
>   include/linux/hugetlb.h       |  51 ++--------
>   mm/gup.c                      |  80 +++------------
>   mm/hugetlb.c                  | 182 ++++++++++++----------------------
>   5 files changed, 87 insertions(+), 278 deletions(-)
> 

> diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> index fe4944f89d34..275e554dd365 100644
> --- a/include/linux/hugetlb.h
> +++ b/include/linux/hugetlb.h
> @@ -264,6 +255,13 @@ static inline void adjust_range_if_pmd_sharing_possible(
>   {
>   }
>   
> +static inline struct page *hugetlb_follow_page_mask(struct vm_area_struct *vma,
> +				unsigned long address, unsigned int flags)
> +{
> +	WARN_ON_ONCE(1); /* should never be called if !CONFIG_HUGETLB_PAGE*/
> +	return ERR_PTR(-EINVAL);

This function is called only when is_vm_hugetlb_page() is true.

When !CONFIG_HUGETLB_PAGE is_vm_hugetlb_page() always returns false, so 
the call to hugetlb_follow_page_mask() should never be compiled in.

Use BUILD_BUG() to catch it at buildtime.

> +}
> +
>   static inline long follow_hugetlb_page(struct mm_struct *mm,
>   			struct vm_area_struct *vma, struct page **pages,
>   			struct vm_area_struct **vmas, unsigned long *position,

> diff --git a/mm/gup.c b/mm/gup.c
> index 7691c65233c3..1515892a9d98 100644
> --- a/mm/gup.c
> +++ b/mm/gup.c
> @@ -530,18 +530,6 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
>   	if (WARN_ON_ONCE((flags & (FOLL_PIN | FOLL_GET)) ==
>   			 (FOLL_PIN | FOLL_GET)))
>   		return ERR_PTR(-EINVAL);
> -
> -	/*
> -	 * Considering PTE level hugetlb, like continuous-PTE hugetlb on
> -	 * ARM64 architecture.
> -	 */
> -	if (is_vm_hugetlb_page(vma)) {
> -		page = follow_huge_pmd_pte(vma, address, flags);
> -		if (page)
> -			return page;
> -		return no_page_table(vma, flags);
> -	}
> -
>   retry:
>   	if (unlikely(pmd_bad(*pmd)))
>   		return no_page_table(vma, flags);