| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <43b6ef51-c88f-4b0c-b512-c7195afdd31b@linux.alibaba.com>
Date: Thu, 25 Dec 2025 09:01:47 +0800
From: Baolin Wang <baolin.wang@...ux.alibaba.com>
To: Ryan Roberts <ryan.roberts@....com>, akpm@...ux-foundation.org,
david@...nel.org, catalin.marinas@....com, will@...nel.org
Cc: lorenzo.stoakes@...cle.com, Liam.Howlett@...cle.com, vbabka@...e.cz,
rppt@...nel.org, surenb@...gle.com, mhocko@...e.com, riel@...riel.com,
harry.yoo@...cle.com, jannh@...gle.com, willy@...radead.org,
baohua@...nel.org, dev.jain@....com, linux-mm@...ck.org,
linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v4 1/5] mm: rmap: support batched checks of the references
for large folios
Hi Ryan,
On 2025/12/24 21:24, Ryan Roberts wrote:
> On 23/12/2025 05:48, Baolin Wang wrote:
>> Currently, folio_referenced_one() always checks the young flag for each PTE
>> sequentially, which is inefficient for large folios. This inefficiency is
>> especially noticeable when reclaiming clean file-backed large folios, where
>> folio_referenced() is observed as a significant performance hotspot.
>>
>> Moreover, on Arm64 architecture, which supports contiguous PTEs, there is already
>> an optimization to clear the young flags for PTEs within a contiguous range.
>> However, this is not sufficient. We can extend this to perform batched operations
>> for the entire large folio (which might exceed the contiguous range: CONT_PTE_SIZE).
>>
>> Introduce a new API: clear_flush_young_ptes() to facilitate batched checking
>> of the young flags and flushing TLB entries, thereby improving performance
>> during large folio reclamation. And it will be overridden by the architecture
>> that implements a more efficient batch operation in the following patches.
>>
>> Signed-off-by: Baolin Wang <baolin.wang@...ux.alibaba.com>
>
> With the 2 niggles below addressed:
>
> Reviewed-by: Ryan Roberts <ryan.roberts@....com>
Thanks.
>> ---
>> include/linux/mmu_notifier.h | 9 +++++----
>> include/linux/pgtable.h | 35 +++++++++++++++++++++++++++++++++++
>> mm/rmap.c | 29 +++++++++++++++++++++++++++--
>> 3 files changed, 67 insertions(+), 6 deletions(-)
>>
>> diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h
>> index d1094c2d5fb6..dbbdcef4abf1 100644
>> --- a/include/linux/mmu_notifier.h
>> +++ b/include/linux/mmu_notifier.h
>> @@ -515,16 +515,17 @@ static inline void mmu_notifier_range_init_owner(
>> range->owner = owner;
>> }
>>
>> -#define ptep_clear_flush_young_notify(__vma, __address, __ptep) \
>> +#define ptep_clear_flush_young_notify(__vma, __address, __ptep, __nr) \
>
> I think I previously suggested that this should be renamed to
> clear_flush_young_ptes_notify() given that it is now a batch operation. Were
> others against that or did you forget?
Sorry, I missed this comment. Yes, sounds reasonable to me. Will do in
the next version.
>> ({ \
>> int __young; \
>> struct vm_area_struct *___vma = __vma; \
>> unsigned long ___address = __address; \
>> - __young = ptep_clear_flush_young(___vma, ___address, __ptep); \
>> + unsigned int ___nr = __nr; \
>> + __young = clear_flush_young_ptes(___vma, ___address, __ptep, ___nr); \
>> __young |= mmu_notifier_clear_flush_young(___vma->vm_mm, \
>> ___address, \
>> ___address + \
>> - PAGE_SIZE); \
>> + ___nr * PAGE_SIZE); \
>> __young; \
>> })
>>
>> @@ -650,7 +651,7 @@ static inline void mmu_notifier_subscriptions_destroy(struct mm_struct *mm)
>>
>> #define mmu_notifier_range_update_to_read_only(r) false
>>
>> -#define ptep_clear_flush_young_notify ptep_clear_flush_young
>> +#define ptep_clear_flush_young_notify clear_flush_young_ptes
>> #define pmdp_clear_flush_young_notify pmdp_clear_flush_young
>> #define ptep_clear_young_notify ptep_test_and_clear_young
>> #define pmdp_clear_young_notify pmdp_test_and_clear_young
>> diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
>> index 2f0dd3a4ace1..fcf7a7820061 100644
>> --- a/include/linux/pgtable.h
>> +++ b/include/linux/pgtable.h
>> @@ -1087,6 +1087,41 @@ static inline void wrprotect_ptes(struct mm_struct *mm, unsigned long addr,
>> }
>> #endif
>>
>> +#ifndef clear_flush_young_ptes
>> +/**
>> + * clear_flush_young_ptes - Clear the access bit and perform a TLB flush for PTEs
>> + * that map consecutive pages of the same folio.
>> + * @vma: The virtual memory area the pages are mapped into.
>> + * @addr: Address the first page is mapped at.
>> + * @ptep: Page table pointer for the first entry.
>> + * @nr: Number of entries to clear access bit.
>> + *
>> + * May be overridden by the architecture; otherwise, implemented as a simple
>> + * loop over ptep_clear_flush_young().
>> + *
>> + * Note that PTE bits in the PTE range besides the PFN can differ. For example,
>> + * some PTEs might be write-protected.
>> + *
>> + * Context: The caller holds the page table lock. The PTEs map consecutive
>> + * pages that belong to the same folio. The PTEs are all in the same PMD.
>> + */
>> +static inline int clear_flush_young_ptes(struct vm_area_struct *vma,
>> + unsigned long addr, pte_t *ptep,
>> + unsigned int nr)
>> +{
>> + int young;
>> +
>> + young = ptep_clear_flush_young(vma, addr, ptep);
>> + while (--nr) {
>> + ptep++;
>> + addr += PAGE_SIZE;
>> + young |= ptep_clear_flush_young(vma, addr, ptep);
>> + }
>
> I think it's better to avoid the two ptep_clear_flush_young() calls if we can.
> Personally I think we should just go for the simple:
>
> for (i = 0; i < nr; ++i, ++ptep, addr += PAGE_SIZE)
> young |= ptep_clear_flush_young(vma, addr, ptep);
ACK. Thanks for reviewing.
Powered by blists - more mailing lists