| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <80a7b182-6bd4-49f5-ba3c-1ef53f51557c@redhat.com>
Date: Thu, 28 Aug 2025 22:22:51 +0200
From: David Hildenbrand <david@...hat.com>
To: Matthew Brost <matthew.brost@...el.com>
Cc: Balbir Singh <balbirs@...dia.com>, dri-devel@...ts.freedesktop.org,
linux-mm@...ck.org, linux-kernel@...r.kernel.org,
Andrew Morton <akpm@...ux-foundation.org>, Zi Yan <ziy@...dia.com>,
Joshua Hahn <joshua.hahnjy@...il.com>, Rakie Kim <rakie.kim@...com>,
Byungchul Park <byungchul@...com>, Gregory Price <gourry@...rry.net>,
Ying Huang <ying.huang@...ux.alibaba.com>,
Alistair Popple <apopple@...dia.com>, Oscar Salvador <osalvador@...e.de>,
Lorenzo Stoakes <lorenzo.stoakes@...cle.com>,
Baolin Wang <baolin.wang@...ux.alibaba.com>,
"Liam R. Howlett" <Liam.Howlett@...cle.com>, Nico Pache <npache@...hat.com>,
Ryan Roberts <ryan.roberts@....com>, Dev Jain <dev.jain@....com>,
Barry Song <baohua@...nel.org>, Lyude Paul <lyude@...hat.com>,
Danilo Krummrich <dakr@...nel.org>, David Airlie <airlied@...il.com>,
Simona Vetter <simona@...ll.ch>, Ralph Campbell <rcampbell@...dia.com>,
Mika Penttilä <mpenttil@...hat.com>,
Francois Dugast <francois.dugast@...el.com>
Subject: Re: [v3 02/11] mm/thp: zone_device awareness in THP handling code
On 28.08.25 22:17, Matthew Brost wrote:
> On Thu, Aug 28, 2025 at 10:12:53PM +0200, David Hildenbrand wrote:
>> On 28.08.25 22:05, Matthew Brost wrote:
>>> On Tue, Aug 12, 2025 at 12:40:27PM +1000, Balbir Singh wrote:
>>>> Make THP handling code in the mm subsystem for THP pages aware of zone
>>>> device pages. Although the code is designed to be generic when it comes
>>>> to handling splitting of pages, the code is designed to work for THP
>>>> page sizes corresponding to HPAGE_PMD_NR.
>>>>
>>>> Modify page_vma_mapped_walk() to return true when a zone device huge
>>>> entry is present, enabling try_to_migrate() and other code migration
>>>> paths to appropriately process the entry. page_vma_mapped_walk() will
>>>> return true for zone device private large folios only when
>>>> PVMW_THP_DEVICE_PRIVATE is passed. This is to prevent locations that are
>>>> not zone device private pages from having to add awareness. The key
>>>> callback that needs this flag is try_to_migrate_one(). The other
>>>> callbacks page idle, damon use it for setting young/dirty bits, which is
>>>> not significant when it comes to pmd level bit harvesting.
>>>>
>>>> pmd_pfn() does not work well with zone device entries, use
>>>> pfn_pmd_entry_to_swap() for checking and comparison as for zone device
>>>> entries.
>>>>
>>>> Support partial unmapping of zone device private entries, which happens
>>>> via munmap(). munmap() causes the device private entry pmd to be split,
>>>> but the corresponding folio is not split. Deferred split does not work for
>>>> zone device private folios due to the need to split during fault
>>>> handling. Get migrate_vma_collect_pmd() to handle this case by splitting
>>>> partially unmapped device private folios.
>>>>
>>>> Cc: Andrew Morton <akpm@...ux-foundation.org>
>>>> Cc: David Hildenbrand <david@...hat.com>
>>>> Cc: Zi Yan <ziy@...dia.com>
>>>> Cc: Joshua Hahn <joshua.hahnjy@...il.com>
>>>> Cc: Rakie Kim <rakie.kim@...com>
>>>> Cc: Byungchul Park <byungchul@...com>
>>>> Cc: Gregory Price <gourry@...rry.net>
>>>> Cc: Ying Huang <ying.huang@...ux.alibaba.com>
>>>> Cc: Alistair Popple <apopple@...dia.com>
>>>> Cc: Oscar Salvador <osalvador@...e.de>
>>>> Cc: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
>>>> Cc: Baolin Wang <baolin.wang@...ux.alibaba.com>
>>>> Cc: "Liam R. Howlett" <Liam.Howlett@...cle.com>
>>>> Cc: Nico Pache <npache@...hat.com>
>>>> Cc: Ryan Roberts <ryan.roberts@....com>
>>>> Cc: Dev Jain <dev.jain@....com>
>>>> Cc: Barry Song <baohua@...nel.org>
>>>> Cc: Lyude Paul <lyude@...hat.com>
>>>> Cc: Danilo Krummrich <dakr@...nel.org>
>>>> Cc: David Airlie <airlied@...il.com>
>>>> Cc: Simona Vetter <simona@...ll.ch>
>>>> Cc: Ralph Campbell <rcampbell@...dia.com>
>>>> Cc: Mika Penttilä <mpenttil@...hat.com>
>>>> Cc: Matthew Brost <matthew.brost@...el.com>
>>>> Cc: Francois Dugast <francois.dugast@...el.com>
>>>>
>>>> Signed-off-by: Matthew Brost <matthew.brost@...el.com>
>>>> Signed-off-by: Balbir Singh <balbirs@...dia.com>
>>>> ---
>>>> include/linux/rmap.h | 2 +
>>>> include/linux/swapops.h | 17 ++++
>>>> lib/test_hmm.c | 2 +-
>>>> mm/huge_memory.c | 214 +++++++++++++++++++++++++++++++---------
>>>> mm/migrate_device.c | 47 +++++++++
>>>> mm/page_vma_mapped.c | 13 ++-
>>>> mm/pgtable-generic.c | 6 ++
>>>> mm/rmap.c | 24 ++++-
>>>> 8 files changed, 272 insertions(+), 53 deletions(-)
>>>>
>>>> diff --git a/include/linux/rmap.h b/include/linux/rmap.h
>>>> index 6cd020eea37a..dfb7aae3d77b 100644
>>>> --- a/include/linux/rmap.h
>>>> +++ b/include/linux/rmap.h
>>>> @@ -927,6 +927,8 @@ struct page *make_device_exclusive(struct mm_struct *mm, unsigned long addr,
>>>> #define PVMW_SYNC (1 << 0)
>>>> /* Look for migration entries rather than present PTEs */
>>>> #define PVMW_MIGRATION (1 << 1)
>>>> +/* Look for device private THP entries */
>>>> +#define PVMW_THP_DEVICE_PRIVATE (1 << 2)
>>>> struct page_vma_mapped_walk {
>>>> unsigned long pfn;
>>>> diff --git a/include/linux/swapops.h b/include/linux/swapops.h
>>>> index 64ea151a7ae3..2641c01bd5d2 100644
>>>> --- a/include/linux/swapops.h
>>>> +++ b/include/linux/swapops.h
>>>> @@ -563,6 +563,7 @@ static inline int is_pmd_migration_entry(pmd_t pmd)
>>>> {
>>>> return is_swap_pmd(pmd) && is_migration_entry(pmd_to_swp_entry(pmd));
>>>> }
>>>> +
>>>> #else /* CONFIG_ARCH_ENABLE_THP_MIGRATION */
>>>> static inline int set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw,
>>>> struct page *page)
>>>> @@ -594,6 +595,22 @@ static inline int is_pmd_migration_entry(pmd_t pmd)
>>>> }
>>>> #endif /* CONFIG_ARCH_ENABLE_THP_MIGRATION */
>>>> +#if defined(CONFIG_ZONE_DEVICE) && defined(CONFIG_ARCH_ENABLE_THP_MIGRATION)
>>>> +
>>>> +static inline int is_pmd_device_private_entry(pmd_t pmd)
>>>> +{
>>>> + return is_swap_pmd(pmd) && is_device_private_entry(pmd_to_swp_entry(pmd));
>>>> +}
>>>> +
>>>> +#else /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */
>>>> +
>>>> +static inline int is_pmd_device_private_entry(pmd_t pmd)
>>>> +{
>>>> + return 0;
>>>> +}
>>>> +
>>>> +#endif /* CONFIG_ZONE_DEVICE && CONFIG_ARCH_ENABLE_THP_MIGRATION */
>>>> +
>>>> static inline int non_swap_entry(swp_entry_t entry)
>>>> {
>>>> return swp_type(entry) >= MAX_SWAPFILES;
>>>> diff --git a/lib/test_hmm.c b/lib/test_hmm.c
>>>> index 761725bc713c..297f1e034045 100644
>>>> --- a/lib/test_hmm.c
>>>> +++ b/lib/test_hmm.c
>>>> @@ -1408,7 +1408,7 @@ static vm_fault_t dmirror_devmem_fault(struct vm_fault *vmf)
>>>> * the mirror but here we use it to hold the page for the simulated
>>>> * device memory and that page holds the pointer to the mirror.
>>>> */
>>>> - rpage = vmf->page->zone_device_data;
>>>> + rpage = folio_page(page_folio(vmf->page), 0)->zone_device_data;
>>>> dmirror = rpage->zone_device_data;
>>>> /* FIXME demonstrate how we can adjust migrate range */
>>>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>>>> index 9c38a95e9f09..2495e3fdbfae 100644
>>>> --- a/mm/huge_memory.c
>>>> +++ b/mm/huge_memory.c
>>>> @@ -1711,8 +1711,11 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
>>>> if (unlikely(is_swap_pmd(pmd))) {
>>>> swp_entry_t entry = pmd_to_swp_entry(pmd);
>>>> - VM_BUG_ON(!is_pmd_migration_entry(pmd));
>>>> - if (!is_readable_migration_entry(entry)) {
>>>> + VM_WARN_ON(!is_pmd_migration_entry(pmd) &&
>>>> + !is_pmd_device_private_entry(pmd));
>>>> +
>>>> + if (is_migration_entry(entry) &&
>>>> + is_writable_migration_entry(entry)) {
>>>> entry = make_readable_migration_entry(
>>>> swp_offset(entry));
>>>> pmd = swp_entry_to_pmd(entry);
>>>> @@ -1722,6 +1725,32 @@ int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm,
>>>> pmd = pmd_swp_mkuffd_wp(pmd);
>>>> set_pmd_at(src_mm, addr, src_pmd, pmd);
>>>> }
>>>> +
>>>> + if (is_device_private_entry(entry)) {
>>>> + if (is_writable_device_private_entry(entry)) {
>>>> + entry = make_readable_device_private_entry(
>>>> + swp_offset(entry));
>>>> + pmd = swp_entry_to_pmd(entry);
>>>> +
>>>> + if (pmd_swp_soft_dirty(*src_pmd))
>>>> + pmd = pmd_swp_mksoft_dirty(pmd);
>>>> + if (pmd_swp_uffd_wp(*src_pmd))
>>>> + pmd = pmd_swp_mkuffd_wp(pmd);
>>>> + set_pmd_at(src_mm, addr, src_pmd, pmd);
>>>> + }
>>>> +
>>>> + src_folio = pfn_swap_entry_folio(entry);
>>>> + VM_WARN_ON(!folio_test_large(src_folio));
>>>> +
>>>> + folio_get(src_folio);
>>>> + /*
>>>> + * folio_try_dup_anon_rmap_pmd does not fail for
>>>> + * device private entries.
>>>> + */
>>>> + VM_WARN_ON(folio_try_dup_anon_rmap_pmd(src_folio,
>>>> + &src_folio->page, dst_vma, src_vma));
>>>
>>> VM_WARN_ON compiles out in non-debug builds. I hit this running the
>>> fork self I shared with a non-debug build.
>>
>>
>> folio_try_dup_anon_rmap_pmd() will never fail for
>> folio_is_device_private(folio) -- unless something is deeply messed up that
>> we wouldn't identify this folio as being device-private.
>>
>> Can you elaborate, what were you able to trigger, and in what kind of
>> environment?
>>
>
> Maybe this was bad phrasing. I compilied the kernel with a non-debug
> build and fork() broke for THP device pages because the above call to
> folio_try_dup_anon_rmap_pmd compiled out (i.e., it wasn't called).
Ah, yes!
As I said in my reply, we should not do any kind of WARN here, like in
the PTE case.
--
Cheers
David / dhildenb
Powered by blists - more mailing lists