linux-kernel - Re: [PATCH] mm/migrate: fix hugetlbfs deadlock by respecting lock ordering

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <a14a7d81-7f87-42a0-9b3c-1a71e765cfe0@kernel.org>
Date: Fri, 9 Jan 2026 16:41:07 +0100
From: "David Hildenbrand (Red Hat)" <david@...nel.org>
To: Jinchao Wang <wangjinchao600@...il.com>
Cc: Matthew Wilcox <willy@...radead.org>,
 Andrew Morton <akpm@...ux-foundation.org>, Zi Yan <ziy@...dia.com>,
 Matthew Brost <matthew.brost@...el.com>,
 Joshua Hahn <joshua.hahnjy@...il.com>, Rakie Kim <rakie.kim@...com>,
 Byungchul Park <byungchul@...com>, Gregory Price <gourry@...rry.net>,
 Ying Huang <ying.huang@...ux.alibaba.com>,
 Alistair Popple <apopple@...dia.com>, linux-mm@...ck.org,
 linux-kernel@...r.kernel.org,
 syzbot+2d9c96466c978346b55f@...kaller.appspotmail.com
Subject: Re: [PATCH] mm/migrate: fix hugetlbfs deadlock by respecting lock
 ordering

On 1/9/26 16:32, Jinchao Wang wrote:
> On Fri, Jan 09, 2026 at 03:18:37PM +0100, David Hildenbrand (Red Hat) wrote:
>> On 1/9/26 15:16, Jinchao Wang wrote:
>>> On Fri, Jan 09, 2026 at 02:39:08PM +0100, David Hildenbrand (Red Hat) wrote:
>>>> On 1/9/26 04:47, Jinchao Wang wrote:
>>>>> Fix an AB-BA deadlock between hugetlbfs_punch_hole() and page migration.
>>>>>
>>>>> The deadlock occurs because migration violates the lock ordering defined
>>>>> in mm/rmap.c for hugetlbfs:
>>>>>
>>>>>      * hugetlbfs PageHuge() take locks in this order:
>>>>>      * hugetlb_fault_mutex
>>>>>      * vma_lock
>>>>>      * mapping->i_mmap_rwsem
>>>>>      * folio_lock
>>>>>
>>>>> The following trace illustrates the inversion:
>>>>>
>>>>> Task A (punch_hole):             Task B (migration):
>>>>> --------------------             -------------------
>>>>> 1. i_mmap_lock_write(mapping)    1. folio_lock(folio)
>>>>> 2. folio_lock(folio)             2. i_mmap_lock_read(mapping)
>>>>>       (blocks waiting for B)           (blocks waiting for A)
>>>>>
>>>>> Task A is blocked in the punch-hole path:
>>>>>      hugetlbfs_fallocate
>>>>>        hugetlbfs_punch_hole
>>>>>          hugetlbfs_zero_partial_page
>>>>>            folio_lock
>>>>>
>>>>> Task B is blocked in the migration path:
>>>>>      migrate_pages
>>>>>        unmap_and_move_huge_page
>>>>>          remove_migration_ptes
>>>>>            __rmap_walk_file
>>>>>              i_mmap_lock_read
>>>>>
>>>>> To fix this, adjust unmap_and_move_huge_page() to respect the established
>>>>> hierarchy. If i_mmap_rwsem is acquired during try_to_migrate(), hold it
>>>>
>>>>
>>>> I'm confused. Isn't it unmap_and_move_huge_page() that grabs the
>>>> i_mmap_rwsem during hugetlb_page_mapping_lock_write() (where we do a
>>>> try-lock)?
>>> Yes, but the lock is released before remove_migration_ptes().
>>>
>>> Task A can enter the race window between
>>> 	i_mmap_unlock_write(mapping)
>>> and
>>> 	remove_migration_ptes() -> i_mmap_lock_read(mapping).
>>>
>>> This window was introduced by the change below:
>>> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/mm/migrate.c?id=336bf30eb765
>>
>> try_to_migrate() is not the problem, but remove_migration_ptes() ?
>>
>> Anyhow, I saw that Willy sent out a version.
> Thank you for letting me know.

For reference:

https://lkml.kernel.org/r/20260109041345.3863089-1-willy@infradead.org

-- 
Cheers

David