linux-kernel - [RFC 00/11] cleanup vma_merge and improve mergeability tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 24 Feb 2023 11:00:11 +0100
From:   Vlastimil Babka <vbabka@...e.cz>
To:     linux-mm@...ck.org, "Liam R. Howlett" <Liam.Howlett@...cle.com>
Cc:     linux-kernel@...r.kernel.org, patches@...ts.linux.dev,
        maple-tree@...ts.infradead.org, Vlastimil Babka <vbabka@...e.cz>
Subject: [RFC 00/11] cleanup vma_merge and improve mergeability tests

Also available in git:
https://git.kernel.org/vbabka/h/vma_merge_cleanup-v1r1

My initial goal here was to try making the check for vm_ops->close in
is_mergeable_vma() only be applied for vma's that would be truly removed
as part of the merge (see Patch 10). This would then allow reverting the
quick fix d014cd7c1c35 ("mm, mremap: fix mremap() expanding for vma's
with vm_ops->close()"). This was successful enough to allow the revert.
Checks using can_vma_merge_before() are still pessimistic about possible
vma removal, and making them precise would probably complicate the
vma_merge() code too much.

Liam's recent simplification of vma_merge() and removal of
__vma_adjust() was very much helpful in understanding the vma_merge()
implementation and especially when vma removals can happen, which is now
very obvious. While studing the code, I've found ways to make it
hopefully even more easy to follow, so that's the patches 2-9. That made
me notice a bug fixed in patch 1, which is already in the mm tree and
here just for completeness.

Vlastimil Babka (11):
  mm/mremap: fix dup_anon_vma() in vma_merge() case 4
  mm/mmap/vma_merge: use only primary pointers for preparing merge
  mm/mmap/vma_merge: use the proper vma pointer in case 3
  mm/mmap/vma_merge: use the proper vma pointers in cases 1 and 6
  mm/mmap/vma_merge: use the proper vma pointer in case 4
  mm/mmap/vma_merge: initialize mid and next in natural order
  mm/mmap/vma_merge: set mid to NULL if not applicable
  mm/mmap/vma_merge: rename adj_next to adj_start
  mm/mmap/vma_merge: convert mergeability checks to return bool
  mm/mmap: start distinguishing if vma can be removed in mergeability
    test
  mm/mremap: simplify vma expansion again

 mm/mmap.c   | 130 +++++++++++++++++++++++++++++-----------------------
 mm/mremap.c |  20 ++------
 2 files changed, 76 insertions(+), 74 deletions(-)

-- 
2.39.2