linux-kernel - Re: [PATCH 3/4] mm: prevent KSM from completely breaking VMA merging

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <e2910260-8deb-44ce-b6c9-376b4917ecea@redhat.com>
Date: Mon, 19 May 2025 20:59:32 +0200
From: David Hildenbrand <david@...hat.com>
To: Lorenzo Stoakes <lorenzo.stoakes@...cle.com>
Cc: Andrew Morton <akpm@...ux-foundation.org>,
 Alexander Viro <viro@...iv.linux.org.uk>,
 Christian Brauner <brauner@...nel.org>, Jan Kara <jack@...e.cz>,
 "Liam R . Howlett" <Liam.Howlett@...cle.com>,
 Vlastimil Babka <vbabka@...e.cz>, Jann Horn <jannh@...gle.com>,
 Pedro Falcato <pfalcato@...e.de>, Xu Xin <xu.xin16@....com.cn>,
 Chengming Zhou <chengming.zhou@...ux.dev>, linux-mm@...ck.org,
 linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org
Subject: Re: [PATCH 3/4] mm: prevent KSM from completely breaking VMA merging

>>
>> I am not 100% sure why we bail out on special mappings: all we have to do is
>> reliably identify anon pages, and we should be able to do that.
> 
> But they map e.g. kernel memory (at least for VM_PFNMAP, purely and by
> implication really VM_IO), it wouldn't make sense for KSM to be asked to
> try to merge these right?
> 
> And of course no underlying struct page to pin, no reference counting
> either, so I think you'd end up in trouble potentially here wouldn't you?
> And how would the CoW work?

KSM only operates on anonymous pages. It cannot de-duplicate anything 
else. (therefore, only MAP_PRIVATE applies)

Anything else (no struct page, not a CoW anon folio in such a mapping) 
is skipped.

Take a look at scan_get_next_rmap_item() where we do

folio = folio_walk_start(&fw, vma, ksm_scan.address, 0);
if (folio) {
	if (!folio_is_zone_device(folio) &&
	    folio_test_anon(folio)) {
		folio_get(folio);
		tmp_page = fw.page;
	}
	folio_walk_end(&fw, vma)
}

Before I changed that code, we were using GUP. And GUP just always 
refuses VM_IO|VM_PFNMAP because it cannot handle it properly.

>>
>> So, assuming we could remove the VM_PFNMAP | VM_IO | VM_DONTEXPAND |
>> VM_MIXEDMAP constraint from vma_ksm_compatible(), could we simplify?
> 
> Well I question removing this constraint for above reasons.
> 
> At any rate, even if we _could_ this feels like a bigger change that we	
> should come later.

"bigger" -- it might just be removing these 4 flags from the check ;)

I'll dig a bit more.

-- 
Cheers,

David / dhildenb