linux-kernel - Re: [PATCH 0/3] ksm: fix incorrect count of merged pages when enabling use_zero

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 30 Sep 2022 11:40:18 +0200
From:   David Hildenbrand <david@...hat.com>
To:     Claudio Imbrenda <imbrenda@...ux.ibm.com>
Cc:     xu.xin.sc@...il.com, akpm@...ux-foundation.org, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org, xu xin <xu.xin16@....com.cn>
Subject: Re: [PATCH 0/3] ksm: fix incorrect count of merged pages when
 enabling use_zero_pages

>>>    
>>>>   
>>>>>
>>>>> second, once a page is merged with a zero page, it's not really handled
>>>>> by KSM anymore. if you have a big allocation, of which you only touch a
>>>>> few pages, would the rest be considered "merged"? no, it's just zero
>>>>> pages, right?
>>>>
>>>> If you haven't touched memory, there is nothing populated -- no shared
>>>> zeropage.
>>>>
>>>> We only populate shared zeropages in private anonymous mappings on read
>>>> access without prior write.
>>>
>>> that's what I meant. if you read without writing, you get zero pages.
>>> you don't consider those to be "shared" from a KSM point of view
>>>
>>> does it make a difference if some pages that have been written to but
>>> now only contain zeroes are discarded and mapped back to the zero pages?
>>
>> That's a good question. When it comes to unmerging, you'd might expect
>> that whatever was deduplicated will get duplicated again -- and your
>> memory consumption will adjust accordingly. The stats might give an
>> admin an idea regarding how much memory is actually overcommited. See
>> below on the important case where we essentially never see the shared
>> zeropage.
>>
>> The motivation behind these patches would be great -- what is the KSM
>> user and what does it want to achieve with these numbers?
> 
> anyone who works on big amounts of very sparse data, especially in a VM
> (as I explained above, with KSM without use_zero_pages KVM guests lose
> the zero page colouring)

I meant the patches in question here :)

> 
>>
>>>    
>>>>   
>>>>> this is the same, except that we take present pages with zeroes in it
>>>>> and we discard them and map them to zero pages. it's kinda like if we
>>>>> had never touched them.
>>>>
>>>> MADV_UNMERGEABLE
>>>>
>>>> "Undo  the effect of an earlier MADV_MERGEABLE operation on the
>>>> specified address range; KSM unmerges whatever pages it had merged in
>>>> the address range specified  by  addr  and length."
>>>>
>>>> Now please explain to me how not undoing a zeropage merging is correct
>>>> according to this documentation.
>>>>   
>>>
>>> because once it's discarded and replaced with a zero page, the page is
>>> not handled by KSM anymore.
>>>
>>> I understand what you mean, that KSM did an action that now cannot be
>>> undone, but how would you differentiate between zero pages that were
>>> never written to and pages that had been written to and then discarded
>>> and mapped back to a zero page because they only contained zeroes?
>>
>> An application that always properly initializes (write at least some
>> part once) all its memory will never have the shared zeropage mapped. VM
>> guest memory comes to mind, probably still the most important KSM use case.
>>
>> There are currently some remaining issues when taking a GUP R/O longterm
>> pin on such a page (e.g., vfio). In contrast to KSM pages, such pins are
>> not reliable for the shared zeropage, but I have fixes for them pending.
>> However, that is rather a corner case (it didn't work at all correctly a
>> while ago) and will be sorted out soon.
>>
>> So the question is if MADV_UNMERGEABLE etc. (stats) should be adjusted
>> to document the behavior with use_zero_pages accordingly.
> 
> we can count how many times a page full of zeroes was merged with a
> zero-page, but we can't count how many time one of those pages was then
> unmerged. once it's merged it becomes a zero-page, like the others.

Right. We could special case on MADV_MERGEABLE ("how many zero pages do 
we have mapped into MADV_MERGEABLE VMAs"), but it gets tricky once we 
enable MADV_MERGEABLE on a region where the shared zeropage was already 
populated. Probably not worth the effort.


-- 
Thanks,

David / dhildenb