Date: Wed, 24 Apr 2024 22:09:02 +0200
From: David Hildenbrand <david@...hat.com>
To: Guillaume Morin <guillaume@...infr.org>
Cc: oleg@...hat.com, linux-kernel@...r.kernel.org,
 linux-trace-kernel@...r.kernel.org, muchun.song@...ux.dev
Subject: Re: [RFC][PATCH] uprobe: support for private hugetlb mappings

On 22.04.24 22:53, Guillaume Morin wrote:
> On 22 Apr 20:59, David Hildenbrand wrote:
>>> The benefit - to me - is very clear. People do use hugetlb mappings to
>>> run code in production environments. The perf benefits are there for some
>>> workloads. Intel has published a whitepaper about it etc.
>>> Uprobes are a very good tool to do live tracing. If you can restart the
>>> process and reproduce, you should be able to disable hugetlb remapping
>>> but if you need to look at a live process, there are not many options.
>>> Not being able to use uprobes is crippling.
>>
>> Please add all that as motivation to the patch description or cover letter.
>>
>>>> Yes, libhugetlbfs exists. But why do we have to support uprobes with it?
>>>> Nobody cared until now, why care now?
>>>
>>> I think you could ask the same question for every new feature patch :)
>>
>> I have to, because it usually indicates a lack of motivation in the
>> cover-letter/patch description :P
> 
> My cover letter was indeed lacking. I will make sure to add this kind of
> details next time.
>   
>>> Since the removal a few releases ago of the __morecore() hook in glibc,
>>> the main feature of libhugetlbfs is ELF segments remapping. I think
>>> there are definitely a lot of users that simply deal with this
>>> unnecessary limitation.
>>>
>>> I am certainly not shoving this patch through anyone's throat if there
>>> is no interest. But we definitely find it a very useful feature ...
>>
>> Let me try to see if we can get this done cleaner.
>>
>> One ugly part (in general here) is the custom page replacement in the
>> registration part.
>>
>> We are guaranteed to have a MAP_PRIVATE mapping. Instead of replacing pages
>> ourselves (which we likely shouldn't do ...) ... maybe we could use
>> FAULT_FLAG_UNSHARE faults such that we will get an anonymous folio
>> populated. (like KSM does nowadays)
>>
>> Punching FOLL_PIN|FOLL_LONGTERM into GUP would achieve the same thing, but
>> using FOLL_WRITE would not work on many file systems. So maybe we have to
>> trigger an unsharing fault ourselves.

^ realizing that we already use FOLL_FORCE, so we can just use 
FOLL_WRITE to break COW.

>>
>> That would do the page replacement for us and we "should" be able to lookup
>> an anonymous folio that we can then just modify, like ptrace would.
>>
>> But then, there is also unregistration part, with weird conditional page
>> replacement. Zapping the anon page if the content matches the content of the
>> original page is one thing. But why are we placing an existing anonymous
>> page by a new anonymous page when the content from the original page differs
>> (but matches the one from the just copied page?)?
>>
>> I'll have to further think about that one. It's all a bit nasty.
>
> Sounds good to me. I am willing to help with the code when you have a
> plan or testing as you see fit. Let me know.

I'm hacking on a redesign that removes the manual COW breaking logic and 
*might* make it easier to integrate hugetlb. (very likely, but until I 
have the redesign running I cannot promise anything :) )

I'll let you know once I have something ready so you could integrate the 
hugetlb portion.

-- 
Cheers,

David / dhildenb

Powered by blists - more mailing lists