linux-kernel - Re: [RESEND PATCH v7 00/10] Small-sized THP for anonymous memory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <F516C7FB-3B40-421B-B0DA-E865806EE6B9@nvidia.com>
Date:   Thu, 23 Nov 2023 20:34:24 -0500
From:   Zi Yan <ziy@...dia.com>
To:     David Hildenbrand <david@...hat.com>
Cc:     Matthew Wilcox <willy@...radead.org>,
        Ryan Roberts <ryan.roberts@....com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Yin Fengwei <fengwei.yin@...el.com>,
        Yu Zhao <yuzhao@...gle.com>,
        Catalin Marinas <catalin.marinas@....com>,
        Anshuman Khandual <anshuman.khandual@....com>,
        Yang Shi <shy828301@...il.com>,
        "\"Huang, Ying\"" <ying.huang@...el.com>,
        Luis Chamberlain <mcgrof@...nel.org>,
        Itaru Kitayama <itaru.kitayama@...il.com>,
        "\"Kirill A. Shutemov\"" <kirill.shutemov@...ux.intel.com>,
        John Hubbard <jhubbard@...dia.com>,
        David Rientjes <rientjes@...gle.com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Hugh Dickins <hughd@...gle.com>,
        Kefeng Wang <wangkefeng.wang@...wei.com>, linux-mm@...ck.org,
        linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org
Subject: Re: [RESEND PATCH v7 00/10] Small-sized THP for anonymous memory

On 23 Nov 2023, at 11:50, David Hildenbrand wrote:

> On 23.11.23 17:18, Matthew Wilcox wrote:
>> On Thu, Nov 23, 2023 at 05:05:37PM +0100, David Hildenbrand wrote:
>>> On 23.11.23 16:59, Matthew Wilcox wrote:
>>>> On Wed, Nov 22, 2023 at 04:29:40PM +0000, Ryan Roberts wrote:
>>>>> Note: I'm resending this at Andrew's suggestion due to having originally sent
>>>>> it during LPC. I'm hoping its in a position where the feedback is minor enough
>>>>> that I can rework in time for v6.8, but so far haven't had any.
>>>>>
>>>>> Hi All,
>>>>>
>>>>> This is v7 of a series to implement small-sized THP for anonymous memory
>>>>> (previously called "large anonymous folios"). The objective of this is to
>>>>
>>>> I'm still against small-sized THP.  We've now got people asking whether
>>>> the THP counters should be updated when dealing with large folios that
>>>> are smaller than PMD sized.  It's sowing confusion, and we should go
>>>> back to large anon folios as a name.
>>>>
>>>
>>> I disagree.
>>>
>>> https://lore.kernel.org/all/65dbdf2a-9281-a3c3-b7e3-a79c5b60b357@redhat.com/
>>
>> And yet:
>> https://lore.kernel.org/linux-mm/20231106193315.GB3661273@cmpxchg.org/
>>
>> "This is a small THP so we don't account it as a THP, we only account
>> normal THPs as THPs" is a bizarre position to take.
>>
>> Not to mention that saying a foo is a small huge baz is just bizarre.
>> Am I a small giant?  Or just a large human?
>
> I like that analogy. Yet, "small giant" sounds "bigger" in some way IMHO ;)
>
> I'll note that "small-sized THP" is just a temporary feature name, it won't be exposed as such to the user in sysfs etc. In a couple of years, it will be forgotten.
>
> To me it makes sense: it's a hugepage (not a page) but smaller compared to what we previously had. But again, there won't be a "small_thp" toggle anywhere.
>
> Long-term it's simply going to be a THP. Quoting from my writeup:
>
> "Nowadays, when somebody says that they are using hugetlb huge pages, the first question frequently is "which huge page size?". The same will
> happen with transparent huge pages I believe.".

I agree. Especially our ultimate goal is to auto-tune THP sizes to give the best
performance to user. Having a separate name for small sized THP is beneficial to
kernel developers, since we want to use the right THP size for right
workloads/scenarios. But for average user, it is better to keep interface
as simple as possible, so that they can just turn on THP and get good performance
boost. For ninja users, I assume they know differences between THP sizes to not
confuse themselves and we can expose fine tune interfaces if really necessary.

>
>
> Regarding the accounting: as I said a couple of times, "AnonHugePages" should have been called "AnonPmdMapped" or similar; that's what it really is: as soon as a THP is PTE-mapped, it's not accounted there. But we can't fix that I guess, unless we add some "world switch" for any workloads that would care about a different accounting.
>
> So we're really only concerned about:
> * AnonHugePages
> * ShmemHugePages
> * FileHugePages
>
> The question is if we really want to continue extending/adjusting the old meminfo interfaces and talk about how to perform accounting there.
>
> Because, as we learned, we might get a new file-based sysfs based interface, because Greg seems to be against exposing new values in the old single-file-based one.

I am not aware of this. And it is interesting. Do you have a pointer?

>
> In a new one, we have all freedom to expose what we actually want nowadays, and can just document that the old interface was designed with the assumption that there is only a single THP size.

This sounds like a good strategy and hopefully we could design the new THP interface
more future proof.

>
> ... like hugetlb, where we also only expose the "default hugetlb size" parameters for legacy reasons:
>
> HugePages_Total:       0
> HugePages_Free:        0
> HugePages_Rsvd:        0
> HugePages_Surp:        0
> Hugepagesize:       2048 kB
>
> -- 
> Cheers,
>
> David / dhildenb


--
Best Regards,
Yan, Zi

Download attachment "signature.asc" of type "application/pgp-signature" (855 bytes)