linux-kernel - Re: [RESEND PATCH v7 00/10] Small-sized THP for anonymous memory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 29 Nov 2023 11:46:53 -0800
From:   John Hubbard <jhubbard@...dia.com>
To:     Ryan Roberts <ryan.roberts@....com>,
        David Hildenbrand <david@...hat.com>,
        Alistair Popple <apopple@...dia.com>
Cc:     Matthew Wilcox <willy@...radead.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Yin Fengwei <fengwei.yin@...el.com>,
        Yu Zhao <yuzhao@...gle.com>,
        Catalin Marinas <catalin.marinas@....com>,
        Anshuman Khandual <anshuman.khandual@....com>,
        Yang Shi <shy828301@...il.com>,
        "Huang, Ying" <ying.huang@...el.com>, Zi Yan <ziy@...dia.com>,
        Luis Chamberlain <mcgrof@...nel.org>,
        Itaru Kitayama <itaru.kitayama@...il.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        David Rientjes <rientjes@...gle.com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Hugh Dickins <hughd@...gle.com>,
        Kefeng Wang <wangkefeng.wang@...wei.com>, linux-mm@...ck.org,
        linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org
Subject: Re: [RESEND PATCH v7 00/10] Small-sized THP for anonymous memory

On 11/29/23 01:59, Ryan Roberts wrote:
...
>>>> Regarding new stats, maybe an interface that indicates the actual sizes would be
>>>> best. As discussed, extending the existing single-large-file statistics might
>>>> not be possible and we'd have to come up with a new interface, that maybe
>>>> completely lacks "AnonHugePages" and directly goes for the individual sizes.
>>>
>>> Yes, but I think we are agreed this is future work.
>>>
>>
>> We do want to have at least some way to verify that mTHP is active from
>> day 0, though.
> 
> Could you clarify what you mean by "active"?

I was thinking of the *pte* counters that we had in v6, in /proc/vmstat and
/proc/meminfo. I missed those, they were helpful in confirming that the test
was actually using the new feature. It's easy to misconfigure these tests
because there are so many settings (in addition to kernel settings), and
people were having some difficulty.

> 
> Current plan is that there will be a per-size
> transparent_hugepage/hugepages-<size>kB/enabled sysfs file that can be querried
> to see if the size is enabled (available for the kernel to use).
> 
> But for this initial submission, we previously agreed (well, at least David and
> I) that not having a full set of stats is not a problem - they can come later.
> So the only way to verify that the kernel is allocating and mapping a particular
> THP size is to parse /proc/<pid>pagemap and look at the PFNs for now. Is that
> sufficient?
> 

ugh, that's a little rough for just a command line sysadmin or QA 
person, isn't it?
Still, I expect we can survive without it for an initial release.

thanks,
-- 
John Hubbard
NVIDIA