linux-kernel - Re: [PATCH v5 3/5] mm: LARGE_ANON

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <341a66e1-c71a-24e8-3eba-6c2fa16babe0@redhat.com>
Date:   Fri, 1 Sep 2023 16:40:59 +0200
From:   David Hildenbrand <david@...hat.com>
To:     Matthew Wilcox <willy@...radead.org>
Cc:     "Huang, Ying" <ying.huang@...el.com>,
        Ryan Roberts <ryan.roberts@....com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Yin Fengwei <fengwei.yin@...el.com>,
        Yu Zhao <yuzhao@...gle.com>,
        Catalin Marinas <catalin.marinas@....com>,
        Anshuman Khandual <anshuman.khandual@....com>,
        Yang Shi <shy828301@...il.com>, Zi Yan <ziy@...dia.com>,
        Luis Chamberlain <mcgrof@...nel.org>,
        Itaru Kitayama <itaru.kitayama@...il.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        linux-mm@...ck.org, linux-kernel@...r.kernel.org,
        linux-arm-kernel@...ts.infradead.org,
        David Rientjes <rientjes@...gle.com>
Subject: Re: [PATCH v5 3/5] mm: LARGE_ANON_FOLIO for improved performance

On 31.08.23 14:29, Matthew Wilcox wrote:
> On Thu, Aug 31, 2023 at 09:57:46AM +0200, David Hildenbrand wrote:
>> As raised in another mail, we can then discuss
>> * how we want to call this feature (transparent large pages? there is
>>    the concern that "THP" might confuse users. Maybe we can consider
>>    "large" the more generic version and "huge" only PMD-size, TBD)
>> * how to expose it in stats towards the user (e.g., /proc/meminfo)
>> * which minimal toggles we want
>>
>> I think there *really* has to be a way to disable it for a running system,
>> otherwise no distro will dare pulling it in, even after we figured out the
>> other stuff.
>>
>> Note that for the pagecache, large folios can be disabled and distributions
>> are actively making use of that.
> 
> You can't.  Well, you can for shmem/tmpfs, but you have to edit the
> source code or disable CONFIG_TRANSPARENT_HUGEPAGE to disable it for XFS.

While you cannot currently control the exact allocation granularity, you 
can limit the effect it has on apps that are sensitive to rss (memcg) 
changes. See

See as an example:

https://www.suse.com/support/kb/doc/?id=000019017


For the pagecache you arguably don't care, because the assumption is 
that you can reclaim that memory anytime. So even if you allocated 2 MiB 
and only mapped 4 KiB into the process, so far you can work around that 
breaking existing setups by setting fault_around_bytes.

For anon memory that's quite different (as of now and in the forseable 
future). For that reason, we have all these knobs to teach THP to not 
over-allocate memory (e.g., no thp on page fault, don't fill holes in 
khugepaged).

I know that Dave R. can share quite some details about that.

-- 
Cheers,

David / dhildenb