linux-kernel - [RFC 0/2] mm: introduce THP deferred setting

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240729222727.64319-1-npache@redhat.com>
Date: Mon, 29 Jul 2024 16:27:25 -0600
From: Nico Pache <npache@...hat.com>
To: linux-kernel@...r.kernel.org,
	linux-doc@...r.kernel.org,
	linux-mm@...ck.org
Cc: Andrew Morton <akpm@...ux-foundation.org>,
	David Hildenbrand <david@...hat.com>,
	Matthew Wilcox <willy@...radead.org>,
	Barry Song <baohua@...nel.org>,
	Ryan Roberts <ryan.roberts@....com>,
	Baolin Wang <baolin.wang@...ux.alibaba.com>,
	Lance Yang <ioworker0@...il.com>,
	Peter Xu <peterx@...hat.com>,
	Zi Yan <ziy@...dia.com>,
	Rafael Aquini <aquini@...hat.com>,
	Andrea Arcangeli <aarcange@...hat.com>,
	Jonathan Corbet <corbet@....net>
Subject: [RFC 0/2] mm: introduce THP deferred setting

We've seen cases were customers switching from RHEL7 to RHEL8 see a
significant increase in the memory footprint for the same workloads.

Through our investigations we found that a large contributing factor to
the increase in RSS was an increase in THP usage.

For workloads like MySQL, or when using allocators like jemalloc, it is
often recommended to set /transparent_hugepages/enabled=never. This is
in part due to performance degradations and increased memory waste.

This series introduces enabled=defer, this setting acts as a middle
ground between always and madvise. If the mapping is MADV_HUGEPAGE, the
page fault handler will act normally, making a hugepage if possible. If
the allocation is not MADV_HUGEPAGE, then the page fault handler will
default to the base size allocation. The caveat is that khugepaged can
still operate on pages thats not MADV_HUGEPAGE.

This allows for two things... one, applications specifically designed to
use hugepages will get them, and two, applications that don't use
hugepages can still benefit from them without aggressively inserting
THPs at every possible chance. This curbs the memory waste, and defers
the use of hugepages to khugepaged. Khugepaged can then scan the memory
for eligible collapsing.

Admins may want to lower max_ptes_none, if not, khugepaged may 
aggressively collapse single allocations into hugepages.

RFC note
==========
Im not sure if im missing anything related to the mTHP
changes. I think now that we have hugepage_pmd_enabled in
commit 00f58104202c ("mm: fix khugepaged activation policy") everything
should work as expected.

Nico Pache (2):
  mm: defer THP insertion to khugepaged
  mm: document transparent_hugepage=defer usage

 Documentation/admin-guide/mm/transhuge.rst | 18 ++++++++++---
 include/linux/huge_mm.h                    | 15 +++++++++--
 mm/huge_memory.c                           | 31 +++++++++++++++++++---
 3 files changed, 55 insertions(+), 9 deletions(-)

Cc: Andrew Morton <akpm@...ux-foundation.org>
Cc: David Hildenbrand <david@...hat.com>
Cc: Matthew Wilcox <willy@...radead.org>
Cc: Barry Song <baohua@...nel.org>
Cc: Ryan Roberts <ryan.roberts@....com>
Cc: Baolin Wang <baolin.wang@...ux.alibaba.com>
Cc: Lance Yang <ioworker0@...il.com>
Cc: Peter Xu <peterx@...hat.com>
Cc: Zi Yan <ziy@...dia.com>
Cc: Rafael Aquini <aquini@...hat.com>
Cc: Andrea Arcangeli <aarcange@...hat.com>
Cc: Jonathan Corbet <corbet@....net>
-- 
2.45.2