linux-kernel - Re: [PATCH v3 3/6] mm: shmem: add multi-size THP sysfs interface for anonymous shmem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <bba3ab3d-6e9c-4c62-9c85-3a3d338d9230@linux.alibaba.com>
Date: Sun, 2 Jun 2024 12:36:19 +0800
From: Baolin Wang <baolin.wang@...ux.alibaba.com>
To: wang wei <a929244872@....com>
Cc: akpm@...ux-foundation.org, hughd@...gle.com, willy@...radead.org,
 david@...hat.com, wangkefeng.wang@...wei.com, ying.huang@...el.com,
 21cnbao@...il.com, ryan.roberts@....com, shy828301@...il.com,
 ziy@...dia.com, ioworker0@...il.com, da.gomez@...sung.com,
 p.raghav@...sung.com, linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v3 3/6] mm: shmem: add multi-size THP sysfs interface for
 anonymous shmem



On 2024/6/1 11:29, wang wei wrote:
> At 2024-05-30 10:04:14, "Baolin Wang" <baolin.wang@...ux.alibaba.com> wrote:
> 
>>To support the use of mTHP with anonymous shmem, add a new sysfs interface
>>'shmem_enabled' in the '/sys/kernel/mm/transparent_hugepage/hugepages-kB/'
>>directory for each mTHP to control whether shmem is enabled for that mTHP,
>>with a value similar to the top level 'shmem_enabled', which can be set to:
>>"always", "inherit (to inherit the top level setting)", "within_size", "advise",
>>"never", "deny", "force". These values follow the same semantics as the top
>>level, except the 'deny' is equivalent to 'never', and 'force' is equivalent
>>to 'always' to keep compatibility.
>>
>>By default, PMD-sized hugepages have enabled="inherit" and all other hugepage
>>sizes have enabled="never" for '/sys/kernel/mm/transparent_hugepage/hugepages-xxkB/shmem_enabled'.
>>
>>In addition, if top level value is 'force', then only PMD-sized hugepages
>>have enabled="inherit", otherwise configuration will be failed and vice versa.
>>That means now we will avoid using non-PMD sized THP to override the global
>>huge allocation.
>>
>>Signed-off-by: Baolin Wang <baolin.wang@...ux.alibaba.com>
>>---
>> Documentation/admin-guide/mm/transhuge.rst | 29 +++++++
>> include/linux/huge_mm.h                    | 10 +++
>> mm/huge_memory.c                           | 11 +--
>> mm/shmem.c                                 | 96 ++++++++++++++++++++++
>> 4 files changed, 138 insertions(+), 8 deletions(-)
>>
>>diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst
>>index d414d3f5592a..659459374032 100644
>>--- a/Documentation/admin-guide/mm/transhuge.rst
>>+++ b/Documentation/admin-guide/mm/transhuge.rst
>>@@ -332,6 +332,35 @@ deny
>> force
>>     Force the huge option on for all - very useful for testing;
>> 
>>+Anonymous shmem can also use "multi-size THP" (mTHP) by adding a new sysfs knob
>>+to control mTHP allocation: /sys/kernel/mm/transparent_hugepage/hugepages-<size>kB/shmem_enabled.
>>+Its value for each mTHP is essentially consistent with the global setting, except
>>+for the addition of 'inherit' to ensure compatibility with the global settings.
>>+always
>>+    Attempt to allocate <size> huge pages every time we need a new page;
>>+
>>+inherit
>>+    Inherit the top-level "shmem_enabled" value. By default, PMD-sized hugepages
>>+    have enabled="inherit" and all other hugepage sizes have enabled="never";
>>+
>>+never
>>+    Do not allocate <size> huge pages;
>>+
>>+within_size
>>+    Only allocate <size> huge page if it will be fully within i_size.
>>+    Also respect fadvise()/madvise() hints;
>>+
>>+advise
>>+    Only allocate <size> huge pages if requested with fadvise()/madvise();
>>+
>>+deny
>>+    Has the same semantics as 'never', now mTHP allocation policy is only
>>+    used for anonymous shmem and no not override tmpfs.
>>+
>>+force
>>+    Has the same semantics as 'always', now mTHP allocation policy is only
>>+    used for anonymous shmem and no not override tmpfs.
>  >+
> 
> I just briefly reviewed the discussion about the value of 
> hugepages-<size>kB/shmem_enabled
> in V1 [PATCH 5/8]. Is there a conclusion now? Maybe I left out some 
> important information.

You can refer to the this patch's commit message and documentation, 
which are based on the conclusions of previous discussions.

In addition, you can also read more discussions from the last bi-weekly 
MM meeting[1], summarized by David.

[1] 
https://lore.kernel.org/all/f1783ff0-65bd-4b2b-8952-52b6822a0835@redhat.com/#t