| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <279d29ad-cbd6-4a0e-b904-0a19326334d1@gmail.com>
Date: Wed, 7 May 2025 17:09:30 +0100
From: Usama Arif <usamaarif642@...il.com>
To: Zi Yan <ziy@...dia.com>
Cc: Andrew Morton <akpm@...ux-foundation.org>, david@...hat.com,
linux-mm@...ck.org, hannes@...xchg.org, shakeel.butt@...ux.dev,
riel@...riel.com, baolin.wang@...ux.alibaba.com, lorenzo.stoakes@...cle.com,
Liam.Howlett@...cle.com, npache@...hat.com, ryan.roberts@....com,
linux-kernel@...r.kernel.org, kernel-team@...a.com,
Yafang Shao <laoar.shao@...il.com>
Subject: Re: [PATCH 0/1] prctl: allow overriding system THP policy to always
On 07/05/2025 16:57, Zi Yan wrote:
> On 7 May 2025, at 11:12, Usama Arif wrote:
>
>> On 07/05/2025 15:57, Zi Yan wrote:
>>> +Yafang, who is also looking at changing THP config at cgroup/container level.
>>>
>>> On 7 May 2025, at 10:00, Usama Arif wrote:
>>>
>>>> Allowing override of global THP policy per process allows workloads
>>>> that have shown to benefit from hugepages to do so, without regressing
>>>> workloads that wouldn't benefit. This will allow such types of
>>>> workloads to be run/stacked on the same machine.
>>>>
>>>> It also helps in rolling out hugepages in hyperscaler configurations
>>>> for workloads that benefit from them, where a single THP policy is
>>>> likely to be used across the entire fleet, and prctl will help override it.
>>>>
>>>> An advantage of doing it via prctl vs creating a cgroup specific
>>>> option (like /sys/fs/cgroup/test/memory.transparent_hugepage.enabled) is
>>>> that this will work even when there are no cgroups present, and my
>>>> understanding is there is a strong preference of cgroups controls being
>>>> hierarchical which usually means them having a numerical value.
>>>
>>> Hi Usama,
>>>
>>> Do you mind giving an example on how to change THP policy for a set of
>>> processes running in a container (under a cgroup)?
>>
>> Hi Zi,
>>
>> In our case, we create the processes in the cgroup via systemd. The way we will enable THP=always
>> for processes in a cgroup is in the same way we enable KSM for the cgroup.
>> The change in systemd would be very similar to the line in [1], where we would set prctl PR_SET_THP_ALWAYS
>> in exec-invoke.
>> This is at the start of the process, but you would already know at the start of the process
>> whether you want THP=always for it or not.
>>
>> [1] https://github.com/systemd/systemd/blob/2e72d3efafa88c1cb4d9b28dd4ade7c6ab7be29a/src/core/exec-invoke.c#L5045
>
> You also need to add a new systemd.directives, e.g., MemoryTHP, to
> pass the THP enablement or disablement info from a systemd config file.
> And if you find those processes do not benefit from using THPs,
> you can just change the new "MemoryTHP" config and restart the processes.
>
> Am I getting it? Thanks.
>
Yes, thats right. They would exactly the same as what we (Meta) do
for KSM. So have MemoryTHP similar to MemroryKSM [1] and if MemoryTHP is set,
the ExecContext->memory_thp would be set similar to memory_ksm [2], and when
that is set, the prctl will be called at exec_invoke of the process [3].
The systemd changes should be quite simple to do.
[1] https://github.com/systemd/systemd/blob/2e72d3efafa88c1cb4d9b28dd4ade7c6ab7be29a/man/systemd.exec.xml#L1978
[2] https://github.com/systemd/systemd/blob/2e72d3efafa88c1cb4d9b28dd4ade7c6ab7be29a/src/core/dbus-execute.c#L2151
[3] https://github.com/systemd/systemd/blob/2e72d3efafa88c1cb4d9b28dd4ade7c6ab7be29a/src/core/exec-invoke.c#L5045
>>>
>>> Yafang mentioned that the prctl approach would require restarting all running
>>> services[1] and other inflexiblities, so he proposed to use BPF to change THP
>>> policy[2]. I wonder if Yafang's issues also apply to your case and if you
>>> have a solution to them.
>>>
>>> Thanks.
>>>
>>> [1] https://lore.kernel.org/linux-mm/CALOAHbCXMi2GaZdHJaNLXxGsJf-hkDTrztsQiceaBcJ8d8p3cA@mail.gmail.com/
>>> [2] https://lore.kernel.org/linux-mm/20250429024139.34365-1-laoar.shao@gmail.com/
>>>>
>>>>
>>>> The output and code of test program is below:
>>>>
>>>> [root@vm4 vmuser]# echo madvise > /sys/kernel/mm/transparent_hugepage/enabled
>>>> [root@vm4 vmuser]# echo inherit > /sys/kernel/mm/transparent_hugepage/hugepages-2048kB/enabled
>>>> [root@vm4 vmuser]# ./a.out
>>>> Default THP setting:
>>>> THP is not set to 'always'.
>>>> PR_SET_THP_ALWAYS = 1
>>>> THP is set to 'always'.
>>>> PR_SET_THP_ALWAYS = 0
>>>> THP is not set to 'always'.
>>>>
>>>>
>>>> #include <stdio.h>
>>>> #include <stdlib.h>
>>>> #include <string.h>
>>>> #include <unistd.h>
>>>> #include <sys/mman.h>
>>>> #include <sys/prctl.h>
>>>>
>>>> #define PR_SET_THP_ALWAYS 78
>>>> #define SIZE 12 * (2 * 1024 * 1024) // 24 MB
>>>>
>>>> void check_smaps(void) {
>>>> FILE *file = fopen("/proc/self/smaps", "r");
>>>> if (!file) {
>>>> perror("fopen");
>>>> return;
>>>> }
>>>>
>>>> char line[256];
>>>> int is_hugepage = 0;
>>>> while (fgets(line, sizeof(line), file)) {
>>>> // if (strstr(line, "AnonHugePages:"))
>>>> // printf("%s\n", line);
>>>> if (strstr(line, "AnonHugePages:") && strstr(line, "24576 kB"))
>>>> {
>>>> // printf("%s\n", line);
>>>> is_hugepage = 1;
>>>> break;
>>>> }
>>>> }
>>>> fclose(file);
>>>> if (is_hugepage) {
>>>> printf("THP is set to 'always'.\n");
>>>> } else {
>>>> printf("THP is not set to 'always'.\n");
>>>> }
>>>> }
>>>>
>>>> void test_mmap_thp(void) {
>>>> char *buffer = (char *)mmap(NULL, SIZE, PROT_READ | PROT_WRITE,
>>>> MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
>>>> if (buffer == MAP_FAILED) {
>>>> perror("mmap");
>>>> return;
>>>> }
>>>> // Touch the memory to ensure it's allocated
>>>> memset(buffer, 0, SIZE);
>>>> check_smaps();
>>>> munmap(buffer, SIZE);
>>>> }
>>>>
>>>> int main() {
>>>> printf("Default THP setting: \n");
>>>> test_mmap_thp();
>>>> printf("PR_SET_THP_ALWAYS = 1 \n");
>>>> prctl(PR_SET_THP_ALWAYS, 1, NULL, NULL, NULL);
>>>> test_mmap_thp();
>>>> printf("PR_SET_THP_ALWAYS = 0 \n");
>>>> prctl(PR_SET_THP_ALWAYS, 0, NULL, NULL, NULL);
>>>> test_mmap_thp();
>>>>
>>>> return 0;
>>>> }
>>>>
>>>>
>>>> Usama Arif (1):
>>>> prctl: allow overriding system THP policy to always per process
>>>>
>>>> include/linux/huge_mm.h | 3 ++-
>>>> include/linux/mm_types.h | 7 ++-----
>>>> include/uapi/linux/prctl.h | 3 +++
>>>> kernel/sys.c | 16 ++++++++++++++++
>>>> tools/include/uapi/linux/prctl.h | 3 +++
>>>> .../perf/trace/beauty/include/uapi/linux/prctl.h | 3 +++
>>>> 6 files changed, 29 insertions(+), 6 deletions(-)
>>>>
>>>> --
>>>> 2.47.1
>>>
>>>
>>> --
>>> Best Regards,
>>> Yan, Zi
>
>
> --
> Best Regards,
> Yan, Zi
Powered by blists - more mailing lists