linux-kernel - Re: THP backed thread stacks

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZAc96GZXnGugA5ML@kernel.org>
Date:   Tue, 7 Mar 2023 15:36:40 +0200
From:   Mike Rapoport <rppt@...nel.org>
To:     Mike Kravetz <mike.kravetz@...cle.com>
Cc:     linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: THP backed thread stacks

Hi Mike,

On Mon, Mar 06, 2023 at 03:57:30PM -0800, Mike Kravetz wrote:
> One of our product teams recently experienced 'memory bloat' in their
> environment.  The application in this environment is the JVM which
> creates hundreds of threads.  Threads are ultimately created via
> pthread_create which also creates the thread stacks.  pthread attributes
> are modified so that stacks are 2MB in size.  It just so happens that
> due to allocation patterns, all their stacks are at 2MB boundaries.  The
> system has THP always set, so a huge page is allocated at the first
> (write) fault when libpthread initializes the stack.
> 
> It would seem that this is expected behavior.  If you set THP always,
> you may get huge pages anywhere.
> 
> However, I can't help but think that backing stacks with huge pages by
> default may not be the right thing to do.  Stacks by their very nature
> grow in somewhat unpredictable ways over time.  Using a large virtual
> space so that memory is allocated as needed is the desired behavior.
> 
> The only way to address their 'memory bloat' via thread stacks today is
> by switching THP to madvise.
> 
> Just wondering if there is anything better or more selective that can be
> done?  Does it make sense to have THP backed stacks by default?  If not,
> who would be best at disabling?  A couple thoughts:
> - The kernel could disable huge pages on stacks.  libpthread/glibc pass
>   the unused flag MAP_STACK.  We could key off this and disable huge pages.
>   However, I'm sure there is somebody somewhere today that is getting better
>   performance because they have huge pages backing their stacks.
> - We could push this to glibc/libpthreads and have them use
>   MADV_NOHUGEPAGE on thread stacks.  However, this also has the potential
>   of regressing performance if somebody somewhere is getting better
>   performance due to huge pages.
> - Other thoughts?

Push this to the application? :)

Something like pthread_attr_getstack() + madvice(MADV_NOHUGEPAGE) will do
the job, no?
 
> Perhaps this is just expected behavior of THP always which is unfortunate
> in this situation.
> -- 
> Mike Kravetz
> 

-- 
Sincerely yours,
Mike.