linux-kernel - Re: [PATCH v4] mm,thp,shmem: limit shmem THP alloc gfp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:   Sat, 24 Oct 2020 13:53:03 -0400
From:   Rik van Riel <riel@...riel.com>
To:     Matthew Wilcox <willy@...radead.org>
Cc:     Hugh Dickins <hughd@...gle.com>, Yu Xu <xuyu@...ux.alibaba.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Mel Gorman <mgorman@...e.de>,
        Andrea Arcangeli <aarcange@...hat.com>,
        linux-kernel@...r.kernel.org, kernel-team@...com,
        linux-mm@...ck.org, Vlastimil Babka <vbabka@...e.cz>,
        Michal Hocko <mhocko@...e.com>
Subject: Re: [PATCH v4] mm,thp,shmem: limit shmem THP alloc gfp_mask

On Sat, 2020-10-24 at 03:09 +0100, Matthew Wilcox wrote:
> On Fri, Oct 23, 2020 at 08:48:04PM -0400, Rik van Riel wrote:
> > The allocation flags of anonymous transparent huge pages can be
> > controlled
> > through the files in /sys/kernel/mm/transparent_hugepage/defrag,
> > which can
> > help the system from getting bogged down in the page reclaim and
> > compaction
> > code when many THPs are getting allocated simultaneously.
> > 
> > However, the gfp_mask for shmem THP allocations were not limited by
> > those
> > configuration settings, and some workloads ended up with all CPUs
> > stuck
> > on the LRU lock in the page reclaim code, trying to allocate dozens
> > of
> > THPs simultaneously.
> > 
> > This patch applies the same configurated limitation of THPs to
> > shmem
> > hugepage allocations, to prevent that from happening.
> > 
> > This way a THP defrag setting of "never" or "defer+madvise" will
> > result
> > in quick allocation failures without direct reclaim when no 2MB
> > free
> > pages are available.
> > 
> > With this patch applied, THP allocations for tmpfs will be a little
> > more aggressive than today for files mmapped with MADV_HUGEPAGE,
> > and a little less aggressive for files that are not mmapped or
> > mapped without that flag.
> 
> How about this code path though?
> 
> shmem_get_pages() [ in i915 ]
>   shmem_read_mapping_page_gfp(__GFP_NORETRY | __GFP_NOWARN)
>     shmem_getpage_gfp()
>       shmem_alloc_and_acct_page()
>         shmem_alloc_hugepage()
> 
> I feel like the NORETRY from i915 should override whatever is set
> in sysfs for anon THPs.  What do others think?

It looks like currently the only way to get a THP
allocation with __GFP_DIRECT_RECLAIM and without
__GFP_NORETRY (which does nothing without
__GFP_DIRECT_RECLAIM) is to explicitly do an
madvise MADV_HUGEPAGE on a VMA.

I am not convinced the i915 driver should
override a userspace madvise.

-- 
All Rights Reversed.

Download attachment "signature.asc" of type "application/pgp-signature" (489 bytes)