linux-kernel - Re: [rfc] mm, hugetlb: allow hugepage allocations to excessively reclaim

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <3c220179-09cc-569f-8cfa-8e2ab5212faa@oracle.com>
Date:   Mon, 7 Oct 2019 15:15:32 -0700
From:   Mike Kravetz <mike.kravetz@...cle.com>
To:     David Rientjes <rientjes@...gle.com>,
        Michal Hocko <mhocko@...nel.org>
Cc:     Vlastimil Babka <vbabka@...e.cz>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Andrea Arcangeli <aarcange@...hat.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Mel Gorman <mgorman@...e.de>,
        "Kirill A. Shutemov" <kirill@...temov.name>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Linux-MM <linux-mm@...ck.org>
Subject: Re: [rfc] mm, hugetlb: allow hugepage allocations to excessively
 reclaim

On 10/4/19 11:02 AM, David Rientjes wrote:
> On Fri, 4 Oct 2019, Michal Hocko wrote:
> 
>> Requesting the userspace to drop _all_ page cache in order allocate a
>> number of hugetlb pages or any other affected __GFP_RETRY_MAYFAIL
>> requests is simply not reasonable IMHO.
> 
> It can be used as a fallback when writing to nr_hugepages and the amount 
> allocated did not match expectation.  Again, I'll defer all of this to 
> Mike when he returns: he expressed his preference, I suggested an 
> alternative to consider, and he can make the decision to ack or nack this 
> patch because he has a better understanding of that expectation from users 
> who use hugetlb pages.

I believe these modifications to commit b39d0ee2632d are absolutely necessary
to maintain expected hugetlbfs functionality.  Michal's simple test in the
rewritten commit message shows the type of regressions that I expect some
hugetlbfs users to experience.  The expectation today is that the kernel will
try hard to allocate the requested number of hugetlb pages.  These pages are
often used for very long running processes.  Therefore, the tradeoff of more
reclaim (and compaction) activity up front to create the pages is generally
acceptable.

My apologies if the 'testing' I did in [1] was taken as an endorsement of
b39d0ee2632d working well with hugetlbfs.  It was a limited test that I knew
did not cover all cases.  Therefore, I suggested that if b39d0ee2632d went
forward there should be an exception for __GFP_RETRY_MAYFAIL requests.

[1] https://lkml.kernel.org/r/3468b605-a3a9-6978-9699-57c52a90bd7e@oracle.com
-- 
Mike Kravetz