linux-kernel - Re: [PATCH v2] mm: Reduce memory bloat with THP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20180119124957.GA6584@dhcp22.suse.cz>
Date:   Fri, 19 Jan 2018 13:49:57 +0100
From:   Michal Hocko <mhocko@...nel.org>
To:     Nitin Gupta <nitingupta910@...il.com>
Cc:     steven.sistare@...cle.com, Nitin Gupta <nitin.m.gupta@...cle.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Ingo Molnar <mingo@...nel.org>, Mel Gorman <mgorman@...e.de>,
        Nadav Amit <namit@...are.com>,
        Minchan Kim <minchan@...nel.org>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Vegard Nossum <vegard.nossum@...cle.com>,
        "Levin, Alexander (Sasha Levin)" <alexander.levin@...izon.com>,
        Mike Rapoport <rppt@...ux.vnet.ibm.com>,
        Hillf Danton <hillf.zj@...baba-inc.com>,
        Shaohua Li <shli@...com>,
        Anshuman Khandual <khandual@...ux.vnet.ibm.com>,
        Andrea Arcangeli <aarcange@...hat.com>,
        David Rientjes <rientjes@...gle.com>,
        Rik van Riel <riel@...hat.com>, Jan Kara <jack@...e.cz>,
        Dave Jiang <dave.jiang@...el.com>,
        Jérôme Glisse <jglisse@...hat.com>,
        Matthew Wilcox <willy@...ux.intel.com>,
        Ross Zwisler <ross.zwisler@...ux.intel.com>,
        Hugh Dickins <hughd@...gle.com>, Tobin C Harding <me@...in.cc>,
        linux-kernel@...r.kernel.org, linux-mm@...ck.org
Subject: Re: [PATCH v2] mm: Reduce memory bloat with THP

On Thu 18-01-18 15:33:16, Nitin Gupta wrote:
> From: Nitin Gupta <nitin.m.gupta@...cle.com>
> 
> Currently, if the THP enabled policy is "always", or the mode
> is "madvise" and a region is marked as MADV_HUGEPAGE, a hugepage
> is allocated on a page fault if the pud or pmd is empty.  This
> yields the best VA translation performance, but increases memory
> consumption if some small page ranges within the huge page are
> never accessed.

Yes, this is true but hardly unexpected for MADV_HUGEPAGE or THP always
users.
 
> An alternate behavior for such page faults is to install a
> hugepage only when a region is actually found to be (almost)
> fully mapped and active.  This is a compromise between
> translation performance and memory consumption.  Currently there
> is no way for an application to choose this compromise for the
> page fault conditions above.

Is that really true? We have /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none
This is not reflected during the PF of course but you can control the
behavior there as well. Either by the global setting or a per proces
prctl.

> With this change, whenever an application issues MADV_DONTNEED on a
> memory region, the region is marked as "space-efficient". For such
> regions, a hugepage is not immediately allocated on first write.

Kirill didn't like it in the previous version and I do not like this
either. You are adding a very subtle side effect which might completely
unexpected. Consider userspace memory allocator which uses MADV_DONTNEED
to free up unused memory. Now you have put it out of THP usage
basically.

If the memory is used really scarce then we have MADV_NOHUGEPAGE.
-- 
Michal Hocko
SUSE Labs