linux-kernel - Re: [PATCH v2] mm: Reduce memory bloat with THP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 25 Jan 2018 23:40:29 +0100
From:   Andrea Arcangeli <aarcange@...hat.com>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     Nitin Gupta <nitin.m.gupta@...cle.com>,
        Nitin Gupta <nitingupta910@...il.com>,
        steven.sistare@...cle.com,
        Andrew Morton <akpm@...ux-foundation.org>,
        Ingo Molnar <mingo@...nel.org>, Mel Gorman <mgorman@...e.de>,
        Nadav Amit <namit@...are.com>,
        Minchan Kim <minchan@...nel.org>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Vegard Nossum <vegard.nossum@...cle.com>,
        "Levin, Alexander (Sasha Levin)" <alexander.levin@...izon.com>,
        Mike Rapoport <rppt@...ux.vnet.ibm.com>,
        Hillf Danton <hillf.zj@...baba-inc.com>,
        Shaohua Li <shli@...com>,
        Anshuman Khandual <khandual@...ux.vnet.ibm.com>,
        David Rientjes <rientjes@...gle.com>,
        Rik van Riel <riel@...hat.com>, Jan Kara <jack@...e.cz>,
        Dave Jiang <dave.jiang@...el.com>,
        Jérôme Glisse <jglisse@...hat.com>,
        Matthew Wilcox <willy@...ux.intel.com>,
        Ross Zwisler <ross.zwisler@...ux.intel.com>,
        Hugh Dickins <hughd@...gle.com>, Tobin C Harding <me@...in.cc>,
        linux-kernel@...r.kernel.org, linux-mm@...ck.org
Subject: Re: [PATCH v2] mm: Reduce memory bloat with THP

On Thu, Jan 25, 2018 at 10:58:32AM +0100, Michal Hocko wrote:
> Ohh, absolutely. And that is why we have changed the default in upstream
> 444eb2a449ef ("mm: thp: set THP defrag by default to madvise and add a
> stall-free defrag option")

Agreed, that direct compaction change should already address the cases
quoted in the other URLs.

One of the URL is about using fork() to snapshot a nosql db state,
that one can't be helped by the above commit but it's still unrelated
to MADV_DONTNEED or memory bloat.

It would be possible to fully fix the use of fork() for snapshotting
without userfaultfd WP mode, by just adding an madvise that forces 4k
CoWs on top of 2M THP and to call it in the parent that keeps writing
to memory while the child is writing the readonly copy to disk, but I
believe userfaultfd WP will be way more optimal as it provides so many
other advantages (i.e. avoid fork() in the first place and use
pthread_create and be able to throttle on I/O and limit the max memory
usage to something less than x2 RAM without the risk of triggering the
OOM killer and have a ring that is written immediately to keep the mem
utilization low etc..).

Thanks,
Andrea