Date:   Wed, 24 Jan 2018 19:47:36 -0500
From:   "Zi Yan" <zi.yan@...rutgers.edu>
To:     "Nitin Gupta" <nitin.m.gupta@...cle.com>
Cc:     "Michal Hocko" <mhocko@...nel.org>,
        "Nitin Gupta" <nitingupta910@...il.com>, steven.sistare@...cle.com,
        "Andrew Morton" <akpm@...ux-foundation.org>,
        "Ingo Molnar" <mingo@...nel.org>, "Mel Gorman" <mgorman@...e.de>,
        "Nadav Amit" <namit@...are.com>,
        "Minchan Kim" <minchan@...nel.org>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        "Peter Zijlstra" <peterz@...radead.org>,
        "Vegard Nossum" <vegard.nossum@...cle.com>,
        "Levin, Alexander" <alexander.levin@...izon.com>,
        "Mike Rapoport" <rppt@...ux.vnet.ibm.com>,
        "Hillf Danton" <hillf.zj@...baba-inc.com>,
        "Shaohua Li" <shli@...com>,
        "Anshuman Khandual" <khandual@...ux.vnet.ibm.com>,
        "Andrea Arcangeli" <aarcange@...hat.com>,
        "David Rientjes" <rientjes@...gle.com>,
        "Rik van Riel" <riel@...hat.com>, "Jan Kara" <jack@...e.cz>,
        "Dave Jiang" <dave.jiang@...el.com>,
        "Jérôme Glisse" <jglisse@...hat.com>,
        "Matthew Wilcox" <willy@...ux.intel.com>,
        "Ross Zwisler" <ross.zwisler@...ux.intel.com>,
        "Hugh Dickins" <hughd@...gle.com>, "Tobin C Harding" <me@...in.cc>,
        linux-kernel@...r.kernel.org, linux-mm@...ck.org
Subject: Re: [PATCH v2] mm: Reduce memory bloat with THP


>
>>> With this change, whenever an application issues MADV_DONTNEED on a
>>> memory region, the region is marked as "space-efficient". For such
>>> regions, a hugepage is not immediately allocated on first write.
>>
>> Kirill didn't like it in the previous version and I do not like this
>> either. You are adding a very subtle side effect which might completely
>> unexpected. Consider userspace memory allocator which uses MADV_DONTNEED
>> to free up unused memory. Now you have put it out of THP usage
>> basically.
>>
>
> Userpsace may want a region to be considered by khugepaged while opting
> out of hugepage allocation on first touch. Asking userspace memory
> allocators to have to track and reclaim unused parts of a THP allocated
> hugepage does not seems right, as the kernel can use simple userspace
> hints to avoid allocating extra memory in the first place.
>
> I agree that this patch is adding a subtle side-effect which may take
> some applications by surprise. However, I often see the opposite too:
> for many workloads, disabling THP is the first advise as this aggressive
> allocation of hugepages on first touch is unexpected and is too
> wasteful. For e.g.:
>
> 1) Disabling THP for TokuDB (Storage engine for MySQL, MariaDB)
> http://www.chriscalender.com/disabling-transparent-hugepages-for-tokudb/
>
> 2) Disable THP on MongoDB
> https://docs.mongodb.com/manual/tutorial/transparent-huge-pages/
>
> 3) Disable THP for Couchbase Server
> https://blog.couchbase.com/often-overlooked-linux-os-tweaks/
>
> 4) Redis
> http://antirez.com/news/84
>
>
>> If the memory is used really scarce then we have MADV_NOHUGEPAGE.
>>
>
> It's not really about memory scarcity but a more efficient use of it.
> Applications may want hugepage benefits without requiring any changes to
> app code which is what THP is supposed to provide, while still avoiding
> memory bloat.
>

I read these links and find that there are mainly two complains:
1. THP causes latency spikes, because direction compaction slows down THP allocation,
2. THP bloats memory footprint when jemalloc uses MADV_DONTNEED to return memory ranges smaller than
   THP size and fails because of THP.

The first complain is not related to this patch.

For second one, at least with recent kernels, MADV_DONTNEED splits THPs and returns the memory range you
specified in madvise(). Am I missing anything?


—
Best Regards,
Yan Zi

Download attachment "signature.asc" of type "application/pgp-signature" (558 bytes)

Powered by blists - more mailing lists