linux-kernel - Re: [RFC 0/3] mm: Discard lazily freed pages when migrating

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 3 Mar 2020 09:58:05 +0100
From:   Michal Hocko <mhocko@...nel.org>
To:     "Huang, Ying" <ying.huang@...el.com>
Cc:     Mel Gorman <mgorman@...e.de>, David Hildenbrand <david@...hat.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Matthew Wilcox <willy@...radead.org>,
        Andrew Morton <akpm@...ux-foundation.org>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org, Vlastimil Babka <vbabka@...e.cz>,
        Zi Yan <ziy@...dia.com>, Peter Zijlstra <peterz@...radead.org>,
        Dave Hansen <dave.hansen@...ux.intel.com>,
        Minchan Kim <minchan@...nel.org>,
        Hugh Dickins <hughd@...gle.com>,
        Alexander Duyck <alexander.duyck@...il.com>
Subject: Re: [RFC 0/3] mm: Discard lazily freed pages when migrating

On Tue 03-03-20 16:47:54, Huang, Ying wrote:
> Michal Hocko <mhocko@...nel.org> writes:
> 
> > On Tue 03-03-20 09:51:56, Huang, Ying wrote:
> >> Mel Gorman <mgorman@...e.de> writes:
> >> > On Mon, Mar 02, 2020 at 07:23:12PM +0800, Huang, Ying wrote:
> >> >> If some applications cannot tolerate the latency incurred by the memory
> >> >> allocation and zeroing.  Then we cannot discard instead of migrate
> >> >> always.  While in some situations, less memory pressure can help.  So
> >> >> it's better to let the administrator and the application choose the
> >> >> right behavior in the specific situation?
> >> >> 
> >> >
> >> > Is there an application you have in mind that benefits from discarding
> >> > MADV_FREE pages instead of migrating them?
> >> >
> >> > Allowing the administrator or application to tune this would be very
> >> > problematic. An application would require an update to the system call
> >> > to take advantage of it and then detect if the running kernel supports
> >> > it. An administrator would have to detect that MADV_FREE pages are being
> >> > prematurely discarded leading to a slowdown and that is hard to detect.
> >> > It could be inferred from monitoring compaction stats and checking
> >> > if compaction activity is correlated with higher minor faults in the
> >> > target application. Proving the correlation would require using the perf
> >> > software event PERF_COUNT_SW_PAGE_FAULTS_MIN and matching the addresses
> >> > to MADV_FREE regions that were freed prematurely. That is not an obvious
> >> > debugging step to take when an application detects latency spikes.
> >> >
> >> > Now, you could add a counter specifically for MADV_FREE pages freed for
> >> > reasons other than memory pressure and hope the administrator knows about
> >> > the counter and what it means. That type of knowledge could take a long
> >> > time to spread so it's really very important that there is evidence of
> >> > an application that suffers due to the current MADV_FREE and migration
> >> > behaviour.
> >> 
> >> OK.  I understand that this patchset isn't a universal win, so we need
> >> some way to justify it.  I will try to find some application for that.
> >> 
> >> Another thought, as proposed by David Hildenbrand, it's may be a
> >> universal win to discard clean MADV_FREE pages when migrating if there are
> >> already memory pressure on the target node.  For example, if the free
> >> memory on the target node is lower than high watermark?
> >
> > This is already happening because if the target node is short on memory
> > it will start to reclaim and if MADV_FREE pages are at the tail of
> > inactive file LRU list then they will be dropped. Please note how that
> > follows proper aging and doesn't introduce any special casing. Really
> > MADV_FREE is an inactive cache for anonymous memory and we treat it like
> > inactive page cache. This is not carved in stone of course but it really
> > requires very good justification to change.
> 
> If my understanding were correct, the newly migrated clean MADV_FREE
> pages will be put at the head of inactive file LRU list instead of the
> tail.  So it's possible that some useful file cache pages will be
> reclaimed.

This is the case also when you migrate other pages, right? We simply
cannot preserve the aging.

-- 
Michal Hocko
SUSE Labs