Message-ID: <CAGsJ_4xhcjt6QMOfqsGK8_EpPHORAeHmPLusmirS1vg_ss1=tA@mail.gmail.com>
Date: Tue, 27 Aug 2024 14:21:42 +1200
From: Barry Song <21cnbao@...il.com>
To: Lokesh Gidra <lokeshgidra@...gle.com>
Cc: Suren Baghdasaryan <surenb@...gle.com>, Nicolas Geoffray <ngeoffray@...gle.com>, 
	Michal Hocko <mhocko@...e.com>, gaoxu <gaoxu2@...or.com>, 
	Andrew Morton <akpm@...ux-foundation.org>, "linux-mm@...ck.org" <linux-mm@...ck.org>, 
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>, Shaohua Li <shli@...com>, 
	yipengxiang <yipengxiang@...or.com>, fengbaopeng <fengbaopeng@...or.com>, 
	Kalesh Singh <kaleshsingh@...gle.com>
Subject: Re: [PATCH v2] mm: add lazyfree folio to lru tail

On Tue, Aug 27, 2024 at 12:12 PM Lokesh Gidra <lokeshgidra@...gle.com> wrote:
>
> On Mon, Aug 26, 2024 at 12:55 PM Barry Song <21cnbao@...il.com> wrote:
> >
> > On Tue, Aug 27, 2024 at 4:37 AM Lokesh Gidra <lokeshgidra@...gle.com> wrote:
> > >
> > > Thanks Suren for looping in
> > >
> > > On Fri, Aug 23, 2024 at 4:39 PM Suren Baghdasaryan <surenb@...gle.com> wrote:
> > > >
> > > > On Wed, Aug 21, 2024 at 2:47 PM Barry Song <21cnbao@...il.com> wrote:
> > > > >
> > > > > On Wed, Aug 21, 2024 at 8:46 PM Michal Hocko <mhocko@...e.com> wrote:
> > > > > >
> > > > > > On Fri 16-08-24 07:48:01, gaoxu wrote:
> > > > > > > Replace lruvec_add_folio with lruvec_add_folio_tail in the lru_lazyfree_fn:
> > > > > > > 1. The lazy-free folio is added to the LRU_INACTIVE_FILE list. If it's
> > > > > > >    moved to the LRU tail, it allows for faster release lazy-free folio and
> > > > > > >    reduces the impact on file refault.
> > > > > >
> > > > > > This has been discussed when MADV_FREE was introduced. The question was
> > > > > > whether this memory has a lower priority than other inactive memory that
> > > > > > has been marked that way longer ago. Also consider several MADV_FREE
> > > > > > users should they be LIFO from the reclaim POV?
> > >
> > > Thinking from the user's perspective, it seems to me that FIFO within
> > > MADV_FREE'ed pages makes more sense. As a user I expect the longer a
> > > MADV_FREE'ed page hasn't been touched, the chances are higher that it
> > > may not be around anymore.
> > > > >
> >
> > Hi Lokesh,
> > Thanks!
> >
> > > > > The priority of this memory compared to other inactive memory that has been
> > > > > marked for a longer time likely depends on the user's expectations - How soon
> > > > > do users expect MADV_FREE to be reclaimed compared with old file folios.
> > > > >
> > > > > art guys moved to MADV_FREE from MADV_DONTNEED without any
> > > > > useful performance data and reason in the changelog:
> > > > > https://android-review.googlesource.com/c/platform/art/+/2633132
> > > > >
> > > > > Since art is the Android Java heap, it can be quite large. This increases the
> > > > > likelihood of packing the file LRU and reduces the chances of reclaiming
> > > > > anonymous memory, which could result in more file re-faults while helping
> > > > > anonymous folio persist longer in memory.
> > >
> > > Individual heaps of android apps are not big, and even in there we
> > > don't call MADV_FREE on the entire heap.
> >
> > How do you define "Individual heaps of android apps", do you know the usual
> > total_size for a phone with memory pressure by running multiple apps and how
> > much for each app?
> >
> Every app is a separate process and therefore has its own private ART
> heap. Those numbers that you are asking vary drastically. But here's
> what I can tell you:
>
> Max heap size for an app is 512MB typically. But it is rarely entirely

On my phone, I am seeing a VMA named "dalvik-main space", its
virtual address is 0x14000000-0x34000000 and its size is 0x20000000
(512MB). I guess this is exactly the ART heap we are talking about?

> used. Typical heap usage is 50MB to 250MB. But as I said, not all of

Thank you! Considering we might have dozens of apps running in the
background and foreground, will the total size of the ART heaps on a
phone still be large?

> it is MADV_FREE'ed. Only those pages which are freed after GC
> compaction are.

Are you saying that some memory in the ART heap is marked with
MADV_DONTNEED instead of MADV_FREE? If so, when did this happen?
Alternatively, is my understanding incorrect and you are referring to memory
that is neither MADV_DONTNEED nor MADV_FREE?

> > > > >
> > > > > I am really curious why art guys have moved to MADV_FREE if we have
> > > > > an approach to reach them.
> > >
> > > Honestly, it makes little sense as a user that calling MADV_FREE on an
> > > anonymous mapping will impact file LRU. That was never the intention
> > > with our ART change.
> > >
> >
> > This is just how MADV_FREE is implemented in the kernel, this kind of lazyfree
> > anon folios are moved to file but *NOT* anon LRU.
> >
> > > From our perspective, once a set of pages are MADV_FREE'ed, they are
> > > like a page-cache. It gives an opportunity, without hurting memory
> > > use, to avoid overhead of page-faults, which happen frequently after
> > > GC is done on running apps.
> > >
> > > IMHO, within LRU_INACTIVE_FILE, MADV_FREE'ed pages should be
> > > prioritized for reclamation over file ones.
> >
> > This is exactly what this patch is doing, putting lazyfree anon folios
> > to the tail of file LRU so that they can be reclaimed earlier than file
> > folios. But the question is: is the requirement "MADV_FREE'ed pages
> > should be prioritized for reclamation over file ones" universally true for
> > all other non-Android users?
> >
> That's definitely an important question to get answered. But putting
> my users hat on again, by explicitly MADV_FREE'ing we ask for that
> behavior. IMHO, MADV_FREE'ed pages should be the first ones to be
> reclaimed on memory pressure.

Thanks for clarification! I'd also like to collect some performance data with
this patch.

> > > >
> > > > Adding Lokesh.
> > > > Lokesh, could you please comment on the reasoning behind the above
> > > > mentioned change?
> > >
> > > Adding Nicolas as well, in case he wants to add something.
> > > >
> > > > >
> > > > > >
> > > > > > --
> > > > > > Michal Hocko
> > > > > > SUSE Labs
> > > > > >
> > > > >
> >

Thanks
Barry

Powered by blists - more mailing lists