linux-kernel - Re: [PATCH RFC] mm: mglru: provide a separate list for lazyfree anon folios

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAGsJ_4xF6FuncfErMmMotkOYNGZcqPXqS20zORAyVZ4LYbO0_w@mail.gmail.com>
Date: Wed, 16 Oct 2024 09:10:21 +1300
From: Barry Song <21cnbao@...il.com>
To: gaoxu <gaoxu2@...or.com>
Cc: David Hildenbrand <david@...hat.com>, 
	"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>, "linux-mm@...ck.org" <linux-mm@...ck.org>, 
	"mhocko@...e.com" <mhocko@...e.com>, "hailong.liu@...o.com" <hailong.liu@...o.com>, 
	"kaleshsingh@...gle.com" <kaleshsingh@...gle.com>, 
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>, 
	"lokeshgidra@...gle.com" <lokeshgidra@...gle.com>, "ngeoffray@...gle.com" <ngeoffray@...gle.com>, 
	"shli@...com" <shli@...com>, "surenb@...gle.com" <surenb@...gle.com>, "yuzhao@...gle.com" <yuzhao@...gle.com>, 
	"minchan@...nel.org" <minchan@...nel.org>, Barry Song <v-songbaohua@...o.com>
Subject: Re: [PATCH RFC] mm: mglru: provide a separate list for lazyfree anon folios

On Tue, Oct 15, 2024 at 11:03 PM gaoxu <gaoxu2@...or.com> wrote:
>
> >
> > On Wed, Sep 18, 2024 at 12:02 AM David Hildenbrand <david@...hat.com>
> > wrote:
> > >
> > > On 14.09.24 08:37, Barry Song wrote:
> > > > From: Barry Song <v-songbaohua@...o.com>
> > > >
> > > > This follows up on the discussion regarding Gaoxu's work[1]. It's
> > > > unclear if there's still interest in implementing a separate LRU
> > > > list for lazyfree folios, but I decided to explore it out of
> > > > curiosity.
> > > >
> > > > According to Lokesh, MADV_FREE'd anon folios are expected to be
> > > > released earlier than file folios. One option, as implemented by Gao
> > > > Xu, is to place lazyfree anon folios at the tail of the file's
> > > > `min_seq` generation. However, this approach results in lazyfree
> > > > folios being released in a LIFO manner, which conflicts with LRU
> > > > behavior, as noted by Michal.
> > > >
> > > > To address this, this patch proposes maintaining a separate list for
> > > > lazyfree anon folios while keeping them classified under the "file"
> > > > LRU type to minimize code changes. These lazyfree anon folios will
> > > > still be counted as file folios and share the same generation with
> > > > regular files. In the eviction path, the lazyfree list will be
> > > > prioritized for scanning before the actual file LRU list.
> > > >
> > >
> > > What's the downside of another LRU list? Do we have any experience on that?
> >
> > Essentially, the goal is to address the downsides of using a single LRU list for files
> > and lazyfree anonymous pages - seriously more files re-faults.
> >
> > I'm not entirely clear on the downsides of having an additional LRU list. While it
> > does increase complexity, it doesn't seem to be significant.
> >
> > Let's wait for Gaoxu's test results before deciding on the next steps.
> > I was just
> > curious about how difficult it would be to add a separate list, so I took two hours
> > to explore it :-)
> Hi song,
> I'm very sorry, various reasons combined have caused the delay in the results.
>
> Basic version：android V (enable Android ART use MADV_FREE)
> Test cases: 60 apps repeatedly restarted, tested for 8 hours;
> The test results are as follows:
>         workingset_refault_anon   workingset_refault_file
> base        42016805                92010542
> patch       19834873                49383572
> % diff       -52.79%                  -46.33%
>
> Additionally, a comparative test was conducted on
> add-lazyfree-folio-to-lru-tail.patch[1], and the results are as follows:
>                workingset_refault_anon   workingset_refault_file
> lazyfree-tail        20313395                 52203061
> patch             19834873                 49383572
> % diff              -2.36%                    -5.40%
>
> From the results, it can be seen that this patch is very beneficial and
> better than the results in [1]; it can solve the performance issue of high
> IO caused by extensive use of MADV_FREE on the Android platform.
>

Thank you for the testing and data. The results look promising. Would you
mind if I send a v2 with the test data and your tag included in the changelog?
I mean:

Tested-by: Gao Xu <gaoxu2@...onor.com>

> Test case notes: There is a discrepancy between the test results mentioned in
> [1] and the current test results because the test cases are different. The test
> case used in [1] involves actions such as clicking and swiping within the app
> after it starts; For the sake of convenience and result stability, the current
> test case only involves app startup without clicking and swiping, and the number
> of apps has been increased (30->60).
>
> 1. https://lore.kernel.org/all/f29f64e29c08427b95e3df30a5770056@honor.com/T/#u
> >
> > >
> > > --
> > > Cheers,
> > >
> > > David / dhildenb
> > >
> >

Thanks
Barry