linux-kernel - Re: [PATCH 1/1] mm/madvise: enhance lazyfreeing with mTHP in madvise

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAGsJ_4x6Otb9LUvnxAaPLnQ2MPPng0xpG-vJmFL7pNm10FDhZA@mail.gmail.com>
Date: Tue, 27 Feb 2024 20:11:07 +1300
From: Barry Song <21cnbao@...il.com>
To: Yin Fengwei <fengwei.yin@...el.com>
Cc: Ryan Roberts <ryan.roberts@....com>, Lance Yang <ioworker0@...il.com>, 
	David Hildenbrand <david@...hat.com>, akpm@...ux-foundation.org, linux-kernel@...r.kernel.org, 
	linux-mm@...ck.org, mhocko@...e.com, minchan@...nel.org, peterx@...hat.com, 
	shy828301@...il.com, songmuchun@...edance.com, wangkefeng.wang@...wei.com, 
	zokeefe@...gle.com
Subject: Re: [PATCH 1/1] mm/madvise: enhance lazyfreeing with mTHP in madvise_free

On Tue, Feb 27, 2024 at 8:02 PM Yin Fengwei <fengwei.yin@...el.com> wrote:
>
>
>
> On 2/27/24 14:40, Barry Song wrote:
> > On Tue, Feb 27, 2024 at 7:14 PM Yin Fengwei <fengwei.yin@...el.com> wrote:
> >>
> >>
> >>
> >> On 2/27/24 10:17, Barry Song wrote:
> >>>> Like if we hit folio which is partially mapped to the range, don't split it but
> >>>> just unmap the mapping part from the range. Let page reclaim decide whether
> >>>> split the large folio or not (If it's not mapped to any other range,it will be
> >>>> freed as whole large folio. If part of it still mapped to other range,page reclaim
> >>>> can decide whether to split it or ignore it for current reclaim cycle).
> >>> Yes, we can. but we still have to play the ptes check game to avoid adding
> >>> folios multiple times to reclaim the list.
> >>>
> >>> I don't see too much difference between splitting in madvise and splitting
> >>> in vmscan.  as our real purpose is avoiding splitting entirely mapped
> >>> large folios. for partial mapped large folios, if we split in madvise, then
> >>> we don't need to play the game of skipping folios while iterating PTEs.
> >>> if we don't split in madvise, we have to make sure the large folio is only
> >>> added in reclaimed list one time by checking if PTEs belong to the
> >>> previous added folio.
> >>
> >> If the partial mapped large folio is unmapped from the range, the related PTE
> >> become none. How could the folio be added to reclaimed list multiple times?
> >
> > in case we have 16 PTEs in a large folio.
> > PTE0 present
> > PTE1 present
> > PTE2 present
> > PTE3  none
> > PTE4 present
> > PTE5 none
> > PTE6 present
> > ....
> > the current code is scanning PTE one by one.
> > while scanning PTE0, we have added the folio. then PTE1, PTE2, PTE4, PTE6...
> No. Before detect the folio is fully mapped to the range, we can't add folio
> to reclaim list because the partial mapped folio shouldn't be added. We can
> only scan PTE15 and know it's fully mapped.

you never know PTE15 is the last one mapping to the large folio, PTE15 can
be mapping to a completely different folio with PTE0.

>
> So, when scanning PTE0, we will not add folio. Then when hit PTE3, we know
> this is a partial mapped large folio. We will unmap it. Then all 16 PTEs
> become none.

I don't understand why all 16PTEs become none as we set PTEs to none.
we set PTEs to swap entries till try_to_unmap_one called by vmscan.

>
> If the large folio is fully mapped, the folio will be added to reclaim list
> after scan PTE15 and know it's fully mapped.

our approach is calling pte_batch_pte while meeting the first pte, if
pte_batch_pte = 16,
then we add this folio to reclaim_list and skip the left 15 PTEs.

>
> Regards
> Yin, Fengwei
>
> >
> > there are all kinds of possibilities for unmapping.
> >
> > so what we can do is recording we have added the folio while scanning PTE0,
> > then skipping this folios for all other PTEs.
> >
> > otherwise, we can split it while scanning PTE0, then we will meet
> > different folios
> > afterwards.
> >
> >>
> >>
> >> Regards
> >> Yin, Fengwei
> >

Thanks
Barry