linux-kernel - Re: DAMON VA regions don't split on an large Android APP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAGsJ_4y+w-i7OhE9Xk94W8p6srkMsVxTFxwb-+8g_ShGPEj5pw@mail.gmail.com>
Date:   Mon, 30 May 2022 17:04:10 +1200
From:   Barry Song <21cnbao@...il.com>
To:     SeongJae Park <sj@...nel.org>
Cc:     Rongwei Wang <rongwei.wang@...ux.alibaba.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Linux-MM <linux-mm@...ck.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Matthew Wilcox <willy@...radead.org>, shuah@...nel.org,
        Brendan Higgins <brendanhiggins@...gle.com>,
        foersleo@...zon.de, sieberf@...zon.com,
        Shakeel Butt <shakeelb@...gle.com>, sjpark@...zon.de,
        tuhailong@...il.com, Song Jiang <sjiang88@...il.com>,
        张诗明(Simon Zhang) 
        <zhangshiming@...o.com>,
        李培锋(wink) <lipeifeng@...o.com>,
        xhao@...ux.alibaba.com, damon@...ts.linux.dev
Subject: Re: DAMON VA regions don't split on an large Android APP

On Mon, May 30, 2022 at 7:54 AM SeongJae Park <sj@...nel.org> wrote:
>
> On Wed, 27 Apr 2022 17:50:49 +0000 sj@...nel.org wrote:
>
> > Hello Rongwei and Barry,
> >
> > On Wed, 27 Apr 2022 19:44:23 +1200 Barry Song <21cnbao@...il.com> wrote:
> >
> > > On Wed, Apr 27, 2022 at 6:56 PM Rongwei Wang
> > > <rongwei.wang@...ux.alibaba.com> wrote:
> > > >
> > > >
> > > >
> > > > On 4/27/22 7:19 AM, Barry Song wrote:
> [...]
> > >
> > > I guess the cause might be:
> > > in case a region is very big like 10GiB, we have only 1MiB hot pages
> > > in this large region.
> > > damon will randomly pick one page to sample, but the page has only
> > > 1MiB/10GiB, thus
> > > less than 1/10000 chance to hit the hot 1MiB. so probably we need
> > > 10000 sample periods
> > > to hit the hot 1MiB in order to split this large region?
> > >
> > > @SeongJae, please correct me if I am wrong.
> >
> > I think your theory makes sense.  There was a similar concern, so we made DAMON
> > to split regions into 3 sub-regions when we don't see advance[1].  My current
> > rough idea for improving DAMON accuracy is making it more aggressive while
> > keeping the monitoring overhead low.
> >
> > [1] https://git.kernel.org/pub/scm/linux/kernel/git/sj/linux.git/tree/mm/damon/core.c?h=damon/next-2022-04-21-08-31-on-v5.18-rc3-mmots-2022-04-20-17-37#n1053
> >
> > >
> > > >
> > > > I'm not sure whether sets init_regions can deal with the above problem,
> > > > or dynamic choose one or limited number VMA to monitor.
> > > >
> > >
> > > I won't set a limited number of VMA as this will make the damon too hard to use
> > > as nobody wants to make such complex operations, especially an Android
> > > app might have more than 8000 VMAs.
> > >
> > > I agree init_regions might be the right place to enhance the situation.
> >
> > 'init_regions' has developed for the purpose, where user space knows some good
> > information for starting point of the regions adjustment, and thus want to hint
> > DAMON.  Nevertheless, it might not work as expected, because DAMON
> > automatically updates the target regions to cover all VMAs as much as it can.
> > I have posted a patchset for the use case yesterday[1].
> >
> > [1] https://lore.kernel.org/linux-mm/20220426231750.48822-1-sj@kernel.org/
>
> FWIW, the patchset for the fixed virtual address space ranges monitoring has
> merged in the mainline[1].
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=98931dd95fd489fcbfa97da563505a6f071d7c77
>

nice to know that. Thanks, though it doesn't fix my problem as I am
looking for a solution to
collect precise monitoring data  automatically and economically.

>
> Thanks,
> SJ

Thanks
Barry