linux-kernel - Re: [PATCH 1/2] mm/damon/sysfs: Implement recording feature

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20231130091426.GA13946@cuiyangpei>
Date:   Thu, 30 Nov 2023 17:14:26 +0800
From:   cuiyangpei <cuiyangpei@...il.com>
To:     SeongJae Park <sj@...nel.org>, akpm@...ux-foundation.org,
        damon@...ts.linux.dev, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org
Cc:     xiongping1@...omi.com
Subject: Re: [PATCH 1/2] mm/damon/sysfs: Implement recording feature

Hi SeongJae,

We also investigated the operation schemes you mentioned, but we don't
think it can fit our needs.

On android, user will open many apps and switch between these apps as
needs. We hope to monitor apps' memory access only when they are on
foreground and record the memory access pattern when they are switched
to the background. 

When avaliable memory reaches a threshold, we will use these access
patterns with some strategies to recognize those memory that will have
little impact on user experience and to reclaim them proactively.  

I'm not sure I have clarified it clearly, if you still have questions
on this, please let us know.

Thanks.

On Wed, Nov 29, 2023 at 05:10:58PM +0000, SeongJae Park wrote:
> Hi Cuiyangpei,
> 
> On Wed, 29 Nov 2023 21:13:15 +0800 cuiyangpei <cuiyangpei@...il.com> wrote:
> 
> > Hi SeongJae,
> > 
> > We are using damon on the Android operating system. It starts monitoring
> > when app comes to the foreground, stops monitoring and save the
> > monitoring results when app goes to the background.
> 
> Thank you so much for sharing this detailed use case.  This will be very
> helpful for us at understanding real usage of DAMON and making it better for
> that together.
> 
> > 
> > The two methods that you mentioned,
> > 
> > 1.tracepoint events
> > This method requires opening the tracepoint event and using the
> > 'perf-record' tool to generate the perf.data file. Then parsing the
> > perf.data file. However, the user's phone is not enabled tracepoint
> > events. Additionally, the generated file is quite complex, and we only
> > need memory addresses and access frequency informations.
> 
> That's fair points, thank you for kindly explaining this.
> 
> > 
> > 2. damos
> > There is no direct Python runtime environment on android phones.
> > 
> > Both of these methods provide results that are not very intuitive and
> > require complex parsing. We save the results in the format of starting
> > address, region size, and access frequency. When the available memory
> > reaches a threshold, the user space reclaim memory with low access
> > frequency by calling 'process_madvise' function.
> 
> Again, very fair points.  So, if I understood correctly, you want to reclaim
> cold pages proactively when the available memory reaches a threshold, right?
> DAMON could do that directly instead of you[1].  Using that, you don't need to
> save the access pattern and parse but just ask DAMON to find memory regions of
> specific access frequency range and reclaim.  Have you also considered using
> that but found some problems?
> 
> I understand the feature may not perfectly fit for your use case, and I want to
> learn from you how it could be better.
> 
> [1] https://docs.kernel.org/mm/damon/design.html#operation-schemes
> 
> 
> Thanks,
> SJ
> 
> > 
> > Thanks.
> > 
> [...]