linux-kernel - Re: [RFC v2] mm: Multi-Gen LRU: fix use mm/page

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20240112044026.58580-1-henry.hj@antgroup.com>
Date: Fri, 12 Jan 2024 12:40:22 +0800
From: "Henry Huang" <henry.hj@...group.com>
To: yuanchu@...gle.com
Cc:  <akpm@...ux-foundation.org>,
  "Henry Huang" <henry.hj@...group.com>,
  "谈鉴锋" <henry.tjf@...group.com>,
   <linux-kernel@...r.kernel.org>,
   <linux-mm@...ck.org>,
   <rientjes@...gle.com>,
  "朱辉(茶水)" <teawater@...group.com>,
   <yuzhao@...gle.com>
Subject: Re: [RFC v2] mm: Multi-Gen LRU: fix use mm/page_idle/bitmap

Thanks for replying.

On Thu, Jan 11, 2024 at 03:24 AM Yuanchu Xie <yuanchu@...gle.com> wrote:
> Does this present a problem with setting memcg limits or OOMs? It
> seems like deterministically charging shared pages would be highly
> desirable. Mina Almasry previously proposed a memcg= mount option to
> implement deterministic charging[1], but it wasn't a generic sharing
> mechanism. Nonetheless, the problem remains, and it would be
> interesting to learn if this presents any issues for you.
>
> [1] https://lore.kernel.org/linux-mm/20211120045011.3074840-1-almasrymina@google.com/

In this case, total size of shared memory usually is small(< 100MB).
What's more, almost shared pages were file cache. So it doesn't present any problems.

> I'm working on a kernel driver/per-memcg interface to perform aging
> with MGLRU, including configuration for the MGLRU page scanning
> optimizations. I suspect scanning the PTE accessed bits for pages
> charged to a foreign memcg ad-hoc has some performance implications,
> and the more general solution is to charge in a predetermined way,
> which makes the scanning on behalf of the foreign memcg a bit cleaner.
> This is possible nonetheless, but a bit hacky. Let me know you have
> any ideas.

Wow! per-memcg interface is also what we need.
It's hardly to control user's behaviors in our envrionment. We can't promise that all
process who share memory would be in same memcg.
Maybe kernel should provide new interface to make shared page charge more predictable.
I think it would take some overhead to do this.

The key problem of us is that we can't check whether page is accesed(no gen or ref changed) 
in this case.
page belongs to A, but maybe process in B read/write this page more frequently.
we may treat this page as cold page but accutly hot page.
Maybe just call folio_mark_accessed instead of folio_update_gen(should hold memcg lru lock)
for those remote memcg pages?

-- 
2.43.0