linux-kernel - Re: [PATCH v1 00/14] Multigenerational LRU

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 15 Mar 2021 11:00:06 -0700
From:   Dave Hansen <dave.hansen@...el.com>
To:     Yu Zhao <yuzhao@...gle.com>, linux-mm@...ck.org
Cc:     Alex Shi <alex.shi@...ux.alibaba.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Dave Hansen <dave.hansen@...ux.intel.com>,
        Hillf Danton <hdanton@...a.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Joonsoo Kim <iamjoonsoo.kim@....com>,
        Matthew Wilcox <willy@...radead.org>,
        Mel Gorman <mgorman@...e.de>, Michal Hocko <mhocko@...e.com>,
        Roman Gushchin <guro@...com>, Vlastimil Babka <vbabka@...e.cz>,
        Wei Yang <richard.weiyang@...ux.alibaba.com>,
        Yang Shi <shy828301@...il.com>,
        Ying Huang <ying.huang@...el.com>,
        linux-kernel@...r.kernel.org, page-reclaim@...gle.com
Subject: Re: [PATCH v1 00/14] Multigenerational LRU

On 3/12/21 11:57 PM, Yu Zhao wrote:
> Background
> ==========
> DRAM is a major factor in total cost of ownership, and improving
> memory overcommit brings a high return on investment. Over the past
> decade of research and experimentation in memory overcommit, we
> observed a distinct trend across millions of servers and clients: the
> size of page cache has been decreasing because of the growing
> popularity of cloud storage. Nowadays anon pages account for more than
> 90% of our memory consumption and page cache contains mostly
> executable pages.

This makes a compelling argument that current reclaim is not well
optimized for anonymous memory with low rates of sharing.  Basically,
anonymous rmap is very powerful, but we're not getting enough bang for
our buck out of it.

I also understand that the workloads you reference are anonymous-heavy
and that page cache isn't a *major* component.

But, what does happens to page-cache-heavy workloads?  Does this just
effectively force databases that want to use shmem over to hugetlbfs?
How bad does this scanning get in the worst case if there's a lot of
sharing?

I'm kinda surprised by this, but my 16GB laptop has a lot more page
cache than I would have guessed:

> Active(anon):    4065088 kB
> Inactive(anon):  3981928 kB
> Active(file):    2260580 kB
> Inactive(file):  3738096 kB
> AnonPages:       6624776 kB
> Mapped:           692036 kB
> Shmem:            776276 kB

Most of it isn't mapped, but it's far from all being used for text.