linux-kernel - Re: [PATCH v3 1/2] mm: collect the number of anon large folios

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <7f9e6bf8-ec58-4525-ace6-98ba58de3172@redhat.com>
Date: Fri, 23 Aug 2024 13:31:55 +0200
From: David Hildenbrand <david@...hat.com>
To: Barry Song <21cnbao@...il.com>, akpm@...ux-foundation.org,
 linux-mm@...ck.org
Cc: baolin.wang@...ux.alibaba.com, chrisl@...nel.org, hanchuanhua@...o.com,
 ioworker0@...il.com, kaleshsingh@...gle.com, kasong@...cent.com,
 linux-kernel@...r.kernel.org, ryan.roberts@....com, v-songbaohua@...o.com,
 yuanshuai@...o.com, ziy@...dia.com, usamaarif642@...il.com
Subject: Re: [PATCH v3 1/2] mm: collect the number of anon large folios

On 23.08.24 00:40, Barry Song wrote:
> From: Barry Song <v-songbaohua@...o.com>
> 

Nit: "anon large folios" come in two flavors: THP and hugetlb.

I suggest to just call it "anon THP" in the context of both patches 
subjects/descriptions (sorry, I should have realized that earlier) to 
make it clearer.

This patch I would call

"mm: count the number of anonymous THPs per size"

Here I would start with:

'
Let's track for each anonymous THP size, how many of them are currently 
allocated. We'll track the complete lifespan of an anon THP, starting 
when it becomes an anon THP ("large anon folio") (->mapping gets set), 
until it gets freed (->mapping gets cleared).

Introduce a new "nr_anon" counter per THP size and adjust the 
corresponding counter in the following cases:
* We allocate a new THP and call folio_add_new_anon_rmap() to map
   it the first time and turn it into an anon THP.
* We split an anon THP into multiple smaller ones.
* We migrate an anon THP, when we prepare the destination.
* We free an anon THP back to the buddy.
'

> Anon large folios come from three places:
> 1. new allocated large folios in page faults, they will call
>     folio_add_new_anon_rmap() for rmap;
> 2. a large folio is split into multiple lower-order large folios;
> 3. a large folio is migrated to a new large folio.
> 
> In all above three counts, we increase nr_anon by 1;
> 
> Anon large folios might go either because of be split or be put
> to free, in these cases, we reduce the count by 1.
> 
> Folios added to the swap cache without an anonymous mapping won't
> be counted. This aligns with the AnonPages statistics in /proc/meminfo.
> However, folios that have been fully unmapped but not yet freed are
> counted. Unlike AnonPages, this can help identify anonymous memory
> leaks, such as when an anon folio is still pinned after being unmapped.

I would just mention here:

"Note that AnonPages in /proc/meminfo currently tracks the total number 
of *mapped* anonymous *pages*, and therefore has slightly different 
semantics. In the future, we might also want to track "nr_anon_mapped" 
for each THP size, which might be helpful when comparing it to the 
number of allocated anon THPs (long-term pinning, stuck in swapcache, 
memory leaks, ...)."

Further note that for now, we only track anon THPs after they got their 
->mapping set, for example via folio_add_new_anon_rmap(). If we would 
allocate some in the swapcache, they will only show up in the statistics 
for now after they have been mapped to user space the first time, where 
we call folio_add_new_anon_rmap().
"

> 
> Signed-off-by: Barry Song <v-songbaohua@...o.com>
> Acked-by: David Hildenbrand <david@...hat.com>
> ---
>   Documentation/admin-guide/mm/transhuge.rst |  5 +++++
>   include/linux/huge_mm.h                    | 15 +++++++++++++--
>   mm/huge_memory.c                           | 13 ++++++++++---
>   mm/migrate.c                               |  4 ++++
>   mm/page_alloc.c                            |  5 ++++-
>   mm/rmap.c                                  |  1 +
>   6 files changed, 37 insertions(+), 6 deletions(-)
> 
> diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst
> index 79435c537e21..b78f2148b242 100644
> --- a/Documentation/admin-guide/mm/transhuge.rst
> +++ b/Documentation/admin-guide/mm/transhuge.rst
> @@ -551,6 +551,11 @@ split_deferred
>           it would free up some memory. Pages on split queue are going to
>           be split under memory pressure, if splitting is possible.
>   
> +nr_anon
> +       the number of transparent anon huge pages we have in the whole system.
> +       These huge pages could be entirely mapped or have partially
> +       unmapped/unused subpages.

To be clearer "This includes THPs that might or might not be currently 
mapped to user space".

-- 
Cheers,

David / dhildenb