linux-kernel - Re: [PATCH 28/28] mm, swap: implement dynamic allocation of swap table

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAKEwX=MmD___ukRrx=hLo7d_m1J_uG_Ke+us7RQgFUV2OSg38w@mail.gmail.com>
Date: Wed, 21 May 2025 11:36:09 -0700
From: Nhat Pham <nphamcs@...il.com>
To: Kairui Song <kasong@...cent.com>
Cc: linux-mm@...ck.org, Andrew Morton <akpm@...ux-foundation.org>, 
	Matthew Wilcox <willy@...radead.org>, Hugh Dickins <hughd@...gle.com>, Chris Li <chrisl@...nel.org>, 
	David Hildenbrand <david@...hat.com>, Yosry Ahmed <yosryahmed@...gle.com>, 
	"Huang, Ying" <ying.huang@...ux.alibaba.com>, Johannes Weiner <hannes@...xchg.org>, 
	Baolin Wang <baolin.wang@...ux.alibaba.com>, Baoquan He <bhe@...hat.com>, 
	Barry Song <baohua@...nel.org>, Kalesh Singh <kaleshsingh@...gle.com>, 
	Kemeng Shi <shikemeng@...weicloud.com>, Tim Chen <tim.c.chen@...ux.intel.com>, 
	Ryan Roberts <ryan.roberts@....com>, linux-kernel@...r.kernel.org
Subject: Re: [PATCH 28/28] mm, swap: implement dynamic allocation of swap table

On Wed, May 14, 2025 at 1:20 PM Kairui Song <ryncsn@...il.com> wrote:
>
> From: Kairui Song <kasong@...cent.com>
>
> Now swap table is cluster based, which means free clusters can free its
> table since no one should modify it.
>
> There could be speculative readers, like swap cache look up, protect
> them by making them RCU safe. All swap table should be filled with null
> entries before free, so such readers will either see a NULL pointer or
> a null filled table being lazy freed.
>
> On allocation, allocate the table when a cluster is used by any order.
>
> This way, we can reduce the memory usage of large swap device
> significantly.
>
> This idea to dynamically release unused swap cluster data was initially
> suggested by Chris Li while proposing the cluster swap allocator and
> I found it suits the swap table idea very well.
>
> Suggested-by: Chris Li <chrisl@...nel.org>
> Signed-off-by: Kairui Song <kasong@...cent.com>

Nice optimization!

However, please correct me if I'm wrong - but we are only dynamically
allocating the swap table with this patch. What we are getting here is
the dynamic allocation of the swap entries' metadata (through the swap
table), which my virtual swap prototype already provides. The cluster
metadata struct (struct swap_cluster_info) itself is statically
allocated still (at swapon time), correct? That will not work for a
large virtual swap space :( So unfortunately, even with this swap
table series, swap virtualization is still not trivial - definitely
not as trivial as a new swap device type...

Reading your physical swapfile allocator gives me some ideas though -
let me build it into my prototype :) I'll send it out once it's ready.