netdev - Re: [PATCH v2 43/47] mm: shrinker: add a secondary array for shrinker_info::{map, nr

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <B421DD89-09B5-4488-BEC1-D6F88C6DE75A@linux.dev>
Date: Wed, 26 Jul 2023 17:30:53 +0800
From: Muchun Song <muchun.song@...ux.dev>
To: Qi Zheng <zhengqi.arch@...edance.com>
Cc: Andrew Morton <akpm@...ux-foundation.org>,
 david@...morbit.com,
 tkhai@...ru,
 Vlastimil Babka <vbabka@...e.cz>,
 Roman Gushchin <roman.gushchin@...ux.dev>,
 djwong@...nel.org,
 Christian Brauner <brauner@...nel.org>,
 "Paul E. McKenney" <paulmck@...nel.org>,
 tytso@....edu,
 steven.price@....com,
 cel@...nel.org,
 Sergey Senozhatsky <senozhatsky@...omium.org>,
 yujie.liu@...el.com,
 Greg KH <gregkh@...uxfoundation.org>,
 LKML <linux-kernel@...r.kernel.org>,
 Linux Memory Management List <linux-mm@...ck.org>,
 x86@...nel.org,
 kvm@...r.kernel.org,
 xen-devel@...ts.xenproject.org,
 linux-erofs@...ts.ozlabs.org,
 linux-f2fs-devel@...ts.sourceforge.net,
 cluster-devel@...hat.com,
 linux-nfs@...r.kernel.org,
 linux-mtd@...ts.infradead.org,
 rcu@...r.kernel.org,
 netdev <netdev@...r.kernel.org>,
 dri-devel@...ts.freedesktop.org,
 linux-arm-msm@...r.kernel.org,
 dm-devel@...hat.com,
 linux-raid@...r.kernel.org,
 linux-bcache@...r.kernel.org,
 virtualization@...ts.linux-foundation.org,
 linux-fsdevel@...r.kernel.org,
 linux-ext4@...r.kernel.org,
 linux-xfs@...r.kernel.org,
 linux-btrfs@...r.kernel.org
Subject: Re: [PATCH v2 43/47] mm: shrinker: add a secondary array for
 shrinker_info::{map, nr_deferred}

> On Jul 24, 2023, at 17:43, Qi Zheng <zhengqi.arch@...edance.com> wrote:
> 
> Currently, we maintain two linear arrays per node per memcg, which are
> shrinker_info::map and shrinker_info::nr_deferred. And we need to resize
> them when the shrinker_nr_max is exceeded, that is, allocate a new array,
> and then copy the old array to the new array, and finally free the old
> array by RCU.
> 
> For shrinker_info::map, we do set_bit() under the RCU lock, so we may set
> the value into the old map which is about to be freed. This may cause the
> value set to be lost. The current solution is not to copy the old map when
> resizing, but to set all the corresponding bits in the new map to 1. This
> solves the data loss problem, but bring the overhead of more pointless
> loops while doing memcg slab shrink.
> 
> For shrinker_info::nr_deferred, we will only modify it under the read lock
> of shrinker_rwsem, so it will not run concurrently with the resizing. But
> after we make memcg slab shrink lockless, there will be the same data loss
> problem as shrinker_info::map, and we can't work around it like the map.
> 
> For such resizable arrays, the most straightforward idea is to change it
> to xarray, like we did for list_lru [1]. We need to do xa_store() in the
> list_lru_add()-->set_shrinker_bit(), but this will cause memory
> allocation, and the list_lru_add() doesn't accept failure. A possible
> solution is to pre-allocate, but the location of pre-allocation is not
> well determined.
> 
> Therefore, this commit chooses to introduce a secondary array for
> shrinker_info::{map, nr_deferred}, so that we only need to copy this
> secondary array every time the size is resized. Then even if we get the
> old secondary array under the RCU lock, the found map and nr_deferred are
> also true, so no data is lost.
> 
> [1]. https://lore.kernel.org/all/20220228122126.37293-13-songmuchun@bytedance.com/
> 
> Signed-off-by: Qi Zheng <zhengqi.arch@...edance.com>

Reviewed-by: Muchun Song <songmuchun@...edance.com>