linux-ext4 - Re: [PATCH v2 43/47] mm: shrinker: add a secondary array for shrinker_info::{map, nr

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <B421DD89-09B5-4488-BEC1-D6F88C6DE75A@linux.dev>
Date:   Wed, 26 Jul 2023 17:30:53 +0800
From:   Muchun Song <muchun.song@...ux.dev>
To:     Qi Zheng <zhengqi.arch@...edance.com>
Cc:     Andrew Morton <akpm@...ux-foundation.org>, david@...morbit.com,
        tkhai@...ru, Vlastimil Babka <vbabka@...e.cz>,
        Roman Gushchin <roman.gushchin@...ux.dev>, djwong@...nel.org,
        Christian Brauner <brauner@...nel.org>,
        "Paul E. McKenney" <paulmck@...nel.org>, tytso@....edu,
        steven.price@....com, cel@...nel.org,
        Sergey Senozhatsky <senozhatsky@...omium.org>,
        yujie.liu@...el.com, Greg KH <gregkh@...uxfoundation.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Linux Memory Management List <linux-mm@...ck.org>,
        x86@...nel.org, kvm@...r.kernel.org,
        xen-devel@...ts.xenproject.org, linux-erofs@...ts.ozlabs.org,
        linux-f2fs-devel@...ts.sourceforge.net, cluster-devel@...hat.com,
        linux-nfs@...r.kernel.org, linux-mtd@...ts.infradead.org,
        rcu@...r.kernel.org, netdev <netdev@...r.kernel.org>,
        dri-devel@...ts.freedesktop.org, linux-arm-msm@...r.kernel.org,
        dm-devel@...hat.com, linux-raid@...r.kernel.org,
        linux-bcache@...r.kernel.org,
        virtualization@...ts.linux-foundation.org,
        linux-fsdevel@...r.kernel.org, linux-ext4@...r.kernel.org,
        linux-xfs@...r.kernel.org, linux-btrfs@...r.kernel.org
Subject: Re: [PATCH v2 43/47] mm: shrinker: add a secondary array for
 shrinker_info::{map, nr_deferred}



> On Jul 24, 2023, at 17:43, Qi Zheng <zhengqi.arch@...edance.com> wrote:
> 
> Currently, we maintain two linear arrays per node per memcg, which are
> shrinker_info::map and shrinker_info::nr_deferred. And we need to resize
> them when the shrinker_nr_max is exceeded, that is, allocate a new array,
> and then copy the old array to the new array, and finally free the old
> array by RCU.
> 
> For shrinker_info::map, we do set_bit() under the RCU lock, so we may set
> the value into the old map which is about to be freed. This may cause the
> value set to be lost. The current solution is not to copy the old map when
> resizing, but to set all the corresponding bits in the new map to 1. This
> solves the data loss problem, but bring the overhead of more pointless
> loops while doing memcg slab shrink.
> 
> For shrinker_info::nr_deferred, we will only modify it under the read lock
> of shrinker_rwsem, so it will not run concurrently with the resizing. But
> after we make memcg slab shrink lockless, there will be the same data loss
> problem as shrinker_info::map, and we can't work around it like the map.
> 
> For such resizable arrays, the most straightforward idea is to change it
> to xarray, like we did for list_lru [1]. We need to do xa_store() in the
> list_lru_add()-->set_shrinker_bit(), but this will cause memory
> allocation, and the list_lru_add() doesn't accept failure. A possible
> solution is to pre-allocate, but the location of pre-allocation is not
> well determined.
> 
> Therefore, this commit chooses to introduce a secondary array for
> shrinker_info::{map, nr_deferred}, so that we only need to copy this
> secondary array every time the size is resized. Then even if we get the
> old secondary array under the RCU lock, the found map and nr_deferred are
> also true, so no data is lost.
> 
> [1]. https://lore.kernel.org/all/20220228122126.37293-13-songmuchun@bytedance.com/
> 
> Signed-off-by: Qi Zheng <zhengqi.arch@...edance.com>

Reviewed-by: Muchun Song <songmuchun@...edance.com>