linux-kernel - Re: [PATCH V2] mm/slab: introduce kvfree_rcu_barrier_on

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <aS-d1OaCZ0TBVewg@hyeyoo>
Date: Wed, 3 Dec 2025 11:17:56 +0900
From: Harry Yoo <harry.yoo@...cle.com>
To: vbabka@...e.cz
Cc: surenb@...gle.com, Liam.Howlett@...cle.com, cl@...two.org,
        rientjes@...gle.com, roman.gushchin@...ux.dev, urezki@...il.com,
        sidhartha.kumar@...cle.com, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org, rcu@...r.kernel.org,
        maple-tree@...ts.infradead.org, linux-modules@...r.kernel.org,
        mcgrof@...nel.org, petr.pavlu@...e.com, samitolvanen@...gle.com,
        atomlin@...mlin.com, lucas.demarchi@...el.com,
        akpm@...ux-foundation.org, jonathanh@...dia.com,
        stable@...r.kernel.org, Daniel Gomez <da.gomez@...sung.com>
Subject: Re: [PATCH V2] mm/slab: introduce kvfree_rcu_barrier_on_cache() for
 cache destruction

On Tue, Dec 02, 2025 at 07:16:26PM +0900, Harry Yoo wrote:
> Currently, kvfree_rcu_barrier() flushes RCU sheaves across all slab
> caches when a cache is destroyed. This is unnecessary; only the RCU
> sheaves belonging to the cache being destroyed need to be flushed.
> 
> As suggested by Vlastimil Babka, introduce a weaker form of
> kvfree_rcu_barrier() that operates on a specific slab cache.
> 
> Factor out flush_rcu_sheaves_on_cache() from flush_all_rcu_sheaves() and
> call it from flush_all_rcu_sheaves() and kvfree_rcu_barrier_on_cache().
> 
> Call kvfree_rcu_barrier_on_cache() instead of kvfree_rcu_barrier() on
> cache destruction.
> 
> The performance benefit is evaluated on a 12 core 24 threads AMD Ryzen
> 5900X machine (1 socket), by loading slub_kunit module.
> 
> Before:
>   Total calls: 19
>   Average latency (us): 18127
>   Total time (us): 344414
> 
> After:
>   Total calls: 19
>   Average latency (us): 10066
>   Total time (us): 191264
> 
> Two performance regression have been reported:
>   - stress module loader test's runtime increases by 50-60% (Daniel)

So I took a look at why this regression is fixed. I didn't expect this
is going to be fixed because Daniel said CONFIG_CODE_TAGGING is enabled,
and there is still a heavy kvfree_rcu_barrier() call during module unloading.

As Vlastimil pointed out off-list, there should be kmem_cache_destroy()
calls somewhere.

So I ran kmod.sh and traced kmem_cache_destroy() calls:
> === kmem_cache_destroy Latency Statistics ===
> Total calls: 6346
> Average latency (us): 5156
> Total time (us): 32725981

Oh, it's called 6346 times during the test? That's impressive.
It also spent 32.725 seconds just for kmem_cache_destroy(), out of total
runtime of 96 seconds.

> === Top 2 stack traces involving kmem_cache_destroy ===
> 
> @stacks[
>     kmem_cache_destroy+1
>     cleanup_module+118
>     __do_sys_delete_module.isra.0+451
>     __x64_sys_delete_module+18
>     x64_sys_call+7366
>     do_syscall_64+128
>     entry_SYSCALL_64_after_hwframe+118
> ]: 1840

It seems tools/testing/selftests/kmod/kmod.sh is using xfs module for testing
and it creates & destroys many slab caches. (see exit_xfs_fs() ->
xfs_destroy_caches()).

Mystery solved, I guess :D

> @stacks[
>     kmem_cache_destroy+1
>     rcbagbt_init_cur_cache+4219734
>     __do_sys_delete_module.isra.0+451
>     __x64_sys_delete_module+18
>     x64_sys_call+7366
>     do_syscall_64+128
>     entry_SYSCALL_64_after_hwframe+118
> ]: 1955

I don't get this one though. Why is the rcbagbt init function (also
from xfs) called during module unloading?

>   - internal graphics test's runtime on Tegra23 increases by 35% (Jon)
> 
> They are fixed by this change.
> 
> Suggested-by: Vlastimil Babka <vbabka@...e.cz>
> Fixes: ec66e0d59952 ("slab: add sheaf support for batching kfree_rcu() operations")
> Cc: <stable@...r.kernel.org>
> Link: https://lore.kernel.org/linux-mm/1bda09da-93be-4737-aef0-d47f8c5c9301@suse.cz
> Reported-and-tested-by: Daniel Gomez <da.gomez@...sung.com>
> Closes: https://lore.kernel.org/linux-mm/0406562e-2066-4cf8-9902-b2b0616dd742@kernel.org
> Reported-and-tested-by: Jon Hunter <jonathanh@...dia.com>
> Closes: https://lore.kernel.org/linux-mm/e988eff6-1287-425e-a06c-805af5bbf262@nvidia.com
> Signed-off-by: Harry Yoo <harry.yoo@...cle.com>
> ---

-- 
Cheers,
Harry / Hyeonggon