linux-kernel - Re: [PATCH] workqueue: Reduce expensive locks for unbound workqueue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAJhGHyBbhJHqAyfQ6x9BSkLdkMr0P5Fqx8T_2XoW-+X1AQobhg@mail.gmail.com>
Date: Fri, 15 Nov 2024 14:38:40 +0800
From: Lai Jiangshan <jiangshanlai@...il.com>
To: Wangyang Guo <wangyang.guo@...el.com>
Cc: Tejun Heo <tj@...nel.org>, linux-kernel@...r.kernel.org, 
	Tim Chen <tim.c.chen@...ux.intel.com>, tianyou.li@...el.com, pan.deng@...el.com
Subject: Re: [PATCH] workqueue: Reduce expensive locks for unbound workqueue

On Fri, Nov 15, 2024 at 2:00 PM Wangyang Guo <wangyang.guo@...el.com> wrote:
>
> For unbound workqueue, pwqs usually map to just a few pools. Most of
> the time, pwqs will be linked sequentially to wq->pwqs list by cpu
> index.  Usually, consecutive CPUs have the same workqueue attribute
> (e.g. belong to the same NUMA node). This makes pwqs with the same
> pool cluster together in the pwq list.
>
> Only do lock/unlock if the pool has changed in flush_workqueue_prep_pwqs().
> This reduces the number of expensive lock operations.
>
> The performance data shows this change boosts FIO by 65x in some cases
> when multiple concurrent threads write to xfs mount points with fsync.
>
> FIO Benchmark Details
> - FIO version: v3.35
> - FIO Options: ioengine=libaio,iodepth=64,norandommap=1,rw=write,
>   size=128M,bs=4k,fsync=1
> - FIO Job Configs: 64 jobs in total writing to 4 mount points (ramdisks
>   formatted as xfs file system).
> - Kernel Codebase: v6.12-rc5
> - Test Platform: Xeon 8380 (2 sockets)
>
> Reviewed-by: Tim Chen <tim.c.chen@...ux.intel.com>
> Signed-off-by: Wangyang Guo <wangyang.guo@...el.com>
> ---
>  kernel/workqueue.c | 22 ++++++++++++++++++----
>  1 file changed, 18 insertions(+), 4 deletions(-)

Reviewed-by: Lai Jiangshan <jiangshanlai@...il.com>

This is a problem caused by 636b927eba5b("workqueue: Make unbound
workqueues to use per-cpu pool_workqueues").

Before the said commit, there is much less likely that two or more PWQs
in the same WQs share the same pool. After the commit, it becomes a common case.

I planned to make the PWQs shared for different CPUs if possible.
But the patch[1] has a problem which is easy to fix.
I will update it if it is needed.

Thanks
Lai

[1] https://lore.kernel.org/lkml/20231227145143.2399-3-jiangshanlai@gmail.com/