linux-kernel - Re: [PATCH RFC v2 0/5] blk-mq-sched: support request batch dispatching for sq elevator

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <9d3aee10-9eb3-4f2d-bb9a-1721c05ec3aa@kernel.org>
Date: Mon, 16 Jun 2025 13:03:59 +0900
From: Damien Le Moal <dlemoal@...nel.org>
To: Yu Kuai <yukuai1@...weicloud.com>, ming.lei@...hat.com,
 yukuai3@...wei.com, tj@...nel.org, josef@...icpanda.com, axboe@...nel.dk
Cc: linux-block@...r.kernel.org, cgroups@...r.kernel.org,
 linux-kernel@...r.kernel.org, yi.zhang@...wei.com, yangerkun@...wei.com,
 johnny.chenyi@...wei.com
Subject: Re: [PATCH RFC v2 0/5] blk-mq-sched: support request batch
 dispatching for sq elevator

On 6/14/25 18:25, Yu Kuai wrote:
> From: Yu Kuai <yukuai3@...wei.com>
> 
> Before this patch, each dispatch context will hold a global lock to
> dispatch one request at a time, which introduce intense lock competition:
> 
> lock
> ops.dispatch_request
> unlock
> 
> Hence support dispatch a batch of requests while holding the lock to
> reduce lock contention.
> 
> nullblk setup:
> modprobe null_blk nr_devices=0 &&
>     udevadm settle &&
>     cd /sys/kernel/config/nullb &&
>     mkdir nullb0 &&
>     cd nullb0 &&
>     echo 0 > completion_nsec &&
>     echo 512 > blocksize &&
>     echo 0 > home_node &&
>     echo 0 > irqmode &&
>     echo 128 > submit_queues &&
>     echo 1024 > hw_queue_depth &&
>     echo 1024 > size &&
>     echo 0 > memory_backed &&
>     echo 2 > queue_mode &&
>     echo 1 > power ||
>     exit $?
> 
> Test script:
> fio -filename=/dev/$disk -name=test -rw=randwrite -bs=4k -iodepth=32 \
>   -numjobs=16 --iodepth_batch_submit=8 --iodepth_batch_complete=8 \
>   -direct=1 -ioengine=io_uring -group_reporting -time_based -runtime=30
> 
> Test result(elevator is deadline): iops
> |                 | null_blk | scsi hdd |
> | --------------- | -------- | -------- |
> | before this set | 263k     | 24       |
> | after this set  | 475k     | 272      |

For the HDD, these numbers are very low, and I do not understand how you can get
any improvement from reducing lock contention, since contention should not be an
issue with this kind of performance. What HW did you use for testing ? Was this
a VM ?

I tested this null_blk setup and your fio command on a bare-metal 16-cores Xeon
machine. For the scsi disk, I used a 26TB SATA HDD connected to an AHCI port).
With this setup, results are like this:

|                 | null_blk | hdd (write) | hdd (read) |
| --------------- | -------- | ----------- | ---------- |
| before this set | 613k     | 1088        | 211        |
| after this set  | 940k     | 1093        | 212        |

So not surprisingly, there is no improvement for the SATA HDD because of the low
max IOPS these devices can achieve: lock contention is not really an issue when
you are dealing with a slow device. And a SAS HDD will be the same. Gains may
likely be more significant with a fast SAS/FC RAID array but I do not have
access to that.

But the improvement for a fast device like null_blk is indeed excellent (+53%).

With LOCKDEP & KASAN disabled, the results are like this:

|                 | null_blk | hdd (write) | hdd (read) |
| --------------- | -------- | ----------- | ---------- |
| before this set | 625k     | 1092        | 213        |
| after this set  | 984k     | 1095        | 215        |

No real changes for the HDD, as expected, and the improvement for null_blk is
still good.

So maybe drop the RFC tag on these patches and repost after cleaning things up ?

-- 
Damien Le Moal
Western Digital Research