linux-kernel - Re: [RFC PATCH v3 2/3] blk-mq: Freeze and quiesce all queues for tagset in elevator

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YElrSFGyim3rjDN+@T590>
Date:   Thu, 11 Mar 2021 08:58:48 +0800
From:   Ming Lei <ming.lei@...hat.com>
To:     John Garry <john.garry@...wei.com>
Cc:     hare@...e.de, bvanassche@....org, axboe@...nel.dk, hch@....de,
        linux-block@...r.kernel.org, linux-kernel@...r.kernel.org,
        pragalla@...eaurora.org, kashyap.desai@...adcom.com,
        yuyufen@...wei.com
Subject: Re: [RFC PATCH v3 2/3] blk-mq: Freeze and quiesce all queues for
 tagset in elevator_exit()

On Fri, Mar 05, 2021 at 11:14:53PM +0800, John Garry wrote:
> A use-after-free may occur if blk_mq_queue_tag_busy_iter() is run on a
> queue when another queue associated with the same tagset is switching IO
> scheduler:
> 
> BUG: KASAN: use-after-free in bt_iter+0xa0/0x120
> Read of size 8 at addr ffff0410285e7e00 by task fio/2302
> 
> CPU: 24 PID: 2302 Comm: fio Not tainted 5.12.0-rc1-11925-g29a317e228d9 #747
> Hardware name: Huawei Taishan 2280 /D05, BIOS Hisilicon D05 IT21 Nemo 2.0 RC0 04/18/2018 
>  Call trace:
> dump_backtrace+0x0/0x2d8 
> show_stack+0x18/0x68
> dump_stack+0x124/0x1a0
> print_address_description.constprop.13+0x68/0x30c
> kasan_report+0x1e8/0x258 
> __asan_load8+0x9c/0xd8
> bt_iter+0xa0/0x120 
> blk_mq_queue_tag_busy_iter+0x348/0x5d8
> blk_mq_in_flight+0x80/0xb8
> part_stat_show+0xcc/0x210
> dev_attr_show+0x44/0x90
> sysfs_kf_seq_show+0x120/0x1c0
> kernfs_seq_show+0x9c/0xb8
> seq_read_iter+0x214/0x668
> kernfs_fop_read_iter+0x204/0x2c0
> new_sync_read+0x1ec/0x2d0
> vfs_read+0x18c/0x248
> ksys_read+0xc8/0x178
> __arm64_sys_read+0x44/0x58
> el0_svc_common.constprop.1+0xc8/0x1a8
> do_el0_svc+0x90/0xa0
> el0_svc+0x24/0x38
> el0_sync_handler+0x90/0xb8
> el0_sync+0x154/0x180
> 
> Indeed, blk_mq_queue_tag_busy_iter() already does take a reference to its
> queue usage counter when called, and the queue cannot be frozen to switch
> IO scheduler until all refs are dropped. This ensures no stale references
> to IO scheduler requests will be seen by blk_mq_queue_tag_busy_iter().
> 
> However, there is nothing to stop blk_mq_queue_tag_busy_iter() being
> run for another queue associated with the same tagset, and it seeing
> a stale IO scheduler request from the other queue after they are freed.
> 
> To stop this happening, freeze and quiesce all queues associated with the
> tagset as the elevator is exited.

I think this way can't be accepted since switching one queue's scheduler
is nothing to do with other request queues attached to same HBA.

This patch will cause performance regression because userspace may
switch scheduler according to medium or workloads, at that time other
LUNs will be affected by this patch.


-- 
Ming