linux-kernel - Re: [PATCHSET v2] blk-mq: reimplement timeout handling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1513813261.2603.36.camel@wdc.com>
Date:   Wed, 20 Dec 2017 23:41:02 +0000
From:   Bart Van Assche <Bart.VanAssche@....com>
To:     "tj@...nel.org" <tj@...nel.org>,
        "axboe@...nel.dk" <axboe@...nel.dk>
CC:     "kernel-team@...com" <kernel-team@...com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "peterz@...radead.org" <peterz@...radead.org>,
        "osandov@...com" <osandov@...com>,
        "linux-block@...r.kernel.org" <linux-block@...r.kernel.org>,
        "oleg@...hat.com" <oleg@...hat.com>, "hch@....de" <hch@....de>
Subject: Re: [PATCHSET v2] blk-mq: reimplement timeout handling

On Tue, 2017-12-12 at 11:01 -0800, Tejun Heo wrote:
> Currently, blk-mq timeout path synchronizes against the usual
> issue/completion path using a complex scheme involving atomic
> bitflags, REQ_ATOM_*, memory barriers and subtle memory coherence
> rules.  Unfortunatley, it contains quite a few holes.

Hello Tejun,

An attempt to run SCSI I/O with this patch series applied resulted in
the following:

BUG: unable to handle kernel NULL pointer dereference at           (null)
IP: scsi_times_out+0x1c/0x2d0
PGD 0 P4D 0
Oops: 0000 [#1] PREEMPT SMP
CPU: 1 PID: 437 Comm: kworker/1:1H Tainted: G        W        4.15.0-rc4-dbg+ #1
Hardware name: Dell Inc. PowerEdge R720/0VWT90, BIOS 2.5.4 01/22/2016
Workqueue: kblockd blk_mq_timeout_work
RIP: 0010:scsi_times_out+0x1c/0x2d0
RSP: 0018:ffffc90007ef3d58 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff880878eab000 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff880878eab000
RBP: ffff880878eab1a0 R08: ffffffffffffffff R09: 0000000000000001
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000004
R13: 0000000000000000 R14: ffff88085e4a5ce8 R15: ffff880878e9f848
FS:  0000000000000000(0000) GS:ffff88093f600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000000 CR3: 0000000001c0f002 CR4: 00000000000606e0
Call Trace:
 blk_mq_terminate_expired+0x36/0x70
 bt_iter+0x43/0x50
 blk_mq_queue_tag_busy_iter+0xee/0x200
 blk_mq_timeout_work+0x186/0x2e0
 process_one_work+0x221/0x6e0
 worker_thread+0x3a/0x390
 kthread+0x11c/0x140
 ret_from_fork+0x24/0x30
RIP: scsi_times_out+0x1c/0x2d0 RSP: ffffc90007ef3d58
CR2: 0000000000000000

(gdb) list *(scsi_times_out+0x1c)
0xffffffff8147adbc is in scsi_times_out (drivers/scsi/scsi_error.c:285).
280      */
281     enum blk_eh_timer_return scsi_times_out(struct request *req)
282     {
283             struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(req);
284             enum blk_eh_timer_return rtn = BLK_EH_NOT_HANDLED;
285             struct Scsi_Host *host = scmd->device->host;
286
287             trace_scsi_dispatch_cmd_timeout(scmd);
288             scsi_log_completion(scmd, TIMEOUT_ERROR);
289

(gdb) disas /s scsi_times_out
[ ... ]
283             struct scsi_cmnd *scmd = blk_mq_rq_to_pdu(req);
284             enum blk_eh_timer_return rtn = BLK_EH_NOT_HANDLED;
285             struct Scsi_Host *host = scmd->device->host;
   0xffffffff8147adb2 <+18>:    mov    0x1d8(%rdi),%rax
   0xffffffff8147adb9 <+25>:    mov    %rdi,%rbx
   0xffffffff8147adbc <+28>:    mov    (%rax),%r13
   0xffffffff8147adbf <+31>:    nopl   0x0(%rax,%rax,1)

Bart.