linux-kernel - Re: REGRESSION: 37f4a24c2469: blk-mq: centralise related handling into blk_mq_get_driver

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200927173834.GA23474@mit.edu>
Date:   Sun, 27 Sep 2020 13:38:34 -0400
From:   "Theodore Y. Ts'o" <tytso@....edu>
To:     Shakeel Butt <shakeelb@...gle.com>
Cc:     Roman Gushchin <guro@...com>, Ming Lei <ming.lei@...hat.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Jens Axboe <axboe@...nel.dk>,
        Ext4 Developers List <linux-ext4@...r.kernel.org>,
        linux-block <linux-block@...r.kernel.org>,
        Vlastimil Babka <vbabka@...e.cz>,
        Linux MM <linux-mm@...ck.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: REGRESSION: 37f4a24c2469: blk-mq: centralise related handling
 into blk_mq_get_driver_tag

On Fri, Sep 25, 2020 at 02:18:48PM -0700, Shakeel Butt wrote:
> 
> Yes, you are right. Let's first get this patch tested and after
> confirmation we can update the commit message.

Thanks Shakeel!  I've tested your patch, as well as reverting the
three commits that Linus had suggested, and both seem to address the
problem for me as well.  I did see a small number of failures
immediately as soon as the VM has booted, when testing with the
"revert the three commits" but this appears to be a different failure,
which I had been seeing periodically during the bisect as well which
was no doubt introducing noise in my testing:

[   28.545018] watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [swapper/1:0]
[   28.545018] Modules linked in:
[   28.545018] irq event stamp: 4517759
[   28.545018] hardirqs last  enabled at (4517758): [<ffffffffaa600b9e>] asm_common_interrupt+0x1e/0x40
[   28.545018] hardirqs last disabled at (4517759): [<ffffffffaa5ee55b>] sysvec_apic_timer_interrupt+0xb/0x90
[   28.545018] softirqs last  enabled at (10634): [<ffffffffa9ac00fd>] irq_enter_rcu+0x6d/0x70
[   28.545018] softirqs last disabled at (10635): [<ffffffffaa600f72>] asm_call_on_stack+0x12/0x20
[   28.545018] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.9.0-rc6-xfstests-00007-g3f3cb48a7d90 #1916
[   28.545018] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
[   28.545018] RIP: 0010:__do_softirq+0xa3/0x435
[   28.545018] Code: 00 83 80 ac 07 00 00 01 48 89 44 24 08 c7 44 24 1c 0a 00 00 00 65 66 c7 05 a8 ae 9e 55 00 00 e8 d3 92 3b ff fb b8 ff ff ff ff <48> c7 c3 40 51 00 ab 41 0f bc c7 89 c6 83 c6 01 89 74 24 04 75 6a
[   28.545018] RSP: 0000:ffffb89f000e0f98 EFLAGS: 00000202
[   28.545018] RAX: 00000000ffffffff RBX: 0000000000000000 RCX: 000000000000298a
[   28.545018] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffaa80009d
[   28.545018] RBP: ffffb89f000abda0 R08: 0000000000000001 R09: 0000000000000000
[   28.545018] R10: 0000000000000001 R11: 0000000000000046 R12: 0000000000000001
[   28.545018] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000080
[   28.545018] FS:  0000000000000000(0000) GS:ffff998e59200000(0000) knlGS:0000000000000000
[   28.545018] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   28.545018] CR2: 0000000000000000 CR3: 000000023e012001 CR4: 00000000001706e0
[   28.545018] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   28.545018] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[   28.545018] Call Trace:
[   28.545018]  <IRQ>
[   28.545018]  asm_call_on_stack+0x12/0x20
[   28.545018]  </IRQ>
[   28.545018]  do_softirq_own_stack+0x4e/0x60
[   28.545018]  irq_exit_rcu+0x9f/0xe0
[   28.545018]  sysvec_call_function_single+0x43/0x90
[   28.545018]  asm_sysvec_call_function_single+0x12/0x20
[   28.545018] RIP: 0010:acpi_idle_do_entry+0x54/0x70
[   28.545018] Code: ed c3 e9 cf fe ff ff 65 48 8b 04 25 00 6e 01 00 48 8b 00 a8 08 75 ea e8 ba c0 5b ff e9 07 00 00 00 0f 00 2d f8 3d 4e 00 fb f4 <9c> 58 fa f6 c4 02 74 cf e9 5f c2 5b ff cc cc cc cc cc cc cc cc cc
[   28.545018] RSP: 0000:ffffb89f000abe88 EFLAGS: 00000202
[   28.545018] RAX: 000000000000293b RBX: ffff998e55640000 RCX: 0000000000001a12
[   28.545018] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffaa5fd2b6
[   28.545018] RBP: ffffffffab163760 R08: 0000000000000001 R09: 00000000000e003c
[   28.545018] R10: ffff998e582e2340 R11: 0000000000000046 R12: 0000000000000001
[   28.545018] R13: 0000000000000001 R14: ffffffffab1637e0 R15: 0000000000000000
[   28.545018]  ? acpi_idle_do_entry+0x46/0x70
[   28.545018]  ? acpi_idle_do_entry+0x46/0x70
[   28.545018]  acpi_idle_enter+0x7d/0xb0
[   28.545018]  cpuidle_enter_state+0x84/0x2c0
[   28.545018]  cpuidle_enter+0x29/0x40
[   28.545018]  cpuidle_idle_call+0x111/0x180
[   28.545018]  do_idle+0x7b/0xd0
[   28.545018]  cpu_startup_entry+0x19/0x20
[   28.545018]  secondary_startup_64+0xb6/0xc0

I think this was an issue relating to acpi_idle that others have
reported, but I thought this was fixed before -rc6 was released?  In
any case, this is post -rc6, so apparently there is something else
going on here, and this is probably unrelated to the regression which
Shakeel's patch is addressing.

					- Ted