linux-kernel - Re: possible 6.6 regression: Deadlock involving super

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <b1d67dd6-4326-4247-96a3-283ab1975a50@molgen.mpg.de>
Date: Sun, 24 Mar 2024 11:50:47 +0100
From: Donald Buczek <buczek@...gen.mpg.de>
To: Christian Brauner <brauner@...nel.org>
Cc: Alexander Viro <viro@...iv.linux.org.uk>, linux-fsdevel@...r.kernel.org,
 linux-kernel@...r.kernel.org, it+linux@...gen.mpg.de
Subject: Re: possible 6.6 regression: Deadlock involving super_lock()

On 3/21/24 14:35, Christian Brauner wrote:
>> Also, one writeback thread was blocked. I mention that, because I
>> don't get how these these two threads could depend on each other:
> 
> Writeback holds s_umount. So writeback seems to not make progress and
> blocks the mount. So right now it seems unlikely that this is related.

Oh, yes, super_trylock_shared() in __writeback_inodes_wb(). Sorry, I've missed that and blamed the victim.

> Any chance you can try and reproduce this with v6.7 and newer?

We can't really reproduce anything yet, we can only wait and see if it happens again on the production system. We are willing to risk that, so we've just switched the system to 6.6.22 and made sure, that drgn is available to further analyze a possible frozen state. This was not the case on the reported freeze due to a configuration error. Although, currently, I would have no idea how to find out, why the folio is locked forever, even with drgn available.

We can switch to 6.7. Why do you propose it? Is there reason to believe that the problem might go away or would that help in analysis if we hit it again on that series?

I'm a bit hesitating, because this is a production system and we have the experience that switching from one mainline release to the next (or from one stable series to another) produces some new problems most of the times.

Thanks

   Donald

>> # # /proc/39359/task/39359: kworker/u268:5+flush-0:58 :
>> # cat /proc/39359/task/39359/stack
>>
>> [<0>] folio_wait_bit_common+0x135/0x350
>> [<0>] write_cache_pages+0x1a0/0x3a0
>> [<0>] nfs_writepages+0x12a/0x1e0 [nfs]
>> [<0>] do_writepages+0xcf/0x1e0
>> [<0>] __writeback_single_inode+0x46/0x3a0
>> [<0>] writeback_sb_inodes+0x1f5/0x4d0
>> [<0>] __writeback_inodes_wb+0x4c/0xf0
>> [<0>] wb_writeback+0x1f5/0x320
>> [<0>] wb_workfn+0x350/0x4f0
>> [<0>] process_one_work+0x142/0x300
>> [<0>] worker_thread+0x2f5/0x410
>> [<0>] kthread+0xe8/0x120
>> [<0>] ret_from_fork+0x34/0x50
>> [<0>] ret_from_fork_asm+0x1b/0x30

-- 
Donald Buczek
buczek@...gen.mpg.de
Tel: +49 30 8413 1433