linux-kernel - Re: [syzbot] inconsistent lock state in kmem_cache

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <0f7a2712-5252-260c-3b0f-ec584e1066a3@kernel.dk>
Date:   Thu, 29 Sep 2022 08:07:39 -0600
From:   Jens Axboe <axboe@...nel.dk>
To:     Jan Kara <jack@...e.cz>, Vlastimil Babka <vbabka@...e.cz>
Cc:     syzbot <syzbot+dfcc5f4da15868df7d4d@...kaller.appspotmail.com>,
        akpm@...ux-foundation.org, keescook@...omium.org,
        linux-kernel@...r.kernel.org, mark.rutland@....com,
        mhiramat@...nel.org, rostedt@...dmis.org,
        syzkaller-bugs@...glegroups.com,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        Michal Hocko <mhocko@...e.com>,
        Amir Goldstein <amir73il@...il.com>,
        Matthew Bobrowski <repnop@...gle.com>,
        Linux-FSDevel <linux-fsdevel@...r.kernel.org>,
        Sebastian Andrzej Siewior <bigeasy@...utronix.de>,
        Matthew Wilcox <willy@...radead.org>, io-uring@...r.kernel.org
Subject: Re: [syzbot] inconsistent lock state in kmem_cache_alloc

On 9/29/22 7:56 AM, Jan Kara wrote:
> On Thu 29-09-22 15:24:22, Vlastimil Babka wrote:
>> On 9/26/22 18:33, syzbot wrote:
>>> Hello,
>>>
>>> syzbot found the following issue on:
>>>
>>> HEAD commit:    105a36f3694e Merge tag 'kbuild-fixes-v6.0-3' of git://git...
>>> git tree:       upstream
>>> console+strace: https://syzkaller.appspot.com/x/log.txt?x=152bf540880000
>>> kernel config:  https://syzkaller.appspot.com/x/.config?x=7db7ad17eb14cb7
>>> dashboard link: https://syzkaller.appspot.com/bug?extid=dfcc5f4da15868df7d4d
>>> compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
>>> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=1020566c880000
>>> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=104819e4880000
>>>
>>> IMPORTANT: if you fix the issue, please add the following tag to the commit:
>>> Reported-by: syzbot+dfcc5f4da15868df7d4d@...kaller.appspotmail.com
>>
>> +CC more folks
>>
>> I'm not fully sure what this report means but I assume it's because there's
>> a GFP_KERNEL kmalloc() allocation from softirq context? Should it perhaps
>> use memalloc_nofs_save() at some well defined point?
> 
> Thanks for the CC. The problem really is that io_uring is calling into
> fsnotify_access() from softirq context. That isn't going to work. The
> allocation is just a tip of the iceberg. Fsnotify simply does not expect to
> be called from softirq context. All the dcache locks are not IRQ safe, it
> can even obtain some sleeping locks and call to userspace if there are
> suitable watches set up.
> 
> So either io_uring needs to postpone fsnotify calls to a workqueue or we
> need a way for io_uring code to tell iomap dio code that the completion
> needs to always happen from a workqueue (as it currently does for writes).
> Jens?

Something like this should probably work - I'll write a test case and
vet it.


diff --git a/io_uring/rw.c b/io_uring/rw.c
index 1ae1e52ab4cb..a25cd44cd415 100644
--- a/io_uring/rw.c
+++ b/io_uring/rw.c
@@ -236,14 +236,6 @@ static void kiocb_end_write(struct io_kiocb *req)
 
 static bool __io_complete_rw_common(struct io_kiocb *req, long res)
 {
-	struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw);
-
-	if (rw->kiocb.ki_flags & IOCB_WRITE) {
-		kiocb_end_write(req);
-		fsnotify_modify(req->file);
-	} else {
-		fsnotify_access(req->file);
-	}
 	if (unlikely(res != req->cqe.res)) {
 		if ((res == -EAGAIN || res == -EOPNOTSUPP) &&
 		    io_rw_should_reissue(req)) {
@@ -270,6 +262,20 @@ static inline int io_fixup_rw_res(struct io_kiocb *req, long res)
 	return res;
 }
 
+static void io_req_rw_complete(struct io_kiocb *req, bool *locked)
+{
+	struct io_rw *rw = io_kiocb_to_cmd(req, struct io_rw);
+
+	if (rw->kiocb.ki_flags & IOCB_WRITE) {
+		kiocb_end_write(req);
+		fsnotify_modify(req->file);
+	} else {
+		fsnotify_access(req->file);
+	}
+
+	io_req_task_complete(req, locked);
+}
+
 static void io_complete_rw(struct kiocb *kiocb, long res)
 {
 	struct io_rw *rw = container_of(kiocb, struct io_rw, kiocb);
@@ -278,7 +284,7 @@ static void io_complete_rw(struct kiocb *kiocb, long res)
 	if (__io_complete_rw_common(req, res))
 		return;
 	io_req_set_res(req, io_fixup_rw_res(req, res), 0);
-	req->io_task_work.func = io_req_task_complete;
+	req->io_task_work.func = io_req_rw_complete;
 	io_req_task_work_add(req);
 }
 

-- 
Jens Axboe