linux-kernel - Re: [fuse-devel] [PATCH] fuse: avoid deadlock when write fuse inode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAJfpegvR1khmFbQ99EHguBTQqG8cPgnQHSyoxEi6brf=+-U7QQ@mail.gmail.com>
Date:   Mon, 25 Apr 2022 15:15:53 +0200
From:   Miklos Szeredi <miklos@...redi.hu>
To:     Rokudo Yan <wu-yan@....com>
Cc:     Ed Tsai (蔡宗軒) <Ed.Tsai@...iatek.com>,
        guoweichao@...o.com, Huang Jianan <huangjianan@...o.com>,
        linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-mm <linux-mm@...ck.org>, zhangshiming@...o.com
Subject: Re: [fuse-devel] [PATCH] fuse: avoid deadlock when write fuse inode

On Thu, 10 Mar 2022 at 12:11, Rokudo Yan <wu-yan@....com> wrote:
>
> Hi, Miklos
>
> The similar issue occurs in our Android device(4G RAM + 3G zram + 8 arm cores + kernel-4.14) too.
> Under the monkey test, kswapd and fuse daemon thread deadlocked when free pages is extreme low
> (less than 1/2 of the min watermark), the backtrace of the 2 threads is as follows. kswapd
> try to evict inode to free some memory(blocked at inode_wait_for_writeback), and fuse daemon thread
> handle the fuse inode write request, which is throttled when do direct reclaim in page allocation
> slow path(blocked at throttle_direct_reclaim). As the __GFP_FS is set, the thread is throttled until
> kswapd free enough pages until watermark ok(check allow_direct_reclaim), which cause the deadlock.
> Although the kernel version is 4.14, the same issue exists in the upstream kernel too.
>
> kswapd0         D 26485194.538158 157 1287917 23577482 0x1a20840 0x0 157 438599862461462
> <ffffff8beec866b4> __switch_to+0x134/0x150
> <ffffff8befb838cc> __schedule+0xd5c/0x1100
> <ffffff8befb83ce0> schedule+0x70/0x90
> <ffffff8befb849b4> bit_wait+0x14/0x54
> <ffffff8befb84350> __wait_on_bit+0x74/0xe0
> <ffffff8beeeae0b4> inode_wait_for_writeback+0xa0/0xe4

This is the one I don't understand.  Fuse inodes must never be dirty
on eviction for the reason stated in my previous reply:

> > I don't see how it can happen on upstream kernels, since there's a
> >"write_inode_now(inode, 1)" call in fuse_release() and nothing can
> > dirty the inode after the file has been released.

If you could trace the source of this dirtyness I think that would
explain this deadlock.

Thanks,
Miklos