Date:   Mon, 11 Jul 2022 10:10:51 +0200
From:   Greg KH <gregkh@...uxfoundation.org>
To:     Tetsuo Handa <penguin-kernel@...ove.sakura.ne.jp>
Cc:     Oliver Neukum <oneukum@...e.com>,
        Wedson Almeida Filho <wedsonaf@...gle.com>,
        "Rafael J. Wysocki" <rjw@...k.pl>,
        Arjan van de Ven <arjan@...ux.intel.com>,
        Len Brown <len.brown@...el.com>,
        Dmitry Vyukov <dvyukov@...gle.com>, linux-pm@...r.kernel.org,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v2 1/4] char: misc: allow calling open() callback without
 misc_mtx held

On Sun, Jul 10, 2022 at 11:23:33AM +0900, Tetsuo Handa wrote:
> syzbot is reporting hung task at misc_open() [1], for there is a race
> window of AB-BA deadlock which involves probe_count variable.
> 
> Currently, wait_for_device_probe() from snapshot_open() from misc_open()
> can sleep forever with misc_mtx held if probe_count cannot become 0. When
> a device is probed, probe_count is incremented before the probe function
> starts, and probe_count is decremented after the probe function completed.
> 
> When USB storage device "sddr09" is probed by hub_event() work in
> usb_hub_wq workqueue, sddr09_probe() is called with elevated probe_count.
> 
> Inside sddr09_probe(), usb_stor_msg_common() from usb_stor_ctrl_transfer()
> calls wait_for_completion_interruptible_timeout() (w_f_c_i_t() afterward)
> with no timeout.
> 
> If sddr09 device does not respond (when using real hardware, or cannot
> respond when using emulated hardware), w_f_c_i_t() cannot return, which
> means that probe_count cannot be decremented.
> 
> According to Oliver Neukum, we can't pass some timeout when calling
> usb_stor_msg_common() from usb_stor_ctrl_transfer(), for the timeout is
> supposed to come from the SCSI layer in the general case.
> 
> The reason why syzkaller processes cannot make w_f_c_i_t() return is that,
> w_f_c_i_t() can return when a syzkaller process which is emulating a USB
> device calls fput() on /dev/raw-gadget due to process termination.
> 
> When we run the reproducer, the syzkaller process which is emulating a USB
> device cannot call fput() on /dev/raw-gadget because that process is
> blocked at mutex_lock(&misc_mtx) in misc_open().
> 
> The process which is holding misc_mtx is waiting for probe_count to become
> 0, but the probe function which is called from hub_event() is waiting for
> the processes which are blocked at mutex_lock(&misc_mtx) to call close()
> on /dev/raw-gadget. This is the phenomenon syzbot is reporting.
> 
> Therefore, as one of steps for making it possible to recover from such
> situation, this patch allows miscdev to call its open() callback without
> misc_mtx held.
> 
> Wedson Almeida Filho worried that this change breaks the invariants of
> miscdev that driver's open() callback will not be made after once
> misc_deregister() is called. But since /dev/snapshot driver does not call
> misc_deregister(), I consider that this change is safe for allowing
> snapshot_open() to be called from misc_open() without misc_mtx held.
> 
> Note that lock_system_sleep() from snapshot_open() has the same problem
> with mutex_lock(&misc_mtx) from misc_open(). This patch alone makes more
> hard to debug, for khungtaskd no longer complains about lock_system_sleep()
> because lock_system_sleep() sets PF_FREEZER_SKIP flag before calling
> mutex_lock(&system_transition_mutex). How to avoid unbounded
> uninterruptible sleeping on system_transition_mutex with PF_FREEZER_SKIP
> flag set deserves different patches.
> 
> Link: https://syzkaller.appspot.com/bug?extid=358c9ab4c93da7b7238c [1]
> Reported-by: syzbot <syzbot+358c9ab4c93da7b7238c@...kaller.appspotmail.com>
> Signed-off-by: Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>
> Cc: Greg KH <gregkh@...uxfoundation.org>
> Cc: Oliver Neukum <oneukum@...e.com>
> Cc: Wedson Almeida Filho <wedsonaf@...gle.com>
> Cc: Rafael J. Wysocki <rjw@...k.pl>
> Cc: Arjan van de Ven <arjan@...ux.intel.com>
> ---
> Changes in v2:
>   Call open() without misc_mtx, instread of making misc_mtx killable.
>   Split into 3 (+ 1 optional) patches.
> 
> v1 is at at https://lkml.kernel.org/r/72e74af9-f1b6-e383-a2c3-6ee8a0aea5e0@I-love.SAKURA.ne.jp .
> 
>  drivers/char/misc.c        | 4 ++++
>  include/linux/miscdevice.h | 1 +
>  kernel/power/user.c        | 1 +
>  3 files changed, 6 insertions(+)

You talk a lot about scsi devices, yet this change has nothing to do
with scsi devices, so that's very confusing and odd.

You also do not document the new misc device flag, and it's totally not
obvious as to why a driver would, or would not, want to set that flag.
How are you handling the loss of the locking order in the misc driver
with the flag not set, it seems like "we can just drop it" is the wrong
thing here.

thanks,

greg k-h

Powered by blists - more mailing lists