linux-kernel - Re: Needed help: dummy_hcd: Fix stalls/inconsistent_lock

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Mon, 24 Jun 2024 15:19:23 +0200
From: Thomas Gleixner <tglx@...utronix.de>
To: Marcello Sylvester Bauer <marcello.bauer@...ements.com>,
 linux-kernel@...r.kernel.org, linux-usb@...r.kernel.org, x86@...nel.org,
 syzbot+c793a7eca38803212c61@...kaller.appspotmail.com,
 syzbot+5127feb52165f8ab165b@...kaller.appspotmail.com,
 oe-lkp@...ts.linux.dev, bp@...en8.de, dave.hansen@...ux.intel.com,
 syzkaller-bugs@...glegroups.com
Cc: Anna-Maria Behnsen <anna-maria@...utronix.de>, Frederic Weisbecker
 <frederic@...nel.org>, Uwe
 Kleine-Koenig <u.kleine-koenig@...gutronix.de>, gregkh@...uxfoundation.org,
 hpa@...or.com, mingo@...hat.com, stern@...land.harvard.edu, Alan Stern
 <stern@...land.harvard.edu>, Matthias Stoeckl
 <matthias.stoeckl@...unet.com>
Subject: Re: Needed help: dummy_hcd: Fix stalls/inconsistent_lock_state due
 to hrtimer migration

On Mon, Jun 24 2024 at 12:25, Marcello Sylvester Bauer wrote:
> I need some help evaluating and fixing a regression due to migration to
> hztimer scheduler in dummy_hcd.
>
> About two months ago I was investigating poor performance for the mass
> storage gadget (g_mass_storage) due to slow timings in the loopback hcd
> driver (dummy_hcd). One of the reasons was that dummy_hcd used the old
> timer API, where the interval is tied to the internal kernel timer
> frequency. So I submitted the patch to migrate to the hrtimer API[^1],
> which was quickly approved.
>
> Since then, syzbot[^2][^3] and intel's kernel test bot[^4] are
> detecting rcu stalls/inconsistent_lock_state due to my patch, and I'm
> trying to figure out how to fix it.
>
> Both bots indicate that the problem is around the usb_hcd_giveback_urb
> function call and it's locking mechanism. 
>
> My patch just replaces the timer API calls without changing anything
> else in the code, so I'm not sure if my patch is actually the root
> cause here. And following the instructions to reproduce syzbot
> regressions[^5] even with the provided assets (bzImage, disk image,
> repro.c) it is quite inconsistent to cause this stall. I have also
> tried to follow Alex Stern's advice, but have not been able to cause a
> stall manually.
>
> So I don't know what to do next. Can someone with more expertise in
> timers look into this?
> Any hints or help in investigating or fixing this regression would be
> greatly appreciated.

The main difference between the timer list timer and the hrtimer is that
the timer list timer callback runs in soft interrupt context, but the
hrtimer callback runs in hard interrupt context.

I really don't know whether it matters, but you can make the hrtimer
expire in soft interrupt context with HRTIMER_MODE_REL_SOFT instead of
HRTIMER_MODE_REL. If you have a reproducer then that should tell you
quickly.

Thanks,

        tglx