linux-kernel - Re: [Question] softlockup in run_timer

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CANDhNCqfBdh8zUd+LseTTQKpmJ27Uid+ZV_+FNckZPNc2Oy3-w@mail.gmail.com>
Date:   Mon, 13 Feb 2023 12:00:56 -0800
From:   John Stultz <jstultz@...gle.com>
To:     "liujian (CE)" <liujian56@...wei.com>
Cc:     "tglx@...utronix.de" <tglx@...utronix.de>,
        "sboyd@...nel.org" <sboyd@...nel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        peterz@...radead.org, "Paul E. McKenney" <paulmck@...nel.org>
Subject: Re: [Question] softlockup in run_timer_softirq

On Fri, Feb 10, 2023 at 1:51 AM liujian (CE) <liujian56@...wei.com> wrote:
>
> During the syz test, we encountered many problems with various timer handler
> functions softlockup.
>
> We analyze __run_timers() and find the following problem.
>
> In the while loop of __run_timers(), because there are too many timers or
> improper timer handler functions, if the processing time of the expired
> timers is always greater than the time wheel's next_expiry, the function
> will loop infinitely.
>
> The following extreme test case can be used to reproduce the problem.
> An extreme test case[1] is constructed to reproduce the problem.

Thanks for reporting and sending out this data:

First, any chance you might submit this as a in-kernel-stress test?
Maybe utilizing the kernel/torture.c framework?

(Though the test may need to occasionally take a break so the system
can eventually catch up)

> Is this a problem or an unreasonable use?
>
> Can we limit the running time of __run_timers() [2]?
>
> Does anyone have a good idea to solve this problem?

So your patch reminds me of Peter's softirq_needs_break() logic:
  https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=core/softirq

Maybe it could extend that series for the timer softirq as well?

thanks
-john