linux-kernel - Re: [PATCH v2 3/3] softirq: Use a dedicated thread for timer wakeups on PREEMPT

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Zx-ZUyMLWWsxR8nL@localhost.localdomain>
Date: Mon, 28 Oct 2024 15:01:55 +0100
From: Frederic Weisbecker <frederic@...nel.org>
To: Sebastian Andrzej Siewior <bigeasy@...utronix.de>
Cc: linux-kernel@...r.kernel.org, rcu@...r.kernel.org,
	"Paul E. McKenney" <paulmck@...nel.org>,
	Anna-Maria Behnsen <anna-maria@...utronix.de>,
	Davidlohr Bueso <dave@...olabs.net>, Ingo Molnar <mingo@...nel.org>,
	Josh Triplett <josh@...htriplett.org>,
	Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [PATCH v2 3/3] softirq: Use a dedicated thread for timer wakeups
 on PREEMPT_RT.

Le Thu, Oct 24, 2024 at 04:55:51PM +0200, Sebastian Andrzej Siewior a écrit :
> A timer/ hrtimer softirq is raised in-IRQ context. With threaded
> interrupts enabled or on PREEMPT_RT this leads to waking the ksoftirqd
> for the processing of the softirq. ksoftirqd runs as SCHED_OTHER which
> means it will compete with other tasks for CPU ressources.
> This can introduce long delays for timer processing on heavy loaded
> systems and is not desired.
> 
> Split the TIMER_SOFTIRQ and HRTIMER_SOFTIRQ processing into a dedicated
> timers thread and let it run at the lowest SCHED_FIFO priority.
> Wake-ups for RT tasks happen from hardirq context so only timer_list timers
> and hrtimers for "regular" tasks are processed here. The higher priority
> ensures that wakeups are performed before scheduling SCHED_OTHER tasks.
> 
> Using a dedicated variable to store the pending softirq bits values
> ensure that the timer are not accidentally picked up by ksoftirqd and
> other threaded interrupts.
> It shouldn't be picked up by ksoftirqd since it runs at lower priority.
> However if ksoftirqd is already running while a timer fires, then
> ksoftird will be PI-boosted due to the BH-lock to ktimer's priority.
> Ideally we try to avoid having ksoftirqd running.
> 
> The timer thread can pick up pending softirqs from ksoftirqd but only
> if the softirq load is high. It is not be desired that the picked up
> softirqs are processed at SCHED_FIFO priority under high softirq load
> but this can already happen by a PI-boost by a force-threaded interrupt.
> 
> [ frederic@...nel.org: rcutorture.c fixes, storm fix by introduction of
>   local_timers_pending() for tick_nohz_next_event() ]
> 
> [ junxiao.chang@...el.com: Ensure ktimersd gets woken up even if a
>   softirq is currently served. ]
> 
> Reviewed-by: Paul E. McKenney <paulmck@...nel.org> [rcutorture]
> Signed-off-by: Sebastian Andrzej Siewior <bigeasy@...utronix.de>

Reviewed-by: Frederic Weisbecker <frederic@...nel.org>

Just a few nits:


> ---
>  include/linux/interrupt.h | 44 +++++++++++++++++++++++++
>  kernel/rcu/rcutorture.c   |  6 ++++
>  kernel/softirq.c          | 69 ++++++++++++++++++++++++++++++++++++++-
>  kernel/time/hrtimer.c     |  4 +--
>  kernel/time/tick-sched.c  |  2 +-
>  kernel/time/timer.c       |  2 +-
>  6 files changed, 122 insertions(+), 5 deletions(-)
> 
> diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h
> index 457151f9f263d..9637af78087f3 100644
> --- a/include/linux/interrupt.h
> +++ b/include/linux/interrupt.h
> @@ -616,6 +616,50 @@ extern void __raise_softirq_irqoff(unsigned int nr);
>  extern void raise_softirq_irqoff(unsigned int nr);
>  extern void raise_softirq(unsigned int nr);
>  
> +/*
> + * Handle timers in a dedicated thread at a low SCHED_FIFO priority instead in
> + * ksoftirqd as to be prefred over SCHED_NORMAL tasks.
> + */

This doesn't parse. How about, inspired by your changelog:

"""
Wake-ups for RT tasks happen from hardirq context so only timer_list timers
and hrtimers for SCHED_OTHER tasks are processed from softirq. As they are
raised from hardirq, their processing would normally happen from ksoftirqd
which runs as SCHED_OTHER and compete with other tasks. Moving timers softirqs
to a low-prio SCHED_FIFO kthread instead ensures that wakeups from timers are
performed before scheduling the target SCHED_OTHER tasks.
"""

Thanks.