linux-kernel - Re: [PATCH 4/5] irq_work: Handle some irq_work in SOFTIRQ on PREEMPT

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YVbjxjzVM5Dx4Mv4@hirez.programming.kicks-ass.net>
Date:   Fri, 1 Oct 2021 12:32:38 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     Sebastian Andrzej Siewior <bigeasy@...utronix.de>
Cc:     linux-kernel@...r.kernel.org, Thomas Gleixner <tglx@...utronix.de>,
        Jiri Olsa <jolsa@...nel.org>
Subject: Re: [PATCH 4/5] irq_work: Handle some irq_work in SOFTIRQ on
 PREEMPT_RT

On Thu, Sep 30, 2021 at 06:38:58PM +0200, Sebastian Andrzej Siewior wrote:
> On 2021-09-30 16:39:51 [+0200], Peter Zijlstra wrote:

> > Runing them all at the same prio still sucks (much like the single
> > net-RX thing), but at least a kthread is somewhat controllable.
> 
> I could replace the softirq processing with a per-CPU thread. This
> should work. But I would have to (still) delay the wake-up of the thread
> to the timer tick - or - we try the wake from the irqwork-self-IPI.

That, just wake the thread from the hardirq.

> I
> just don't know how many will arrive back-to-back. The RCU callback
> (rcu_preempt_deferred_qs_handler()) pops up a lot. By my naive guesswork
> I would say that the irqwork is not needed since preempt-enable
> somewhere should do needed scheduling. But then commit
>   0864f057b050b ("rcu: Use irq_work to get scheduler's attention in clean context")
> 
> claims it is not enough.

Oh gawd, that was something really nasty. I'm not sure that Changelog
captures all (at least I'm not sure I fully understand the problem again
reading it).

But basically that thing wants to reschedule, but suffers the same
problem as:

	preempt_disable();

	<TIF_NEED_RESCHED gets set>

	local_irq_disable();
	preempt_enable();
	  // cannea schedule because IRQs are disabled
	local_irq_enable();
	// lost a reschedule

Yes, that will _eventually_ reschedule, but violates the PREEMPT rules
because there is an unspecified amount of time until it does actually do
reschedule.

So what RCU does there is basically trigger a self-IPI, which guarantees
that we reschedule after IRQs are finally enabled, which then triggers a
resched.

I see no problem marking that particular irq_work as HARD tho, it really
doesn't do anything (other than tell RCU the GP is no longer blocked)
and triggering the return-from-interrupt path.

There's also a fun comment in perf_lock_task_context() that possibly
predates the above RCU fix.