linux-kernel - Re: [RFC][PATCH 4/7] smp: Optimize send_call_function_single

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200527101513.GJ325303@hirez.programming.kicks-ass.net>
Date:   Wed, 27 May 2020 12:15:13 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     tglx@...utronix.de, frederic@...nel.org
Cc:     linux-kernel@...r.kernel.org, x86@...nel.org, cai@....pw,
        mgorman@...hsingularity.net, paulmck@...nel.org,
        joel@...lfernandes.org
Subject: Re: [RFC][PATCH 4/7] smp: Optimize send_call_function_single_ipi()

On Wed, May 27, 2020 at 11:56:45AM +0200, Peter Zijlstra wrote:

> This is rcu_is_cpu_rrupt_from_idle()'s lockdep_assert_in_irq() tripping
> up (it's comment is obviously a bit antiquated).
> 
> Now, if I read that code correctly, it actually relies on
> rcu_irq_enter() and thus really wants to be in an interrupt. Is there
> any way this code can be made to work from the new context too?
> 
> After all, all that really is different is not having gone throught he
> bother of setting up the IRQ context, nothing else changed -- it just so
> happens you actually relied on that ;/

At first glance, something like the below could work. But obviously I
might have overlooked something more subtle than a brick :-)

---

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index 90c8be22d57a..0792c032a972 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -426,8 +426,11 @@ EXPORT_SYMBOL_GPL(rcu_momentary_dyntick_idle);
  */
 static int rcu_is_cpu_rrupt_from_idle(void)
 {
-	/* Called only from within the scheduling-clock interrupt */
-	lockdep_assert_in_irq();
+	/*
+	 * Usually called from the tick; but also used from smp_call_function()
+	 * for expedited grace periods.
+	 */
+	lockdep_assert_irqs_disabled();
 
 	/* Check for counter underflows */
 	RCU_LOCKDEP_WARN(__this_cpu_read(rcu_data.dynticks_nesting) < 0,
@@ -435,8 +438,11 @@ static int rcu_is_cpu_rrupt_from_idle(void)
 	RCU_LOCKDEP_WARN(__this_cpu_read(rcu_data.dynticks_nmi_nesting) <= 0,
 			 "RCU dynticks_nmi_nesting counter underflow/zero!");
 
-	/* Are we at first interrupt nesting level? */
-	if (__this_cpu_read(rcu_data.dynticks_nmi_nesting) != 1)
+	/*
+	 * Are we at first interrupt nesting level? -- or below, when running
+	 * directly from the idle loop itself.
+	 */
+	if (__this_cpu_read(rcu_data.dynticks_nmi_nesting) > 1)
 		return false;
 
 	/* Does CPU appear to be idle from an RCU standpoint? */