Message-ID: <ac753dd6-8b78-4fbc-96fe-353c9be2935f@paulmck-laptop>
Date: Mon, 11 Mar 2024 17:01:28 -0700
From: "Paul E. McKenney" <paulmck@...nel.org>
To: Thomas Gleixner <tglx@...utronix.de>
Cc: Joel Fernandes <joel@...lfernandes.org>,
	Ankur Arora <ankur.a.arora@...cle.com>,
	linux-kernel@...r.kernel.org, peterz@...radead.org,
	torvalds@...ux-foundation.org, akpm@...ux-foundation.org,
	luto@...nel.org, bp@...en8.de, dave.hansen@...ux.intel.com,
	hpa@...or.com, mingo@...hat.com, juri.lelli@...hat.com,
	vincent.guittot@...aro.org, willy@...radead.org, mgorman@...e.de,
	jpoimboe@...nel.org, mark.rutland@....com, jgross@...e.com,
	andrew.cooper3@...rix.com, bristot@...nel.org,
	mathieu.desnoyers@...icios.com, geert@...ux-m68k.org,
	glaubitz@...sik.fu-berlin.de, anton.ivanov@...bridgegreys.com,
	mattst88@...il.com, krypton@...ich-teichert.org,
	rostedt@...dmis.org, David.Laight@...lab.com, richard@....at,
	mjguzik@...il.com, jon.grimm@....com, bharata@....com,
	raghavendra.kt@....com, boris.ostrovsky@...cle.com,
	konrad.wilk@...cle.com, rcu@...r.kernel.org
Subject: Re: [PATCH 15/30] rcu: handle quiescent states for PREEMPT_RCU=n,
 PREEMPT_COUNT=y

On Mon, Mar 11, 2024 at 09:29:14PM +0100, Thomas Gleixner wrote:
> Paul!
> 
> On Mon, Mar 11 2024 at 12:53, Paul E. McKenney wrote:
> > On Mon, Mar 11, 2024 at 08:12:58PM +0100, Thomas Gleixner wrote:
> >> I'd rather spend brain cycles on figuring out whether RCU can be flipped
> >> over between PREEMPT_RCU=n/y at boot or obviously run-time.
> >
> > Well, it is just software, so anything is possible.  But there can
> > be a wide gap between "possible" and "sensible".  ;-)
> 
> Indeed.
> 
> > In theory, one boot-time approach would be build preemptible RCU,
> > and then to boot-time binary-rewrite calls to __rcu_read_lock()
> > and __rcu_read_unlock() to preempt_disable() and preempt_enable(),
> > respectively.  Because preemptible RCU has to treat preemption-disabled
> > regions of code as RCU readers, this Should Just Work.  However, there
> > would then be a lot of needless branches in the grace-period code.
> > Only the ones on fastpaths (for example, context switch) would need
> > to be static-branchified, but there would likely need to be other
> > restructuring, given the need for current preemptible RCU to do a better
> > job of emulating non-preemptible RCU.  (Emulating non-preemptible RCU
> > is of course currently a complete non-goal for preemptible RCU.)
> 
> Sure, but that might be a path to have a more unified RCU model in the
> longer term with the tweak of patching the flavor once at boot.

I am pretty sure that it can be made to happen.  The big question in my
mind is "Is it worth it?"

> > But this one needs careful design and review up front, as in step through
> > all the code and check assumptions and changes in behavior.  After all,
> > this stuff is way easier to break than to debug and fix.  ;-)
> 
> Isn't that where all the real fun is? :)
> 
> > On the other hand, making RCU switch at runtime is...  Tricky.
> 
> I was only half serious about that. Just thought to mention if for
> completeness sake and of course to make you write a novel. :)

A short novel.  ;-)

> > For example, if the system was in non-preemptible mode at rcu_read_lock()
> > time, the corresponding rcu_read_unlock() needs to be aware that it needs
> > to act as if the system was still in non-preemptible mode, and vice versa.
> > Grace period processing during the switch needs to be aware that different
> > CPUs will be switching at different times.  Also, it will be common for a
> > given CPU's switch to span more than one grace period.  So any approach
> > based on either binary rewrite or static branches will need to be set
> > up in a multi-phase multi-grace-period state machine.  Sort of like
> > Frederic's runtime-switched callback offloading, but rather more complex,
> > and way more performance sensitive.
> 
> Of course it would be a complex endavour at the scale of run-time
> switching to or from PREEMPT_RT, which will never happen. Even the boot
> time switch for RT would be way harder than the RCU one :)

Run-time switching of either RCU or PREEMPT_RT would likely provide many
opportunities along the way for black hats.  ;-)

> > But do we even need to switch RCU at runtime, other than to say that
> > we did it?  What is the use case?  Or is this just a case of "it would
> > be cool!"?  Don't get me wrong, I am a sucker for "it would be cool",
> > as you well know, but even for me there are limits.  ;-)
> 
> There is no need for runtime switching other than "it would be cool" and
> I'm happy that even you have limits. :)

So *that* was the point of this whole email exchange.  ;-)

> > At the moment, I would prioritize improving quiescent-state forcing for
> > existing RCU over this, especially perhaps given the concerns from the
> > MM folks.
> >
> > But what is motivating the desire to boot-time/run-time switch RCU
> > between preemptible and non-preemptible?
> 
> The goal of PREEMPT_AUTO is to get rid of the zoo of preemption models
> and their associated horrors. As we debated some time ago we still need
> to have the distinction of preemptible and non-preemptible RCU even with
> that.
> 
> So it's a pretty obvious consequence to make this decision at boot time
> to reduce the number of kernel flavors for distros to build.

Very well, we can look to the distros for requirements, when and if.

> Nothing urgent, but it just came to my mind when reading through this
> thread and replying to Joel.

Got it, thank you!

							Thanx, Paul

Powered by blists - more mailing lists