linux-kernel - Re: [PATCH 2/2] rcutorture: Nudge ksoftirqd priority for RCU boost testing

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20210804225332.GB4397@paulmck-ThinkPad-P17-Gen-1>
Date:   Wed, 4 Aug 2021 15:53:32 -0700
From:   "Paul E. McKenney" <paulmck@...nel.org>
To:     Valentin Schneider <valentin.schneider@....com>
Cc:     linux-kernel@...r.kernel.org, rcu@...r.kernel.org,
        linux-rt-users@...r.kernel.org, Ingo Molnar <mingo@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Steven Rostedt <rostedt@...dmis.org>,
        Daniel Bristot de Oliveira <bristot@...hat.com>,
        Frederic Weisbecker <frederic@...nel.org>,
        Josh Triplett <josh@...htriplett.org>,
        Davidlohr Bueso <dave@...olabs.net>,
        Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
        Lai Jiangshan <jiangshanlai@...il.com>,
        Joel Fernandes <joel@...lfernandes.org>
Subject: Re: [PATCH 2/2] rcutorture: Nudge ksoftirqd priority for RCU boost
 testing

On Wed, Aug 04, 2021 at 11:18:11AM +0100, Valentin Schneider wrote:
> On 03/08/21 16:42, Paul E. McKenney wrote:
> > On Tue, Aug 03, 2021 at 11:54:37PM +0100, Valentin Schneider wrote:
> >> diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
> >> index 680f66b65f14..3dd5fa75f469 100644
> >> --- a/kernel/rcu/rcutorture.c
> >> +++ b/kernel/rcu/rcutorture.c
> >> @@ -948,12 +948,26 @@ static int rcu_torture_boost(void *arg)
> >>      unsigned long endtime;
> >>      unsigned long oldstarttime;
> >>      struct rcu_boost_inflight rbi = { .inflight = 0 };
> >> +	struct task_struct *ksoftirqd = this_cpu_ksoftirqd();
> >>
> >>      VERBOSE_TOROUT_STRING("rcu_torture_boost started");
> >>
> >>      /* Set real-time priority. */
> >>      sched_set_fifo_low(current);
> >>
> >> +	/*
> >> +	 * Boost testing requires TIMER_SOFTIRQ to run at a higher priority
> >> +	 * than the CPU-hogging torture kthreads, otherwise said threads
> >> +	 * will never let timer expiry for the RCU GP kthread happen, which will
> >> +	 * prevent any boosting.
> >> +	 */
> >> +	if (current->normal_prio < ksoftirqd->normal_prio) {
> >
> > Would it make sense to add IS_ENABLED(CONFIG_PREEMPT_RT) to the above
> > condition?
> >
> 
> Hm so v5.13-rt1 has this commit:
> 
>   5e59fba573e6 ("rcutorture: Fix testing of RCU priority boosting")
> 
> which gates RCU boost torture testing under CONFIG_PREEMPT_RT. Now, AFAICT
> the TIMER_SOFTIRQ priority problem is there regardless of
> CONFIG_PREEMPT_RT, so this patch would (should?) make sense even on
> !CONFIG_PREEMPT_RT.

What rcutorture scenario TREE03 does is to boot with tree.use_softirq=0
and threadirqs.  I see your point about timers and softirq, but this
does run reliably for me.

Ah, I see why.  Commit ea6d962e80b6 ("rcutorture: Judge RCU priority
boosting on grace periods, not callbacks") includes boosting the priority
of the ksoftirqd kthreads.  But only when running rcutorture builtin,
not as a module.  Here is the code in rcu_torture_init():

		// Testing RCU priority boosting requires rcutorture do
		// some serious abuse.  Counter this by running ksoftirqd
		// at higher priority.
		if (IS_BUILTIN(CONFIG_RCU_TORTURE_TEST)) {
			for_each_online_cpu(cpu) {
				struct sched_param sp;
				struct task_struct *t;

				t = per_cpu(ksoftirqd, cpu);
				WARN_ON_ONCE(!t);
				sp.sched_priority = 2;
				sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
			}
		}

I take it that you were running rcutorture as a module?

This describes how to run it built-in, if that works for you:

https://paulmck.livejournal.com/61432.html

More specifically: https://paulmck.livejournal.com/57769.html

Alternatively, the "IS_BUILTIN(CONFIG_RCU_TORTURE_TEST)" check could be
removed in the above code, and the ksoftirqd kthreads could have their
original priority restored in rcu_torture_cleanup().

Thoughts?

							Thanx, Paul

> >> +		struct sched_param sp = { .sched_priority = 2 };
> >> +
> >> +		pr_alert("%s(): Adjusting %s priority\n", __func__, ksoftirqd->comm);
> >> +		sched_setscheduler_nocheck(ksoftirqd, SCHED_FIFO, &sp);
> >> +	}
> >> +
> >>      init_rcu_head_on_stack(&rbi.rcu);
> >>      /* Each pass through the following loop does one boost-test cycle. */
> >>      do {
> >> --
> >> 2.25.1
> >>