linux-kernel - Re: [lkp-robot] [rcutorture] 46e26223e3: WARNING:at_kernel/rcu/rcutorture.c:#rcu_torture_stats

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 18 Jun 2018 18:36:06 -0700
From:   Joel Fernandes <joelaf@...gle.com>
To:     paulmck@...ux.vnet.ibm.com,
        "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
        Joel Fernandes <joel@...lfernandes.org>
CC:     kernel test robot <xiaolong.ye@...el.com>,
        linux-kernel@...r.kernel.org, kernel-team@...roid.com,
        Josh Triplett <josh@...htriplett.org>,
        Lai Jiangshan <jiangshanlai@...il.com>,
        Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
        mingo@...hat.com, Steven Rostedt <rostedt@...dmis.org>,
        tglx@...utronix.de, lkp@...org
Subject: Re: [lkp-robot] [rcutorture]  46e26223e3: WARNING:at_kernel/rcu/rcutorture.c:#rcu_torture_stats_print



On June 18, 2018 6:08:03 PM PDT, "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com> wrote:
>On Mon, Jun 18, 2018 at 03:26:47PM -0700, Joel Fernandes wrote:
>> On Mon, Jun 18, 2018 at 09:56:46AM -0700, Paul E. McKenney wrote:
>> > > The reason for the rcutorture test failure could be that the
>default
>> > > kthread_prio for the system's RCU threads is set to 1 (unless
>overridden by
>> > > rcutree.kthread_prio) which is also equal to the priority of the
>rcutorture's
>> > > boost threads. Due to this the rcutorture test could starve the
>RCU threads
>> > > as well and defeat the boosting mechanism. I was able to solve a
>similar
>> > > issue by just passing rcutree.kthread_prio of 50 on the kernel
>command line.
>> > > 
>> > > Paul, would it be ok if we changed the default kthread_prio to
>something > 1
>> > > so that rcutorture can test properly without needing to pass any
>extra
>> > > rcutree.* parameters?
>> > > 
>> > > so something like this in kernel/rcu/tree.c ?
>> > > 
>> > > static int kthread_prio = IS_ENABLED(CONFIG_RCU_BOOST) ? 2 : 0;
>> > 
>> > Would it be possible to also condition this on rcutorture being
>built
>> > in?  Or are they doing modprobes for rcutorture?
>> 
>> They seem to be doing built-in rcutorture tests. But I believe the
>same
>> problem would occur even if you used modules? I believe the fact that
>> rcutorture is a module or built-in wouldn't matter to the underlying
>issue
>> which is the RCU subsystems's threads are at too low of a priority
>> (rcutree.kthread_prio = 1).
>
>Understood...
>
>> If you agree with changing the default priority, I have included a
>patch
>> below for rcu/dev.
>
>The problem is that without rcutorture, rcutree.kthread_prio=1 is a
>legitimate choice, and changing the default globally could be breaking
>someone.  So it would be far better to up the priority only during
>known
>rcutorture testing.

Oh I see what you're saying. I'll work on a patch along these lines then. Thanks!

- Joel




>
>							Thanx, Paul
>
>> thanks,
>> 
>>  - Joel
>> 
>> ---8<-----------------------
>> 
>> >From b0f4111ef1abd1c481c269fadb3535c83ab43c93 Mon Sep 17 00:00:00
>2001
>> From: "Joel Fernandes (Google)" <joel@...lfernandes.org>
>> Date: Mon, 18 Jun 2018 15:13:10 -0700
>> Subject: [PATCH] rcu: Change default RCU kthread priority to 2
>> 
>> The current RT priority of 1 for RCU kthreads makes rcutorture's
>boost test
>> fail on systems where rcutree.kthread_prio isn't passed.
>> 
>> The rcutorture boost kthreads have the same priority as well (RT
>priority of
>> 1). Due to this, the rcutorture kthreads starve the RCU subsystem's
>kthreads
>> and causes rcutorture failures. This patch changes the priority of
>the RCU
>> subsystem's threads to a default RT priority of 2 so that
>rcutorture's
>> threads get preempted by them. Verified that the boost tests will
>pass with
>> this change.
>> 
>> Reported-by: Xiaolong Ye (via lkp-robot) <xiaolong.ye@...el.com>
>> Signed-off-by: Joel Fernandes (Google) <joel@...lfernandes.org>
>> ---
>>  kernel/rcu/tree.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
>> index deb2508be923..920c39e3f871 100644
>> --- a/kernel/rcu/tree.c
>> +++ b/kernel/rcu/tree.c
>> @@ -171,7 +171,7 @@ static void rcu_report_exp_rdp(struct rcu_state
>*rsp,
>>  static void sync_sched_exp_online_cleanup(int cpu);
>> 
>>  /* rcuc/rcub kthread realtime priority */
>> -static int kthread_prio = IS_ENABLED(CONFIG_RCU_BOOST) ? 1 : 0;
>> +static int kthread_prio = IS_ENABLED(CONFIG_RCU_BOOST) ? 2 : 0;
>>  module_param(kthread_prio, int, 0644);
>> 
>>  /* Delay in jiffies for grace-period initialization delays, debug
>only. */
>> -- 
>> 2.18.0.rc1.244.gcf134e6275-goog
>> 

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.