linux-kernel - RE: [PATCH] rcu: Remove duplicate settings for rcu boost kthreads affinity

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sat, 17 Dec 2022 13:02:03 +0000
From:   "Zhang, Qiang1" <qiang1.zhang@...el.com>
To:     "Zhang, Qiang1" <qiang1.zhang@...el.com>,
        "paulmck@...nel.org" <paulmck@...nel.org>
CC:     "frederic@...nel.org" <frederic@...nel.org>,
        "quic_neeraju@...cinc.com" <quic_neeraju@...cinc.com>,
        "joel@...lfernandes.org" <joel@...lfernandes.org>,
        "rcu@...r.kernel.org" <rcu@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: RE: [PATCH] rcu: Remove duplicate settings for rcu boost kthreads
 affinity

On Thu, Dec 15, 2022 at 05:00:29PM +0800, Zqiang wrote:
> Currently, when CPUs going offline, the rcu boost kthreads CPU
> affinity has been reset in rcutree_offline_cpu(), therefore this
> commit remove duplicate settings in rcutree_dead_cpu().
> 
> Signed-off-by: Zqiang <qiang1.zhang@...el.com>
>
>One question...
>
>Given that rcutree_offline_cpu() causes rcu_boost_kthread_setaffinity()
>to be invoked with the number of the outgoing CPU, but rcutree_dead_cpu()
>instead passes in -1, are these two invocations really redundant?
>
>Although -1 is passed in, but at this time, the outgoing CPU has been removed from 
>rcu_rnp_online_cpus(rnp),  in other words, we will eliminate the same CPU, which is
>also  we passed in cpu in the rcutree_offline_cpu() before.
>

I also used " qemu -smp 4 "for a simple test.

1) rcutree_offline_cpu()

(gdb) bt
#0  set_cpus_allowed_ptr (p=p@...ry=0xffff888100bd9cc0, new_mask=new_mask@...ry=0xffff888100ce7cd0)
    at kernel/sched/core.c:2939
#1  0xffffffff811be169 in rcu_boost_kthread_setaffinity (rnp=0xffffffff837a4c00 <rcu_state>,
    outgoingcpu=outgoingcpu@...ry=3) at kernel/rcu/tree_plugin.h:1250
#2  0xffffffff811cfa9f in rcutree_affinity_setting (outgoing=3, cpu=3) at kernel/rcu/tree.c:4297
#3  rcutree_offline_cpu (cpu=3) at kernel/rcu/tree.c:4341
#4  0xffffffff810c4e6f in cpuhp_invoke_callback (cpu=cpu@...ry=3, state=state@...ry=CPUHP_AP_RCUTREE_ONLINE,
    bringup=bringup@...ry=false, node=<optimized out>, lastp=lastp@...ry=0xffff88815b027568) at kernel/cpu.c:192
#5  0xffffffff810c6e24 in cpuhp_thread_fun (cpu=<optimized out>) at kernel/cpu.c:785
#6  0xffffffff8111784e in smpboot_thread_fn (data=0xffff88810014c3c0) at kernel/smpboot.c:164
#7  0xffffffff8110c88e in kthread (_create=0xffff8881009b8b00) at kernel/kthread.c:376
#8  0xffffffff81002902 in ret_from_fork () at arch/x86/entry/entry_64.S:306
#9  0x0000000000000000 in ?? ()

(gdb) p/x *0xffff888100ce7cd0
$1 = 0x7

2) rcutree_dead_cpu()

#0  set_cpus_allowed_ptr (p=p@...ry=0xffff888100bd9cc0, new_mask=new_mask@...ry=0xffff888109fc7948)
    at kernel/sched/core.c:2939
#1  0xffffffff811be169 in rcu_boost_kthread_setaffinity (rnp=0xffffffff837a4c00 <rcu_state>,
    outgoingcpu=outgoingcpu@...ry=-1) at kernel/rcu/tree_plugin.h:1250
#2  0xffffffff811cf4a1 in rcutree_dead_cpu (cpu=3) at kernel/rcu/tree.c:4176
#3  0xffffffff810c4e6f in cpuhp_invoke_callback (cpu=cpu@...ry=3, state=CPUHP_RCUTREE_PREP, bringup=bringup@...ry=false,
    node=node@...ry=0x0 <fixed_percpu_data>, lastp=lastp@...ry=0x0 <fixed_percpu_data>) at kernel/cpu.c:192
#4  0xffffffff810c58f5 in cpuhp_invoke_callback_range (bringup=bringup@...ry=false, cpu=cpu@...ry=3,
    st=st@...ry=0xffff88815b027540, target=target@...ry=CPUHP_OFFLINE) at kernel/cpu.c:675
#5  0xffffffff827a5f6a in cpuhp_down_callbacks (target=CPUHP_OFFLINE, st=0xffff88815b027540, cpu=3) at kernel/cpu.c:1170
#6  _cpu_down (cpu=cpu@...ry=3, tasks_frozen=tasks_frozen@...ry=0, target=target@...ry=CPUHP_OFFLINE) at kernel/cpu.c:1170
#7  0xffffffff810c6ff8 in cpu_down_maps_locked (target=CPUHP_OFFLINE, cpu=3) at kernel/cpu.c:1202
#8  cpu_down (cpu=3, target=target@...ry=CPUHP_OFFLINE) at kernel/cpu.c:1210
#9  0xffffffff810c7906 in cpu_device_down (dev=dev@...ry=0xffff88815b026988) at kernel/cpu.c:1227

(gdb) p/x *0xffff888109fc7948
$2 = 0x7

As can be seen from the above, we have set the same cpumask twice for the same boost kthreads.

Thanks
Zqiang


>Thanks
>Zqiang

>
>							Thanx, paul

> ---
>  kernel/rcu/tree.c | 5 -----
>  1 file changed, 5 deletions(-)
> 
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index 65f3dd2fd3ae..20de83ed0c7e 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -4122,15 +4122,10 @@ static void rcu_cleanup_dead_rnp(struct rcu_node *rnp_leaf)
>   */
>  int rcutree_dead_cpu(unsigned int cpu)
>  {
> -	struct rcu_data *rdp = per_cpu_ptr(&rcu_data, cpu);
> -	struct rcu_node *rnp = rdp->mynode;  /* Outgoing CPU's rdp & rnp. */
> -
>  	if (!IS_ENABLED(CONFIG_HOTPLUG_CPU))
>  		return 0;
>  
>  	WRITE_ONCE(rcu_state.n_online_cpus, rcu_state.n_online_cpus - 1);
> -	/* Adjust any no-longer-needed kthreads. */
> -	rcu_boost_kthread_setaffinity(rnp, -1);
>  	// Stop-machine done, so allow nohz_full to disable tick.
>  	tick_dep_clear(TICK_DEP_BIT_RCU);
>  	return 0;
> -- 
> 2.25.1
>