Message-Id: <20210111152638.2417-1-jiangshanlai@gmail.com>
Date:   Mon, 11 Jan 2021 23:26:30 +0800
From:   Lai Jiangshan <jiangshanlai@...il.com>
To:     linux-kernel@...r.kernel.org
Cc:     Valentin Schneider <valentin.schneider@....com>,
        Peter Zijlstra <peterz@...radead.org>,
        Qian Cai <cai@...hat.com>,
        Vincent Donnefort <vincent.donnefort@....com>,
        Tejun Heo <tj@...nel.org>,
        "Paul E . McKenney" <paulmck@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Hillf Danton <hdanton@...a.com>,
        Lai Jiangshan <laijs@...ux.alibaba.com>
Subject: [PATCH -tip V4 0/8] workqueue: break affinity initiatively

From: Lai Jiangshan <laijs@...ux.alibaba.com>

06249738a41a ("workqueue: Manually break affinity on hotplug")
said that scheduler will not force break affinity for us.

But workqueue highly depends on the old behavior. Many parts of the codes
relies on it, 06249738a41a ("workqueue: Manually break affinity on hotplug")
is not enough to change it, and the commit has flaws in itself too.

It doesn't handle for worker detachment.
It doesn't handle for worker attachement, especially worker creation
  which is handled by Valentin Schneider's patch [1].
It doesn't handle for unbound workers which might be possible
per-cpu-kthread.

We need to thoroughly update the way workqueue handles affinity
in cpu hot[un]plug, what is this patchset intends to do and
replace the Valentin Schneider's patch [1].  The equivalent patch
is patch 8 here.

The patchset is based on tip/master rather than workqueue tree,
because the patchset is a complement for 06249738a41a ("workqueue:
Manually break affinity on hotplug") which is only in tip/master by now.

And TJ acked to route the series through tip.

Changed from V3:
	split hotplug callbacks

	introduce break_unbound_workers_cpumask() rather than resuing
	restore_unbound_workers_cpumask().

Changed from V2:
	Drop V2's patch4, which causes warning about setting cpumask
	online&!active to kthread reported by several people:
		Dexuan Cui <decui@...rosoft.com>
		kernel test robot <oliver.sang@...el.com>

	Drop V2's patch 1, which can also cause warning about setting
	cpumask online&!active to kthread.  restore_unbound_workers_cpumask()
	is changed when we are bring cpu online.  And it cause V2's patch7
	(V3's patch5) to be changed accordingly.

	Marked patch8 Reviewed-by: Valentin Schneider <valentin.schneider@....com>

Changed from V1:
	Add TJ's acked-by for the whole patchset

	Add more words to the comments and the changelog, mainly derived
	from discussion with Peter.

	Update the comments as TJ suggested.
	
	Update a line of code as Valentin suggested.

	Add Valentin's ack for patch 10 because "Seems alright to me." and
	add Valentin's comments to the changelog which is integral.

[1]: https://lore.kernel.org/r/ff62e3ee994efb3620177bf7b19fab16f4866845.camel@redhat.com
[V1 patchset]: https://lore.kernel.org/lkml/20201214155457.3430-1-jiangshanlai@gmail.com/
[V2 patchset]: https://lore.kernel.org/lkml/20201218170919.2950-1-jiangshanlai@gmail.com/
[V3 patchset]: https://lore.kernel.org/lkml/20201226025117.2770-1-jiangshanlai@gmail.com/

Lai Jiangshan (8):
  workqueue: split cpuhotplug callbacks for unbound workqueue
  workqueue: set pool->attr->cpumask to workers when cpu online
  workqueue: use cpu_possible_mask instead of cpu_active_mask to break
    affinity
  workqueue: Manually break affinity on pool detachment
  workqueue: introduce wq_unbound_online_cpumask
  workqueue: use wq_unbound_online_cpumask in
    restore_unbound_workers_cpumask()
  workqueue: Manually break affinity on hotplug for unbound pool
  workqueue: Fix affinity of kworkers when attaching into pool

 include/linux/cpuhotplug.h |   4 +
 include/linux/workqueue.h  |   2 +
 kernel/cpu.c               |   5 +
 kernel/workqueue.c         | 192 +++++++++++++++++++++++++++++--------
 4 files changed, 165 insertions(+), 38 deletions(-)

-- 
2.19.1.6.gb485710b

Powered by blists - more mailing lists