linux-kernel - Re: [PATCH wq/for-4.5-fixes] workqueue: handle NUMA_NO_NODE for unbound pool

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160204084005.GA14430@dhcp22.suse.cz>
Date:	Thu, 4 Feb 2016 09:40:05 +0100
From:	Michal Hocko <mhocko@...nel.org>
To:	Tejun Heo <tj@...nel.org>
Cc:	Mike Galbraith <umgwanakikbuti@...il.com>,
	LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH wq/for-4.5-fixes] workqueue: handle NUMA_NO_NODE for
 unbound pool_workqueue lookup

On Wed 03-02-16 13:54:25, Tejun Heo wrote:
> When looking up the pool_workqueue to use for an unbound workqueue,
> workqueue assumes that the target CPU is always bound to a valid NUMA
> node.  However, currently, when a CPU goes offline, the mapping is
> destroyed and cpu_to_node() returns NUMA_NO_NODE.  This has always
> been broken but hasn't triggered until recently.
> 
> After 874bbfe600a6 ("workqueue: make sure delayed work run in local
> cpu"), workqueue forcifully assigns the local CPU for delayed work
> items without explicit target CPU to fix a different issue.  This
> widens the window where CPU can go offline while a delayed work item
> is pending causing delayed work items dispatched with target CPU set
> to an already offlined CPU.  The resulting NUMA_NO_NODE mapping makes
> workqueue try to queue the work item on a NULL pool_workqueue and thus
> crash.
> 
> Fix it by mapping NUMA_NO_NODE to the default pool_workqueue from
> unbound_pwq_by_node().  This is a temporary workaround.  The long term
> solution is keeping CPU -> NODE mapping stable across CPU off/online
> cycles which is in the works.
> 
> Signed-off-by: Tejun Heo <tj@...nel.org>
> Reported-by: Mike Galbraith <umgwanakikbuti@...il.com>
> Cc: Tang Chen <tangchen@...fujitsu.com>
> Cc: Rafael J. Wysocki <rafael@...nel.org>
> Cc: Len Brown <len.brown@...el.com>
> Cc: stable@...r.kernel.org # v4.3+
> Fixes: 874bbfe600a6 ("workqueue: make sure delayed work run in local cpu")
> Link: http://lkml.kernel.org/g/1454424264.11183.46.camel@gmail.com
> Link: http://lkml.kernel.org/g/1453702100-2597-1-git-send-email-tangchen@cn.fujitsu.com

Reviewed-by: Michal Hocko <mhocko@...e.com>

> ---
>  kernel/workqueue.c |   10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/kernel/workqueue.c b/kernel/workqueue.c
> index 61a0264..f748eab 100644
> --- a/kernel/workqueue.c
> +++ b/kernel/workqueue.c
> @@ -570,6 +570,16 @@ static struct pool_workqueue *unbound_pwq_by_node(struct workqueue_struct *wq,
>  						  int node)
>  {
>  	assert_rcu_or_wq_mutex_or_pool_mutex(wq);
> +
> +	/*
> +	 * XXX: @node can be NUMA_NO_NODE if CPU goes offline while a
> +	 * delayed item is pending.  The plan is to keep CPU -> NODE
> +	 * mapping valid and stable across CPU on/offlines.  Once that
> +	 * happens, this workaround can be removed.
> +	 */
> +	if (unlikely(node == NUMA_NO_NODE))
> +		return wq->dfl_pwq;
> +
>  	return rcu_dereference_raw(wq->numa_pwq_tbl[node]);
>  }
>  

-- 
Michal Hocko
SUSE Labs