linux-kernel - Re: [RFC PATCH 0/3] workqueue: Control cpu affinity of !WQ

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <7h4n30k641.fsf@paris.lan>
Date:	Fri, 14 Mar 2014 11:06:06 -0700
From:	Kevin Hilman <khilman@...aro.org>
To:	Frederic Weisbecker <fweisbec@...il.com>
Cc:	LKML <linux-kernel@...r.kernel.org>,
	Christoph Lameter <cl@...ux.com>,
	Mike Galbraith <bitbucket@...ine.de>,
	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	Tejun Heo <tj@...nel.org>,
	Viresh Kumar <viresh.kumar@...aro.org>
Subject: Re: [RFC PATCH 0/3] workqueue: Control cpu affinity of !WQ_SYSFS unbound workqueues

Frederic Weisbecker <fweisbec@...il.com> writes:

> There are several types of workqueues. Some of them are bound to specific
> CPUs, some others are unbound and can be executed on any CPU.
>
> A tiny subset of the unbound workqueues have a sysfs representation
> in /sys/devices/virtual/workqueue/ and have a cpumask file than can
> be used to tweak their CPU affinity.
>
> But the vast majority of unbound workqueues aren't visible in sysfs.
>
> They are a problem nowaday because people who work on CPU isolation
> (HPC, Real time, etc...) want to be able to migrate all the unbound
> workqueues away from specific CPUs.
>
> There are several possible solutions to solve this:
>
> 1) Affine the !WQ_SYSFS unbound workqueues to the CPUs outside the
> full dynticks mask. Full dynticks is expected to be a component in
> many CPU isolation configurations and its CPU mapping can be a good
> way to retrieve the desired set of isolated CPUs.
>
> On the drawbacks though we can notice the lack of consistency with
> WQ_SYSFS workqueue affinity interface, issues with ordering between
> workqueue and dynticks subsystem initialization, intrusion from
> the workqueue subsystem on dynticks internals.
>
> 2) Implement a sysfs directory for each unbound !WQ_SYSFS. That sounds
> like a very nice solution as it uses existing and known interface.
> But workqueues appearing in the sysfs hierarchy are subject to become
> stable ABIs. And this is definetly not what we want.
>
> This could be worked around with a specific Kconfig to make sure that
> these workqueues won't be considered as a stable ABI. But we all know
> that all distros will enable this Kconfig symbol and that nobody
> reads, nor care about, warnings in Kconfig help text which thereby won't
> protect us against anything.
>
> 3) Implement a single sysfs directory to map properties of all !WQ_SYSFS
> unbound workqueues. It would contain only the cpumask file to control
> the affinity of all these workqueues. But more can be added later.
>
> This complexifies the code a bit although not that much compared to
> solution 2 which requires some plumbling to cope with workqueues created
> before sysfs, as reported by Mike (I played with that a bit as well, as I
> took that direction initially). But it deals with all issues previously
> described.

Since I also tinkered with (1) and (2) and ran into some of the same
issues, I think (3) is a good way.  It also doesn't tie the affinity to
full_nohz, and leaves it up to userspace which addresses a concern of
Mike's in earlier proposals as well.

Kevin
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/