linux-kernel - Re: [patch] sched: prevent bound kthreads from changing cpus

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 10 Jun 2008 08:38:16 -0700
From:	Max Krasnyansky <maxk@...lcomm.com>
To:	Peter Zijlstra <peterz@...radead.org>
CC:	David Rientjes <rientjes@...gle.com>, Paul Jackson <pj@....com>,
	mingo@...e.hu, menage@...gle.com, linux-kernel@...r.kernel.org,
	Oleg Nesterov <oleg@...sign.ru>
Subject: Re: [patch] sched: prevent bound kthreads from changing	cpus_allowed



Peter Zijlstra wrote:
> On Mon, 2008-06-09 at 13:59 -0700, Max Krasnyanskiy wrote:
>> David Rientjes wrote:
>>>>  2) Sometimes calls to kthread_bind are binding to any online cpu, such as in:
>>>>
>>>> 	drivers/infiniband/hw/ehca/ehca_irq.c:	kthread_bind(cct->task, any_online_cpu(cpu_online_map));
>>>>
>>>>     In such cases, the PF_THREAD_BOUND seems inappropriate.  The caller of
>>>>     kthread_bind() really doesn't seem to care where that thread is bound;
>>>>     they just want it on a CPU that is still online.
>>>>
>>> This particular case is simply moving the thread to any online cpu so that 
>>> it survives long enough for the subsequent kthread_stop() in 
>>> destroy_comp_task().  So I don't see a problem with this instance.
>>>
>>> A caller to kthread_bind() can always remove PF_THREAD_BOUND itself upon 
>>> return, but I haven't found any cases in the tree where that is currently 
>>> necessary.  And doing that would defeat the semantics of kthread_bind() 
>>> where these threads are supposed to be bound to a specific cpu and not 
>>> allowed to run on others.
>> Actually I have another use case here. Above example in particular may be ok 
>> but it does demonstrate the issue nicely. Which is that in some cases kthreads 
>> are bound to a CPU but do not have a strict "must run here" requirement and 
>> could be moved if needed.
>> For example I need an ability to move workqueue threads. Workqueue threads do 
>> kthread_bind().
> 
> Per cpu workqueues should stay on their cpu.
> 
> What you're really looking for is a more fine grained alternative to
> flush_workqueue().
Actually I had a discussion on that with Oleg Nesterov. If you remember my
original solution (ie centralized cpu_isolate_map) was to completely redirect
work onto other cpus. Then you pointed out that it's the flush_() that really
makes the box stuck. So I started thinking about redoing the flush. While
looking at the code I realized that if I only change the flush_() then queued
work can get stale so to speak. ie Machine does not get stuck but some work
submitted on the isolated cpus will sit there for a long time. Oleg pointed
out exact same thing. So the simplest solution that does not require any
surgery to the workqueue is to just move the threads to other cpus. I did not
want to get into too much detail on the workqueue stuff here. I'll start a
separate thread on this.
As I pointed out, there are a bunch of other kthreads like: kswapd, kacpid,
pdflush, khubd, etc, etc, that clearly do not need any pinning but still
violate cpuset constraints they inherit from kthreadd.

Max



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/