linux-kernel - Re: [PATCH -tip V3 0/8] workqueue: break affinity initiatively

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <f5b0f650-5dc1-c67d-f548-340e7ca54d41@kernel.dk>
Date:   Tue, 12 Jan 2021 07:57:26 -0700
From:   Jens Axboe <axboe@...nel.dk>
To:     Valentin Schneider <valentin.schneider@....com>,
        Peter Zijlstra <peterz@...radead.org>,
        Thomas Gleixner <tglx@...utronix.de>
Cc:     Lai Jiangshan <jiangshanlai@...il.com>,
        linux-kernel@...r.kernel.org, Qian Cai <cai@...hat.com>,
        Vincent Donnefort <vincent.donnefort@....com>,
        Dexuan Cui <decui@...rosoft.com>,
        Lai Jiangshan <laijs@...ux.alibaba.com>,
        Paul McKenney <paulmck@...nel.org>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Steven Rostedt <rostedt@...dmis.org>
Subject: Re: [PATCH -tip V3 0/8] workqueue: break affinity initiatively

On 1/11/21 12:21 PM, Valentin Schneider wrote:
> On 11/01/21 18:16, Peter Zijlstra wrote:
>> Sadly it appears like io_uring() uses kthread_create_on_cpu() without
>> then having any hotplug crud on, so that needs additinoal frobbing.
>>
> 
> I noticed that as well sometime ago, and I believed then (still do) this
> usage is broken. I don't think usage of kthread_create_on_cpu() outside
> of smpboot makes sense, because without any hotplug step to park the
> thread, its affinity can end up being reset after its dedicated CPU gets
> offlined.
> 
> I'm clueless about io_uring, but if it *actually* has a good reason to
> use some pcpu kthreads (it seems it doesn't have to be on all CPUs?),
> then it needs to register some hotplug step to park them / do something
> sensible on hotplug.

For io_uring, it's purely used by the SQPOLL mode, which sets aside a
kernel thread for submissions so the application doesn't have to do
anything but write new SQE entries to submit. The thread then notices
these, and submits them. There's an option to affinitize that thread
to a single CPU, which often makes sense for setups like that. Think
of it like a strong hint.

Things aren't going to break if this CPU goes away and we end up being
affinitized to some other CPU, though it is suboptimal. So I guess we
might need some notifiers to ensure that we reset the CPU back again
if it's gone offline+online again? I can take a look at that.

-- 
Jens Axboe