linux-kernel - Re: [PATCH 11/20] sched: Handle CPU isolation on last resort fallback rq selection

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20241028162514.GA2709@willie-the-truck>
Date: Mon, 28 Oct 2024 16:25:15 +0000
From: Will Deacon <will@...nel.org>
To: Frederic Weisbecker <frederic@...nel.org>
Cc: LKML <linux-kernel@...r.kernel.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Vincent Guittot <vincent.guittot@...aro.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	Michal Hocko <mhocko@...nel.org>, Vlastimil Babka <vbabka@...e.cz>,
	"Paul E. McKenney" <paulmck@...nel.org>,
	Neeraj Upadhyay <neeraj.upadhyay@...nel.org>,
	Joel Fernandes <joel@...lfernandes.org>,
	Boqun Feng <boqun.feng@...il.com>,
	Zqiang <qiang.zhang1211@...il.com>,
	Uladzislau Rezki <urezki@...il.com>, rcu@...r.kernel.org,
	Michal Hocko <mhocko@...e.com>
Subject: Re: [PATCH 11/20] sched: Handle CPU isolation on last resort
 fallback rq selection

Hi Frederic,

Thanks for having a crack at this, but I'm pretty confused now so please
prepare for a bunch of silly questions!

On Tue, Oct 15, 2024 at 03:48:55PM +0200, Frederic Weisbecker wrote:
> Le Tue, Oct 08, 2024 at 11:54:35AM +0100, Will Deacon a écrit :
> > On Fri, Sep 27, 2024 at 12:48:59AM +0200, Frederic Weisbecker wrote:
> > > When a kthread or any other task has an affinity mask that is fully
> > > offline or unallowed, the scheduler reaffines the task to all possible
> > > CPUs as a last resort.
> > > 
> > > This default decision doesn't mix up very well with nohz_full CPUs that
> > > are part of the possible cpumask but don't want to be disturbed by
> > > unbound kthreads or even detached pinned user tasks.
> > > 
> > > Make the fallback affinity setting aware of nohz_full. This applies to
> > > all architectures supporting nohz_full except arm32. However this
> > > architecture that overrides the task possible mask is unlikely to be
> > > willing to integrate new development.
> > 
> > I'm not sure I understand this last sentence. The possible mask is
> > overridden for 32-bit tasks on an *arm64* kernel when running on an SoC
> > featuring some CPUs that can execute only 64-bit tasks. Who is unwilling
> > to integrate what?

I should've been clearer in my reply, but I think the most important thing
here for the arm64 heterogeneous SoCs is that we document whatever the
behaviour is in Documentation/arch/arm64/asymmetric-32bit.rst. There are
a few other kernel features that don't play well (e.g. SCHED_DEADLINE),
so it might be sufficient just to call out the limitations relating to
CPU isolation there.

However:

> Will, how does the (untested) following look like? The rationale is that
> we must deal with the fact that CPU supporting 32-bits el0 may appear at
> any time and those may not intersect housekeeping CPUs (housekeeping CPUs
> are CPUs that are not part of nohz_full=.

In the funky SoCs, all CPUs support 64-bit and we have a 64-bit kernel.
Some CPUs additionally support 32-bit but that should only be a concern
for the scheduling of user tasks.

> If nohz_full= isn't used then
> it's cpu_possible_mask). If there is a housekeeping CPU supporting el0 32bits
> then it will be disallowed to be ever offlined. But if the first mismatching
> CPU supporting el0 that pops up is not housekeeping then we may end up
> with that CPU disallowed to be offlined + later if a housekeeping CPU appears
> that also supports 32bits el0 will also be disallowed to be offlined. Ideally
> it should turn back the previous CPU to be offlinable but there may be
> other things that have forbidden that CPU to be offline so...

I'd have thought the bigger problem would be if the set of nohz_full=
CPUs was defined as the set of CPUs that support 32-bit. In that case,
executing a 32-bit task will give the scheduler no choice but to run
the task on a !housekeeping core.

So perhaps we could turn this on its head and explicitly mark the first
32-bit capable CPU as a housekeeping core when the mismatched mode is
enabled? We're already preventing CPU hotplug for the thing, so it's
"special" already. If that conflicts with the nohz_full_option, we can
emit a warning message that we're overriding it. I think that's ok, as
the user will have had to specify 'allow_mismatched_32bit_el0' as well.

Will