[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <6b068916-5e1b-a943-1aad-554964d8b746@redhat.com>
Date: Tue, 31 Jan 2023 23:14:27 -0500
From: Waiman Long <longman@...hat.com>
To: Will Deacon <will@...nel.org>, linux-kernel@...r.kernel.org
Cc: kernel-team@...roid.com, Peter Zijlstra <peterz@...radead.org>,
Zefan Li <lizefan.x@...edance.com>, Tejun Heo <tj@...nel.org>,
Johannes Weiner <hannes@...xchg.org>, cgroups@...r.kernel.org
Subject: Re: [PATCH 1/2] cpuset: Fix cpuset_cpus_allowed() to not filter
offline CPUs
On 1/31/23 17:17, Will Deacon wrote:
> From: Peter Zijlstra <peterz@...radead.org>
>
> There is a difference in behaviour between CPUSET={y,n} that is now
> wrecking havoc with {relax,force}_compatible_cpus_allowed_ptr().
>
> Specifically, since commit 8f9ea86fdf99 ("sched: Always preserve the
> user requested cpumask") relax_compatible_cpus_allowed_ptr() is
> calling __sched_setaffinity() unconditionally.
>
> But the underlying problem goes back a lot further, possibly to
> commit: ae1c802382f7 ("cpuset: apply cs->effective_{cpus,mems}") which
> switched cpuset_cpus_allowed() from cs->cpus_allowed to
> cs->effective_cpus.
>
> The problem is that for CPUSET=y cpuset_cpus_allowed() will filter out
> all offline CPUs. For tasks that are part of a (!root) cpuset this is
> then later fixed up by the cpuset hotplug notifiers that re-evaluate
> and re-apply cs->effective_cpus, but for (normal) tasks in the root
> cpuset this does not happen and they will forever after be excluded
> from CPUs onlined later.
>
> As such, rewrite cpuset_cpus_allowed() to return a wider mask,
> including the offline CPUs.
>
> Fixes: 8f9ea86fdf99 ("sched: Always preserve the user requested cpumask")
> Reported-by: Will Deacon <will@...nel.org>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@...radead.org>
> Link: https://lkml.kernel.org/r/20230117160825.GA17756@willie-the-truck
> Signed-off-by: Will Deacon <will@...nel.org>
Before cgroup v2, cpuset had only one cpumask - cpus_allowed. It only
tracked online cpus and ignored the offline ones. It behaves more like
effective_cpus in cpuset v2. With v2, we have 2 cpumasks - cpus_allowed
and effective_cpus. When cpuset v1 is mounted, cpus_allowed and
effective_cpus are effectively the same and track online cpus. With
cpuset v2, cpus_allowed contains what the user has written into and it
won't be changed until another write happen. However, what the user
written may not be what the system can give it and effective_cpus is
what the system decides a cpuset can use.
Cpuset v2 is able to handle hotplug correctly and update the task's
cpumask accordingly. So missing previously offline cpus won't happen
with v2.
Since v1 keeps the old behavior, previously offlined cpus are lost in
the cpuset's cpus_allowed. However tasks in the root cpuset will still
be fine with cpu hotplug as its cpus_allowed should track
cpu_online_mask. IOW, only tasks in a non-root cpuset suffer this problem.
It was a known issue in v1 and I believe is one of the major reasons of
the cpuset v2 redesign.
A major concern I have is the overhead of creating a poor man version of
v2 cpus_allowed. This issue can be worked around even for cpuset v1 if
it is mounted with the cpuset_v2_mode option to behave more like v2 in
its cpumask handling. Alternatively we may be able to provide a config
option to make this the default for v1 without the special mount option,
if necessary.
Cheers,
Longman
Powered by blists - more mailing lists