linux-kernel - Re: [PATCH] sched/nohz: Fix NOHZ imbalance by adding options for ILB CPU

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <9c8a2b87-5062-08a6-5a27-f53d986b1be2@gentwo.org>
Date: Thu, 4 Sep 2025 08:34:34 -0700 (PDT)
From: "Christoph Lameter (Ampere)" <cl@...two.org>
To: Frederic Weisbecker <frederic@...nel.org>
cc: Valentin Schneider <vschneid@...hat.com>, 
    Adam Li <adamli@...amperecomputing.com>, mingo@...hat.com, 
    peterz@...radead.org, juri.lelli@...hat.com, vincent.guittot@...aro.org, 
    dietmar.eggemann@....com, rostedt@...dmis.org, bsegall@...gle.com, 
    mgorman@...e.de, linux-kernel@...r.kernel.org, patches@...erecomputing.com
Subject: Re: [PATCH] sched/nohz: Fix NOHZ imbalance by adding options for
 ILB CPU

On Thu, 4 Sep 2025, Frederic Weisbecker wrote:

> Le Wed, Aug 20, 2025 at 10:31:24AM -0700, Christoph Lameter (Ampere) a écrit :
> > On Wed, 20 Aug 2025, Valentin Schneider wrote:
> >
> > > My first question would be: is NOHZ_FULL really right for your workload?
> >
> > Yes performance is improved. AI workloads are like HPC workloads in that
> > they need to do compute and then rendezvous for data exchange.
>
> Ok, I was about to say that this is the first known (for me) usecase of
> nohz_full that is about performance and doesn't strictly require low-latency
> guarantee. But...

For me it was always about both. Low latency is required for a high number
of compute cycles in HPC apps. It is a requiremen for high performance
parallelized compute.

> > The more frequent rendezvous can be performed the better the performance
> > numbers will be.
>
> ...that is low-latency requirement...for performance :-)

Yea thats why we want this in HPC/HFT and AI applications.

> That's an argument _not_ in favour of dynamic balancing such as ILB, even for
> this usecase in nohz_full (all the other usecases of nohz_full I know really
> want static affinity and no balancing at all).
>
> So I have to ask, what would be wrong with static affinities to these tasks?

Static affinities are great but they keep the tick active and thus the
rendevous can be off off one or the other compute thread.

> > hohz full has been reworked somewhat since the early days and works in a
> > more general way today.
>
> Not sure about that. Although it was not initially intended to, it has
> been very single purpose since the early days: ie: run a single task in
> userspace without being disturbed.

The restrictions have been reduced from what I see in the code and
syscalls are possible without incurring a 2 second penalty of ticks.


> > > Here AIUI you're relying on the scheduler load balancing to distribute work
> > > to the NOHZ_FULL CPUs, so you're going to be penalized a lot by the
> > > NOHZ_FULL context switch overheads. What's the point? Wouldn't you have
> > > less overhead with just NOHZ_IDLE?
> >
> > The benchmarks show a regression of 10-20% if the tick is operational.
>
> Impressive!
>
> > The context switch overhead is negligible since the cpus are doing compute
> > and not system calls.
>
> And not many syscalls, right?

Periodically the data needs to be saved but that can be done from special
threads or after a large number of compute cycles is complete.