linux-kernel - Re: Kernel 6.14.11 dl_server_timer(...) causing IPI/Function Call Interrupts on isolcpu/nohz

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAKJHwtMAHpOpm4QJLFKES9Uz_9XPa9gX_QvLYCzvWEjMEjv6YA@mail.gmail.com>
Date: Tue, 25 Nov 2025 13:33:47 -0600
From: David Haufe <dhaufe@...plextrading.com>
To: Juri Lelli <juri.lelli@...hat.com>
Cc: linux-kernel@...r.kernel.org
Subject: Re: Kernel 6.14.11 dl_server_timer(...) causing IPI/Function Call
 Interrupts on isolcpu/nohz_full cores, performance regression

f237e524f3c7 ("sched/deadline: Make dl-server nohz full aware") <--
patch applied
219a63335b67 ("sched/deadline: Don't count nr_running twice for
dl_server proxy tasks") <-- exists in mainline 6.17
7620177e8108 ("sched/deadline: Fix RT task potential starvation when
expiry time passed") <-- exists in mainline 6.17
cccb45d7c429 ("sched/deadline: Less agressive dl_server handling") <--
slightly different code but exists in mainline 6.17

The "nohz full aware" patch does stop the hrtimer interrupts. I will
watch for that merge and can patch locally for now. Thank you!

On Tue, Nov 25, 2025 at 7:58 AM Juri Lelli <juri.lelli@...hat.com> wrote:
>
> Hi David,
>
> On 24/11/25 14:30, David Haufe wrote:
> > Hi Juri,
> > Working with 6.17.7 (code appears the same in .8 and 6.18rc), we are
> > still having isolated/nohz cores interrupted by deadline
> > functionality. The dl_task_timer is firing once a second with a single
> > SCHED_OTHER process spinning on the core. I am no longer seeing the
> > IPI interrupts, but the hrtime activity is still causing a performance
> > regression compared to kernels prior to the deadline merge.
>
> Where you able to check if the changes on the branch below made things
> any better? It is hopefully not that hard to apply to newer/different
> baselines.
>
> I have to apologize in advance, but I'm going to be on pto for a few
> days and then I have to travel for LPC26. So, unfortunately I am going
> to have very limited availability until end of the year.
>
> Best,
> Juri
>
> > On Mon, Aug 4, 2025 at 11:59 AM Juri Lelli <juri.lelli@...hat.com> wrote:
> > >
> > > On 04/08/25 10:44, David Haufe wrote:
> > > > My apologies, I see what you mean now. add_nr_running() is not being
> > > > invoked if it is the dl_server. We are still trying to get this branch
> > > > to boot to verify ourselves. We will be on the lookout for this to be
> > > > merged for release.
> > >
> > > No worries, guess I wasn't clear the first time. :)
> > >
> > > I added a very much experimental commit on
> > >
> > > https://github.com/jlelli/linux/tree/upstream/fix-dlserver-1
> > >
> > > that seems to be able to remove the one per second dl_server_timer and
> > > start it back as needed. But, I just played briefly with it, so I am not
> > > fully convinced is what we want. Anyway, if you could test with it as
> > > well it would be a useful data point. In principle you could try porting
> > > the following commits to your current tree and check if they do improve
> > > things (in reverse order starting from the bottom from the branch above):
> > >
> > > f237e524f3c7 ("sched/deadline: Make dl-server nohz full aware")
> > > 219a63335b67 ("sched/deadline: Don't count nr_running twice for dl_server proxy tasks")
> > > 7620177e8108 ("sched/deadline: Fix RT task potential starvation when expiry time passed")
> > > cccb45d7c429 ("sched/deadline: Less agressive dl_server handling")
>

-- 
DISCLAIMER: NOTICE REGARDING PRIVACY AND CONFIDENTIALITY 

The information 
contained in and/or accompanying this communication is intended only for 
use by the addressee(s) named herein and may contain legally privileged 
and/or confidential information. If you are not the intended recipient of 
this e-mail, you are hereby notified that any dissemination, distribution 
or copying of this information, and any attachments thereto, is strictly 
prohibited. If you have received this e-mail in error, please immediately 
notify the sender and permanently delete the original and any copy of any 
e-mail and any printout thereof. Electronic transmissions cannot be 
guaranteed to be secure or error-free. The sender therefore does not accept 
liability for any errors or omissions in the contents of this message which 
arise as a result of e-mail transmission. Simplex Trading, LLC and its 
affiliates reserves the right to intercept, monitor, and retain electronic 
communications to and from its system as permitted by law. Simplex Trading, 
LLC is a registered Broker Dealer with CBOE and a Member of SIPC.