linux-kernel - Fwd: [BUG] nohz_full: Different behavior for RT vs CFS task on isolated CPU (6.16.*)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <CAO3najBmEK4r_UjMhmvxf4wBF+5zbncMKhA-jKHp8WaM8Ndz6Q@mail.gmail.com>
Date: Sun, 21 Sep 2025 19:48:16 +0200
From: Mario Roß <marioross3503@...il.com>
To: frederic@...nel.org, mingo@...nel.org
Cc: linux-kernel@...r.kernel.org
Subject: Fwd: [BUG] nohz_full: Different behavior for RT vs CFS task on
 isolated CPU (6.16.*)

Hi,
I tried to send this bug report earlier, but my mail server seems to have
issues with the kernel.org domain.
Please find the report below.
Thanks,
Mario

-------- Forwarded Message --------
Subject: [BUG] nohz_full: Different behavior for RT vs CFS task on isolated
CPU (6.16.*)
Date: Sat, 20 Sep 2025 14:30:31 +0200
From: Mario Roß <mail@...ioross.de> <mail@...ioross.de>
To: anna-maria@...utronix.de, frederic@...nel.org, mingo@...nel.org,
tglx@...utronix.de
CC: linux-kernel@...r.kernel.org

Hi together,

we are currently exploring sources of latency on completely isolated CPU
cores
with the following kernel parameters enabled:

  isolcpus, nohz_full, rcu_nocbs

We have observed odd behavior on Linux 6.16.* where it makes a difference
whether a process has a realtime priority or not, despite being the only
runnable process on the isolated core.

Test setup
==========

- CPU: isolated with isolcpus, nohz_full, rcu_nocbs
- Userspace workload: simple `while(1)` loop

Case 1: RT priority

  sudo taskset -c 2 chrt -f 1 ./a.out

When traced with the function tracer (ftrace), we see no output → expected
behavior, since nothing should interrupt the running task.

Case 2: no RT priority

  sudo taskset -c 2 ./a.out

In this case we see trace output, indicating interrupts/scheduler activity
still occur. See attached Ftrace_Log.txt for details.

This behavior is not present on a tested Linux 6.8.*, so it appears to be a
regression
introduced at some point to 6.16.*. We could reproduce it on multiple
systems, including
6.16.7.

System information
==================

  Linux rocky 6.16.3-custom-rt #1 SMP PREEMPT_RT Wed Aug 27 22:21:22 CEST
2025 x86_64 GNU/Linux

Current tasks on isolated CPU (ps -eLo pid,psr,comm,pri,state | awk
'$2==2'):

    39   2 cpuhp/2          19 S
    40   2 idle_inject/2    90 S
    41   2 irq_work/2       41 S
    42   2 migration/2     139 S
    43   2 rcuc/2           41 S
    44   2 ktimers/2        41 S
    45   2 ksoftirqd/2      19 S
    46   2 kworker/2:0-eve  19 I
    47   2 kworker/2:0H-kb  39 I
    83   2 backlog_napi/2   19 S
    86   2 kworker/2:1-eve  19 I
  8564   2 kworker/2:1H-kb  39 I
  9211   2 a.out            19 R

Kernel cmdline:

  BOOT_IMAGE=/vmlinuz-6.16.3-custom-rt root=/dev/mapper/rl-root ro
crashkernel=2G-64G:256M,64G-:512M
resume=UUID=5c610720-dc2d-476e-9f9d-f91679e9f6ff \
  rd.lvm.lv=rl/root rd.lvm.lv=rl/swap rhgb quiet isolcpus=2,3 nohz_full=2,3
rcu_nocbs=2,3 cpufreq.default_governor=performance irqaffinity=0-1 \
  mitigations=off processor.max_cstate=1 intel_idle.max_cstate=0
rcu_nocb_poll nosoftlookup mce=ignore_ce audit=0 idle=poll skew_tick=1 \
  tsc=reliable rcupdate.rcu_normal_after_boot=1

Any insights would be very welcome. Please let us know if further details
would help to reproduce.

Thanks,
Mario

Content of type "text/html" skipped

View attachment "Ftrace_Log.txt" of type "text/plain" (33710 bytes)