linux-kernel - Latency spikes on V6.15.1 Preempt RT and maybe related to intel IGB

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <cb0d6ffc-acb3-4974-8f93-c86a6479109b@eltropuls.de>
Date: Tue, 10 Jun 2025 13:23:13 +0200
From: Marc Strämke <marc.straemke@...ropuls.de>
To: linux-kernel@...r.kernel.org
Cc: linux-rt-users@...r.kernel.org
Subject: Latency spikes on V6.15.1 Preempt RT and maybe related to intel IGB

Hello Everyone, I am reposting to LKML as I am not sure the rt-users 
mailinglist is read by many people, (I hope that is okay)

On an AMD Ryzen Embedded machine I am experiencing strange Latency 
spikes in cyclictest and need some hints how to debug that further.

The system typically has max latencys of  88 us and averages of 4-8 
which is more then sufficient for my application, but I saw some spikes 
of many hundred us in testing.

I can provoke latenciess of more then 500-1000 us by invoking "ip l set 
enp1s0 promisc off" on the first network interfaces. The network 
interface is an "Intel Corporation I210 Gigabit Network Connection" 
using the IGB driver.

I tried more or less all tracers but am not knowledgeable enough to make 
sense of the output. IRQSoff and wakeup_rt trace output attached.

Can anyone point me in the right direction? I am not sure how to 
interpret the function tracers and function_graph tracers output in a 
meaningful way. As mainly a user of of the kernel I am a bit overwhelmed 
by the interaction of the scheduler, RCU and so on..

I have attached my config for reference.

Kind Regards

Marc

View attachment "irqsoff_trace" of type "text/plain" (2114 bytes)

View attachment "wakeup_rt_trace" of type "text/plain" (9760 bytes)

Download attachment "config.gz" of type "application/x-gzip" (34225 bytes)