| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Thu, 16 Nov 2023 19:44:06 -0800
From: Stephen Hemminger <stephen@...workplumber.org>
To: netdev@...r.kernel.org
Subject: Fw: [Bug 218152] New: net rx interrupts causing a high load average
on system
Report of high cpu usage, maybe RT related
Begin forwarded message:
Date: Thu, 16 Nov 2023 06:20:25 +0000
From: bugzilla-daemon@...nel.org
To: stephen@...workplumber.org
Subject: [Bug 218152] New: net rx interrupts causing a high load average on system
https://bugzilla.kernel.org/show_bug.cgi?id=218152
Bug ID: 218152
Summary: net rx interrupts causing a high load average on
system
Product: Networking
Version: 2.5
Hardware: ARM
OS: Linux
Status: NEW
Severity: high
Priority: P3
Component: IPV4
Assignee: stephen@...workplumber.org
Reporter: rhmcruiser@...il.com
Regression: No
-> Specifically the net rx load is observed to cause a high load average on the
system running kernel 5.15.93-rt58 as compared to 4.9.214.
-> When checking the net rx interrupts they seems to quite high as compared to
similar workloads running on 4.9.214-rt137-g1fc72f4. ( I mean I tested with
same hardware for both kernels).
=> Note: When all ifaces are made down or disconnected from network, load
average drops from 1.4 to 0.2 or 0.1 , so most of the load is induced by net rx
interrupts in some manner.
Is there any known issue reported on 5.15.93-xx specific to net rx handling
that causes this load ?
Environment: kernel 5.15.93-rt58
Hardware: ARM
Observations of high load on 5.15.93-rt58
# cat /proc/interrupts
CPU0 CPU1
25: 0 0 GICv2 29 Level arch_timer
26: 3696358 3059507 GICv2 30 Level arch_timer
31: 0 0 GICv2 196 Edge arm-irq1
32: 0 0 GICv2 197 Edge arm-irq2
35: 96168 0 GICv2 107 Level fsl-ifc
36: 128 0 GICv2 126 Level mmc0
39: 5343 0 GICv2 120 Level 2180000.i2c
41: 0 0 GICv2 130 Level gpio-cascade
42: 0 0 GICv2 131 Level gpio-cascade
43: 0 0 GICv2 132 Level gpio-cascade
44: 0 0 GICv2 198 Level gpio-cascade
46: 428 0 GICv2 112 Level fsl-lpuart
47: 45740 0 GICv2 113 Level fsl-lpuart
49: 0 0 GICv2 147 Level 2ad0000.watchdog
50: 0 0 GICv2 167 Level eDMA
54: 17218 0 GICv2 158 Level can0
55: 0 0 GICv2 203 Level fsl-usb2-udc
56: 4436 0 GICv2 125 Level xhci-hcd:usb1
58: 0 0 GICv2 123 Level aerdrv
59: 1251756 0 MSI 524288 Edge xdma
60: 6987 0 GICv2 176 Level eth0_g0_tx
61: 389169 0 GICv2 177 Level eth0_g0_rx <<<<< RX
interrupts
62: 0 0 GICv2 178 Level eth0_g0_er
63: 1838 0 GICv2 179 Level eth0_g1_tx
64: 0 0 GICv2 180 Level eth0_g1_rx
65: 0 0 GICv2 181 Level eth0_g1_er
72: 263 0 GICv2 135 Level 1710000.jr
73: 0 0 GICv2 136 Level 1720000.jr
74: 0 0 GICv2 137 Level 1730000.jr
75: 0 0 GICv2 138 Level 1740000.jr
IPI0: 0 0 CPU wakeup interrupts
IPI1: 0 0 Timer broadcast interrupts
IPI2: 1254818 1669575 Rescheduling interrupts <<<<<<
IPI3: 97630 21898 Function call interrupts
IPI4: 0 0 CPU stop interrupts
IPI5: 109629 83567 IRQ work interrupts
IPI6: 0 0 completion interrupts
# top
top - 06:46:03 up 32 min, 1 user, load average: 1.44, 1.36, 1.19
^^^^^^^^^--------------->>>>>>>> load avg over 1 5 and 15 mins ( 2 CPU system)
Tasks: 142 total, 1 running, 141 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.5 us, 2.5 sy, 0.0 ni, 97.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 1001.9 total, 336.0 free, 521.5 used, 144.4 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 446.4 avail Mem
# cat /proc/cpuinfo
processor : 0
model name : ARMv7 Processor rev 5 (v7l)
BogoMIPS : 25.00
Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
vfpd32 lpae evtstrm
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xc07
CPU revision : 5
processor : 1
model name : ARMv7 Processor rev 5 (v7l)
BogoMIPS : 25.00
Features : half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt
vfpd32 lpae evtstrm
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xc07
CPU revision : 5
Hardware : Freescale LS1021A
Revision : 0000
Serial : 0000000000000000
#
==> Nothing cumbersome in the iptables rules, are minimal
root@...021atwr:~# iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-A INPUT -i eth0 -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i eth1 -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i bt0 -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i rsi_wlan -p tcp -m tcp --dport 80 -j DROP
-A INPUT -i ppp0 -p tcp -m tcp --dport 80 -j DROP
root@...021atwr:~#
==== LOAD AVERAGE WHEN NET IFACES ARE brought DOWN
Load average dropped from 1.44 to 0.06
top - 06:20:09 up 1:59, 1 user, load average: 0.06, 0.45, 0.86
<<<<<<
Tasks: 149 total, 1 running, 148 sleeping, 0 stopped, 0 zombie
%Cpu(s): 4.8 us, 11.9 sy, 0.0 ni, 83.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 1003.9 total, 396.0 free, 529.6 used, 78.2 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 452.9 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5865 root 20 0 3008 1716 1484 R 16.7 0.2 0:00.05 top
760 root -3 0 9716 3916 3424 S 5.6 0.4 0:17.46 smp
1 root 20 0 1780 476 428 S 0.0 0.0 0:09.23 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.02 kthreadd
3 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_gp
4 root 0 -20 0 0 0 I 0.0 0.0 0:00.00 rcu_par_gp
5 root 0 -20 0 0 0 I 0.0 0.0 0:00.00
slub_flushwq
7 root 0 -20 0 0 0 I 0.0 0.0 0:00.00
kworker/0:0H-events_highpri
9 root 0 -20 0 0 0 I 0.0 0.0 0:00.00
mm_percpu_wq
10 root 20 0 0 0 0 S 0.0 0.0 0:00.00
rcu_tasks_kthre
11 root 20 0 0 0 0 S 0.0 0.0 0:00.00
rcu_tasks_rude_
--
You may reply to this email to add a comment.
You are receiving this mail because:
You are the assignee for the bug.
Powered by blists - more mailing lists