netdev - TC: HTB module over limiting when CPU is under load

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAC-fF8Sv3rEx3-st-vHWqcOGerSN66-6qv4Xv1Sh2wDLQ2yNmg@mail.gmail.com>
Date: Sat, 13 Jan 2024 14:40:56 +0200
From: Isaac Boukris <iboukris@...il.com>
To: netdev@...r.kernel.org
Subject: TC: HTB module over limiting when CPU is under load

Hello,

While experimenting with HTB I noticed it sometimes overlimits the
bandwidth way lower than I've configured (~1/4). After lots of tests I
narrowed it to the case where the CPU is under heavy load.

I do realize you can't be too precise under such circumstances, but
such gaps make the module not very useful, and most importantly other
modules such as HSFC and NETEM still work correctly in the same test,
so it doesn't seem to be an impossible task.

The attached script reproduces it for me, when run normally the curl
command reports ~1800kbps for all modules HTB, HSFC and NETEM, but
when I run the below command to incur heavy cpu load, I only get
~400kbps when using HTB while HSFC and NETEM still work well.

$ watch -n 0.1 openssl speed -multi $(nproc --all)

See below dmesg related entries and tc stats from the last test.s
Thanks a lot for your thoughts.

[ 5487.205876] HTB: quantum of class 10001 is big. Consider r2q change.
[ 5514.158475] htb: too many events!

tc -s -d -n server qdisc show dev server0
qdisc htb 1: root refcnt 129 r2q 10 default 0x1 direct_packets_stat 0
ver 3.17 direct_qlen 1000
 Sent 26140201 bytes 17273 pkt (dropped 0, overlimits 6440 requeues 0)
 backlog 0b 0p requeues 0

tc -s -d -n server class show dev server0
class htb 1:1 root prio 0 quantum 200000 rate 8Gbit ceil 8Gbit
linklayer ethernet burst 0b/1mpu 0b cburst 0b/1mpu 0b level 0
 Sent 182 bytes 3 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
 lended: 3 borrowed: 0 giants: 0
 tokens: 13 ctokens: 13

class htb 1:2 root prio 0 quantum 200000 rate 16Mbit ceil 16Mbit
linklayer ethernet burst 1600b/1mpu 0b cburst 1600b/1mpu 0b level 0
 Sent 26140019 bytes 17270 pkt (dropped 0, overlimits 6440 requeues 0)
 backlog 0b 0p requeues 0
 lended: 6539 borrowed: 0 giants: 0
 tokens: 1964 ctokens: 1964

View attachment "htb_reproducer.sh" of type "text/x-sh" (1258 bytes)