netdev - Bad shaping at low rates, after commit 56b765b79 (htb: improved accuracy at high rates)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130604141342.00c8eb9f@redhat.com>
Date:	Tue, 4 Jun 2013 14:13:42 +0200
From:	Jesper Dangaard Brouer <brouer@...hat.com>
To:	Stephen Hemminger <stephen@...workplumber.org>,
	Eric Dumazet <eric.dumazet@...il.com>,
	Jiri Benc <jbenc@...hat.com>
Cc:	Jesper Dangaard Brouer <brouer@...hat.com>,
	David Miller <davem@...hat.com>, j.vimal@...il.com,
	Michal Soltys <soltys@....info>,
	Mike Frysinger <vapier@...too.org>,
	Jussi Kivilinna <jussi.kivilinna@...et.fi>,
	Patrick McHardy <kaber@...sh.net>,
	Jiri Pirko <jpirko@...hat.com>,
	Toke Høiland-Jørgensen <toke@...e.dk>,
	Dave Taht <dave.taht@...il.com>, netdev@...r.kernel.org,
	bloat@...ts.bufferbloat.net, Dan Siemon <dan@...erfire.com>,
	Jim Gettys <jg@...edesktop.org>,
	Steven Barth <cyrus@...nwrt.org>, Felix Fietkau <nbd@....name>
Subject: Bad shaping at low rates, after commit 56b765b79 (htb: improved
 accuracy at high rates)

Hi again,

I found another regression by commit 56b765b79 (htb: improved accuracy
at high rates).

After the commit HTB does not honor network rate limiting below 500kbps.

I have found that the bandwidth problem is related to GSO being enabled
on the device.  To test the situation, I had to use a real NIC (as I
were not allowed to disable GSO on dev "lo", test below).

Testing with 100kbit/s:
-----------------------
A localhost dev "lo" reproducer commands:
 tc qdisc add dev lo root handle 1: htb default 1
 tc class add dev lo parent 1: classid 1:1 htb rate 100kbit
 netserver
 netperf -t TCP_STREAM -l 60 -H 127.0.0.1 -- -m 1024

Measuring the qdisc "rate" via:
 tc -s -d class show dev lo classid 1:1

I know the qdisc "rate" is not not very reliable/accurate, but I do
see a large difference (and so do netperf reported throughput).
 - Kernel 2.6.32-352.el6 gives rate 98752bit
 - Kernel 3.10.0-rc1     gives rate 390208bit

My real-NIC test show:
 With GSO: rate 153248bit
 Without GSO: rate 99320bit

Now, we just have to figure out why commit 56b765b79e (htb: improved
accuracy at high rates) breaks bandwidth shaping at low rate with GSO
enabled...

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Sr. Network Kernel Developer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

Data: real NIC
==============

With GSO: rate 153248bit
---------
class htb 1:1 root prio 0 quantum 1250 rate 100000bit ceil 100000bit
burst 1600b/1 mpu 0b overhead 0b cburst 1600b/1 mpu 0b overhead 0b
level 0 Sent 844892 bytes 568 pkt (dropped 0, overlimits 0 requeues 0)
rate 153248bit 13pps backlog 0b 7p requeues 0 lended: 87 borrowed: 0
giants: 0 tokens: -847776554 ctokens: -847776554

Without GSO: rate 99320bit
------------
(disabled via: ethtool -K eth63 gso off gro off tso off)

class htb 1:1 root prio 0 quantum 1250 rate 100000bit ceil 100000bit
burst 1600b/1 mpu 0b overhead 0b cburst 1600b/1 mpu 0b overhead 0b
level 0 Sent 1706968 bytes 1164 pkt (dropped 0, overlimits 0 requeues
0) rate 99320bit 8pps backlog 0b 51p requeues 0 lended: 664 borrowed: 0
giants: 0 tokens: -121054491 ctokens: -121054491

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html