netdev - Re: inaccurate packet scheduling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 18 Dec 2012 18:54:09 +0100
From:	Jiri Pirko <jiri@...nulli.us>
To:	Stephen Hemminger <stephen.hemminger@...tta.com>
Cc:	jhs@...atatu.com, davem@...emloft.net, edumazet@...gle.com,
	tgraf@...g.ch, netdev@...r.kernel.org
Subject: Re: inaccurate packet scheduling

Tue, Dec 18, 2012 at 05:26:31PM CET, stephen.hemminger@...tta.com wrote:
>
>
>----- Original Message -----
>> Hi all.
>> 
>> Run one of the following 2 scripts on machine A:
>> 
>> #!/bin/bash
>> tc qdisc del dev eth0 root
>> sleep 1
>> tc -batch << EOF
>> qdisc add dev eth0 root handle 1: prio bands 2 priomap 0 0 0 0 0 0 0
>> 0 0 0 0 0 0 0 0 0
>> qdisc add dev eth0 parent 1:1 handle 10: pfifo limit 50
>> qdisc add dev eth0 parent 1:2 handle 20 tbf latency 100ms rate 4mbit
>> burst 2m
>> filter add dev eth0 parent 1: protocol ip u32 match ip dst
>> $machineB_ip flowid 1:2
>> EOF
>> 
>> #!/bin/bash
>> tc qdisc del dev eth0 root
>> sleep 1
>> tc -batch << EOF
>> qdisc add dev eth0 root handle 1: prio bands 2 priomap 0 0 0 0 0 0 0
>> 0 0 0 0 0 0 0 0 0
>> qdisc add dev eth0 parent 1:1 handle 10: pfifo limit 20
>> qdisc add dev eth0 parent 1:2 handle 20: pfifo limit 20
>> filter add dev eth0 parent 1: protocol ip pref 10 \
>> u32 match ip dst $machineB_ip \
>> flowid 1:2 \
>> police rate 4Mbit burst 2m conform-exceed drop
>> EOF
>> 
>> And run:
>> [machineB ~]# iperf -s
>> [machineA ~]# iperf -c machineB_ip -t 60
>> 
>> Expected results are: ~3.8-4.2 Mbits/s
>> But actual results are: ~130-170 Kbits/s with tbf, ~70-300 Kbits/s
>> with policy rate
>> 
>> [machineA ~]# tc -s qdisc list dev eth0
>> qdisc prio 1: root refcnt 9 bands 2 priomap  0 0 0 0 0 0 0 0 0 0 0 0
>> 0 0 0 0
>>  Sent 1512384 bytes 1032 pkt (dropped 729, overlimits 0 requeues 0)
>>  backlog 0b 0p requeues 0
>> qdisc pfifo 10: parent 1:1 limit 50p
>>  Sent 4560 bytes 32 pkt (dropped 0, overlimits 0 requeues 0)
>>  backlog 0b 0p requeues 0
>> qdisc tbf 20: parent 1:2 rate 4000Kbit burst 2Mb lat 100.0ms
>>  Sent 1507824 bytes 1000 pkt (dropped 729, overlimits 0 requeues 0)
>>  backlog 0b 0p requeues 0
>> 
>> 
>> Tested with kernel pulled from linus's git today. This happens with
>> older
>> kernels as well (I tried 2.6.32-based rhel6 kernels).
>> 
>> This happens to me on following machines:
>> HP DL360G8 (x86_64) http://people.redhat.com/jpirko/aThoo2Ei/dl380g8/
>> HP DL360G3 (i686)
>> IBM JS22 (ppc64) http://people.redhat.com/jpirko/aThoo2Ei/ibmjs22/
>> 
>> On following machines, I do not observe this issue:
>> qemu kvm (x86_64)
>> IBM Zseries (s390x) http://people.redhat.com/jpirko/aThoo2Ei/ibmz/
>> 
>> Please ask in case you need me to provide any other details.
>> 
>> Thanks.
>
>Check kernel log for messages about clock. It could be that on the
>machines with issues TSC is not usable for kernel clock.

No such messages in log

>Also turn off TSO since it screws up any form of rate control.

Turning off TSO did not help. However, turning off scatter-gather
helped, on both HP DL360G8 (igb) and IBM JS22 (ehea).


>--
>To unsubscribe from this list: send the line "unsubscribe netdev" in
>the body of a message to majordomo@...r.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html