netdev - Re: [RFC] net: remove busylock

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1463779960.5939.7.camel@edumazet-glaptop3.roam.corp.google.com>
Date:	Fri, 20 May 2016 14:32:40 -0700
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Jesper Dangaard Brouer <brouer@...hat.com>
Cc:	Alexander Duyck <alexander.duyck@...il.com>,
	netdev <netdev@...r.kernel.org>,
	Alexander Duyck <aduyck@...antis.com>,
	John Fastabend <john.r.fastabend@...el.com>,
	Jamal Hadi Salim <jhs@...atatu.com>
Subject: Re: [RFC] net: remove busylock

On Fri, 2016-05-20 at 19:49 +0200, Jesper Dangaard Brouer wrote:
> On Fri, 20 May 2016 07:16:55 -0700
> Eric Dumazet <eric.dumazet@...il.com> wrote:
> 
> > Since bonding pretends to be multiqueue, TCQ_F_ONETXQUEUE is not set
> > on sch->flags when HTB is installed at the bonding device root.
> 
> If would be cool if you could run a test with removed busylock and
> allow HTB to bulk dequeue.

I added a /sys/class/net/eth0/tx_bulk_limit to tune the number of
packets that we could bulk dequeue from a virtual device 
(no BQL on them)

200 TCP_RR through HTB on eth0 (bonding device)

1) busylock enabled
--------------------

With tx_bulk_limit set to 8, we get 12.7 % increase.

lpaa23:~# for f in `seq 1 16`; do echo $f|tee /sys/class/net/eth0/tx_bulk_limit; sar -n DEV 3 3|grep eth0|grep Average; done
1
Average:         eth0 868625.67 868487.44  57055.69  56826.37      0.00      0.00      0.56
2
Average:         eth0 927081.67 926920.78  60923.83  60649.90      0.00      0.00      0.44
3
Average:         eth0 957678.11 957554.00  62877.04  62653.89      0.00      0.00      0.56
4
Average:         eth0 966912.44 966747.33  63532.72  63255.51      0.00      0.00      0.56
5
Average:         eth0 973137.56 972950.44  63958.31  63661.39      0.00      0.00      0.44
6
Average:         eth0 958589.22 958449.44  62961.79  62712.56      0.00      0.00      0.67
7
Average:         eth0 960737.67 960672.22  62968.34  62857.97      0.00      0.00      0.44
8
Average:         eth0 979271.78 979201.67  64199.47  64070.84      0.00      0.00      0.56
9
Average:         eth0 982464.33 982390.33  64418.42  64278.93      0.00      0.00      0.56
10
Average:         eth0 982698.00 982578.22  64495.25  64291.28      0.00      0.00      0.44
11
Average:         eth0 981862.22 981746.00  64438.16  64236.31      0.00      0.00      0.56
12
Average:         eth0 983277.44 983096.33  64632.79  64327.79      0.00      0.00      0.44
13
Average:         eth0 981221.11 981018.00  64521.82  64189.26      0.00      0.00      0.67
14
Average:         eth0 981754.11 981555.89  64553.19  64224.39      0.00      0.00      0.44
15
Average:         eth0 982484.33 982301.67  64572.00  64273.38      0.00      0.00      0.44
16
Average:         eth0 978529.56 978326.67  64350.89  64013.39      0.00      0.00      0.67


2) busylock disabled
--------------------

Well, bulk dequeue helps, but does not close the gap. 
(busylock is needed)

lpaa23:~# for f in `seq 1 16`; do echo $f|tee /sys/class/net/eth0/tx_bulk_limit; sar -n DEV 3 3|grep eth0|grep Average; done
1
Average:         eth0 795408.44 795407.67  52044.66  52045.93      0.00      0.00      0.56
2
Average:         eth0 843411.78 843415.11  55185.23  55184.51      0.00      0.00      0.56
3
Average:         eth0 876175.89 876175.00  57329.50  57327.98      0.00      0.00      0.44
4
Average:         eth0 890631.22 890629.44  58274.58  58274.25      0.00      0.00      0.67
5
Average:         eth0 900672.00 900668.89  58931.29  58930.54      0.00      0.00      0.44
6
Average:         eth0 908325.78 908328.22  59432.97  59431.76      0.00      0.00      0.56
7
Average:         eth0 913895.33 913885.11  59796.89  59795.46      0.00      0.00      0.56
8
Average:         eth0 914429.11 914433.56  59832.26  59831.23      0.00      0.00      0.67
9
Average:         eth0 918701.11 918699.67  60110.68  60110.36      0.00      0.00      0.55
10
Average:         eth0 920382.33 920376.56  60223.31  60220.54      0.00      0.00      0.67
11
Average:         eth0 914341.67 914344.67  59826.25  59825.90      0.00      0.00      0.67
12
Average:         eth0 912697.00 912693.78  59718.77  59717.44      0.00      0.00      0.44
13
Average:         eth0 917392.56 917385.00  60025.79  60024.34      0.00      0.00      0.44
14
Average:         eth0 918232.89 918233.78  60081.04  60079.94      0.00      0.00      0.67
15
Average:         eth0 918377.11 918381.00  60091.14  60089.79      0.00      0.00      0.44
16
Average:         eth0 913817.56 913812.33  59792.09  59790.66      0.00      0.00      0.56