netdev - RE: [PATCH v2 net] ibmveth: Reduce maximum tx queues to 8

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <26dc5891a0244f5f834cfd95f74ee8b4@AcuMS.aculab.com>
Date:   Sat, 5 Nov 2022 11:33:15 +0000
From:   David Laight <David.Laight@...LAB.COM>
To:     'Nick Child' <nnac123@...ux.ibm.com>,
        Jakub Kicinski <kuba@...nel.org>
CC:     "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "nick.child@....com" <nick.child@....com>,
        "bjking1@...ux.ibm.com" <bjking1@...ux.ibm.com>,
        "ricklind@...ibm.com" <ricklind@...ibm.com>,
        "dave.taht@...il.com" <dave.taht@...il.com>
Subject: RE: [PATCH v2 net] ibmveth: Reduce maximum tx queues to 8

From: Nick Child
> Sent: 04 November 2022 18:16
...
> Yes, and Dave can jump in here if I am wrong, but, from my
> understanding, if the NIC cannot send packets at the rate that
> they are queued then these queues will inevitably fill to txqueuelen.
> In this case, having more queues will not mean better throughput but
> will result in a large number of allocations sitting in queues
> (bufferbloat). I believe Dave's point was, if more queues does not
> allow for better performance (and can risk bufferbloat) then why
> have so many at all.

Isn't there another issue (noted in the tg3 driver) that if the
underlying hardware (or other consumer) is doing a round-robin
scan of the transmit queues then (IIRC) a lot of small packet
going through one queue can starve queues sending big packets.
Certainly tg3 has multiple tx queues disabled.

There is an associated problem with drivers that force the
number of transmit and receive rings (or whatever) to be the same.
The receive processing is far more expensive than transmit
(it is also much more critical - it doesn't really matter if
transmits get slightly delayed, but dropping rx packets is a PITA).
If threaded NAPI is enabled (to avoid issues with softint
processing) then you don't really need lots of threads for
transmit queues, but do need ones for the rx queues.
I had to use threaded NAPI with the threads running under
the RT scheduler to avoid packet loss (at 500,000 pkg/sec).

With tg3 four rx queues and one tx worked fine.

	David (not Dave)

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)