netdev - RE: What's the benefit of large Rx rings?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <063D6719AE5E284EB5DD2968C1650D6D1CBDA892@AcuExch.aculab.com>
Date:	Mon, 23 Nov 2015 17:27:56 +0000
From:	David Laight <David.Laight@...LAB.COM>
To:	'Yuval Mintz' <Yuval.Mintz@...gic.com>,
	netdev <netdev@...r.kernel.org>
Subject: RE: What's the benefit of large Rx rings?

From: Yuval Mintz
> Sent: 22 November 2015 20:19
> This might be a dumb question, but I recently touched this
> and felt like I'm missing something basic -
> 
> NAPI is being scheduled from soft-interrupt contex, and it
> has a ~strict quota for handling Rx packets [even though we're
> allowing practically unlimited handling of Tx completions].
> Given these facts, what's the benefit of having arbitrary large
> Rx buffer rings? Assuming quota is 64, I would have expected
> that having more than twice or thrice as many buffers could not
> help in real traffic scenarios - in any given time-unit
> [the time between 2 NAPI runs which should be relatively
> constant] CPU can't handle more than the quota; If HW is
> generating more packets on a regular basis the buffers are bound
> to get exhausted, no matter how many there are.

What you don't want is guaranteed packet loss for common scenarios.

The worst one we I've seen was not having enough buffers for a
single NFS 8k UDP datagram (was a long time ago).

But a 64k send using hardware TSO will most likely give you
about 40 receive frames back to back, unless you can keep up
with line speed (unlikely at high speeds on a slow cpu) you
may need several times that many buffers to handle rx data
on multiple connections.
At some point data will get discarded, but usually the delays
in sending acks will slow down the receive data.

In reality it is all a trade off between a lot of rx buffers
and recovering from rx discards.

	David

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html