netdev - Re: BW regression after "tcp: refine TSO autosizing"

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1421603317.11734.154.camel@edumazet-glaptop2.roam.corp.google.com>
Date:	Sun, 18 Jan 2015 09:48:37 -0800
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Eyal Perry <eyalpe@....mellanox.co.il>
Cc:	Or Gerlitz <gerlitz.or@...il.com>,
	Linux Netdev List <netdev@...r.kernel.org>,
	Amir Vadai <amirv@...lanox.com>,
	Yevgeny Petrilin <yevgenyp@...lanox.com>,
	Saeed Mahameed <saeedm@...lanox.com>,
	Ido Shamay <idos@...lanox.com>,
	Amir Ancel <amira@...lanox.com>,
	Eyal Perry <eyalpe@...lanox.com>
Subject: Re: BW regression after "tcp: refine TSO autosizing"

On Sun, 2015-01-18 at 18:22 +0200, Eyal Perry wrote:

> 
> Please let me know if you see something in the results.

Getting high throughput on a single flow means lot of tweaking.

For a start, mlx4 is known to have interrupt mitigation that can hurt,
as the TX interrupt timer is restarted for every packet that is
delivered to the NIC.

ethtool -c ethX
..
tx-usecs: 16
tx-frames: 16
tx-usecs-irq: 0
tx-frames-irq: 256
...

-> TX IRQ can be delayed by 16*16 = 256 usec.

Can you try :

ethtool -C ethX tx-usecs 2 tx-frames 2

Or even

ethtool -C ethX tx-usecs 1 tx-frames 1

Interrupt mitigation is a trade-off.

If one customer wants high throughput on a single flow, then you might
remove interrupt mitigation.

If another customer wants cpu efficiency with thousand of flows, I guess
current mlx4 defaults are pretty good.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html