netdev - Re: [PATCH net] net: stmmac: Use hrtimer for TX coalescing

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date: Tue, 19 Sep 2023 20:53:20 +0300
From: Maxim Mikityanskiy <maxtram95@...il.com>
To: Vincent Whitchurch <Vincent.Whitchurch@...s.com>
Cc: "davem@...emloft.net" <davem@...emloft.net>,
	"nbd@....name" <nbd@....name>,
	"alexandre.torgue@...com" <alexandre.torgue@...com>,
	"joabreu@...opsys.com" <joabreu@...opsys.com>,
	kernel <kernel@...s.com>,
	"peppe.cavallaro@...com" <peppe.cavallaro@...com>,
	"kuba@...nel.org" <kuba@...nel.org>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: Re: [PATCH net] net: stmmac: Use hrtimer for TX coalescing

On Mon, 18 Sep 2023 at 12:56:31 +0000, Vincent Whitchurch wrote:
> On Wed, 2023-08-30 at 21:05 +0300, Maxim Mikityanskiy wrote:
> > On Wed, 30 Aug 2023 at 14:55:37 +0000, Vincent Whitchurch wrote:
> > > Any test results with this patch on the hardware with the performance
> > > problems would be appreciated.
> > 
> > TL/DR: it's definitely better than without the patch, but still worse
> > than fully reverting hrtimer [1].
> 
> Thank you for testing this.
> 
> Have you also had the chance to try out Felix's suggestion of completely
> disabling coalescing in the driver?  For that you will need to apply the
> patch at [0] (the command below may appear to work without the patch but
> the timer will continue to be programmed and expire immediately if the
> patch is not applied) and then run something like:
> 
>   ethtool -C eth0 tx-frame 1 tx-usecs 0

It's really good, yet not as perfect as reverting the hrtimer patch.
Aggregated summary over 15 20-second iperf3 tests (with XPS, RPS and
flow tables):

        | avg up | avg dn | std up | std dn
--------|--------|--------|--------|--------
Revert  |   931  |   939  |     6  |     2
No-coal |   925  |   916  |    13  |    26

I'd say it's almost as good as the full revert, but with more deviation,
numbers are less stable.

Kind of off topic, but I've also been testing PPPoE, and it's pretty bad
even with the available fixes:

        | avg up | avg dn | std up | std dn
--------|--------|--------|--------|--------
Revert  |   791  |   388  |    13  |    26
No-coal |   806  |   390  |    97  |    60

Thanks,
Max

> 
> [0] https://lore.kernel.org/all/20230907-stmmac-coaloff-v2-1-38ccfac548b9@axis.com/
> 
> With coalescing disabled in the driver, there is also the option of
> playing with the generic software IRQ coalescing options available in
> NAPI in newer kernels (eg. gro_flush_timeout), which may work better for
> you than the one in the driver.