netdev - Re: [RFC] [PATCH] Optimize TCP sendmsg in favour of fast devices?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <OFC641141C.812873FC-ON652576B8.003216B6-652576B8.003440CC@in.ibm.com>
Date:	Wed, 27 Jan 2010 15:12:48 +0530
From:	Krishna Kumar2 <krkumar2@...ibm.com>
To:	Krishna Kumar2 <krkumar2@...ibm.com>
Cc:	David Miller <davem@...emloft.net>, eric.dumazet@...il.com,
	Herbert Xu <herbert@...dor.apana.org.au>,
	ilpo.jarvinen@...sinki.fi, netdev@...r.kernel.org
Subject: Re: [RFC] [PATCH] Optimize TCP sendmsg in favour of fast devices?

> Krishna Kumar2/India/IBM wrote on 01/27/2010 12:42 PM
>
> > Herbert Xu <herbert@...dor.apana.org.au> wrote on 01/21/2010 03:11 PM
>
> Sorry for the late response.
>
> > >> I had to remove the F_SG flag from cxgb3 driver (using ethtool
> > >> didn't show any difference in performance since GSO was enabled
> > >> on the device due to register_netdev setting it). Testing show a
> > >> drop of 25% in performance with this patch for non-SG device,
> > >> the extra alloc/memcpy is showing up.
> > >>
> > >> For the SG driver, I get a good performace gain (not anywhere
> > >> close to 25% though). What do you suggest?
> > >
> > > I don't think we can add your change if it hurts non-SG
> > > devices that much.
> >
> > Wait, we need to be careful when testing this.  Non-SG devices
> > do actually benefit from TSO which they otherwise cannot access.
> >
> > If you unset the F_SG bit, then that would disable TSO too.  So
> > you need to enable GSO to compensate.  So Krishna, did you check
> > with tcpdump to see if GSO was really enabled with SG off?
>
> OK, I unset F_SG and set F_GSO (in driver). With this, tcpdump shows
> GSO is enabled - the tcp packet sizes builds up to 65160 bytes.

I should have mentioned this too - if I unset F_SG in the
cxgb3 driver and nothing else, ethtool -k still shows GSO
is set, and tcpdump shows max packet size is 1448. If I
additionally set GSO in driver, then ethtool still has the
same output, but tcpdump shows max packet size of 65160.

Thanks,

- KK

> I ran 5 serial netperf's with 16K and another 5 serial netperfs
> with 64K I/O sizes, and the aggregate result is:
>
> 0. Driver unsets F_SG but sets F_GSO:
>  Original code with 16K: 19471.65
>  New code with 16K:      19409.70
>  Original code with 64K: 21357.23
>  New code with 64K:      22050.42
>
> To recap the other tests I did today:
>
> 1. Driver unsets F_SG, and with GSO off
>  Original code with 16K: 10123.56
>  New code with 16K:      7111.12
>  Original code with 64K: 11568.99
>  New code with 64K:      7611.37
>
> 2. Driver unsets F_SG and uses ethtool to set GSO:
>  Original code with 16K: 18864.38
>  New code with 16K:      18465.54
>  Original code with 64K: 21005.43
>  New code with 64K:      22529.24
>
> Thanks,
>
> - KK
>
> > IIRC when I did a similar test with e1000 back when I wrote this
> > the performance of GSO with SG off was pretty much the same as
> > no GSO with SG off.

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html