netdev - Re: Performance regressions in TCP_STREAM tests in Linux 4.15 (and later)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <476bfc0f-eb2d-fe57-73d9-ec8a8392ad33@candelatech.com>
Date:   Mon, 30 Apr 2018 09:14:04 -0700
From:   Ben Greear <greearb@...delatech.com>
To:     Steven Rostedt <rostedt@...dmis.org>,
        Michael Wenig <mwenig@...are.com>
Cc:     "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "eric.dumazet@...il.com" <eric.dumazet@...il.com>,
        Shilpi Agarwal <sagarwal@...are.com>,
        Boon Ang <bang@...are.com>, Darren Hart <dvhart@...are.com>,
        Steven Rostedt <srostedt@...are.com>,
        Abdul Anshad Azeez <aazees@...are.com>
Subject: Re: Performance regressions in TCP_STREAM tests in Linux 4.15 (and
 later)

On 04/27/2018 08:11 PM, Steven Rostedt wrote:
>
> We'd like this email archived in netdev list, but since netdev is
> notorious for blocking outlook email as spam, it didn't go through. So
> I'm replying here to help get it into the archives.
>
> Thanks!
>
> -- Steve
>
>
> On Fri, 27 Apr 2018 23:05:46 +0000
> Michael Wenig <mwenig@...are.com> wrote:
>
>> As part of VMware's performance testing with the Linux 4.15 kernel,
>> we identified CPU cost and throughput regressions when comparing to
>> the Linux 4.14 kernel. The impacted test cases are mostly TCP_STREAM
>> send tests when using small message sizes. The regressions are
>> significant (up 3x) and were tracked down to be a side effect of Eric
>> Dumazat's RB tree changes that went into the Linux 4.15 kernel.
>> Further investigation showed our use of the TCP_NODELAY flag in
>> conjunction with Eric's change caused the regressions to show and
>> simply disabling TCP_NODELAY brought performance back to normal.
>> Eric's change also resulted into significant improvements in our
>> TCP_RR test cases.
>>
>>
>>
>> Based on these results, our theory is that Eric's change made the
>> system overall faster (reduced latency) but as a side effect less
>> aggregation is happening (with TCP_NODELAY) and that results in lower
>> throughput. Previously even though TCP_NODELAY was set, system was
>> slower and we still got some benefit of aggregation. Aggregation
>> helps in better efficiency and higher throughput although it can
>> increase the latency. If you are seeing a regression in your
>> application throughput after this change, using TCP_NODELAY might
>> help bring performance back however that might increase latency.

I guess you mean _disabling_ TCP_NODELAY instead of _using_ TCP_NODELAY?

Thanks,
Ben


-- 
Ben Greear <greearb@...delatech.com>
Candela Technologies Inc  http://www.candelatech.com