netdev - Re: [RFC PATCH] net: Introduce a socket option to enable picking tx queue based on rx queue.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 19 Sep 2017 17:48:19 -0700
From:   Tom Herbert <tom@...bertland.com>
To:     "Samudrala, Sridhar" <sridhar.samudrala@...el.com>
Cc:     Eric Dumazet <eric.dumazet@...il.com>,
        Alexander Duyck <alexander.h.duyck@...el.com>,
        Linux Kernel Network Developers <netdev@...r.kernel.org>
Subject: Re: [RFC PATCH] net: Introduce a socket option to enable picking tx
 queue based on rx queue.

On Tue, Sep 19, 2017 at 5:34 PM, Samudrala, Sridhar
<sridhar.samudrala@...el.com> wrote:
> On 9/12/2017 3:53 PM, Tom Herbert wrote:
>>
>> On Tue, Sep 12, 2017 at 3:31 PM, Samudrala, Sridhar
>> <sridhar.samudrala@...el.com> wrote:
>>>
>>>
>>> On 9/12/2017 8:47 AM, Eric Dumazet wrote:
>>>>
>>>> On Mon, 2017-09-11 at 23:27 -0700, Samudrala, Sridhar wrote:
>>>>>
>>>>> On 9/11/2017 8:53 PM, Eric Dumazet wrote:
>>>>>>
>>>>>> On Mon, 2017-09-11 at 20:12 -0700, Tom Herbert wrote:
>>>>>>
>>>>>>> Two ints in sock_common for this purpose is quite expensive and the
>>>>>>> use case for this is limited-- even if a RX->TX queue mapping were
>>>>>>> introduced to eliminate the queue pair assumption this still won't
>>>>>>> help if the receive and transmit interfaces are different for the
>>>>>>> connection. I think we really need to see some very compelling
>>>>>>> results
>>>>>>> to be able to justify this.
>>>>>
>>>>> Will try to collect and post some perf data with symmetric queue
>>>>> configuration.
>
>
> Here is some performance data i collected with memcached workload over
> ixgbe 10Gb NIC with mcblaster benchmark.
> ixgbe is configured with 16 queues and rx-usecs is set to 1000 for a very
> low
> interrupt rate.
>       ethtool -L p1p1 combined 16
>       ethtool -C p1p1 rx-usecs 1000
> and busy poll is set to 1000usecs
>       sysctl net.core.busy_poll = 1000
>
> 16 threads  800K requests/sec
> =============================
>                   rtt(min/avg/max)usecs     intr/sec contextswitch/sec
> -----------------------------------------------------------------------
> Default                2/182/10641            23391 61163
> Symmetric Queues       2/50/6311              20457 32843
>
> 32 threads  800K requests/sec
> =============================
>                  rtt(min/avg/max)usecs     intr/sec contextswitch/sec
> ------------------------------------------------------------------------
> Default                2/162/6390            32168 69450
> Symmetric Queues        2/50/3853            35044 35847
>
No idea what "Default" configuration is. Please report how xps_cpus is
being set, how many RSS queues there are, and what the mapping is
between RSS queues and CPUs and shared caches. Also, whether and
threads are pinned.

Thanks,
Tom