netdev - Re: [PATCH net-next] net: set default rss queues num to physical cores / 2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20220316201007.41aeb5f7@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
Date:   Wed, 16 Mar 2022 20:10:07 -0700
From:   Jakub Kicinski <kuba@...nel.org>
To:     Íñigo Huguet <ihuguet@...hat.com>
Cc:     davem@...emloft.net, edumazet@...gle.com, bigeasy@...utronix.de,
        atenart@...nel.org, imagedong@...cent.com, petrm@...dia.com,
        arnd@...db.de, netdev@...r.kernel.org,
        linux-kernel@...r.kernel.org, michael.chan@...adcom.com,
        Siva Reddy Kallam <siva.kallam@...adcom.com>,
        Prashant Sreedharan <prashant@...adcom.com>,
        Rasesh Mody <rmody@...vell.com>,
        Sunil Goutham <sgoutham@...vell.com>,
        Raju Rangoju <rajur@...lsio.com>,
        Ajit Khaparde <ajit.khaparde@...adcom.com>,
        Sriharsha Basavapatna <sriharsha.basavapatna@...adcom.com>,
        Somnath Kotur <somnath.kotur@...adcom.com>,
        Chris Lee <christopher.lee@...i.com>,
        Jon Mason <jdmason@...zu.us>,
        Simon Horman <simon.horman@...igine.com>,
        Ariel Elior <aelior@...vell.com>,
        Manish Chopra <manishc@...vell.com>,
        Jose Abreu <Jose.Abreu@...opsys.com>
Subject: Re: [PATCH net-next] net: set default rss queues num to physical
 cores / 2

On Tue, 15 Mar 2022 10:18:32 +0100 Íñigo Huguet wrote:
> Network drivers can call to netif_get_num_default_rss_queues to get the
> default number of receive queues to use. Right now, this default number
> is min(8, num_online_cpus()).
> 
> Instead, as suggested by Jakub, use the number of physical cores divided
> by 2 as a way to avoid wasting CPU resources and to avoid using both CPU
> threads, but still allowing to scale for high-end processors with many
> cores.
> 
> As an exception, select 2 queues for processors with 2 cores, because
> otherwise it won't take any advantage of RSS despite being SMP capable.
> 
> Tested: Processor Intel Xeon E5-2620 (2 sockets, 6 cores/socket, 2
> threads/core). NIC Broadcom NetXtreme II BCM57810 (10GBps). Ran some
> tests with `perf stat iperf3 -R`, with parallelisms of 1, 8 and 24,
> getting the following results:
> - Number of queues: 6 (instead of 8)
> - Network throughput: not affected
> - CPU usage: utilized 0.05-0.12 CPUs more than before (having 24 CPUs
>   this is only 0.2-0.5% higher)
> - Reduced the number of context switches by 7-50%, being more noticeable
>   when using a higher number of parallel threads.

Thanks for following up, Inigo!

Heads up for the maintainers of drivers which use
netif_get_num_default_rss_queues() today - please note the above - 
the default number of Rx queues may change for you starting with 
the 5.18 kernel.