netdev - RE: [PATCH v2] net: netvsc: Update default VMBus channels

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID:
 <SN6PR02MB4157330500B79F4E728DFF7FD4872@SN6PR02MB4157.namprd02.prod.outlook.com>
Date: Wed, 14 Aug 2024 23:57:07 +0000
From: Michael Kelley <mhklinux@...look.com>
To: Erni Sri Satya Vennela <ernis@...ux.microsoft.com>, "kys@...rosoft.com"
	<kys@...rosoft.com>, "haiyangz@...rosoft.com" <haiyangz@...rosoft.com>,
	"wei.liu@...nel.org" <wei.liu@...nel.org>, "decui@...rosoft.com"
	<decui@...rosoft.com>, "davem@...emloft.net" <davem@...emloft.net>,
	"edumazet@...gle.com" <edumazet@...gle.com>, "kuba@...nel.org"
	<kuba@...nel.org>, "pabeni@...hat.com" <pabeni@...hat.com>,
	"linux-hyperv@...r.kernel.org" <linux-hyperv@...r.kernel.org>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
CC: "ernis@...rosoft.com" <ernis@...rosoft.com>
Subject: RE: [PATCH v2] net: netvsc: Update default VMBus channels

From: Erni Sri Satya Vennela <ernis@...ux.microsoft.com> Sent: Wednesday, August 14, 2024 9:59 AM
> 
> Change VMBus channels macro (VRSS_CHANNEL_DEFAULT) in
> Linux netvsc from 8 to 16 to align with Azure Windows VM
> and improve networking throughput.
> 
> For VMs having less than 16 vCPUS, the channels depend
> on number of vCPUs. Between 16 to 32 vCPUs, the channels
> default to VRSS_CHANNEL_DEFAULT. For greater than 32 vCPUs,
> set the channels to number of physical cores / 2 as a way
> to optimize CPU resource utilization and scale for high-end
> processors with many cores.
> Maximum number of channels are by default set to 64.

Where in the code is this enforced? It's not part of this patch. It
might be in rndis_set_subchannel(), where a value larger than
64 could be sent to the Hyper-V host, expecting that the Hyper-V
host will limit it to 64. But netvsc driver code is declaring an array
of size VRSS_CHANNEL_MAX, and there's nothing that guarantees
that Hyper-V will always limit the channel count to 64. But maybe
the netvsc driver enforces the limit of VRSS_CHANNEL_MAX in a
place that I didn't immediately see in a quick look at the code.

> 
> Based on this change the subchannel creation would change as follows:
> 
> -------------------------------------------------------------
> |No. of vCPU	|dev_info->num_chn	|subchannel created |
> -------------------------------------------------------------
> |  0-16		|	16		|	vCPU	    |
> | >16 & <=32	|	16		|	16          |
> | >32 & <=128	|	vCPU/2		|	vCPU/2      |
> | >128		|	vCPU/2		|	64          |
> -------------------------------------------------------------

The terminology here is slightly wrong. A VMBus device has one
primary channel plus zero or more subchannels. The chart
above is specifying the total number of channels (primary plus
subchannels), not the number of subchannels.

Michael

> 
> Performance tests showed significant improvement in throughput:
> - 0.54% for 16 vCPUs
> - 0.83% for 32 vCPUs
> - 1.76% for 48 vCPUs
> - 10.35% for 64 vCPUs
> - 13.47% for 96 vCPUs
> 
> Signed-off-by: Erni Sri Satya Vennela <ernis@...ux.microsoft.com>
> ---
> Changes in v2:
> * Set dev_info->num_chn based on vCPU count
> ---
>  drivers/net/hyperv/hyperv_net.h | 2 +-
>  drivers/net/hyperv/netvsc_drv.c | 5 ++++-
>  2 files changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_net.h
> index 810977952f95..e690b95b1bbb 100644
> --- a/drivers/net/hyperv/hyperv_net.h
> +++ b/drivers/net/hyperv/hyperv_net.h
> @@ -882,7 +882,7 @@ struct nvsp_message {
> 
>  #define VRSS_SEND_TAB_SIZE 16  /* must be power of 2 */
>  #define VRSS_CHANNEL_MAX 64
> -#define VRSS_CHANNEL_DEFAULT 8
> +#define VRSS_CHANNEL_DEFAULT 16
> 
>  #define RNDIS_MAX_PKT_DEFAULT 8
>  #define RNDIS_PKT_ALIGN_DEFAULT 8
> diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c
> index 44142245343d..e32eb2997bf7 100644
> --- a/drivers/net/hyperv/netvsc_drv.c
> +++ b/drivers/net/hyperv/netvsc_drv.c
> @@ -27,6 +27,7 @@
>  #include <linux/rtnetlink.h>
>  #include <linux/netpoll.h>
>  #include <linux/bpf.h>
> +#include <linux/cpumask.h>
> 
>  #include <net/arp.h>
>  #include <net/route.h>
> @@ -987,7 +988,9 @@ struct netvsc_device_info *netvsc_devinfo_get(struct
> netvsc_device *nvdev)
>  			dev_info->bprog = prog;
>  		}
>  	} else {
> -		dev_info->num_chn = VRSS_CHANNEL_DEFAULT;
> +		int count = num_online_cpus();
> +
> +		dev_info->num_chn = (count < 32) ? VRSS_CHANNEL_DEFAULT : DIV_ROUND_UP(count, 2);
>  		dev_info->send_sections = NETVSC_DEFAULT_TX;
>  		dev_info->send_section_size = NETVSC_SEND_SECTION_SIZE;
>  		dev_info->recv_sections = NETVSC_DEFAULT_RX;
> --
> 2.34.1
>