netdev - Re: [PATCH RFC v2 net-next 1/2] devlink: Add shared descriptor eswitch attr

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <91629dbc-8fce-4f58-bd9b-b37293c220b8@intel.com>
Date: Thu, 29 Feb 2024 19:46:10 -0600
From: "Samudrala, Sridhar" <sridhar.samudrala@...el.com>
To: William Tu <witu@...dia.com>, <netdev@...r.kernel.org>
CC: <jiri@...dia.com>, <bodong@...dia.com>, <tariqt@...dia.com>,
	<yossiku@...dia.com>, <kuba@...nel.org>
Subject: Re: [PATCH RFC v2 net-next 1/2] devlink: Add shared descriptor
 eswitch attr



On 2/29/2024 7:11 PM, William Tu wrote:
> Add two eswitch attrs: shrdesc_mode and shrdesc_count.
> 
> 1. shrdesc_mode: to enable a sharing memory buffer for
> representor's rx buffer, and 2. shrdesc_count: to control the
> number of buffers in this shared memory pool.
> 
> When using switchdev mode, the representor ports handles the slow path
> traffic, the traffic that can't be offloaded will be redirected to the
> representor port for processing. Memory consumption of the representor
> port's rx buffer can grow to several GB when scaling to 1k VFs reps.
> For example, in mlx5 driver, each RQ, with a typical 1K descriptors,
> consumes 3MB of DMA memory for packet buffer in WQEs, and with four
> channels, it consumes 4 * 3MB * 1024 = 12GB of memory. And since rep
> ports are for slow path traffic, most of these rx DMA memory are idle.
> 
> Add shrdesc_mode configuration, allowing multiple representors
> to share a rx memory buffer pool. When enabled, individual representor
> doesn't need to allocate its dedicated rx buffer, but just pointing
> its rq to the memory pool. This could make the memory being better

I guess the rx buffers are allocated from a page_pool. Does it mean that 
a page pool is now shared across multiple rx queues belonging to 
multiple netdevs?  Do they all share the same napi?

> utilized. The shrdesc_count represents the number of rx ring
> entries, e.g., same meaning as ethtool -g, that's shared across other
> representors. Users adjust it based on how many reps, total system
> memory, or performance expectation.
> 
> The two params are also useful for other vendors such as Intel ICE
> drivers and Broadcom's driver, which also have representor ports for
> slow path traffic.
> 
> An example use case:
> $ devlink dev eswitch show pci/0000:08:00.0
>    pci/0000:08:00.0: mode legacy inline-mode none encap-mode basic \
>    shrdesc-mode none shrdesc-count 0
> $ devlink dev eswitch set pci/0000:08:00.0 mode switchdev \
>    shrdesc-mode basic shrdesc-count 1024
> $ devlink dev eswitch show pci/0000:08:00.0
>    pci/0000:08:00.0: mode switchdev inline-mode none encap-mode basic \
>    shrdesc-mode basic shrdesc-count 1024
> 
> Note that new configurations are set at legacy mode, and enabled at
> switchdev mode.
> 
> Signed-off-by: William Tu <witu@...dia.com>
> ---

<snip>