netdev - Re: [PATCH RFC v2 net-next 1/2] devlink: Add shared descriptor eswitch attr

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <b25bfe54-f3c7-44a2-8c39-d32a0d5f3d47@nvidia.com>
Date: Fri, 1 Mar 2024 09:25:20 -0800
From: William Tu <witu@...dia.com>
To: "Samudrala, Sridhar" <sridhar.samudrala@...el.com>, netdev@...r.kernel.org
Cc: jiri@...dia.com, bodong@...dia.com, tariqt@...dia.com,
 yossiku@...dia.com, kuba@...nel.org
Subject: Re: [PATCH RFC v2 net-next 1/2] devlink: Add shared descriptor
 eswitch attr



On 2/29/24 5:46 PM, Samudrala, Sridhar wrote:
> External email: Use caution opening links or attachments
>
>
> On 2/29/2024 7:11 PM, William Tu wrote:
>> Add two eswitch attrs: shrdesc_mode and shrdesc_count.
>>
>> 1. shrdesc_mode: to enable a sharing memory buffer for
>> representor's rx buffer, and 2. shrdesc_count: to control the
>> number of buffers in this shared memory pool.
>>
>> When using switchdev mode, the representor ports handles the slow path
>> traffic, the traffic that can't be offloaded will be redirected to the
>> representor port for processing. Memory consumption of the representor
>> port's rx buffer can grow to several GB when scaling to 1k VFs reps.
>> For example, in mlx5 driver, each RQ, with a typical 1K descriptors,
>> consumes 3MB of DMA memory for packet buffer in WQEs, and with four
>> channels, it consumes 4 * 3MB * 1024 = 12GB of memory. And since rep
>> ports are for slow path traffic, most of these rx DMA memory are idle.
>>
>> Add shrdesc_mode configuration, allowing multiple representors
>> to share a rx memory buffer pool. When enabled, individual representor
>> doesn't need to allocate its dedicated rx buffer, but just pointing
>> its rq to the memory pool. This could make the memory being better
>
> I guess the rx buffers are allocated from a page_pool. Does it mean that
> a page pool is now shared across multiple rx queues belonging to
> multiple netdevs?  Do they all share the same napi?

yes. The basic sharing scheme is to have all representor netdevs' rx 
queue N sharing 1 pool.
And packets are proceeded by uplink netdev, so they share the same napi.
More detail here:
https://lore.kernel.org/netdev/20240125223617.7298-1-witu@nvidia.com/