netdev - Re: [PATCH v2] net: mana: Use page pool fragments for RX buffers instead of full pages to improve memory efficiency.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20250725175418.553b5b6e@kernel.org>
Date: Fri, 25 Jul 2025 17:54:18 -0700
From: Jakub Kicinski <kuba@...nel.org>
To: Dipayaan Roy <dipayanroy@...ux.microsoft.com>
Cc: horms@...nel.org, kys@...rosoft.com, haiyangz@...rosoft.com,
 wei.liu@...nel.org, decui@...rosoft.com, andrew+netdev@...n.ch,
 davem@...emloft.net, edumazet@...gle.com, pabeni@...hat.com,
 longli@...rosoft.com, kotaranov@...rosoft.com, ast@...nel.org,
 daniel@...earbox.net, hawk@...nel.org, john.fastabend@...il.com,
 sdf@...ichev.me, lorenzo@...nel.org, michal.kubiak@...el.com,
 ernis@...ux.microsoft.com, shradhagupta@...ux.microsoft.com,
 shirazsaleem@...rosoft.com, rosenp@...il.com, netdev@...r.kernel.org,
 linux-hyperv@...r.kernel.org, linux-rdma@...r.kernel.org,
 bpf@...r.kernel.org, linux-kernel@...r.kernel.org,
 ssengar@...ux.microsoft.com, dipayanroy@...rosoft.com
Subject: Re: [PATCH v2] net: mana: Use page pool fragments for RX buffers
 instead of full pages to improve memory efficiency.

On Wed, 23 Jul 2025 12:07:06 -0700 Dipayaan Roy wrote:
> This patch enhances RX buffer handling in the mana driver by allocating
> pages from a page pool and slicing them into MTU-sized fragments, rather
> than dedicating a full page per packet. This approach is especially
> beneficial on systems with large page sizes like 64KB.
> 
> Key improvements:
> 
> - Proper integration of page pool for RX buffer allocations.
> - MTU-sized buffer slicing to improve memory utilization.
> - Reduce overall per Rx queue memory footprint.
> - Automatic fallback to full-page buffers when:
>    * Jumbo frames are enabled (MTU > PAGE_SIZE / 2).
>    * The XDP path is active, to avoid complexities with fragment reuse.
> - Removal of redundant pre-allocated RX buffers used in scenarios like MTU
>   changes, ensuring consistency in RX buffer allocation.
> 
> Testing on VMs with 64KB pages shows around 200% throughput improvement.
> Memory efficiency is significantly improved due to reduced wastage in page
> allocations. Example: We are now able to fit 35 rx buffers in a single 64kb
> page for MTU size of 1500, instead of 1 rx buffer per page previously.

The diff is pretty large and messy, please try to extract some
refactoring patches that make the final transition easier to review.

> - iperf3, iperf2, and nttcp benchmarks.
> - Jumbo frames with MTU 9000.
> - Native XDP programs (XDP_PASS, XDP_DROP, XDP_TX, XDP_REDIRECT) for
>   testing the XDP path in driver.
> - Page leak detection (kmemleak).

kmemleak doesn't detect page leaks AFAIU, just slab objects

> - Driver load/unload, reboot, and stress scenarios.
> 
> Signed-off-by: Dipayaan Roy <dipayanroy@...ux.microsoft.com>
> 
> Reviewed-by: Jacob Keller <jacob.e.keller@...el.com>
> Reviewed-by: Saurabh Sengar <ssengar@...ux.microsoft.com>

> -	if (apc->port_is_up)
> +	if (apc->port_is_up) {
> +		/* Re-create rxq's after xdp prog was loaded or unloaded.
> +		 * Ex: re create rxq's to switch from full pages to smaller
> +		 * size page fragments when xdp prog is unloaded and vice-versa.
> +		 */
> +
> +		err = mana_detach(ndev, false);
> +		if (err) {
> +			netdev_err(ndev, "mana_detach failed at xdp set: %d\n", err);
> +			goto out;
> +		}
> +
> +		err = mana_attach(ndev);
> +		if (err) {
> +			netdev_err(ndev, "mana_attach failed at xdp set: %d\n", err);
> +			goto out;
> +		}

If the system is low on memory you will make it unreachable.
It's a very poor design.

> -/* Release pre-allocated RX buffers */
> -void mana_pre_dealloc_rxbufs(struct mana_port_context *mpc)
> -{
> -	struct device *dev;
> -	int i;
> -

Looks like you're deleting the infrastructure the driver had for
pre-allocating memory. Not even mentioning it in the commit message.
This ability needs to be maintain. Please test with memory allocation
injections and make sure the driver survives failed reconfig requests.
The reconfiguration should be cleanly rejected if mem alloc fails,
and the driver should continue to work with old settings in place.
-- 
pw-bot: cr