linux-kernel - Re: [PATCH v2] net: mana: Use page pool fragments for RX buffers instead of full pages to improve memory efficiency.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <i5o2nzwpd5ommosp4ci5edrozci34v6lfljteldyilsfe463xd@6qts2hifezz3>
Date: Wed, 30 Jul 2025 07:31:50 +0000
From: Dragos Tatulea <dtatulea@...dia.com>
To: Dipayaan Roy <dipayanroy@...ux.microsoft.com>, 
	Jesper Dangaard Brouer <hawk@...nel.org>
Cc: horms@...nel.org, kuba@...nel.org, kys@...rosoft.com, 
	haiyangz@...rosoft.com, wei.liu@...nel.org, decui@...rosoft.com, andrew+netdev@...n.ch, 
	davem@...emloft.net, edumazet@...gle.com, pabeni@...hat.com, longli@...rosoft.com, 
	kotaranov@...rosoft.com, ast@...nel.org, daniel@...earbox.net, john.fastabend@...il.com, 
	sdf@...ichev.me, lorenzo@...nel.org, michal.kubiak@...el.com, 
	ernis@...ux.microsoft.com, shradhagupta@...ux.microsoft.com, shirazsaleem@...rosoft.com, 
	rosenp@...il.com, netdev@...r.kernel.org, linux-hyperv@...r.kernel.org, 
	linux-rdma@...r.kernel.org, bpf@...r.kernel.org, linux-kernel@...r.kernel.org, 
	ssengar@...ux.microsoft.com, dipayanroy@...rosoft.com, Chris Arges <carges@...udflare.com>, 
	kernel-team <kernel-team@...udflare.com>, Tariq Toukan <tariqt@...dia.com>, 
	Saeed Mahameed <saeedm@...dia.com>, Yunsheng Lin <linyunsheng@...wei.com>
Subject: Re: [PATCH v2] net: mana: Use page pool fragments for RX buffers
 instead of full pages to improve memory efficiency.

On Tue, Jul 29, 2025 at 01:20:07PM -0700, Dipayaan Roy wrote:
> On Tue, Jul 29, 2025 at 12:15:23PM +0200, Jesper Dangaard Brouer wrote:
> > 
> > 
> > On 23/07/2025 21.07, Dipayaan Roy wrote:
> > >This patch enhances RX buffer handling in the mana driver by allocating
> > >pages from a page pool and slicing them into MTU-sized fragments, rather
> > >than dedicating a full page per packet. This approach is especially
> > >beneficial on systems with large page sizes like 64KB.
> > >
> > >Key improvements:
> > >
> > >- Proper integration of page pool for RX buffer allocations.
> > >- MTU-sized buffer slicing to improve memory utilization.
> > >- Reduce overall per Rx queue memory footprint.
> > >- Automatic fallback to full-page buffers when:
> > >    * Jumbo frames are enabled (MTU > PAGE_SIZE / 2).
> > >    * The XDP path is active, to avoid complexities with fragment reuse.
> > >- Removal of redundant pre-allocated RX buffers used in scenarios like MTU
> > >   changes, ensuring consistency in RX buffer allocation.
> > >
> > >Testing on VMs with 64KB pages shows around 200% throughput improvement.
> > >Memory efficiency is significantly improved due to reduced wastage in page
> > >allocations. Example: We are now able to fit 35 rx buffers in a single 64kb
> > >page for MTU size of 1500, instead of 1 rx buffer per page previously.
> > >
> > >Tested:
> > >
> > >- iperf3, iperf2, and nttcp benchmarks.
> > >- Jumbo frames with MTU 9000.
> > >- Native XDP programs (XDP_PASS, XDP_DROP, XDP_TX, XDP_REDIRECT) for
> > >   testing the XDP path in driver.
> > >- Page leak detection (kmemleak).
> > >- Driver load/unload, reboot, and stress scenarios.
> > 
> > Chris (Cc) discovered a crash/bug[1] with page pool fragments used
> > from the mlx5 driver.
> > He put together a BPF program that reproduces the issue here:
> > - [2] https://github.com/arges/xdp-redirector
> > 
> > Can I ask you to test that your driver against this reproducer?
> > 
> > 
> > [1] https://lore.kernel.org/all/aIEuZy6fUj_4wtQ6@861G6M3/
> > 
> > --Jesper
> >
> 
> Hi Jesper,
> 
> I was unable to reproduce this issue on mana driver.
>
Please note that I had to make a few adjustments to get reprodduction on
mlx5:

- Make sure that the veth MACs are recognized by the device. Otherwise
  traffic might be dropped by the device.

- Enable GRO on the veth device. Otherwise packets get dropped before
  they reach the devmap BPF program.

Try starting the test program with one thread and see if you see packets
coming through veth1-ns1 end of the veth pair.

Thanks,
Dragos