linux-kernel - Re: [PATCH net-next 10/10] net/mlx5e: Use the 'num

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <a36bc123aa1021a5aeca0494f4b8ea140ef789d0.camel@nvidia.com>
Date: Tue, 16 Sep 2025 08:32:22 +0000
From: Cosmin Ratiu <cratiu@...dia.com>
To: Tariq Toukan <tariqt@...dia.com>, "stfomichev@...il.com"
	<stfomichev@...il.com>
CC: "corbet@....net" <corbet@....net>, "andrew+netdev@...n.ch"
	<andrew+netdev@...n.ch>, "hawk@...nel.org" <hawk@...nel.org>,
	"davem@...emloft.net" <davem@...emloft.net>, "john.fastabend@...il.com"
	<john.fastabend@...il.com>, "leon@...nel.org" <leon@...nel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"edumazet@...gle.com" <edumazet@...gle.com>, "ast@...nel.org"
	<ast@...nel.org>, "jiri@...nulli.us" <jiri@...nulli.us>, "pabeni@...hat.com"
	<pabeni@...hat.com>, "linux-rdma@...r.kernel.org"
	<linux-rdma@...r.kernel.org>, Jiri Pirko <jiri@...dia.com>,
	"linux-doc@...r.kernel.org" <linux-doc@...r.kernel.org>, Dragos Tatulea
	<dtatulea@...dia.com>, Mark Bloch <mbloch@...dia.com>, "daniel@...earbox.net"
	<daniel@...earbox.net>, "bpf@...r.kernel.org" <bpf@...r.kernel.org>, Saeed
 Mahameed <saeedm@...dia.com>, "netdev@...r.kernel.org"
	<netdev@...r.kernel.org>, "kuba@...nel.org" <kuba@...nel.org>, Gal Pressman
	<gal@...dia.com>, "jgg@...pe.ca" <jgg@...pe.ca>
Subject: Re: [PATCH net-next 10/10] net/mlx5e: Use the 'num_doorbells' devlink
 param

On Wed, 2025-09-10 at 09:16 -0700, Stanislav Fomichev wrote:
> > +       - 0: No channel-specific doorbells, use the global one for
> > everything.
> > +       - [1, max_num_channels]: Spread netdev channels equally
> > across these
> > +         doorbells.
> 
> Do you have any guidance on this number? Why would the user want
> `num_doorbells < num_doorbells` vs `num_doorbells == num_channels`?
> 
> IOW, why not allocate the same number of doorbells as the number of
> channels and do it unconditionally without devlink param? Are extra
> doorbells causing any overhead in the non-contended case?

In most cases, additional doorbells are an overhead and not required.
For the last 10+ years, mlx5 has been running with a single doorbell
for all channels. But as the number of cores and channels grew,
bottlenecks were discovered on some platforms. Thus the need for this
series.

This series proposes 8 as the new default and we expect nobody would
need to touch this knob except in extreme cases, but I think it's nice
for it to exist. Regression testing showed little to no impact on most
platforms where 1 doorbell was enough, and a significant improvement on
platforms that showed MMIO bottlenecks.

I'll add some numbers in the next version's cover letter.

Cosmin.