netdev - Re: [PATCH v4 10/10] bnxt: Create an auxiliary device for fwctl

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CACDg6nWU7XXn4X3LGy=jxREYDDVaqy1Pq19kt93wQPn_US9iiQ@mail.gmail.com>
Date: Fri, 7 Feb 2025 18:29:15 -0500
From: Andy Gospodarek <andrew.gospodarek@...adcom.com>
To: Jakub Kicinski <kuba@...nel.org>
Cc: Jason Gunthorpe <jgg@...dia.com>, Aron Silverton <aron.silverton@...cle.com>, 
	Dan Williams <dan.j.williams@...el.com>, Daniel Vetter <daniel.vetter@...ll.ch>, 
	Dave Jiang <dave.jiang@...el.com>, David Ahern <dsahern@...nel.org>, 
	Andy Gospodarek <gospo@...adcom.com>, Christoph Hellwig <hch@...radead.org>, 
	Itay Avraham <itayavr@...dia.com>, Jiri Pirko <jiri@...dia.com>, 
	Jonathan Cameron <Jonathan.Cameron@...wei.com>, Leonid Bloch <lbloch@...dia.com>, 
	Leon Romanovsky <leonro@...dia.com>, linux-cxl@...r.kernel.org, linux-rdma@...r.kernel.org, 
	netdev@...r.kernel.org, Saeed Mahameed <saeedm@...dia.com>, 
	"Nelson, Shannon" <shannon.nelson@....com>, Michael Chan <michael.chan@...adcom.com>
Subject: Re: [PATCH v4 10/10] bnxt: Create an auxiliary device for fwctl_bnxt

On Fri, Feb 7, 2025 at 10:36 AM Jakub Kicinski <kuba@...nel.org> wrote:
>
> On Thu, 6 Feb 2025 22:17:58 -0500 Andy Gospodarek wrote:
> > On Thu, Feb 6, 2025 at 7:44 PM Jakub Kicinski <kuba@...nel.org> wrote:
> > > On Thu,  6 Feb 2025 20:13:32 -0400 Jason Gunthorpe wrote:
> > > > From: Andy Gospodarek <gospo@...adcom.com>
> > > >
> > > > Signed-off-by: Andy Gospodarek <gospo@...adcom.com>
> > > > Signed-off-by: Jason Gunthorpe <jgg@...dia.com>
> > >
> > > This is only needed for RDMA, why can't you make this part of bnxt_re ?
> >
> > This is not just needed for RDMA, so having the aux device for fwctl
> > as part of the base driver is preferred.
>
> Please elaborate. As you well know I have experience using Broadcom
> devices in large TCP/IP networks, without the need for proprietary
> tooling.

I totally get that.  As a user it is not satisfying to have to
download and attempt to compile complicated proprietary tools to use
hardware features that seem like they should just work.  I don't think
fwctl should be used as a crutch to avoid doing the work that is
needed to get support upstream.

> Now, I understand that it may be expedient for Broadcom and nVidia
> to skip the upstream process and ship "features" to customers using
> DOCA and whatever you call your tooling. But let's be honest that
> this is the motivation here. Unified support for proprietary tooling
> across subsystems and product lines for a given vendor. This way
> migrating from in-tree networking to proprietary IPU/DPU networking
> is easier, while migrating to another vendor would require full tooling
> replacement.
>
> I have nothing against RDMA and CXL subsystems adding whatever APIs
> they want. But I don't understand why you think it's okay to force
> this on normal networking, which does not need it.
>
> nVidia is already refusing to add basic minoring features to their
> upstream driver, and keeps asking its customers to migrate to libdoca.
> So the concern that merging this will negatively impact standard
> tooling is no longer theoretical.
>
> Anyway, rant over. Give us some technical details.

The primary use-case that I find valuable is the ability to perform
debug of different parts of a hardware pipeline when devices are
already in the field.  This could be the standard ethernet pipeline,
RoCE, crypto, etc.

We do have the ability to gather all the information we need via tools
like ethtool and devlink, but there are cases where running a tool in
real-time can help us know what is happening in a system on a per
packet basis.  We actually did something like this this week.

When I look at fwctl, I don't see it as something that is valuable
only today -- I see it as something that is valuable 2 years from now.
When someone is still running v6.17 and we have discovered that a
debug counter/infra that was added in v7.0, but they cannot use it
without installing a new kernel or an OOB driver we don't have an
option to easily help narrow down the problem.

If a fairly simple tool can help perform RPC to FW to glean some of
this hardware information we save days of back and forth debugging
with special drivers to try and help narrow down the issue.