Message-ID: <d5241829-bd20-4c41-9dec-d805ce5b9bcc@nvidia.com>
Date: Sun, 4 May 2025 20:46:51 +0300
From: Mark Bloch <mbloch@...dia.com>
To: Jakub Kicinski <kuba@...nel.org>, Moshe Shemesh <moshe@...dia.com>
Cc: netdev@...r.kernel.org, "David S. Miller" <davem@...emloft.net>,
 Eric Dumazet <edumazet@...gle.com>, Paolo Abeni <pabeni@...hat.com>,
 Simon Horman <horms@...nel.org>, Donald Hunter <donald.hunter@...il.com>,
 Jiri Pirko <jiri@...nulli.us>, Jonathan Corbet <corbet@....net>,
 Andrew Lunn <andrew+netdev@...n.ch>, Tariq Toukan <tariqt@...dia.com>
Subject: Re: [RFC net-next 0/5] devlink: Add unique identifier to devlink port
 function



On 02/05/2025 3:39, Jakub Kicinski wrote:
> On Tue, 29 Apr 2025 11:37:51 +0300 Moshe Shemesh wrote:
>>>> UUID is limited, like it has to be 128 bits, while here it is variable
>>>> length up to the vendor.
>>>> We would like to keep it flexible per vendor. If vendor wants to use
>>>> UUID here, it will work too.  
>>>
>>> Could you please provide at least one clear user scenario for
>>> the discussion? Matching up the ports to function is presumably
>>> a means to an end for the user.  
>>
>> Sure. Multi-host system with smart-NIC, on the smart-NIC internal host 
>> we will see a representor for each PF on each of the external hosts.
>> However, we can't tell which representor belongs to which host. 
>> Actually, each host doesn't know about the others or where it is in the 
>> topology. The function uid can help the user match the host PF to the 
>> representor on the smart-NIC internal host and use the right representor 
>> to config the required host function.
> 
> Insufficient information. There are many many hosts deployed with
> multi-host NICs which do not need this sort of matching. I'm not
> saying you don't have a use case. I'm saying you haven't explained it.
> 
> We exchanged so many emails on this topic, counting the emails with
> Jiri. And you still haven't explained to me the use case. This is
> ridiculous.

Hi Jakub,

I'll try to explain the use case more clearly, I realize that some
internal context at NVIDIA may not be obvious externally, and we sometimes
take that for granted.

We're dealing with a multi-host system using a DPU (smart-NIC). In such a
system, each external (x86) host has its own PFs/VFs/SFs, but the E-Switch
manager for each PF resides on the DPU's ARM core (the internal host).

To illustrate, consider a system with two external hosts:

Host 1: PF0
        VF0 on PF0

Host 2: PF0
        VF0 on PF0

Each host is unaware that it's part of a multi-host system, internally,
each sees its PF simply as PF0, with no notion of the global topology.

On the DPU (ARM), we see representors for each BDF. For simplicity,
assume each BDF corresponds to a single devlink port. So the ARM would
expose:

PF0_HOST0_REP
UPLINK0_REP
PF0_HOST1_REP
UPLINK1_REP

In devlink terms, we're referring to the c argument in phys_port_name,
which represents the controller, effectively indicating which host
the BDF belongs to.

The problem we're addressing is matching the PF seen on a host to its
corresponding representor on the DPU. From the ARM side, we know that
this rep X belongs to pf0 on host y, but we don't which host is which.
>From within each host, you can't tell which host you are, because all
see their PF as PF0.

With the proposed feature (along with Jiri's changes), this becomes
trivial, you just match the function UID and you're done.

As a side note, I believe this feature has merit even beyond this
specific use case. It makes the mapping between representors and what
they represent more explicit and straightforward. Which is always a
good thing from a usability and clarity standpoint.

Mark

Powered by blists - more mailing lists