Message-ID: <aa57da6b-bb1b-4d77-bffa-9746c3fe94ba@nvidia.com>
Date: Tue, 6 May 2025 18:34:22 +0300
From: Mark Bloch <mbloch@...dia.com>
To: Jakub Kicinski <kuba@...nel.org>
Cc: Moshe Shemesh <moshe@...dia.com>, netdev@...r.kernel.org,
 "David S. Miller" <davem@...emloft.net>, Eric Dumazet <edumazet@...gle.com>,
 Paolo Abeni <pabeni@...hat.com>, Simon Horman <horms@...nel.org>,
 Donald Hunter <donald.hunter@...il.com>, Jiri Pirko <jiri@...nulli.us>,
 Jonathan Corbet <corbet@....net>, Andrew Lunn <andrew+netdev@...n.ch>,
 Tariq Toukan <tariqt@...dia.com>
Subject: Re: [RFC net-next 0/5] devlink: Add unique identifier to devlink port
 function



On 06/05/2025 18:20, Jakub Kicinski wrote:
> On Tue, 6 May 2025 14:25:10 +0300 Mark Bloch wrote:
>>> Thanks for explaining the setup. Could you please explain the user
>>> scenario now? Perhaps thinking of it as a sequence diagram would
>>> be helpful, but whatever is easiest, just make it concrete.
>>>   
>>
>> It's a rough flow, but I believe it clearly illustrates the use case
>> we're targeting:
>>  
>> Some system configuration info:
>>  
>> - A static mapping file exists that defines the relationship between
>>   a host and the corresponding ARM/DPU host that manages it.
>>  
>> - OVN, OVS and Kubernetes are used to manage network connectivity and
>>   resource allocation.
>>  
>> Flow:
>> 1. A user requests a container with networking connectivity.
>> 2. Kubernetes allocates a VF on host X. An agent on the host handles VF
>>    configuration and sends the PF number and VF index to the central
>>    management software.
> 
> What is "central management software" here? Deployment specific or
> some part of k8s?

It's the k8s API server.

> 
>> 3. An agent on the DPU side detects the changes made on host X. Using
>>    the PF number and VF index, it identifies the corresponding
>>    representor, attaches it to an OVS bridge, and allows OVN to program
>>    the relevant steering rules.
> 
> What does it mean that DPU "detects it", what's the source and 
> mechanism of the notification?
> Is it communicating with the central SW during  the process?

The agent (running in the ARM/DPU) listens for events from the k8s API server.
 
> 
>> This setup works well when the mapping file defines a one-to-one
>> relationship between a host and a single ARM/DPU host.
>> It's already supported in upstream today [1]
>>  
>> However, in a slightly more generic scenario like:
>>  
>> Control Host A: External host X
>>                 External host Y
>>  
>> A single ARM/DPU host manages multiple external hosts. In this case, step
>> 2—where only the PF number and VF index are sent is insufficient. During
>> step 3, the agent on the DPU reads the data but cannot determine which
>> external host created the VF. As a result, it cannot correctly associate
>> the representor with the appropriate OVS bridge.
>>  
>> To resolve this, we plan to modify step 2 to include the VUID along with
>> the PF number and VF index. The DPU-side agent will use the VUID to match
>> it with the FUID, identify the correct PF representor, and then use
>> standard devlink mechanisms to locate the corresponding VF representor.
>>
>> 1: https://github.com/ovn-kubernetes/ovn-kubernetes
>> You can look at: go-controller/pkg/util/dpu_annotations.go for more info.
> 
> A link to the actual file / relevant code would be more helpful :(

This code listens for events on the ARM/DPU from the Kubernetes API server:
https://github.com/ovn-kubernetes/ovn-kubernetes/blob/39e94d80c286f69c5416166d8acda5d1d2f9add5/go-controller/pkg/node/base_node_network_controller_dpu.go#L100

I’m not very familiar with this part of the code, so I asked our
k8s team to help me identify the relevant function. Hopefully, this
is what you were looking for.

Mark

Powered by blists - more mailing lists