netdev - Re: [net-next 10/15] net/mlx5e: Let channels be SD-aware

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240109080036.65634705@kernel.org>
Date: Tue, 9 Jan 2024 08:00:36 -0800
From: Jakub Kicinski <kuba@...nel.org>
To: Gal Pressman <gal@...dia.com>
Cc: Saeed Mahameed <saeed@...nel.org>, "David S. Miller"
 <davem@...emloft.net>, Paolo Abeni <pabeni@...hat.com>, Eric Dumazet
 <edumazet@...gle.com>, Saeed Mahameed <saeedm@...dia.com>,
 netdev@...r.kernel.org, Tariq Toukan <tariqt@...dia.com>
Subject: Re: [net-next 10/15] net/mlx5e: Let channels be SD-aware

On Tue, 9 Jan 2024 16:15:50 +0200 Gal Pressman wrote:
> >> I'm confused, how are RX queues related to XPS?  
> > 
> > Separate sentence, perhaps I should be more verbose..  
> 
> Sorry, yes, your understanding is correct.
> If a packet is received on RQ 0 then it is from PF 0, RQ 1 came from PF
> 1, etc. Though this is all from the same wire/port.
> 
> You can enable arfs for example, which will make sure that packets that
> are destined to a certain CPU will be received by the PF that is closer
> to it.

Got it.

> >> XPS shouldn't be affected, we just make sure that whatever queue XPS
> >> chose will go out through the "right" PF.  
> > 
> > But you said "correct" to queue 0 going to PF 0 and queue 1 to PF 1.
> > The queue IDs in my question refer to the queue mapping form the stacks
> > perspective. If user wants to send everything to queue 0 will it use
> > both PFs?  
> 
> If all traffic is transmitted through queue 0, it will go out from PF 0
> (the PF that is closer to CPU 0 numa).

Okay, but earlier you said: "whatever queue XPS chose will go out
through the "right" PF." - which I read as PF will be chosen based
on CPU locality regardless of XPS logic.

If queue 0 => PF 0, then user has to set up XPS to make CPUs from NUMA
node which has PF 0 use even number queues, and PF 1 to use odd number
queues. Correct?

> >> So for example, XPS will choose a queue according to the CPU, and the
> >> driver will make sure that packets transmitted from this SQ are going
> >> out through the PF closer to that NUMA.  
> > 
> > Sounds like queue 0 is duplicated in both PFs, then?  
> 
> Depends on how you look at it, each PF has X queues, the netdev has 2X
> queues.

I'm asking how it looks from the user perspective, to be clear.
From above I gather than the answer is no - queue 0 maps directly 
to PF 0 / queue 0, nothing on PF 1 will ever see traffic of queue 0.

> >> Can you share a link please?  
> > 
> > commit a90d56049acc45802f67cd7d4c058ac45b1bc26f  
> 
> Thanks, will take a look.
> 
> >> All the logic is internal to the driver, so I expect it to be fine, but
> >> I'd like to double check.
> > 
> > Herm, "internal to the driver" is a bit of a landmine. It will be fine
> > for iperf testing but real users will want to configure the NIC.
> 
> What kind of configuration are you thinking of?

Well, I was hoping you'd do the legwork and show how user configuration
logic has to be augmented for all relevant stack features to work with
multi-PF devices. I can list the APIs that come to mind while writing
this email, but that won't be exhaustive :(