netdev - net-shapers and traffic classes (an update)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <67df1a562614b553dcab043f347a0d7c5393ff83.camel@nvidia.com>
Date: Thu, 6 Feb 2025 10:03:22 +0000
From: Cosmin Ratiu <cratiu@...dia.com>
To: "netdev@...r.kernel.org" <netdev@...r.kernel.org>
CC: "andrew+netdev@...n.ch" <andrew+netdev@...n.ch>, "davem@...emloft.net"
	<davem@...emloft.net>, Tariq Toukan <tariqt@...dia.com>, Gal Pressman
	<gal@...dia.com>, Leon Romanovsky <leonro@...dia.com>, "jiri@...nulli.us"
	<jiri@...nulli.us>, "horms@...nel.org" <horms@...nel.org>,
	"edumazet@...gle.com" <edumazet@...gle.com>, "kuba@...nel.org"
	<kuba@...nel.org>, Saeed Mahameed <saeedm@...dia.com>, "pabeni@...hat.com"
	<pabeni@...hat.com>
Subject: net-shapers and traffic classes (an update)

Hello,

In December there was some discussion on a patch series [1] regarding
supporting traffic classes in net-shapers, whether it's possible and
how. Since then, a few of us from the mlx5 team have been meeting with
Paolo Abeni and Simon Horman to try to figure things out, and I wanted
to give an overview of where things are.

net-shapers [2] is a recently introduced API to shape traffic on a
netdevice or its queues.  It has a hierarchy of levels it can configure
shapers on. One assumption is that the hierarchy is a tree and not a
graph.

devlink rate has an API for shaping traffic for a VF or a group of VFs
[2]. The recent proposal [1] extends it with support for traffic class
shaping, with an example implementation for mlx5.

The question is whether net-shapers should model traffic classes and be
able to affect them.

In the series of meetings we had, we tried to understand how mlx5 is
configured to do traffic shaping and how that can be extrapolated into
a generic way to do it in net-shapers that could apply to other
drivers.

So far, we know that there is a 1:1 mapping between txqs and the
traffic class they service. There are a variety of mechanisms to
configure:
1. The TC for a queue: mqprio, HTB, infiniband user queue pairs, etc.
2. queue mapping for an skb in ndo_select_queue: VLAN tags, DCBX + IP
TOS, skb prio.
But in the end, the skb is assigned a queue and therefore a traffic
class.

Another question is who can set the limit for a TC. In devlink rate,
this limit is set from outside the netdevice being affected ("the other
side of the wire"), mostly from the hypervisor. The VM sending traffic
out on a VF has no say about the limit.

There are some open questions left:
- What happens if the number of queue changes while TC is configured,
how would net-shapers and traffic behave?
- What about some more obscure mechanisms in mlx5 that maybe don't rely
on the 1:1 mapping between txq:tc? The HW can inspect outgoing packets
and can be configured to assign them TCs and schedule them accordingly.
This is on the mlx5 team to properly understand and communicate.

But this is where we're at.
Based on the discussions so far, I believe it should be possible to
model this in net-shapers when TCs appear as disjoint subsets of netdev
queues.
Given this, we will proceed with the ETS submission, continue meeting
with Paolo and Simon to iron out net-shapers TC details and use this
thread for public discussions.

Cosmin.

[1]
https://lore.kernel.org/netdev/20241204220931.254964-1-tariqt@nvidia.com/
[2]
https://lore.kernel.org/netdev/cover.1728460186.git.pabeni@redhat.com/