Message-ID: <Z2CpgqpIR5_MXTO7@lore-desk>
Date: Mon, 16 Dec 2024 23:28:18 +0100
From: Lorenzo Bianconi <lorenzo@...nel.org>
To: Vladimir Oltean <olteanv@...il.com>
Cc: Oleksij Rempel <linux@...pel-privat.de>, netdev@...r.kernel.org,
	andrew@...n.ch, davem@...emloft.net, edumazet@...gle.com,
	kuba@...nel.org, pabeni@...hat.com, horms@...nel.org, nbd@....name,
	sean.wang@...iatek.com, Mark-MC.Lee@...iatek.com,
	lorenzo.bianconi83@...il.com
Subject: Re: [RFC net-next 0/5] Add ETS and TBF Qdisc offload for Airoha
 EN7581 SoC

> On Mon, Dec 16, 2024 at 08:01:33PM +0100, Lorenzo Bianconi wrote:
> > Considering patch [0], we are still offloading the Qdisc on the provided
> > DSA switch port (e.g. LANx) via the port_setup_tc() callback available in
> > dsa_user_setup_qdisc(), but we are introducing even the ndo_setup_tc_conduit()
> > callback in order to use the hw Qdisc capabilities available on the mac chip
> > (e.g. EN7581) for the routed traffic from WAN to LANx. We will still apply
> > the Qdisc defined on LANx for L2 traffic from LANy to LANx. Agree?
> 
> Not quite, no.
> 
> ndo_setup_tc_conduit() does not have the same instruments to offload
> what port_setup_tc() can offload. It is not involved in all data paths
> that port_setup_tc() has to handle. Please ack this. So if port_setup_tc()

Can you please elaborate on this? Both (->ndo_setup_tc_conduit() and
->port_setup_tc()) refer to the same DSA user port (please take a look the
callback signature).

> returns -EOPNOTSUPP, the entire dsa_user_setup_qdisc() should return
> -EOPNOTSUPP, UNLESS you install packet traps on all other offloaded data
> paths in the switch, such that all packets that egress the DSA user port
> are handled by ndo_setup_tc_conduit()'s instruments.

Uhm, do you mean we are changing the user expected result in this way?
It seems to me the only case we are actually changing is if port_setup_tc()
callback is not supported by the DSA switch driver while ndo_setup_tc_conduit()
one is supported by the mac chip. In this case the previous implementation
returns -EOPNOTSUPP while the proposed one does not report any error.
Do we really care about this case? If so, I guess we can rework
dsa_user_setup_qdisc().

> 
> > > How does the introduction of ndo_setup_tc_conduit() help, since the problem
> > > is elsewhere? You are not making "tc qdisc add lanN root ets" work correctly.
> > > It is simply not comparable to the way in which it is offloaded by
> > > drivers/net/dsa/microchip/ksz_common.c, even though the user space
> > > syntax is the same. Unless you're suggesting that for ksz it is not
> > > offloaded correctly?
> > 
> > nope, I am not saying the current Qdisc DSA infrastructure is wrong, it just
> > does not allow to exploit all hw capabilities available on EN7581 when the
> > traffic is routed from the WAN port to a given DSA switch port.
> 
> And I don't believe it should, in this way.

Can you please elaborate on this? IIUC it seems even Oleksij has a use-case
for this.

> 
> > If we do:
> > 
> > $tc qdisc add dev lan0 root handle 1: ets strict 8 priomap ...
> > 
> > in the current upstream implementation we do:
> >   dsa_user_setup_tc():
> >      ...
> >        -> ds->ops->port_setup_tc(ds, dp->index, type, type_data)
> >           (it applies the Qdisc on lan0 configuring the hw switch)
> > 
> > adding the ndo_setup_tc_conduit() callback we have:
> > 
> >   dsa_user_setup_qdisc()
> >     ...
> >       -> conduit->netdev_ops->ndo_setup_tc_conduit(conduit, dp->index, type, type_data)
> >          (it applies the Qdisc on EN7581 mac chip for the routed traffic destinated to lan0)
> > 
> >       -> ds->ops->port_setup_tc(ds, dp->index, type, type_data)
> >          (it applies the Qdisc on lan0 configuring the hw switch)
> > 
> > > Also, flower is a classifier, not an action. It doesn't mangle packets
> > > by the very definition of what a classifier is.
> > 
> > yes, but goal of the series is the Queue scheduler offloading, not
> > classifier/action. Agree?
> 
> Classifiers + flowid are an instrument to direct packets to classes of a
> classful egress Qdisc. They seem perfectly relevant to the discussion,
> given the information I currently have.

yep, sure. We will need a tc classifier to set the flow-id (I used flower during
development). What I mean is the series is taking care just of Qdisc offloading.

> 
> > > Can't you do something like this to guide packets to the correct channels?
> > > 
> > > tc qdisc add dev eth0 clsact
> > > tc qdisc add dev eth0 root handle 1: ets strict 8 priomap ...
> > > tc filter add dev eth0 egress ${u32 or flower filter to match on DSA tagged packets} \
> > > 	flowid 1:1
> > 
> > If we apply the Qdisc directly on the conduit port (eth0) we can just apply the
> > queue scheduler on all the traffic egressing via the DSA switch while I would
> > like to apply it on per DSA port basis (but using the mac chip hw capabilities),
> > got my point?
> 
> We need something as the root Qdisc of the conduit which exposes its
> hardware capabilities. I just assumed that would be a simple (and single)
> ETS, you can correct me if I am wrong.
> 
> On conduit egress, what is the arbitration scheme between the traffic
> destined towards each DSA user port (channel, as the driver calls them)?
> How can this be best represented?

The EN7581 supports up to 32 different 'channels' (each of them support 8
different hw queues). You can define an ETS and/or TBF Qdisc for each channel.
My idea is to associate a channel to each DSA switch port, so the user can
define independent QoS policies for each DSA ports (e.g. shape at 100Mbps lan0,
apply ETS on lan1, ...) configuring the mac chip instead of the hw switch.
The kernel (if the traffic is not offloaded) or the PPE block (if the traffic
is offloaded) updates the channel and queue information in the DMA descriptor
(please take a look to [0] for the first case).

> 
> IIUC, in your patch set, you expose the conduit hardware QoS capabilities
> as if they can be perfectly virtualized among DSA user ports, and as if
> each DSA user port can have its own ETS root Qdisc, completely independent
> of each other, as if the packets do not serialize on the conduit <-> CPU
> port link, and as if that is not a bottleneck. Is that really the case?

correct

> If so (but please explain how), maybe you really need your own root Qdisc
> driver, with one class per DSA user port, and those classes have ETS
> attached to them.

Can you please clarify what do you mean with 'root Qdisc driver'?

Regards,
Lorenzo

[0] https://patchwork.kernel.org/project/netdevbpf/patch/a7d8ec3d70d7a0e2208909189e46a63e769f8f9d.1733930558.git.lorenzo@kernel.org/

Download attachment "signature.asc" of type "application/pgp-signature" (229 bytes)

Powered by blists - more mailing lists