Message-ID: <20201209160440.evuv26c7cnkqdb22@skbuf>
Date:   Wed, 9 Dec 2020 18:04:40 +0200
From:   Vladimir Oltean <olteanv@...il.com>
To:     Tobias Waldekranz <tobias@...dekranz.com>
Cc:     davem@...emloft.net, kuba@...nel.org, andrew@...n.ch,
        vivien.didelot@...il.com, f.fainelli@...il.com,
        j.vosburgh@...il.com, vfalico@...il.com, andy@...yhouse.net,
        netdev@...r.kernel.org
Subject: Re: [PATCH v3 net-next 2/4] net: dsa: Link aggregation support

On Wed, Dec 09, 2020 at 03:11:41PM +0100, Tobias Waldekranz wrote:
> On Wed, Dec 09, 2020 at 12:53, Vladimir Oltean <olteanv@...il.com> wrote:
> > And I think that .port_lag_change passes more arguments than needed to
> > the driver.
> 
> You mean the `struct netdev_lag_lower_state_info`? Fine by me, it was
> mostly to avoid hiding state from the driver if anyone needed it.

There are two approaches really.
Either you extract the useful information from it, which you already do,
and in that case you don't need to provide the structure from the netdev
notifier to the driver, or you let the driver pick it up. Either way is
fine with me, but having both seems redundant.

> >> > I don't think the DSA switch tree is private to anyone.
> >>
> >> Well I need somewhere to store the association from LAG netdev to LAG
> >> ID. These IDs are shared by all chips in the tree. It could be
> >> replicated on each ds of course, but that does not feel quite right.
> >
> > The LAG ID does not have significance beyond the mv88e6xxx driver, does
> > it? And even there, it's just a number. You could recalculate all IDs
> > dynamically upon every join/leave, and they would be consistent by
> > virtue of the fact that you use a formula which ensures consistency
> > without storing the LAG ID anywhere. Like, say, the LAG ID is to be
> > determined by the first struct dsa_port in the DSA switch tree that has
> > dp->bond == lag_dev. The ID itself can be equal to (dp->ds->index *
> > MAX_NUM_PORTS + dp->index). All switches will agree on what is the first
> > dp in dst, since they iterate in the same way, and any LAG join/leave
> > will notify all of them. It has to be like this anyway.
> 
> This will not work for mv88e6xxx. The ID is not just an internal number
> used by the driver. If that was the case we could just as well use the
> LAG netdev pointer for this purpose. This ID is configured in hardware,
> and it is shared between blocks in the switch, we can not just
> dynamically change them. Neither can we use your formula since this is a
> 4-bit field.
> 
> Another issue is how we are going to handle this in the tagger now,
> since we can no longer call dsa_lag_dev_by_id. I.e. with `struct
> dsa_lag` we could resolve the LAG ID (which is the only source
> information we have in the tag) to the corresponding netdev. This
> information is now only available in mv88e6xxx driver. I am not sure how
> I am supposed to conjure it up. Ideas?

Yeah, ok, I get your point. I was going to say that:
(a) accessing the bonding interface in the fast path seems to be a
    mv88e6xxx problem
(b) the LAG IDs that you are making DSA fabricate are not necessarily
    the ones that other drivers will use, due to various other
    constraints (e.g. ocelot)
so basically my point was that I think you are adding a lot of infra in
core DSA that only mv88e6xxx will use.

My practical proposal would have been to keep a 16-entry bonding pointer
array private in mv88e6xxx, which you could retrieve via:

	struct dsa_port *cpu_dp = netdev->dsa_ptr;
	struct dsa_switch *ds = cpu_dp->ds;
	struct mv88e6xxx_chip *chip = ds->priv;

	skb->dev = chip->lags[source_port];

This is just how I would do it. It would not be the first tagger that
accesses driver private data prior to knowing the net device. It would,
however, mean that we know for sure that the mv88e6xxx device will
always be top-of-tree in a cascaded setup. And this is in conflict with
what I initially said, that the dst is not private to anyone.

I think that there is a low practical risk that the assumption will not
hold true basically forever. But I also see why you might like your
approach more. Maybe Vivien, Andrew, Florian could also chime in and we
can see if struct dsa_lag "bothers" anybody else except me (bothers in
the sense that it's an unnecessary complication to hold in DSA). We
could, of course, also take the middle ground, which would be to keep
the 16-entry array of bonding net_device pointers in DSA, and you could
still call your dsa_lag_dev_by_id() and pretend it's generic, and that
would just look up that table. Even with this middle ground, we are
getting rid of the port lists and of the reference counting, which is
still a welcome simplification in my book.

Powered by blists - more mailing lists