netdev - Re: Multi-PHYs and multiple-ports bonding support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20221018112035.5c3e14df@pc-8.home>
Date:   Tue, 18 Oct 2022 11:20:35 +0200
From:   Maxime Chevallier <maxime.chevallier@...tlin.com>
To:     "Russell King (Oracle)" <linux@...linux.org.uk>
Cc:     netdev@...r.kernel.org, linux-arm-kernel@...ts.infradead.org,
        Thomas Petazzoni <thomas.petazzoni@...tlin.com>,
        Antoine Tenart <atenart@...nel.org>,
        "David S. Miller" <davem@...emloft.net>,
        Heiner Kallweit <hkallweit1@...il.com>,
        Florian Fainelli <f.fainelli@...il.com>,
        Vivien Didelot <vivien.didelot@...il.com>,
        Andrew Lunn <andrew@...n.ch>,
        Tobias Waldekranz <tobias@...dekranz.com>,
        Oleksij Rempel <o.rempel@...gutronix.de>,
        Jakub Kicinski <kuba@...nel.org>, Jiri Pirko <jiri@...nulli.us>
Subject: Re: Multi-PHYs and multiple-ports bonding support

On Tue, 18 Oct 2022 09:13:14 +0100
"Russell King (Oracle)" <linux@...linux.org.uk> wrote:

> On Tue, Oct 18, 2022 at 10:02:05AM +0200, Maxime Chevallier wrote:
> > Hello Russell,
> > 
> > On Mon, 17 Oct 2022 10:24:49 +0100
> > "Russell King (Oracle)" <linux@...linux.org.uk> wrote:
> >   
> > > On Mon, Oct 17, 2022 at 10:51:00AM +0200, Maxime Chevallier
> > > wrote:  
> > > > 2) Changes in Phylink
> > > > 
> > > > This might be the tricky part, as we need to track several
> > > > ports, possibly connected to different PHYs, to get their
> > > > state. For now, I haven't prototyped any of this yet.    
> > > 
> > > The problem is _way_ larger than phylink. It's a fundamental
> > > throughout the net layer that there is one-PHY to one-MAC
> > > relationship. Phylink just adopts this because it is the
> > > established norm, and trying to fix it is rather rediculous
> > > without a use case.
> > > 
> > > See code such as the ethtool code, where the MAC and associated
> > > layers are# entirely bypassed with all the PHY-accessing ethtool
> > > commands and the commands are passed directly to phylib for the
> > > PHY registered against the netdev.
> > > 
> > > We do have use cases though - consider a setup such as the mcbin
> > > with the 3310 in SGMII mode on the fibre link and a copper PHY
> > > plugged in with its own PHY - a stacked PHY situation (we don't
> > > support this right now.) Which PHY should the MII ioctls,
> > > ethtool, and possibly the PTP timestamp code be accessing with a
> > > copper SFP module plugged in?
> > > 
> > > This needs to be solved for your multi-PHY case, because you need
> > > to deal with programming e.g. the link advertisement in both
> > > PHYs, not just one - and with the above model, you have no choice
> > > which PHY gets the call, it's always going to be the one
> > > registered with the netdev.
> > > 
> > > The point I'm making is that you're suggesting this is a phylink
> > > issue, but it isn't, it's a generic networking layering bypass
> > > issue. If the net code always forwarded the ethtool etc stuff to
> > > the MAC and let the MAC make appropriate decisions about how
> > > these were handled, then we would have a properly layered
> > > approach where each layer can decide how a particular interface
> > > is implemented - to cope with situations such as the one you
> > > describe.  
> > 
> > I agree with all you say, and indeed this problem is a good
> > opportunity IMO to consider the other use-cases like the one you
> > mention and come up with a nice solution.  
> 
> However, this isn't really "other use-cases" that I'm talking about
> above, but a problem that needs solving for your case.
> 
> > When you mention that ethtool bypasses the MAC layer and talks to
> > phylib, since phylink has the overall view of the link, and
> > abstracts the phy away from the MAC, I would think this is a good
> > place to manage this tree of PHYs/ports, but on the other hand
> > that's adding quite a lot of complexity to phylink.  
> 
> phylink doesn't abstract the PHY from the networking layer. What we
> have are these call paths through the layers:
> 
> net --> mac --> phylink --> phy
>  |                           ^
>  `---------------------------'
>       (bypass call path)
> 
> That bypass call path will be a problem as soon as you start talking
> about having more than one PHY for a MAC.
> 
> Yes, changing phylink fixes some of the issues, but doesn't get away
> from the fundamental issue that both the MAC and phylink are bypassed
> for certain paths.

You're right, that will need to be addressed. With Andrew's proposition
of having an identifier to address each part of the link (end-points or
intermediary PHYs/link-converters/PCSs), that would be a first step
towards fixing this. So, we would have to rewrite that call chain to
make sure we address the proper member, defaulting to the active PHY
for backwards compat as Andrew said.

I'll poke around and try to come-up with what it could look like on the
userspace side, with some ethtool examples maybe, to try to get a
clearer view.

Thanks,

Maxime