Message-ID: <20230512144401.GA10864@ti139.telemetry-investments.com>
Date: Fri, 12 May 2023 10:44:01 -0400
From: "Andrew J. Schorr" <aschorr@...emetry-investments.com>
To: Jay Vosburgh <jay.vosburgh@...onical.com>
Cc: Hangbin Liu <liuhangbin@...il.com>, netdev@...r.kernel.org
Subject: Re: [Issue] Bonding can't show correct speed if lower interface is
 bond 802.3ad

Hi Jay,

On Thu, May 11, 2023 at 06:38:48PM -0700, Jay Vosburgh wrote:
> 	The individual port behavior is part of the LACP standard (IEEE
> 802.1AX, recent editions call this "Solitary"), and is done
> automatically by the kernel.  One of the reasons for it is to permit
> exactly the situation you mention: to enable PXE or "fallback"
> communication to work even if LACP negotiation fails or is not
> configured or implemented at one end.  This is called out explicitly in
> 802.1AX, 6.1.1.j.
> 
> 	The duplex test is only part of the "individual" logic; it comes
> up because LACP negotiation requires the peers to be point-to-point
> links, i.e., full duplex (IEEE 802.1AX-2014, 6.4.8).  That's the norm
> for most everything now, but historically a port in half duplex could be
> on a multiple access topology, e.g., 802.3 CSMA/CD 10BASE2 on a coax
> cable, which is incompatible with LACP aggregation.  This situation
> doesn't come up a lot these days.
> 
> 	The important part of the "individual" logic is whether or not
> the port successfully completes LACP negotiation with a link partner.
> If not, the port is an individual port, which acts essentially like an
> aggregator with just one port in it.  This is separate from
> "is_individual" in the bonding code, and happens in
> ad_port_selection_logic(), after the comment "check if current
> aggregator suits us".  "is_individual" is one element of this test, the
> remaining tests compare the various keys and whether the partner MAC
> address has been populated.

OK. So it sounds like this should just work automatically with no
configuration required to identify which slaves are running in individual
mode. Thanks for clarifying.

> 	As far as documentation goes, the bonding docs[0] describe some
> of the parameters, but doesn't describe the specifics of bonding's
> ability to manage multiple aggregators; I should write that up, since
> this comes up periodically.  The IEEE standard (to which the bonding
> implementation conforms) describes how the whole system works, but
> doesn't really have a simple overview.
> 
> [0] https://www.kernel.org/doc/Documentation/networking/bonding.rst

I noticed the parameters related to this and did do some google searching to
learn about having multiple aggregators, but as you say, it would be
helpful to have a few more clues about how this works in the Bonding Howto,
as well as a mention of this individual port capability.

> 	I'll have to give this some thought.  The best long term
> solution would be to decouple the link monitoring stuff from the mode,
> and thus allow ARP and MII in a wider variety of modes.  I've prototyped
> that out in the past, along with changing the MII monitor to respond to
> carrier state changes in real time instead of polling, and it's fairly
> complicated.
> 
> 	In any event, this does sound like a valid use case for nesting
> the bonds, so simply disabling that facility seems to be off the table.

OK, great. Then I'll stick with this config for now, even though NetworkManager
has some brain damage in this area, since it tries to bring up both bonds
before the MAC addresses have gotten sorted out, which can leave everything
with a random MAC address. I've managed to kludge a solution to this by setting
ONBOOT=no for the active-backup bond, which convinces NetworkManager to start
it a bit later and somehow fixes the race condition.

Regards,
Andy

Powered by blists - more mailing lists