netdev - Re: Failing to attach bond(created with two interfaces from different NICs) to a bridge

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAACQVJqjrBqVqdZN6cnFOSXmVs6vdsrVzOeMd-4aG--NgoLn8g@mail.gmail.com>
Date:   Sun, 6 Sep 2020 16:45:57 +0530
From:   Vasundhara Volam <vasundhara-v.volam@...adcom.com>
To:     Jakub Kicinski <kuba@...nel.org>
Cc:     Jiri Pirko <jiri@...lanox.com>,
        Michael Chan <michael.chan@...adcom.com>, jtoppins@...hat.com,
        Netdev <netdev@...r.kernel.org>,
        Ido Schimmel <idosch@...sch.org>, Andrew Lunn <andrew@...n.ch>
Subject: Re: Failing to attach bond(created with two interfaces from different
 NICs) to a bridge

On Fri, Sep 4, 2020 at 12:44 AM Jakub Kicinski <kuba@...nel.org> wrote:
>
> On Thu, 3 Sep 2020 12:52:25 +0530 Vasundhara Volam wrote:
> > Hello Jiri,
> >
> > After the following set of upstream commits, the user fails to attach
> > a bond to the bridge, if the user creates the bond with two interfaces
> > from different bnxt_en NICs. Previously bnxt_en driver does not
> > advertise the switch_id for legacy mode as part of
> > ndo_get_port_parent_id cb but with the following patches, switch_id is
> > returned even in legacy mode which is causing the failure.
> >
> > ---------------
> > 7e1146e8c10c00f859843817da8ecc5d902ea409 net: devlink: introduce
> > devlink_compat_switch_id_get() helper
> > 6605a226781eb1224c2dcf974a39eea11862b864 bnxt: pass switch ID through
> > devlink_port_attrs_set()
> > 56d9f4e8f70e6f47ad4da7640753cf95ae51a356 bnxt: remove
> > ndo_get_port_parent_id implementation for physical ports
> > ----------------
> >
> > As there is a plan to get rid of ndo_get_port_parent_id in future, I
> > think there is a need to fix devlink_compat_switch_id_get() to return
> > the switch_id only when device is in SWITCHDEV mode and this effects
> > all the NICs.
> >
> > Please let me know your thoughts. Thank you.
>
> I'm not Jiri, but I'd think that hiding switch_id from devices should
> not be the solution here. Especially that no NICs offload bridging
> today.
>
> Could you describe the team/bridge failure in detail, I'm not that
> familiar with this code.

I am copying the Redhat kernel team analysis in crisp to clarify. Hope
this helps.

----
When a new bridge port is being set up in br_add_if(),
nbp_switchdev_mark_set() is called to get the Switch ID of the new
port (in this case, the bond). Prior to devlink_compat_switch_id()
changes, the bond's lower devs (the bnxt ports) do not report a Switch
ID (EOPNOTSUPP) when device is not in SWITCHDEV mode, so this activity
is moot.

Prior to the devlink_compat_switch_id() changes,
switchdev_port_attr_get() is set and points to
bnxt_swdev_port_attr_get(). It simply calls bnxt_port_attr_get():

8358 static const struct switchdev_ops bnxt_switchdev_ops = {
8359         .switchdev_port_attr_get        = bnxt_swdev_port_attr_get
8360 };

8352 static int bnxt_swdev_port_attr_get(struct net_device *dev,
8353                                     struct switchdev_attr *attr)
8354 {
8355         return bnxt_port_attr_get(netdev_priv(dev), attr);
8356 }

- Basically, bnxt_port_attr_get() is either going to copy the switch
ID into the passed-in switchdev_attr struct OR return -EOPNOTSUPP for
legacy mode:

8332 int bnxt_port_attr_get(struct bnxt *bp, struct switchdev_attr *attr)
8333 {
8334         if (bp->eswitch_mode != DEVLINK_ESWITCH_MODE_SWITCHDEV)
8335                 return -EOPNOTSUPP;
8336
8337         /* The PF and it's VF-reps only support the switchdev framework */
8338         if (!BNXT_PF(bp))
8339                 return -EOPNOTSUPP;
8340
8341         switch (attr->id) {
8342         case SWITCHDEV_ATTR_ID_PORT_PARENT_ID:
8343                 attr->u.ppid.id_len = sizeof(bp->switch_id);
8344                 memcpy(attr->u.ppid.id, bp->switch_id,
attr->u.ppid.id_len);
8345                 break;
8346         default:
8347                 return -EOPNOTSUPP;
8348         }
8349         return 0;
8350 }


However, now the bnxt driver provides an ID via its new
ndo_get_devlink_port() handler. Logic in dev_get_port_parent_id()
returns ENODATA if the bond's ports do not all have the same switch
identifier (here, phys_switch_id).

Now nbp_switchdev_mark_set() only uses dev_get_port_parent_id(), which
calls devlink_compat_switch_id_get and when it comes to the bnxt
devices the call to devlink_compat_switch_id_get() actually returns a
useful value. But of course, the Switch ID of two physically separate
cards is not expected to be the same so the overall result is the
ENODATA and the bond is failed to attach to the bridge..
---------

I am typing the above and saw mail from Ido Schimmel in parallel.

Thanks,
Vasundhara