netdev - Re: [net-next PATCH 0/2] net: phylink: Fix issue w/ BMC link flap

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAKgT0UfN_WgpwwkQk0iRWhORUYWiRuTedLq-mCvH6gE30Gofqg@mail.gmail.com>
Date: Wed, 23 Apr 2025 11:38:23 -0700
From: Alexander Duyck <alexander.duyck@...il.com>
To: Andrew Lunn <andrew@...n.ch>
Cc: Jakub Kicinski <kuba@...nel.org>, netdev@...r.kernel.org, linux@...linux.org.uk, 
	hkallweit1@...il.com, davem@...emloft.net, pabeni@...hat.com
Subject: Re: [net-next PATCH 0/2] net: phylink: Fix issue w/ BMC link flap

On Tue, Apr 22, 2025 at 4:06 PM Alexander Duyck
<alexander.duyck@...il.com> wrote:
>
> On Tue, Apr 22, 2025 at 3:26 PM Andrew Lunn <andrew@...n.ch> wrote:
> >
> > On Tue, Apr 22, 2025 at 02:29:48PM -0700, Alexander Duyck wrote:
> > > On Tue, Apr 22, 2025 at 9:50 AM Andrew Lunn <andrew@...n.ch> wrote:
> > > >

...

> > Or is there always 4 MACs, each MAC has its own queues, and you need
> > to place frames into the correct queue, and with a 2x50CR2 you also
> > need to load balance across those two queues?
>
> Are you familiar with the concept of QSFP breakout cables? The general
> idea is that one end of the cable is a QSFP connection and it will
> break out into 4 SFP connections on the other end. That is actually
> pretty close to the concept behind our NIC. We essentially have an
> internalized breakout where the QSFP connection comes in, but we break
> it into either 2 or 4 connections on our end. Our limit is 2 lanes per
> host.
>
> I did a quick search and came up with the following link to a Cisco
> whitepaper that sort of explains the breakout cable concept. I will
> try to see if I can find a spec somewhere that defines how to handle a
> breakout cable:
> https://www.cisco.com/c/en/us/products/collateral/interfaces-modules/transceiver-modules/whitepaper-c11-744077.html
>

So I have done some more digging. I'm wondering if the annex 109B,C
and 135A,C,E,G are meant to essentially explain how to hook up a
breakout style connection without mentioning splitting things up or
sharing.

As far as the QSFP cables I was able to find a bit more. Specifically
in SFF-8636 they have a mention of a "Separable module" in Byte 113
bits 6-4 that was added to support breakout cables. Getting into CMIS
(https://www.oiforum.com/wp-content/uploads/OIF-CMIS-05.2.pdf) there
is quite a bit more mentions of partitioning the module to support
multiple "host interface" instances on the same cable, with section
6.2 really getting into the separation of datapaths for "applications"
and how to go about configuring multiple distinct links sharing the
same module.

The only issue is that CMIS doesn't really apply until we get to
QSFP-DD cables, so the QSFP28 and QSFP+ may end up being a bit of a
wild west when it comes to breaking out the connection. That said
though Byte 113 may be enough as it seems like partitioning it is just
a matter of selecting 1/2/4 lanes versus QSFP-DD where we can swap
between different modulation modes and such.