Message-ID: <20201202100922.GM3055@nanopsycho.orion>
Date:   Wed, 2 Dec 2020 11:09:22 +0100
From:   Jiri Pirko <jiri@...nulli.us>
To:     Edwin Peer <edwin.peer@...adcom.com>
Cc:     Ido Schimmel <idosch@...sch.org>, netdev <netdev@...r.kernel.org>,
        "David S . Miller" <davem@...emloft.net>,
        Jakub Kicinski <kuba@...nel.org>, jiri@...dia.com,
        danieller@...dia.com, andrew@...n.ch, f.fainelli@...il.com,
        mkubecek@...e.cz, mlxsw@...dia.com,
        Ido Schimmel <idosch@...dia.com>
Subject: Re: [PATCH net-next 1/6] ethtool: Extend link modes settings uAPI
 with lanes

Wed, Dec 02, 2020 at 01:32:46AM CET, edwin.peer@...adcom.com wrote:
>On Tue, Dec 1, 2020 at 3:22 AM Jiri Pirko <jiri@...nulli.us> wrote:
>
>> >Consider a physical QSFP connector comprising 4 lanes. Today, if the
>> >speed is forced, we would achieve 100G speeds using all 4 lanes with
>> >NRZ encoding. If we configure the port for PAM4 encoding at the same
>> >speed, then we only require 2 of the available 4 lanes. The remaining 2
>> >lanes are wasted. If we only require 2 of the 4 lanes, why not split the
>> >port and request the same speed of one of the now split out ports? Now,
>> >this same speed is only achievable using PAM4 encoding (it is implied)
>> >and we have a spare, potentially usable, assuming an appropriate break-
>> >out cable, port instead of the 2 unused lanes.
>>
>> I don't see how this dynamic split port could work in real life to be
>> honest. The split is something admin needs to configure and can rely
>> that netdevice exists all the time and not comes and goes under
>> different circumstances. Multiple obvious reasons why.
>
>I'm not suggesting the port split be dynamic at all. I'm suggesting that if
>the admin wants or needs to force PAM4 on a port that would otherwise
>be able to achieve the given speed using more lanes with NRZ, then the
>admin should split the port, so that it has fewer lanes, in order to make
>that intent clear (or otherwise configure the port to have fewer lanes
>attached, if you really don't want to or can't create the additional split
>port).

Okay, I see your point now. The thing is, the port split/unsplit causes
a great distubance. Meaning, the netdevs all of the port
disappear/reappear. Now consider following example:

You have a router you have configured routes on many netdevs
On one of the netdevs (has routes on it), you for any reason
need to force lane number.
In your suggestion, the netdev disappears along with the routes, the
routing is then broken. I don't see how this could be acceptable.

We are talking here about netdev configuration, we have a tool for that,
that is ethtool. What you suggest is to take it to different level,
I don't believe it is correct/doable.


>
>Using this approach, the existing ethtool forced speed interface is
>sufficient to configure all possible lane encodings, because the
>encoding that the driver must select is now implicit (note, we don't
>need to care about media type here). That is, the driver can always
>select the encoding that maximizes utilization of the lanes available
>to the port (as defined by the admin).
>
>> >So concretely, I'm suggesting that if we want to force PAM4 at the lower
>> >speeds, split the port and then we don't need an ethtool interface change
>> >at all to achieve the same goal. Having a spare (potentially usable) port
>> >is better than spare (unusable) lanes.
>>
>> The admin has to decide, define.
>
>I'm not sure I understand. The admin would indeed decide. This paragraph
>merely served to motivate why a rational admin should prefer to have a
>spare port rather than unused lanes he can't use, because they would be
>attached to a port using an encoding that doesn't need them. If he wasn't
>planning on using the additional port, he loses nothing. Otherwise, he gains
>something he would not otherwise have had (it's win-win). From the
>perspective of the original port, two unused lanes is no different than two
>lanes allocated to another logical port.
>
>Regards,
>Edwin Peer

Powered by blists - more mailing lists