lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Thu, 17 Nov 2022 15:42:02 -0800
From:   Tim Harvey <tharvey@...eworks.com>
To:     Sean Anderson <sean.anderson@...o.com>
Cc:     netdev <netdev@...r.kernel.org>,
        Russell King <linux@...linux.org.uk>,
        "David S. Miller" <davem@...emloft.net>
Subject: Re: status of rate adaptation

On Thu, Nov 17, 2022 at 7:38 AM Sean Anderson <sean.anderson@...o.com> wrote:
>
> On 11/16/22 17:37, Tim Harvey wrote:
> > On Mon, Nov 14, 2022 at 11:33 AM Tim Harvey <tharvey@...eworks.com> wrote:
> >>
> >> On Fri, Nov 11, 2022 at 2:38 PM Sean Anderson <sean.anderson@...o.com> wrote:
> >> >
> >> > On 11/11/22 17:14, Tim Harvey wrote:
> >> > > On Fri, Nov 11, 2022 at 1:54 PM Sean Anderson <sean.anderson@...o.com> wrote:
> >> > >>
> >> > >> On 11/11/22 16:20, Tim Harvey wrote:
> >> > >> > On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@...o.com> wrote:
> >> > >> >>
> >> > >> >> On 11/11/22 15:57, Sean Anderson wrote:
> >> > >> >> > Hi Tim,
> >> > >> >> >
> >> > >> >> > On 11/11/22 15:44, Tim Harvey wrote:
> >> > >> >> >> Greetings,
> >> > >> >> >>
> >> > >> >> >> I've noticed some recent commits that appear to add rate adaptation support:
> >> > >> >> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
> >> > >> >> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
> >> > >> >> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
> >> > >> >> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
> >> > >> >> >> 0c3e10cb4423 net: phy: Add support for rate matching
> >> > >> >> >>
> >> > >> >> >> I have a board with an AQR113C PHY over XFI that functions properly at
> >> > >> >> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
> >> > >> >> >>
> >> > >> >> >> Should I expect this to work now at those lower rates
> >> > >> >> >
> >> > >> >> > Yes.
> >> > >> >
> >> > >> > Sean,
> >> > >> >
> >> > >> > Good to hear - thank you for your work on this feature!
> >> > >> >
> >> > >> >> >
> >> > >> >> >> and if so what kind of debug information or testing can I provide?
> >> > >> >> >
> >> > >> >> > Please send
> >> > >> >> >
> >> > >> >> > - Your test procedure (how do you select 1G?)
> >> > >> >> > - Device tree node for the interface
> >> > >> >> > - Output of ethtool (on both ends if possible).
> >> > >> >> > - Kernel logs with debug enabled for drivers/phylink.c
> >> > >> >>
> >> > >> >> Sorry, this should be drivers/net/phy/phylink.c
> >> > >> >>
> >> > >> >> >
> >> > >> >> > That should be enough to get us started.
> >> > >> >> >
> >> > >> >> > --Sean
> >> > >> >>
> >> > >> >
> >> > >> > I'm currently testing by bringing up the network interface while
> >> > >> > connected to a 10gbe switch, verifying link and traffic, then forcing
> >> > >> > the switch port to 1000mbps.
> >> > >> >
> >> > >> > The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
> >> > >> >
> >> > >> > #include "cn9130.dtsi" /* include SoC device tree */
> >> > >> >
> >> > >> > &cp0_xmdio {
> >> > >> >         pinctrl-names = "default";
> >> > >> >         pinctrl-0 = <&cp0_xsmi_pins>;
> >> > >> >         status = "okay";
> >> > >> >
> >> > >> >         phy1: ethernet-phy@8 {
> >> > >> >                 compatible = "ethernet-phy-ieee802.3-c45";
> >> > >> >                 reg = <8>;
> >> > >> >         };
> >> > >> > };
> >> > >> >
> >> > >> > &cp0_ethernet {
> >> > >> >         status = "okay";
> >> > >> > };
> >> > >> >
> >> > >> > /* 10GbE XFI AQR113C */
> >> > >> > &cp0_eth0 {
> >> > >> >         status = "okay";
> >> > >> >         phy = <&phy1>;
> >> > >> >         phy-mode = "10gbase-r";
> >> > >> >         phys = <&cp0_comphy4 0>;
> >> > >> > };
> >> > >> >
> >> > >> > Here are some logs with debug enabled in drivers/net/phy/phylink.c and
> >> > >> > some additional debug in mvpp2.c and aquantia_main.c:
> >> > >> > # ifconfig eth0 192.168.1.22
> >> > >> > [    8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
> >> > >> > speed=-1 26:10gbase-r
> >> > >> > [    8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
> >> > >> > [    8.898165] aqr107_resume
> >> > >> > [    8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
> >> > >> > duplex=255 speed=-1 26:10gbase-r 0:
> >> > >> > [    8.911932] mvpp2 f2000000.ethernet eth0: PHY
> >> > >> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
> >> > >> > [    8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
> >> > >> > supported 00000000,00018000,000e706f advertising
> >> > >> > 00000000,00018000,000e706f
> >> > >> > [    8.934349] mvpp2 f2000000.ethernet eth0: mac link down
> >> > >> > [    8.948812] mvpp2 f2000000.ethernet eth0: configuring for
> >> > >> > phy/10gbase-r link mode
> >> > >> > [    8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
> >> > >> > [    8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
> >> > >> > mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
> >> > >> > pause=00 link=0 an=0
> >> > >> > [    8.976252] mvpp2 f2000000.ethernet eth0: mac link down
> >> > >> > [    8.976267] aqr107_resume
> >> > >> > [    8.988970] mvpp2 f2000000.ethernet eth0: phy link down
> >> > >> > 10gbase-r/10Gbps/Full/none/off
> >> > >> > [    8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
> >> > >> > duplex=1 speed=10000 26:10gbase-r
> >> > >> > [   14.112540] mvpp2 f2000000.ethernet eth0: mac link up
> >> > >> > [   14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
> >> > >> > - flow control off
> >> > >> > [   14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> >> > >> > [   14.118198] mvpp2 f2000000.ethernet eth0: phy link up
> >> > >> > 10gbase-r/10Gbps/Full/none/off
> >> > >> > [   14.139808] aqr107_link_change_notify state=4:running an=1 link=1
> >> > >> > duplex=1 speed=10000 26:10gbase-r
> >> > >> >
> >> > >> > # ethtool eth0
> >> > >> > Settings for eth0:
> >> > >> >         Supported ports: [ ]
> >> > >> >         Supported link modes:   10baseT/Half 10baseT/Full
> >> > >> >                                 100baseT/Half 100baseT/Full
> >> > >>
> >> > >> 10/100 half duplex aren't achievable with rate matching (and we avoid
> >> > >> turning them on), so they must be coming from somewhere else. I wonder
> >> > >> if this is because PHY_INTERFACE_MODE_SGMII is set in
> >> > >> supported_interfaces.
> >> > >>
> >> > >> I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
> >> > >> should support it. I'm not sure if the aquantia driver is set up for it.
> >> > >
> >> > > This appears to trigger an issue from mvpp2:
> >> > > mvpp2 f2000000.ethernet eth0: validation of usxgmii with support
> >> > > 00000000,00018000,000e706f and advertisement
> >> > > 00000000,00018000,000e706f failed: -EINVAL
> >> >
> >> > Ah, I forgot this was a separate phy mode. Disregard this.
> >> >
> >> > >>
> >> > >> >                                 1000baseT/Full
> >> > >> >                                 10000baseT/Full
> >> > >> >                                 1000baseKX/Full
> >> > >> >                                 10000baseKX4/Full
> >> > >> >                                 10000baseKR/Full
> >> > >> >                                 2500baseT/Full
> >> > >> >                                 5000baseT/Full
> >> > >> >         Supported pause frame use: Symmetric Receive-only
> >> > >> >         Supports auto-negotiation: Yes
> >> > >> >         Supported FEC modes: Not reported
> >> > >> >         Advertised link modes:  10baseT/Half 10baseT/Full
> >> > >> >                                 100baseT/Half 100baseT/Full
> >> > >> >                                 1000baseT/Full
> >> > >> >                                 10000baseT/Full
> >> > >> >                                 1000baseKX/Full
> >> > >> >                                 10000baseKX4/Full
> >> > >> >                                 10000baseKR/Full
> >> > >> >                                 2500baseT/Full
> >> > >> >                                 5000baseT/Full
> >> > >> >         Advertised pause frame use: Symmetric Receive-only
> >> > >> >         Advertised auto-negotiation: Yes
> >> > >> >         Advertised FEC modes: Not reported
> >> > >> >         Link partner advertised link modes:  100baseT/Half 100baseT/Full
> >> > >> >                                              1000baseT/Half 1000baseT/Full
> >> > >> >                                              10000baseT/Full
> >> > >> >                                              2500baseT/Full
> >> > >> >                                              5000baseT/Full
> >> > >> >         Link partner advertised pause frame use: No
> >> > >> >         Link partner advertised auto-negotiation: Yes
> >> > >> >         Link partner advertised FEC modes: Not reported
> >> > >> >         Speed: 10000Mb/s
> >> > >> >         Duplex: Full
> >> > >> >         Port: Twisted Pair
> >> > >> >         PHYAD: 8
> >> > >> >         Transceiver: external
> >> > >> >         Auto-negotiation: on
> >> > >> >         MDI-X: Unknown
> >> > >> >         Link detected: yes
> >> > >> > # ping 192.168.1.146 -c5
> >> > >> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
> >> > >> > 64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
> >> > >> > 64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
> >> > >> > 64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
> >> > >> > 64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
> >> > >> > 64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
> >> > >> >
> >> > >> > --- 192.168.1.146 ping statistics ---
> >> > >> > 5 packets transmitted, 5 packets received, 0% packet loss
> >> > >> > round-trip min/avg/max = 0.267/0.416/0.991 ms
> >> > >> > # # force switch port to 1G
> >> > >> > [  193.343494] mvpp2 f2000000.ethernet eth0: phy link down
> >> > >> > 10gbase-r/Unknown/Unknown/none/off
> >> > >> > [  193.343539] mvpp2 f2000000.ethernet eth0: mac link down
> >> > >> > [  193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
> >> > >> > [  193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
> >> > >> > duplex=255 speed=-1 26:10gbase-r
> >> > >> > [  197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
> >> > >>
> >> > >> Well, it looks like we have selected PHY_INTERFACE_MODE_NA. Can you send
> >> > >> the value of MDIO_PHYXS_VEND_IF_STATUS (dev 4, reg 0xe812)? Please also
> >> > >> send the global config registers (dev 0x1e, reg 0x0310 through 0x031f)
> >> > >> and the vendor provisioning registers (dev 4, reg 0xc440 through
> >> > >> 0xc449).
> >> > >
> >> > > yes, this is what I've been looking at as well. When forced to 1000m
> >> > > the register shows a phy type of 11 which according to the aqr113
> >> > > datasheet is XFI 5G:
> >> > > aqr107_read_status STATUS=0x00001258 (type=11) state=4:running an=1
> >> > > link=1 duplex=1 speed=1000 interface=0
> >> >
> >> > That's pretty strange. Seems like it's rate adapting from 5g instead of
> >> > 10g. Is SERDES Mode in the Global System Configuration For 1G register
> >> > set to XFI?
> >>
> >> 1E.31C=0x0106:
> >>   Rate Adaptation Method: 2=Pause Rate Adaptation
> >>   SERDES Mode: 6=XFI/2 (XFI 5G)
> >>
> >
> > The SERDES mode here is not valid and it seems to always be set to
> > XFI/2 unless I init/use the AQR113 in U-Boot. If I manually set SERDES
> > Mode to 0 (XFI) in the driver I find that all rates
> > 10g,5g,2.5g,1g,100m work fine both in Linux 6.0 and in Linux 6.1-rc5.
> > I'm still trying to understand why I would need to set SERDES Mode
> > manually (vs the XFI mode specific firmware setting this) but I am
> > unclear what the rate adaptation in 6.1 provides in this case. Is it
> > that perhaps the AQR113 is handling rate adaptation internally and
> > that's why the non 10gbe rates are working on 6.0?
>
> The changes in 6.1 are
>
> - We now always enable pause frame reception when doing rate adaptation.
>   This is necessary for rate adaptation to work, but wasn't done
>   automatically before.
> - We advertise lower speed modes which are enabled with rate adaptation,
>   even if we would not otherwise be able to support them.
>
> I'm not sure why you'd have XFI/2 selected in 6.1 if it isn't selected
> in 6.0.

Sean,

Thanks for the explanation. The issue I had which resulted in the
wrong SERDES mode was simply that I was using the wrong aquantia
firmware. They customize the firmware to default registers like SERDES
mode specifically for customers and I was unknowingly using the
firmware for XFI/2 instead of XFI.

I suppose it would be worth putting something in the aquantia driver
that verifies SERDES mode matches the phy-mode from dt to throw an
error.

Best Regards,

Tim

Powered by blists - more mailing lists