| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CAJ+vNU2pzk4c5yg1mfw=6m-+z1j3-0ydkvw-uMgYKJC28Dhf+g@mail.gmail.com>
Date: Wed, 16 Nov 2022 14:37:23 -0800
From: Tim Harvey <tharvey@...eworks.com>
To: Sean Anderson <sean.anderson@...o.com>
Cc: netdev <netdev@...r.kernel.org>,
Russell King <linux@...linux.org.uk>,
"David S. Miller" <davem@...emloft.net>
Subject: Re: status of rate adaptation
On Mon, Nov 14, 2022 at 11:33 AM Tim Harvey <tharvey@...eworks.com> wrote:
>
> On Fri, Nov 11, 2022 at 2:38 PM Sean Anderson <sean.anderson@...o.com> wrote:
> >
> > On 11/11/22 17:14, Tim Harvey wrote:
> > > On Fri, Nov 11, 2022 at 1:54 PM Sean Anderson <sean.anderson@...o.com> wrote:
> > >>
> > >> On 11/11/22 16:20, Tim Harvey wrote:
> > >> > On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@...o.com> wrote:
> > >> >>
> > >> >> On 11/11/22 15:57, Sean Anderson wrote:
> > >> >> > Hi Tim,
> > >> >> >
> > >> >> > On 11/11/22 15:44, Tim Harvey wrote:
> > >> >> >> Greetings,
> > >> >> >>
> > >> >> >> I've noticed some recent commits that appear to add rate adaptation support:
> > >> >> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
> > >> >> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
> > >> >> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
> > >> >> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
> > >> >> >> 0c3e10cb4423 net: phy: Add support for rate matching
> > >> >> >>
> > >> >> >> I have a board with an AQR113C PHY over XFI that functions properly at
> > >> >> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
> > >> >> >>
> > >> >> >> Should I expect this to work now at those lower rates
> > >> >> >
> > >> >> > Yes.
> > >> >
> > >> > Sean,
> > >> >
> > >> > Good to hear - thank you for your work on this feature!
> > >> >
> > >> >> >
> > >> >> >> and if so what kind of debug information or testing can I provide?
> > >> >> >
> > >> >> > Please send
> > >> >> >
> > >> >> > - Your test procedure (how do you select 1G?)
> > >> >> > - Device tree node for the interface
> > >> >> > - Output of ethtool (on both ends if possible).
> > >> >> > - Kernel logs with debug enabled for drivers/phylink.c
> > >> >>
> > >> >> Sorry, this should be drivers/net/phy/phylink.c
> > >> >>
> > >> >> >
> > >> >> > That should be enough to get us started.
> > >> >> >
> > >> >> > --Sean
> > >> >>
> > >> >
> > >> > I'm currently testing by bringing up the network interface while
> > >> > connected to a 10gbe switch, verifying link and traffic, then forcing
> > >> > the switch port to 1000mbps.
> > >> >
> > >> > The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
> > >> >
> > >> > #include "cn9130.dtsi" /* include SoC device tree */
> > >> >
> > >> > &cp0_xmdio {
> > >> > pinctrl-names = "default";
> > >> > pinctrl-0 = <&cp0_xsmi_pins>;
> > >> > status = "okay";
> > >> >
> > >> > phy1: ethernet-phy@8 {
> > >> > compatible = "ethernet-phy-ieee802.3-c45";
> > >> > reg = <8>;
> > >> > };
> > >> > };
> > >> >
> > >> > &cp0_ethernet {
> > >> > status = "okay";
> > >> > };
> > >> >
> > >> > /* 10GbE XFI AQR113C */
> > >> > &cp0_eth0 {
> > >> > status = "okay";
> > >> > phy = <&phy1>;
> > >> > phy-mode = "10gbase-r";
> > >> > phys = <&cp0_comphy4 0>;
> > >> > };
> > >> >
> > >> > Here are some logs with debug enabled in drivers/net/phy/phylink.c and
> > >> > some additional debug in mvpp2.c and aquantia_main.c:
> > >> > # ifconfig eth0 192.168.1.22
> > >> > [ 8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
> > >> > speed=-1 26:10gbase-r
> > >> > [ 8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
> > >> > [ 8.898165] aqr107_resume
> > >> > [ 8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
> > >> > duplex=255 speed=-1 26:10gbase-r 0:
> > >> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
> > >> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
> > >> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
> > >> > supported 00000000,00018000,000e706f advertising
> > >> > 00000000,00018000,000e706f
> > >> > [ 8.934349] mvpp2 f2000000.ethernet eth0: mac link down
> > >> > [ 8.948812] mvpp2 f2000000.ethernet eth0: configuring for
> > >> > phy/10gbase-r link mode
> > >> > [ 8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
> > >> > [ 8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
> > >> > mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
> > >> > pause=00 link=0 an=0
> > >> > [ 8.976252] mvpp2 f2000000.ethernet eth0: mac link down
> > >> > [ 8.976267] aqr107_resume
> > >> > [ 8.988970] mvpp2 f2000000.ethernet eth0: phy link down
> > >> > 10gbase-r/10Gbps/Full/none/off
> > >> > [ 8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
> > >> > duplex=1 speed=10000 26:10gbase-r
> > >> > [ 14.112540] mvpp2 f2000000.ethernet eth0: mac link up
> > >> > [ 14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
> > >> > - flow control off
> > >> > [ 14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> > >> > [ 14.118198] mvpp2 f2000000.ethernet eth0: phy link up
> > >> > 10gbase-r/10Gbps/Full/none/off
> > >> > [ 14.139808] aqr107_link_change_notify state=4:running an=1 link=1
> > >> > duplex=1 speed=10000 26:10gbase-r
> > >> >
> > >> > # ethtool eth0
> > >> > Settings for eth0:
> > >> > Supported ports: [ ]
> > >> > Supported link modes: 10baseT/Half 10baseT/Full
> > >> > 100baseT/Half 100baseT/Full
> > >>
> > >> 10/100 half duplex aren't achievable with rate matching (and we avoid
> > >> turning them on), so they must be coming from somewhere else. I wonder
> > >> if this is because PHY_INTERFACE_MODE_SGMII is set in
> > >> supported_interfaces.
> > >>
> > >> I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
> > >> should support it. I'm not sure if the aquantia driver is set up for it.
> > >
> > > This appears to trigger an issue from mvpp2:
> > > mvpp2 f2000000.ethernet eth0: validation of usxgmii with support
> > > 00000000,00018000,000e706f and advertisement
> > > 00000000,00018000,000e706f failed: -EINVAL
> >
> > Ah, I forgot this was a separate phy mode. Disregard this.
> >
> > >>
> > >> > 1000baseT/Full
> > >> > 10000baseT/Full
> > >> > 1000baseKX/Full
> > >> > 10000baseKX4/Full
> > >> > 10000baseKR/Full
> > >> > 2500baseT/Full
> > >> > 5000baseT/Full
> > >> > Supported pause frame use: Symmetric Receive-only
> > >> > Supports auto-negotiation: Yes
> > >> > Supported FEC modes: Not reported
> > >> > Advertised link modes: 10baseT/Half 10baseT/Full
> > >> > 100baseT/Half 100baseT/Full
> > >> > 1000baseT/Full
> > >> > 10000baseT/Full
> > >> > 1000baseKX/Full
> > >> > 10000baseKX4/Full
> > >> > 10000baseKR/Full
> > >> > 2500baseT/Full
> > >> > 5000baseT/Full
> > >> > Advertised pause frame use: Symmetric Receive-only
> > >> > Advertised auto-negotiation: Yes
> > >> > Advertised FEC modes: Not reported
> > >> > Link partner advertised link modes: 100baseT/Half 100baseT/Full
> > >> > 1000baseT/Half 1000baseT/Full
> > >> > 10000baseT/Full
> > >> > 2500baseT/Full
> > >> > 5000baseT/Full
> > >> > Link partner advertised pause frame use: No
> > >> > Link partner advertised auto-negotiation: Yes
> > >> > Link partner advertised FEC modes: Not reported
> > >> > Speed: 10000Mb/s
> > >> > Duplex: Full
> > >> > Port: Twisted Pair
> > >> > PHYAD: 8
> > >> > Transceiver: external
> > >> > Auto-negotiation: on
> > >> > MDI-X: Unknown
> > >> > Link detected: yes
> > >> > # ping 192.168.1.146 -c5
> > >> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
> > >> > 64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
> > >> > 64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
> > >> > 64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
> > >> > 64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
> > >> > 64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
> > >> >
> > >> > --- 192.168.1.146 ping statistics ---
> > >> > 5 packets transmitted, 5 packets received, 0% packet loss
> > >> > round-trip min/avg/max = 0.267/0.416/0.991 ms
> > >> > # # force switch port to 1G
> > >> > [ 193.343494] mvpp2 f2000000.ethernet eth0: phy link down
> > >> > 10gbase-r/Unknown/Unknown/none/off
> > >> > [ 193.343539] mvpp2 f2000000.ethernet eth0: mac link down
> > >> > [ 193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
> > >> > [ 193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
> > >> > duplex=255 speed=-1 26:10gbase-r
> > >> > [ 197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
> > >>
> > >> Well, it looks like we have selected PHY_INTERFACE_MODE_NA. Can you send
> > >> the value of MDIO_PHYXS_VEND_IF_STATUS (dev 4, reg 0xe812)? Please also
> > >> send the global config registers (dev 0x1e, reg 0x0310 through 0x031f)
> > >> and the vendor provisioning registers (dev 4, reg 0xc440 through
> > >> 0xc449).
> > >
> > > yes, this is what I've been looking at as well. When forced to 1000m
> > > the register shows a phy type of 11 which according to the aqr113
> > > datasheet is XFI 5G:
> > > aqr107_read_status STATUS=0x00001258 (type=11) state=4:running an=1
> > > link=1 duplex=1 speed=1000 interface=0
> >
> > That's pretty strange. Seems like it's rate adapting from 5g instead of
> > 10g. Is SERDES Mode in the Global System Configuration For 1G register
> > set to XFI?
>
> 1E.31C=0x0106:
> Rate Adaptation Method: 2=Pause Rate Adaptation
> SERDES Mode: 6=XFI/2 (XFI 5G)
>
The SERDES mode here is not valid and it seems to always be set to
XFI/2 unless I init/use the AQR113 in U-Boot. If I manually set SERDES
Mode to 0 (XFI) in the driver I find that all rates
10g,5g,2.5g,1g,100m work fine both in Linux 6.0 and in Linux 6.1-rc5.
I'm still trying to understand why I would need to set SERDES Mode
manually (vs the XFI mode specific firmware setting this) but I am
unclear what the rate adaptation in 6.1 provides in this case. Is it
that perhaps the AQR113 is handling rate adaptation internally and
that's why the non 10gbe rates are working on 6.0?
Best Regards,
Tim
Powered by blists - more mailing lists