netdev - RE: high latency with imx8mm compared to imx6q

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sat, 18 Feb 2023 12:29:50 +0000
From:   Wei Fang <wei.fang@....com>
To:     Richard Weinberger <richard@....at>
CC:     Andrew Lunn <andrew@...n.ch>,
        David Laight <David.Laight@...lab.com>,
        netdev <netdev@...r.kernel.org>,
        Shenwei Wang <shenwei.wang@....com>,
        Clark Wang <xiaoning.wang@....com>,
        dl-linux-imx <linux-imx@....com>
Subject: RE: high latency with imx8mm compared to imx6q



> -----Original Message-----
> From: Wei Fang
> Sent: 2023年2月18日 20:28
> To: 'Richard Weinberger' <richard@....at>
> Cc: Andrew Lunn <andrew@...n.ch>; David Laight
> <David.Laight@...lab.com>; netdev <netdev@...r.kernel.org>; Shenwei
> Wang <shenwei.wang@....com>; Clark Wang <xiaoning.wang@....com>;
> dl-linux-imx <linux-imx@....com>
> Subject: RE: high latency with imx8mm compared to imx6q
> 
> 
> > -----Original Message-----
> > From: Richard Weinberger <richard@....at>
> > Sent: 2023年2月18日 20:03
> > To: Wei Fang <wei.fang@....com>
> > Cc: Andrew Lunn <andrew@...n.ch>; David Laight
> > <David.Laight@...lab.com>; netdev <netdev@...r.kernel.org>; Shenwei
> > Wang <shenwei.wang@....com>; Clark Wang <xiaoning.wang@....com>;
> > dl-linux-imx <linux-imx@....com>
> > Subject: Re: high latency with imx8mm compared to imx6q
> >
> > ----- Ursprüngliche Mail -----
> > > Von: "wei fang" <wei.fang@....com>
> > >> Hm, I thought my settings are fine (IOW no coalescing at all).
> > >> Coalesce parameters for eth0:
> > >> Adaptive RX: n/a  TX: n/a
> > >> rx-usecs: 0
> > >> rx-frames: 0
> > >> tx-usecs: 0
> > >> tx-frames: 0
> > >>
> > > Unfortunately, the fec driver does not support to set
> > > rx-usecs/rx-frames/tx-usecs/tx-frames
> > > to 0 to disable interrupt coalescing. 0 is an invalid parameters. :(
> >
> > So setting all values to 1 is the most "no coalescing" setting i can get?
> >
> If you use the ethtool cmd, the minimum can only be set to 1.
> But you can set the coalescing registers directly on your console,
> ENET_RXICn[ICEN] (addr: base + F0h offset + (4d × n) where n=0,1,2) and
> ENET_TXICn[ICEN] (addr: base + 100h offset + (4d × n), where n=0d to 2d)
> set the ICEN bit (bit 31) to 0:
> 0 disable Interrupt coalescing.
> 1 disable Interrupt coalescing.
sorry, correct my typo. 
1 enable Interrupt coalescing.

> or modify you fec driver, but remember, the interrupt coalescing feature
> can only be disable by setting the ICEN bit to 0, do not set the tx/rx
> usecs/frames
> to 0.
> 
> > >>
> > >> But I noticed something interesting this morning. When I set
> > >> rx-usecs, tx-usecs, rx-frames and tx-frames to 1, *sometimes* the RTT is
> > good.
> > >>
> > >> PING 192.168.0.52 (192.168.0.52) 56(84) bytes of data.
> > >> 64 bytes from 192.168.0.52: icmp_seq=1 ttl=64 time=0.730 ms
> > >> 64 bytes from 192.168.0.52: icmp_seq=2 ttl=64 time=0.356 ms
> > >> 64 bytes from 192.168.0.52: icmp_seq=3 ttl=64 time=0.303 ms
> > >> 64 bytes from 192.168.0.52: icmp_seq=4 ttl=64 time=2.22 ms
> > >> 64 bytes from 192.168.0.52: icmp_seq=5 ttl=64 time=2.54 ms
> > >> 64 bytes from 192.168.0.52: icmp_seq=6 ttl=64 time=0.354 ms
> > >> 64 bytes from 192.168.0.52: icmp_seq=7 ttl=64 time=2.22 ms
> > >> 64 bytes from 192.168.0.52: icmp_seq=8 ttl=64 time=2.54 ms
> > >> 64 bytes from 192.168.0.52: icmp_seq=9 ttl=64 time=2.53 ms
> > >>
> > >> So coalescing plays a role but it looks like the ethernet controller
> > >> does not always obey my settings.
> > >> I didn't look into the configured registers so far, maybe ethtool
> > >> does not set them correctly.
> > >>
> > > It look a bit weird. I did the same setting with my i.MX8ULP and
> > > didn't have this issue. I'm not sure whether you network is stable or
> > > network node devices also enable interrupt coalescing and the relevant
> > > parameters are set to a bit high.
> >
> > I'm pretty sure my network is good, I've tested also different locations.
> > And as I said, with the imx6q on the very same network everything works as
> > expected.
> >
> > So, with rx-usecs/rx-frames/tx-usecs/tx-frames set to 1, you see a RTT smaller
> > than 1ms?
> >
> Yes, but my platform is i.MX8ULP not i.MX8MM, I'll check i.MX8MM next
> Monday.