[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <trinity-283297c1-e5fc-4d90-9f4b-505ebf8c82cb-1680184695162@3c-app-gmx-bap58>
Date: Thu, 30 Mar 2023 15:58:15 +0200
From: Frank Wunderlich <frank-w@...lic-files.de>
To: Felix Fietkau <nbd@....name>
Cc: netdev@...r.kernel.org, Daniel Golle <daniel@...rotopia.org>
Subject: Aw: Re: Re: Re: Re: [PATCH net] net: ethernet: mtk_eth_soc: fix tx
throughput regression with direct 1G links
something ist still strange...i get a rcu stall again with this patch...reverted it and my r2 boots again.
[ 29.772755] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
[ 29.778689] rcu: 2-...0: (1 GPs behind) idle=547c/1/0x40000000 softirq=251/258 fqs=427
[ 29.786697] rcu: (detected by 1, t=2104 jiffies, g=-875, q=29 ncpus=4)
[ 29.793308] Sending NMI from CPU 1 to CPUs 2:
[ 34.492968] vusb: disabling
[ 34.495828] vmc: disabling
[ 34.498587] vmch: disabling
[ 34.501433] vgp1: disabling
[ 34.504426] vcamaf: disabling
[ 39.797579] rcu: rcu_sched kthread timer wakeup didn't happen for 994 jiffies! g-875 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
[ 39.808619] rcu: Possible timer handling issue on cpu=1 timer-softirq=493
[ 39.815487] rcu: rcu_sched kthread starved for 995 jiffies! g-875 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[ 39.825571] rcu: Unless rcu_sched kthread gets sufficient CPU time, OOM is now expected behavior.
[ 39.834520] rcu: RCU grace-period kthread stack dump:
[ 39.839564] task:rcu_sched state:I stack:0 pid:14 ppid:2 flags:0x00000000
[ 39.847928] __schedule from schedule+0x54/0xe8
[ 39.852472] schedule from schedule_timeout+0x94/0x158
[ 39.857619] schedule_timeout from rcu_gp_fqs_loop+0x12c/0x50c
[ 39.863467] rcu_gp_fqs_loop from rcu_gp_kthread+0x194/0x21c
[ 39.869135] rcu_gp_kthread from kthread+0xc8/0xcc
[ 39.873931] kthread from ret_from_fork+0x14/0x2c
[ 39.878639] Exception stack(0xf0859fb0 to 0xf0859ff8)
[ 39.883690] 9fa0: 00000000 00000000 00000000 00000000
[ 39.891864] 9fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 39.900037] 9fe0: 00000000 00000000 00000000 00000000 00000013 00000000
[ 39.906645] rcu: Stack dump where RCU GP kthread last ran:
[ 39.912125] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.3.0-rc1-bpi-r2-rc-net #2
[ 39.919518] Hardware name: Mediatek Cortex-A7 (Device Tree)
[ 39.925082] PC is at default_idle_call+0x1c/0xb0
[ 39.929698] LR is at ct_kernel_enter.constprop.0+0x48/0x11c
[ 39.935267] pc : [<c0d105ec>] lr : [<c0d0ffa4>] psr: 600e0013
[ 39.941527] sp : f0861fb0 ip : c15721e0 fp : 00000000
[ 39.946746] r10: 00000000 r9 : 410fc073 r8 : 8000406a
[ 39.951964] r7 : c1404f74 r6 : c19e0900 r5 : c15727e0 r4 : c19e0900
[ 39.958486] r3 : 00000000 r2 : 2da0a000 r1 : 00000001 r0 : 00008cfc
[ 39.965007] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none
[ 39.972138] Control: 10c5387d Table: 84f4806a DAC: 00000051
[ 39.977878] default_idle_call from cpuidle_idle_call+0x24/0x68
[ 39.983805] cpuidle_idle_call from do_idle+0x9c/0xd0
[ 39.988863] do_idle from cpu_startup_entry+0x20/0x24
[ 39.993921] cpu_startup_entry from secondary_start_kernel+0x118/0x138
[ 40.000457] secondary_start_kernel from 0x801017a0
maybe i need additional patch or did anything else wrong?
still working on 6.3-rc1
https://github.com/frank-w/BPI-Router-Linux/commits/6.3-rc-net
regards Frank
> Gesendet: Mittwoch, 29. März 2023 um 14:04 Uhr
> Von: "Felix Fietkau" <nbd@....name>
> An: "Frank Wunderlich" <frank-w@...lic-files.de>
> Cc: netdev@...r.kernel.org, "Daniel Golle" <daniel@...rotopia.org>
> Betreff: Re: Aw: Re: Re: Re: [PATCH net] net: ethernet: mtk_eth_soc: fix tx throughput regression with direct 1G links
>
> On 27.03.23 19:28, Frank Wunderlich wrote:
> >
> >> Gesendet: Sonntag, 26. März 2023 um 22:09 Uhr
> >> Von: "Felix Fietkau" <nbd@....name>
> >> On 26.03.23 19:49, Frank Wunderlich wrote:
> >> >> Gesendet: Sonntag, 26. März 2023 um 19:27 Uhr
> >> >> Von: "Felix Fietkau" <nbd@....name>
> >
> >> >> On 26.03.23 19:10, Frank Wunderlich wrote:
> >> >> >> Gesendet: Sonntag, 26. März 2023 um 17:56 Uhr
> >> >> >> Von: "Felix Fietkau" <nbd@....name>
> >
> >> >> >> On 25.03.23 10:28, Frank Wunderlich wrote:
> >> >> >> >> Gesendet: Freitag, 24. März 2023 um 15:04 Uhr
> >> >> >> >> Von: "Felix Fietkau" <nbd@....name>
> >
> >> >> >> > thx for the fix, as daniel already checked it on mt7986/bpi-r3 i tested bpi-r2/mt7623
> >> >> >> >
> >> >> >> > but unfortunately it does not fix issue on bpi-r2 where the gmac0/mt7530 part is affected.
> >> >> >> >
> >> >> >> > maybe it needs a special handling like you do for mt7621? maybe it is because the trgmii mode used on this path?
> >> >> >> Could you please test if making it use the MT7621 codepath brings back
> >> >> >> performance? I don't have any MT7623 hardware for testing right now.
> >
> >> > used the CONFIG_MACH_MT7623 (which is set in my config) boots up fine, but did not fix the 622Mbit-tx-issue
> >> >
> >> > and i'm not sure i have tested it before...all ports of mt7531 are affected, not only wan (i remembered you asked for this)
> >> Does the MAC that's connected to the switch use flow control? Can you
> >> test if changing that makes a difference?
> >
> > it does use flow control/pause on mac and switch-port, disabled it, but it does not change anything (still ~620Mbit on tx)
>
> I finally found a MT7623 device in my stash, so I could run some
> experiments with it. For some reason I could only reproduce your tx
> throughput values after switching off TRGMII. With TRGMII enabled, I got
> around 864 Mbit/s, which is of course still lower than what I get with
> shaping disabled.
> I also experimented with bumping the shaping rate to higher values, but
> got no changes in throughput at all.
> Based on that, I'm beginning to think that the shaper simply can't
> handle rates close to the MAC rate and runs into a fundamental limit
> somehow.
> Now that I think about it, I do remember that shaping to 2.5 Gbps on
> MT7622 also reduced link throughput.
>
> I think we should simply use this patch to deal with it. Do you agree?
>
> - Felix
>
> ---
> --- a/drivers/net/ethernet/mediatek/mtk_eth_soc.c
> +++ b/drivers/net/ethernet/mediatek/mtk_eth_soc.c
> @@ -753,6 +753,7 @@ static void mtk_mac_link_up(struct phylink_config *config,
> MAC_MCR_FORCE_RX_FC);
>
> /* Configure speed */
> + mac->speed = speed;
> switch (speed) {
> case SPEED_2500:
> case SPEED_1000:
> @@ -3235,6 +3236,9 @@ static int mtk_device_event(struct notifier_block *n, unsigned long event, void
> if (dp->index >= MTK_QDMA_NUM_QUEUES)
> return NOTIFY_DONE;
>
> + if (mac->speed > 0 && mac->speed <= s.base.speed)
> + s.base.speed = 0;
> +
> mtk_set_queue_speed(eth, dp->index + 3, s.base.speed);
>
> return NOTIFY_DONE;
>
>
>
>
Powered by blists - more mailing lists