[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <TY2PR01MB3692E49BB082DEF855A3ACBAD8690@TY2PR01MB3692.jpnprd01.prod.outlook.com>
Date: Mon, 6 Jul 2020 09:25:07 +0000
From: Yoshihiro Shimoda <yoshihiro.shimoda.uh@...esas.com>
To: Sergei Shtylyov <sergei.shtylyov@...il.com>,
Sergei Shtylyov <sergei.shtylyov@...entembedded.com>
CC: "REE dirk.behme@...bosch.com" <dirk.behme@...bosch.com>,
"Shashikant.Suguni@...bosch.com" <Shashikant.Suguni@...bosch.com>,
"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
"linux-renesas-soc@...r.kernel.org"
<linux-renesas-soc@...r.kernel.org>,
"davem@...emloft.net" <davem@...emloft.net>,
"kuba@...nel.org" <kuba@...nel.org>
Subject: RE: [PATCH/RFC] net: ethernet: ravb: Try to wake subqueue instead of
stop on timeout
Hello!
> From: Sergei Shtylyov, Sent: Monday, July 6, 2020 5:29 AM
>
> Hello!
>
> On 30.06.2020 8:22, Yoshihiro Shimoda wrote:
>
> >>>>>>> From: Yoshihiro Shimoda, Sent: Tuesday, May 26, 2020 6:47 PM
> >>>>>>>
> >>>>>>> According to the report of [1], this driver is possible to cause
> >>>>>>> the following error in ravb_tx_timeout_work().
> >>>>>>>
> >>>>>>> ravb e6800000.ethernet ethernet: failed to switch device to config mode
> >>>>>>>
> >>>>>>> This error means that the hardware could not change the state
> >>>>>>> from "Operation" to "Configuration" while some tx queue is operating.
> >>>>>>> After that, ravb_config() in ravb_dmac_init() will fail, and then
> >>>>>>> any descriptors will be not allocaled anymore so that NULL porinter
>
> Pointer. :-)
Oops! I should fix it :)
> >>>>>>> dereference happens after that on ravb_start_xmit().
> >>>>>>>
> >>>>>>> Such a case is possible to be caused because this driver supports
> >>>>>>> two queues (NC and BE) and the ravb_stop_dma() is possible to return
> >>>>>>> without any stopping process if TCCR or CSR register indicates
> >>>>>>> the hardware is operating for TX.
>
> Maybe we should just fix those blind assumptions?
Maybe I should have described some facts instead of assumptions like below?
If so, I should modify the code too.
After ravb_stop_dma() was called, the driver assumed any transfers were
stopped. However, the current ravb_tx_timeout_work() doesn't check whether
the ravb_stop_dma() is succeeded without any error or not. So, we should
fix it.
> >>>>>>> To fix the issue, just try to wake the subqueue on
> >>>>>>> ravb_tx_timeout_work() if the descriptors are not full instead
> >>>>>>> of stop all transfers (all queues of TX and RX).
> >>>>>>>
> >>>>>>> [1]
> >>>>>>> https://lore.kernel.org/linux-renesas-soc/20200518045452.2390-1-dirk.behme@de.bosch.com/
> >>>>>>>
> >>>>>>> Reported-by: Dirk Behme <dirk.behme@...bosch.com>
> >>>>>>> Signed-off-by: Yoshihiro Shimoda <yoshihiro.shimoda.uh@...esas.com>
> >>>>>>> ---
> >>>>>>> I'm guessing that this issue is possible to happen if:
> >>>>>>> - ravb_start_xmit() calls netif_stop_subqueue(), and
> >>>>>>> - ravb_poll() will not be called with some reason, and
> >>>>>>> - netif_wake_subqueue() will be not called, and then
> >>>>>>> - dev_watchdog() in net/sched/sch_generic.c calls ndo_tx_timeout().
> >>>>>>>
> >>>>>>> However, unfortunately, I didn't reproduce the issue yet.
> >>>>>>> To be honest, I'm also guessing other queues (SR) of this hardware
> >>>>>>> which out-of tree driver manages are possible to reproduce this issue,
> >>>>>>> but I didn't try such environment for now...
> >>>>>>>
> >>>>>>> So, I marked RFC on this patch now.
> >>>>>>
> >>>>>> I'm afraid, but do you have any comments about this patch?
> >>>>>
> >>>>> I agree that we should now reset only the stuck queue, not both but I
> >>>>> doubt your solution is good enough. Let me have another look...
> >>>>
> >>>> Thank you for your comment! I hope this solution is good enough...
> >>>
> >>> I'm sorry again and again. But, do you have any time to look this patch?
> >>
> >> Yes, in the sense of reviewing -- I don't consider it complete. And no, in
> >> the sense of looking into the issue myself... Can we do a per-queue tear-down
> >> and re-init (not necessarily all in 1 patch)?
>
> In fact, it would ensue many changes...
I think so.
> > Thank you for your comment! I'm not sure this "re-init" mean. But, we can do
>
> Well, I meant the ring re-allocation and re-formatting... but (looking at
> sh_eth) the former is not really necessary, it's enough to just stop the TX
> ring and then re-format it and re-start...
I got it. I also think the ring re-allocation is not really necessary.
> Well, unfortunately, the way I
> structured the code, we can't do *just* that...
I agree. We need refactoring for it.
> > a per-queue tear-down if DMAC is still working. And, we can prepare new descriptors
> > for the queue after tear-down.
> >
> > < Tear-down >
> > 1. Set DT_EOS to the desc_bat[queue].
> > 2. Set DLR.LBAx to 1.
> > 3. Check if DLA.LBAx is cleared.
>
> DLR.LBAx, you mean?
Yes. I heard this procedure from BSP team.
> Well, I was thinking of polling TCCR and CSR like the current
> ravb_stop_dma() does, but if that works...
I'm not sure whether polling TCCR and CSR is enough or not.
Instead of polling those registers, maybe we should poll whether
ravb_stop_dma() is succeeded or not? Especially, result of ravb_config() is
a key point whether the hardware is really stopped or not.
So, I'm thinking that just polling the ravb_stop_dma() in
ravb_tx_timeout_work() is better than the per-queue tear-down and
re-init now. But, what do you think?
> > < Prepare new descriptors and start again >
> > 4. Prepare new descriptors.
>
> That's where the cause for using the workqueue lies -- the descriptors are
> allocated with GFP_KERNEL, not GFP_ATOMIC...
IIUC, we can avoid to use the workqueue if re-allocation is not really necessary.
> if you have time/desire to
> untangle all this, I'd appreciate it; else I'd have to work on this in my
> copious free time... :-)
If we don't need refactoring, I think I can do it :)
> > 5. Set DT_LINKFIX to the desc_bat[queue].
> > 6. Set DLR.LBAx to 1.
> > 7. Check if DLA.LBAx is cleared.
> >
> >
> > I'm thinking doing tear-down and "re-init" of the descriptors is better
> > than just try to wake the subqueue. But, what do you think?
>
> Definitely better, yes...
I got it.
Best regards,
Yoshihiro Shimoda
> > Best regards,
> > Yoshihiro Shimoda
>
> MBR, Sergei
Powered by blists - more mailing lists