linux-kernel - Re: [PATCH RFC] dmaengine: dw: Prevent tx-status calling desc callback (Fix UART deadlock!)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <mdyifrhjufjwp2tko54gfw34riepov5bo4a4tefhrtfmuystao@wpjfsgikebki>
Date: Fri, 23 Aug 2024 19:39:07 +0300
From: Serge Semin <fancer.lancer@...il.com>
To: Andy Shevchenko <andy.shevchenko@...il.com>
Cc: Greg Kroah-Hartman <gregkh@...uxfoundation.org>, 
	Jiri Slaby <jirislaby@...nel.org>, Andy Shevchenko <andy@...nel.org>, 
	Viresh Kumar <vireshk@...nel.org>, Andy Shevchenko <andriy.shevchenko@...ux.intel.com>, 
	Vinod Koul <vkoul@...nel.org>, Ilpo Järvinen <ilpo.jarvinen@...ux.intel.com>, 
	dmaengine@...r.kernel.org, linux-serial@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH RFC] dmaengine: dw: Prevent tx-status calling desc
 callback (Fix UART deadlock!)

Hi Andy

On Fri, Aug 23, 2024 at 04:21:24PM +0300, Andy Shevchenko wrote:
> On Fri, Aug 23, 2024 at 12:48 PM Serge Semin <fancer.lancer@...il.com> wrote:
> >
> > Hi folks
> >
> > Any comments or suggestion about the change? The kernel occasionally
> > _deadlocks_ without it for the DW UART + DW DMAC hardware setup.
> 
> I have no time to look at that, but FWIW with a stress tests on older
> machines I have seen something similar from time to time (less than
> 10% reproducibility ration IIRC).

Thanks for the response. I also used to have the system hanging up at
the very rare occasion, but after I decreased the size of the Rx
DMA-buffer (to fix a platform-specific problem) and sped up the port
the deadlock probability dramatically increased so I managed to debug
the hanging ups.

> 
> P.S. Is there is any possibility to have a step-by-step reproducer?

There is a good chance that my approach might be platform-specific
(but the problem is general for sure), but here is what I did to
make the deadlock reproducible at the reasonable time:
1. Revert the patch in the subject (if it's applied)
2. Decrease the Rx DMA-buffer size:
drivers/tty/serial/8250/8250_dma.c: dma->rx_size = SZ_512;
3. Increase the serial communication baud-rate:
stty -F /dev/ttyS1 1500000 raw -echo -echok -echoe;
4. Loopback the ttyS1 interface: connect Tx and Rx pins.
5. Start pushing data to the ttS1 interface by the chunks someway
greater than 512 bytes. Like this:
while :; do echo -n "-"; head -c 65536 /dev/zero > /dev/ttyS1; done

In my case the system almost always hangs up after 10-50-100-200
iterations of the one-liner above.

> Also can we utilise (and update if needed) the open source project
> https://github.com/cbrake/linux-serial-test?

I guess the utility can be used to reproduce the problem, but the
data integrity check wasn't required in my case. I am also not sure
whether the loopback-test is required to reproduce the denoted
deadlock, since only the Rx code-path causes it. So most likely the
heavy inbound traffic shall be enough.

-Serge(y)

> 
> -- 
> With Best Regards,
> Andy Shevchenko