linux-kernel - Re: mhi resume failure on reboot with 6.13-rc2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20241218183555.qste4gve7vnvdml2@thinkpad>
Date: Thu, 19 Dec 2024 00:05:55 +0530
From: Manivannan Sadhasivam <manivannan.sadhasivam@...aro.org>
To: Johan Hovold <johan@...nel.org>
Cc: mhi@...ts.linux.dev, linux-arm-msm@...r.kernel.org,
	linux-kernel@...r.kernel.org,
	Loic Poulain <loic.poulain@...aro.org>
Subject: Re: mhi resume failure on reboot with 6.13-rc2

On Wed, Dec 18, 2024 at 03:26:38PM +0100, Johan Hovold wrote:
> On Wed, Dec 18, 2024 at 07:39:10PM +0530, Manivannan Sadhasivam wrote:
> > On Wed, Dec 18, 2024 at 02:55:02PM +0100, Johan Hovold wrote:
> > > On Wed, Dec 18, 2024 at 06:00:19PM +0530, Manivannan Sadhasivam wrote:
> > > > On Wed, Dec 18, 2024 at 01:02:39PM +0100, Johan Hovold wrote:
> > > > > On Wed, Dec 18, 2024 at 05:08:30PM +0530, Manivannan Sadhasivam wrote:
> > > > > > On Wed, Dec 18, 2024 at 09:40:45AM +0100, Johan Hovold wrote:
> > > > > 
> > > > > > > I've tracked down the hang to a deadlock on the parent device lock.
> > > > > > > 
> > > > > > > Driver core takes the parent device lock before calling shutdown(), and
> > > > > > > then mhi_pci_shutdown() waits indefinitely for the recovery thread to
> > > > > > > finish.
> > > > > 
> > > > > > Thanks for tracking the deadlock. I think we should use pci_try_reset_function()
> > > > > > instead of pci_reset_function() in mhi_pci_recovery_work().
> > > > > > 
> > > > > > If the pci_dev_lock() is already taken, it will return with -EAGAIN and we do
> > > > > > not need to worry in that case since the host is going to be powered off anyway
> > > > > > (and so the device).
> > > > > 
> > > > > That may work. But note that I've now also seen this deadlock during
> > > > > suspend (i.e. when the device is not going away). The
> > > > > pci_try_reset_function() should avoid the deadlock here too, but we'll
> > > > > end up in funny state.
> > > > 
> > > > Hopefully, recovery_work() started by mhi_pci_runtime_resume() would be able to
> > > > reset the device.
> > > 
> > > But that's not going to happen as that reset is what is currently
> > > causing the deadlock and which would simply be skipped if you switch to
> > > pci_try_reset_function().
> > > 
> > 
> > mhi_pci_runtime_resume() will queue the recovery_work() and return. So I was
> > hoping that by the time pci_try_reset_function() is called, the lock would be
> > available.
> 
> We can't rely on luck with timings, and this is the very reason for the
> deadlock I'm currently seeing (i.e. the recovery thread is still running
> when another thread grabs the lock and waits for the recovery thread to
> finish).
> 
> Perhaps the recovery work should be done synchronously in the resume
> handler to avoid such issues.
> 

Synchronously? How can that help when the recovery_work() cannot acquire the
lock?

Anyhow, even if the lock is not available during resume (worst case), PCI core
should reset the device when it tries to change the state.

I don't know if there is any better solution available.

- Mani

-- 
மணிவண்ணன் சதாசிவம்