netdev - Re: virtio_close() stuck on napi_disable

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CACGkMEtg00ih8tv4LTgxNkUEREF5vzYP=dKrth_eFPbEDZg11w@mail.gmail.com>
Date: Thu, 24 Jul 2025 18:53:57 +0800
From: Jason Wang <jasowang@...hat.com>
To: Paolo Abeni <pabeni@...hat.com>
Cc: Jakub Kicinski <kuba@...nel.org>, Zigit Zo <zuozhijie@...edance.com>, 
	"Michael S. Tsirkin" <mst@...hat.com>, Xuan Zhuo <xuanzhuo@...ux.alibaba.com>, 
	Eugenio Pérez <eperezma@...hat.com>, 
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: Re: virtio_close() stuck on napi_disable_locked()

On Thu, Jul 24, 2025 at 4:43 PM Paolo Abeni <pabeni@...hat.com> wrote:
>
> On 7/23/25 7:14 AM, Jason Wang wrote:
> > On Wed, Jul 23, 2025 at 5:55 AM Jakub Kicinski <kuba@...nel.org> wrote:
> >> On Tue, 22 Jul 2025 13:00:14 +0200 Paolo Abeni wrote:
> >>> The NIPA CI is reporting some hung-up in the stats.py test caused by the
> >>> virtio_net driver stuck at close time.
> >>>
> >>> A sample splat is available here:
> >>>
> >>> https://netdev-3.bots.linux.dev/vmksft-drv-hw-dbg/results/209441/4-stats-py/stderr
> >>>
> >>> AFAICS the issue happens only on debug builds.
> >>>
> >>> I'm wild guessing to something similar to the the issue addressed by
> >>> commit 4bc12818b363bd30f0f7348dd9ab077290a637ae, possibly for tx_napi,
> >>> but I could not spot anything obvious.
> >>>
> >>> Could you please have a look?
> >>
> >> It only hits in around 1 in 5 runs.
> >
> > I tried to reproduce this locally but failed. Where can I see the qemu
> > command line for the VM?
> >
> >> Likely some pre-existing race, but
> >> it started popping up for us when be5dcaed694e ("virtio-net: fix
> >> recursived rtnl_lock() during probe()") was merged.
> >
> > Probably but I didn't see a direct connection with that commit. It
> > looks like the root cause is the deadloop of napi_disable() for some
> > reason as Paolo said.
> >
> >> It never hit before.
> >> If we can't find a quick fix I think we should revert be5dcaed694e for
> >> now, so that it doesn't end up regressing 6.16 final.
>
> I tried hard to reproduce the issue locally - to validate an eventual
> revert before pushing it. But so far I failed quite miserably.
>

I've also tried to follow the instructions of nipai for setup 2 virtio
and make the relevant taps to connect with a bridge on the host. But I
failed to reproduce it locally for several hours.

Is there a log of the execution of nipa test that we can know more
information like:

1) full qemu command line
2) host kernel version
3) Qemu version

> Given the above, I would avoid the revert and just mention the known
> issue in the net PR to Linus.

+1

Thanks

>
> /P
>