netdev - Re: [PATCH net V2] virtio-net: correctly enable callback during start

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20230104014256-mutt-send-email-mst@kernel.org>
Date:   Wed, 4 Jan 2023 01:46:09 -0500
From:   "Michael S. Tsirkin" <mst@...hat.com>
To:     Jason Wang <jasowang@...hat.com>
Cc:     davem@...emloft.net, edumazet@...gle.com, kuba@...nel.org,
        pabeni@...hat.com, virtualization@...ts.linux-foundation.org,
        netdev@...r.kernel.org, linux-kernel@...r.kernel.org,
        xuanzhuo@...ux.alibaba.com
Subject: Re: [PATCH net V2] virtio-net: correctly enable callback during
 start_xmit

On Wed, Jan 04, 2023 at 12:23:07PM +0800, Jason Wang wrote:
> 
> 在 2022/12/23 14:29, Jason Wang 写道:
> > On Fri, Dec 16, 2022 at 11:43 AM Jason Wang <jasowang@...hat.com> wrote:
> > > On Thu, Dec 15, 2022 at 5:35 PM Michael S. Tsirkin <mst@...hat.com> wrote:
> > > > On Thu, Dec 15, 2022 at 05:15:43PM +0800, Jason Wang wrote:
> > > > > On Thu, Dec 15, 2022 at 5:02 PM Michael S. Tsirkin <mst@...hat.com> wrote:
> > > > > > On Thu, Dec 15, 2022 at 11:27:19AM +0800, Jason Wang wrote:
> > > > > > > Commit a7766ef18b33("virtio_net: disable cb aggressively") enables
> > > > > > > virtqueue callback via the following statement:
> > > > > > > 
> > > > > > >          do {
> > > > > > >             ......
> > > > > > >        } while (use_napi && kick &&
> > > > > > >                 unlikely(!virtqueue_enable_cb_delayed(sq->vq)));
> > > > > > > 
> > > > > > > When NAPI is used and kick is false, the callback won't be enabled
> > > > > > > here. And when the virtqueue is about to be full, the tx will be
> > > > > > > disabled, but we still don't enable tx interrupt which will cause a TX
> > > > > > > hang. This could be observed when using pktgen with burst enabled.
> > > > > > > 
> > > > > > > Fixing this by trying to enable tx interrupt after we disable TX when
> > > > > > > we're not using napi or kick is false.
> > > > > > > 
> > > > > > > Fixes: a7766ef18b33 ("virtio_net: disable cb aggressively")
> > > > > > > Signed-off-by: Jason Wang <jasowang@...hat.com>
> > > > > > > ---
> > > > > > > The patch is needed for -stable.
> > > > > > > Changes since V1:
> > > > > > > - enable tx interrupt after we disable tx
> > > > > > > ---
> > > > > > >   drivers/net/virtio_net.c | 2 +-
> > > > > > >   1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > > > 
> > > > > > > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > > > > > > index 86e52454b5b5..dcf3a536d78a 100644
> > > > > > > --- a/drivers/net/virtio_net.c
> > > > > > > +++ b/drivers/net/virtio_net.c
> > > > > > > @@ -1873,7 +1873,7 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct net_device *dev)
> > > > > > >         */
> > > > > > >        if (sq->vq->num_free < 2+MAX_SKB_FRAGS) {
> > > > > > >                netif_stop_subqueue(dev, qnum);
> > > > > > > -             if (!use_napi &&
> > > > > > > +             if ((!use_napi || !kick) &&
> > > > > > >                    unlikely(!virtqueue_enable_cb_delayed(sq->vq))) {
> > > > > > >                        /* More just got used, free them then recheck. */
> > > > > > >                        free_old_xmit_skbs(sq, false);
> > > > > > This will work but the following lines are:
> > > > > > 
> > > > > >                         if (sq->vq->num_free >= 2+MAX_SKB_FRAGS) {
> > > > > >                                  netif_start_subqueue(dev, qnum);
> > > > > >                                  virtqueue_disable_cb(sq->vq);
> > > > > >                          }
> > > > > > 
> > > > > > 
> > > > > > and I thought we are supposed to keep callbacks enabled with napi?
> > > > > This seems to be the opposite logic of commit a7766ef18b33 that
> > > > > disables callbacks for NAPI.
> > > > > 
> > > > > It said:
> > > > > 
> > > > >      There are currently two cases where we poll TX vq not in response to a
> > > > >      callback: start xmit and rx napi.  We currently do this with callbacks
> > > > >      enabled which can cause extra interrupts from the card.  Used not to be
> > > > >      a big issue as we run with interrupts disabled but that is no longer the
> > > > >      case, and in some cases the rate of spurious interrupts is so high
> > > > >      linux detects this and actually kills the interrupt.
> > > > > 
> > > > > My undersatnding is that it tries to disable callbacks on TX.
> > > > I think we want to disable callbacks while polling, yes. here we are not
> > > > polling, and I think we want a callback because otherwise nothing will
> > > > orphan skbs and a socket can be blocked, not transmitting anything - a
> > > > deadlock.
> > > I'm not sure how I got here, did you mean a partial revert of
> > > a7766ef18b33 (the part that disables TX callbacks on start_xmit)?
> > Michael, any idea on this?
> > 
> > Thanks
> 
> 
> Michael, any comment?
> 
> Thanks

Sorry I don't understand the question. What does "how I got here" mean?
To repeat my suggestion:

	I think it is easier to just do a separate branch here. Along the
	lines of:

			if (use_napi) {
				if (unlikely(!virtqueue_enable_cb_delayed(sq->vq)))
					virtqueue_napi_schedule(napi, vq);
			} else {
				... old code ...
			}

we can also backport this minimal safe fix, any refactorings can be done on
top.


-- 
MST