netdev - Re: [PATCH net-next v4 2/5] virtio_net: enable irq for the control vq

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <ZnvI2hJXPJZyveAv@nanopsycho.orion>
Date: Wed, 26 Jun 2024 09:52:58 +0200
From: Jiri Pirko <jiri@...nulli.us>
To: Heng Qi <hengqi@...ux.alibaba.com>
Cc: "Michael S. Tsirkin" <mst@...hat.com>, Jason Wang <jasowang@...hat.com>,
	netdev@...r.kernel.org, virtualization@...ts.linux.dev,
	Xuan Zhuo <xuanzhuo@...ux.alibaba.com>,
	Eugenio Pérez <eperezma@...hat.com>,
	"David S. Miller" <davem@...emloft.net>,
	Eric Dumazet <edumazet@...gle.com>,
	Jakub Kicinski <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>
Subject: Re: [PATCH net-next v4 2/5] virtio_net: enable irq for the control vq

Thu, Jun 20, 2024 at 12:31:34PM CEST, hengqi@...ux.alibaba.com wrote:
>On Thu, 20 Jun 2024 06:11:40 -0400, "Michael S. Tsirkin" <mst@...hat.com> wrote:
>> On Thu, Jun 20, 2024 at 06:10:51AM -0400, Michael S. Tsirkin wrote:
>> > On Thu, Jun 20, 2024 at 05:53:15PM +0800, Heng Qi wrote:
>> > > On Thu, 20 Jun 2024 16:26:05 +0800, Jason Wang <jasowang@...hat.com> wrote:
>> > > > On Thu, Jun 20, 2024 at 4:21 PM Jason Wang <jasowang@...hat.com> wrote:
>> > > > >
>> > > > > On Thu, Jun 20, 2024 at 3:35 PM Heng Qi <hengqi@...ux.alibaba.com> wrote:
>> > > > > >
>> > > > > > On Wed, 19 Jun 2024 17:19:12 -0400, "Michael S. Tsirkin" <mst@...hat.com> wrote:
>> > > > > > > On Thu, Jun 20, 2024 at 12:19:05AM +0800, Heng Qi wrote:
>> > > > > > > > @@ -5312,7 +5315,7 @@ static int virtnet_find_vqs(struct virtnet_info *vi)
>> > > > > > > >
>> > > > > > > >     /* Parameters for control virtqueue, if any */
>> > > > > > > >     if (vi->has_cvq) {
>> > > > > > > > -           callbacks[total_vqs - 1] = NULL;
>> > > > > > > > +           callbacks[total_vqs - 1] = virtnet_cvq_done;
>> > > > > > > >             names[total_vqs - 1] = "control";
>> > > > > > > >     }
>> > > > > > > >
>> > > > > > >
>> > > > > > > If the # of MSIX vectors is exactly for data path VQs,
>> > > > > > > this will cause irq sharing between VQs which will degrade
>> > > > > > > performance significantly.
>> > > > > > >
>> > > > >
>> > > > > Why do we need to care about buggy management? I think libvirt has
>> > > > > been teached to use 2N+2 since the introduction of the multiqueue[1].
>> > > > 
>> > > > And Qemu can calculate it correctly automatically since:
>> > > > 
>> > > > commit 51a81a2118df0c70988f00d61647da9e298483a4
>> > > > Author: Jason Wang <jasowang@...hat.com>
>> > > > Date:   Mon Mar 8 12:49:19 2021 +0800
>> > > > 
>> > > >     virtio-net: calculating proper msix vectors on init
>> > > > 
>> > > >     Currently, the default msix vectors for virtio-net-pci is 3 which is
>> > > >     obvious not suitable for multiqueue guest, so we depends on the user
>> > > >     or management tools to pass a correct vectors parameter. In fact, we
>> > > >     can simplifying this by calculating the number of vectors on realize.
>> > > > 
>> > > >     Consider we have N queues, the number of vectors needed is 2*N + 2
>> > > >     (#queue pairs + plus one config interrupt and control vq). We didn't
>> > > >     check whether or not host support control vq because it was added
>> > > >     unconditionally by qemu to avoid breaking legacy guests such as Minix.
>> > > > 
>> > > >     Reviewed-by: Philippe Mathieu-Daudé <philmd@...hat.com
>> > > >     Reviewed-by: Stefano Garzarella <sgarzare@...hat.com>
>> > > >     Reviewed-by: Stefan Hajnoczi <stefanha@...hat.com>
>> > > >     Signed-off-by: Jason Wang <jasowang@...hat.com>
>> > > 
>> > > Yes, devices designed according to the spec need to reserve an interrupt
>> > > vector for ctrlq. So, Michael, do we want to be compatible with buggy devices?
>> > > 
>> > > Thanks.
>> > 
>> > These aren't buggy, the spec allows this. So don't fail, but
>> > I'm fine with using polling if not enough vectors.
>> 
>> sharing with config interrupt is easier code-wise though, FWIW -
>> we don't need to maintain two code-paths.
>
>Yes, it works well - config change irq is used less before - and will not fail.

Please note I'm working on such fallback for admin queue. I would Like
to send the patchset by the end of this week. You can then use it easily
for cvq.

Something like:
/* the config->find_vqs() implementation */
int vp_find_vqs(struct virtio_device *vdev, unsigned int nvqs,
                struct virtqueue *vqs[], vq_callback_t *callbacks[],
                const char * const names[], const bool *ctx,
                struct irq_affinity *desc)
{
        int err;

        /* Try MSI-X with one vector per queue. */
        err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names,
                               VP_VQ_VECTOR_POLICY_EACH, ctx, desc);
        if (!err)
                return 0;
        /* Fallback: MSI-X with one shared vector for config and
         * slow path queues, one vector per queue for the rest. */
        err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names,
                               VP_VQ_VECTOR_POLICY_SHARED_SLOW, ctx, desc);
        if (!err)
                return 0;
        /* Fallback: MSI-X with one vector for config, one shared for queues. */
        err = vp_find_vqs_msix(vdev, nvqs, vqs, callbacks, names,
                               VP_VQ_VECTOR_POLICY_SHARED, ctx, desc);
        if (!err)
                return 0;
        /* Is there an interrupt? If not give up. */
        if (!(to_vp_device(vdev)->pci_dev->irq))
                return err;
        /* Finally fall back to regular interrupts. */
        return vp_find_vqs_intx(vdev, nvqs, vqs, callbacks, names, ctx);
}




>
>Thanks.
>
>> 
>> > > > 
>> > > > Thanks
>> > > > 
>> > > > >
>> > > > > > > So no, you can not just do it unconditionally.
>> > > > > > >
>> > > > > > > The correct fix probably requires virtio core/API extensions.
>> > > > > >
>> > > > > > If the introduction of cvq irq causes interrupts to become shared, then
>> > > > > > ctrlq need to fall back to polling mode and keep the status quo.
>> > > > >
>> > > > > Having to path sounds a burden.
>> > > > >
>> > > > > >
>> > > > > > Thanks.
>> > > > > >
>> > > > >
>> > > > >
>> > > > > Thanks
>> > > > >
>> > > > > [1] https://www.linux-kvm.org/page/Multiqueue
>> > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > MST
>> > > > > > >
>> > > > > >
>> > > > 
>> 
>