Message-ID: <20180525201907-mutt-send-email-mst@kernel.org>
Date:   Fri, 25 May 2018 20:19:23 +0300
From:   "Michael S. Tsirkin" <mst@...hat.com>
To:     Sridhar Samudrala <sridhar.samudrala@...el.com>
Cc:     stephen@...workplumber.org, davem@...emloft.net,
        netdev@...r.kernel.org, virtualization@...ts.linux-foundation.org,
        virtio-dev@...ts.oasis-open.org, jesse.brandeburg@...el.com,
        alexander.h.duyck@...el.com, kubakici@...pl, jasowang@...hat.com,
        loseweigh@...il.com, jiri@...nulli.us, aaron.f.brown@...el.com,
        anjali.singhai@...el.com
Subject: Re: [PATCH net-next v12 0/5] Enable virtio_net to act as a standby
 for a passthru device

On Thu, May 24, 2018 at 09:55:12AM -0700, Sridhar Samudrala wrote:
> The main motivation for this patch is to enable cloud service providers
> to provide an accelerated datapath to virtio-net enabled VMs in a 
> transparent manner with no/minimal guest userspace changes. This also
> enables hypervisor controlled live migration to be supported with VMs that
> have direct attached SR-IOV VF devices.
> 
> Patch 1 introduces a failover module that provides a generic interface for 
> paravirtual drivers to listen for netdev register/unregister/link change
> events from pci ethernet devices with the same MAC and takeover their
> datapath. The notifier and event handling code is based on the existing
> netvsc implementation. 
> 
> Patch 2 refactors netvsc to use the registration/notification framework
> introduced by failover module.
> 
> Patch 3 introduces a net_failover driver that provides an automated
> failover mechanism to paravirtual drivers via APIs to create and destroy
> a failover master netdev and mananges a primary and standby slave netdevs
> that get registered via the generic failover infrastructure.
> 
> Patch 4 introduces a new feature bit VIRTIO_NET_F_STANDBY to virtio-net
> that can be used by hypervisor to indicate that virtio_net interface
> should act as a standby for another device with the same MAC address.
> 
> Patch 5 extends virtio_net to use alternate datapath when available and
> registered. When STANDBY feature is enabled, virtio_net driver uese the
> net_failover API to create an additional 'failover' netdev that acts as
> a master device and controls 2 slave devices.  The original virtio_net
> netdev is registered as 'standby' netdev and a passthru/vf device with
> the same MAC gets registered as 'primary' netdev. Both 'standby' and
> 'failover' netdevs are associated with the same 'pci' device.  The user
> accesses the network interface via 'failover' netdev. The 'failover'
> netdev chooses 'primary' netdev as default for transmits when it is
> available with link up and running.
> 
> As this patch series is initially focusing on usecases where hypervisor 
> fully controls the VM networking and the guest is not expected to directly 
> configure any hardware settings, it doesn't expose all the ndo/ethtool ops
> that are supported by virtio_net at this time. To support additional usecases,
> it should be possible to enable additional ops later by caching the state
> in failover netdev and replaying when the 'primary' netdev gets registered. 
>  
> At the time of live migration, the hypervisor needs to unplug the VF device
> from the guest on the source host and reset the MAC filter of the VF to
> initiate failover of datapath to virtio before starting the migration. After
> the migration is completed, the destination hypervisor sets the MAC filter
> on the VF and plugs it back to the guest to switch over to VF datapath.
> 
> This patch is based on the discussion initiated by Jesse on this thread.
> https://marc.info/?l=linux-virtualization&m=151189725224231&w=2

Series:

Acked-by: Michael S. Tsirkin <mst@...hat.com>


> v12:
> - Tested live migration with virtio-net/AVF(i40evf) configured in failover
>   mode while running iperf in background. Tried static ip and dhcp
>   configurations using 'network' scripts and Network Manager.
> - Build tested netvsc module.
> Updates:
> - Extended generic failover module to do common functions like setting
>   FAILOVER_SLAVE flag, registering rx-handler and linking to upper dev in
>   the generic register/unregister handlers.
>   This required adding 3 additional failover ops pre_register, pre_unregister
>   and handle_frame.  netvsc and net_failover drivers are updated to support
>   these ops.
> 
> v11:
> - Split net_failover module into 2 components.
>   1. 'failover' module that provides generic failover infrastructure
>   to register a failover instance and listen for slave events.
>   2. 'net_failover' driver that provides APIs to create/destroy upper
>   netdev and supports 3-netdev model used by virtio-net.
> - Added documentation
> 
> v10:
> - fix net_failover_open() to update failover CARRIER correctly based on
>   standby and primary states. 
> - fix net_failover_handle_frame() to handle frames received on standby
>   when primary is present.
> - replace netdev_upper_dev_link with netdev_master_upper_dev_link and
>   handle lower dev state changes.
> - fix net_failver_create() and net_failover_register() interfaces to
>   use ERR_PTR and avoid arg **
> - disable setting mac address when virtio-net in STANDBY mode
> - document exported symbols
> - added entry to MAINTAINERS file
> 
> v9:
> Select NET_FAILOVER automatically when VIRTIO_NET/HYPERV_NET 
> are enabled. (stephen)
> 
> v8:
> - Made the failover managment routines more robust by updating the feature 
>   bits/other fields in the failover netdev when slave netdevs are 
>   registered/unregistered. (mst)
> - added support for handling vlans.
> - Limited the changes in netvsc to only use the notifier/event/lookups
>   from the failover module. The slave register/unregister/link-change 
>   handlers are only updated to use the getbymac routine to get the 
>   upper netdev. There is no change in their functionality. (stephen)
> - renamed structs/function/file names to use net_failover prefix. (mst)
> 
> v7
> - Rename 'bypass/active/backup' terminology with 'failover/primary/standy'
>   (jiri, mst)
> - re-arranged dev_open() and dev_set_mtu() calls in the register routines
>   so that they don't get called for 2-netdev model. (stephen)
> - fixed select_queue() routine to do queue selection based on VF if it is
>   registered as primary. (stephen)
> -  minor bugfixes
> 
> v6 RFC:
>   Simplified virtio_net changes by moving all the ndo_ops of the 
>   bypass_netdev and create/destroy of bypass_netdev to 'bypass' module.
>   avoided 2 phase registration(driver + instances).
>   introduced IFF_BYPASS/IFF_BYPASS_SLAVE dev->priv_flags 
>   replaced mutex with a spinlock
> 
> v5 RFC:
>   Based on Jiri's comments, moved the common functionality to a 'bypass'
>   module so that the same notifier and event handlers to handle child
>   register/unregister/link change events can be shared between virtio_net
>   and netvsc.
>   Improved error handling based on Siwei's comments.
> v4:
> - Based on the review comments on the v3 version of the RFC patch and
>   Jakub's suggestion for the naming issue with 3 netdev solution,
>   proposed 3 netdev in-driver bonding solution for virtio-net.
> v3 RFC:
> - Introduced 3 netdev model and pointed out a couple of issues with
>   that model and proposed 2 netdev model to avoid these issues.
> - Removed broadcast/multicast optimization and only use virtio as
>   backup path when VF is unplugged.
> v2 RFC:
> - Changed VIRTIO_NET_F_MASTER to VIRTIO_NET_F_BACKUP (mst)
> - made a small change to the virtio-net xmit path to only use VF datapath
>   for unicasts. Broadcasts/multicasts use virtio datapath. This avoids
>   east-west broadcasts to go over the PCI link.
> - added suppport for the feature bit in qemu
> 
> Sridhar Samudrala (5):
>   net: Introduce generic failover module
>   netvsc: refactor notifier/event handling code to use the failover
>     framework
>   net: Introduce net_failover driver
>   virtio_net: Introduce VIRTIO_NET_F_STANDBY feature bit
>   virtio_net: Extend virtio to use VF datapath when available
> 
>  Documentation/networking/failover.rst     |  18 +
>  Documentation/networking/net_failover.rst | 116 +++++
>  MAINTAINERS                               |  16 +
>  drivers/net/Kconfig                       |  13 +
>  drivers/net/Makefile                      |   1 +
>  drivers/net/hyperv/Kconfig                |   1 +
>  drivers/net/hyperv/hyperv_net.h           |   2 +
>  drivers/net/hyperv/netvsc_drv.c           | 222 ++------
>  drivers/net/net_failover.c                | 836 ++++++++++++++++++++++++++++++
>  drivers/net/virtio_net.c                  |  40 +-
>  include/linux/netdevice.h                 |  16 +
>  include/net/failover.h                    |  36 ++
>  include/net/net_failover.h                |  40 ++
>  include/uapi/linux/virtio_net.h           |   3 +
>  net/Kconfig                               |  13 +
>  net/core/Makefile                         |   1 +
>  net/core/failover.c                       | 315 +++++++++++
>  17 files changed, 1522 insertions(+), 167 deletions(-)
>  create mode 100644 Documentation/networking/failover.rst
>  create mode 100644 Documentation/networking/net_failover.rst
>  create mode 100644 drivers/net/net_failover.c
>  create mode 100644 include/net/failover.h
>  create mode 100644 include/net/net_failover.h
>  create mode 100644 net/core/failover.c
> 
> -- 
> 2.14.3

Powered by blists - more mailing lists