| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <9369f524-4368-92ca-5342-b9bb62b70559@amd.com>
Date: Tue, 6 Dec 2022 17:06:53 -0800
From: Shannon Nelson <shannon.nelson@....com>
To: Shay Drory <shayd@...dia.com>, netdev@...r.kernel.org,
kuba@...nel.org, davem@...emloft.net
Cc: danielj@...dia.com, yishaih@...dia.com, jiri@...dia.com,
saeedm@...dia.com, parav@...dia.com,
Shannon Nelson <snelson@...sando.io>
Subject: Re: [PATCH net-next V4 7/8] devlink: Expose port function commands to
control migratable
On 12/6/22 10:51 AM, Shay Drory wrote:
> Expose port function commands to enable / disable migratable
> capability, this is used to set the port function as migratable.
>
> Live migration is the process of transferring a live virtual machine
> from one physical host to another without disrupting its normal
> operation.
>
> In order for a VM to be able to perform LM, all the VM components must
> be able to perform migration. e.g.: to be migratable.
> In order for VF to be migratable, VF must be bound to VFIO driver with
> migration support.
>
> When migratable capability is enabled for a function of the port, the
> device is making the necessary preparations for the function to be
> migratable, which might include disabling features which cannot be
> migrated.
>
> Example of LM with migratable function configuration:
> Set migratable of the VF's port function.
>
> $ devlink port show pci/0000:06:00.0/2
> pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0
> vfnum 1
> function:
> hw_addr 00:00:00:00:00:00 migratable disable
>
> $ devlink port function set pci/0000:06:00.0/2 migratable enable
>
> $ devlink port show pci/0000:06:00.0/2
> pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0
> vfnum 1
> function:
> hw_addr 00:00:00:00:00:00 migratable enable
>
> Bind VF to VFIO driver with migration support:
> $ echo <pci_id> > /sys/bus/pci/devices/0000:08:00.0/driver/unbind
> $ echo mlx5_vfio_pci > /sys/bus/pci/devices/0000:08:00.0/driver_override
> $ echo <pci_id> > /sys/bus/pci/devices/0000:08:00.0/driver/bind
>
> Attach VF to the VM.
> Start the VM.
> Perform LM.
>
> Cc: Shannon Nelson <snelson@...sando.io>
Acked-by: Shannon Nelson <shannon.nelson@....com>
(yes, my primary email has recently changed)
> Signed-off-by: Shay Drory <shayd@...dia.com>
> Reviewed-by: Jiri Pirko <jiri@...dia.com>
> ---
> v3->v4:
> - change port_function_mig to port_fn_migratable
> v2->v3:
> - fix documentation warning
> - introduce DEVLINK_PORT_FN_CAP_MIGRATABLE
> v1->v2:
> - fix documentation warning
> ---
> .../networking/devlink/devlink-port.rst | 46 ++++++++++++++++
> include/net/devlink.h | 21 +++++++
> include/uapi/linux/devlink.h | 3 +
> net/core/devlink.c | 55 +++++++++++++++++++
> 4 files changed, 125 insertions(+)
>
> diff --git a/Documentation/networking/devlink/devlink-port.rst b/Documentation/networking/devlink/devlink-port.rst
> index c3302d23e480..3da590953ce8 100644
> --- a/Documentation/networking/devlink/devlink-port.rst
> +++ b/Documentation/networking/devlink/devlink-port.rst
> @@ -125,6 +125,9 @@ this means a MAC address.
> Users may also set the RoCE capability of the function using
> `devlink port function set roce` command.
>
> +Users may also set the function as migratable using
> +'devlink port function set migratable' command.
> +
> Function attributes
> ===================
>
> @@ -194,6 +197,49 @@ VF/SF driver cannot override it.
> function:
> hw_addr 00:00:00:00:00:00 roce disable
>
> +migratable capability setup
> +---------------------------
> +Live migration is the process of transferring a live virtual machine
> +from one physical host to another without disrupting its normal
> +operation.
> +
> +User who want PCI VFs to be able to perform live migration need to
> +explicitly enable the VF migratable capability.
> +
> +When user enables migratable capability for a VF, and the HV binds the VF to VFIO driver
> +with migration support, the user can migrate the VM with this VF from one HV to a
> +different one.
> +
> +However, when migratable capability is enable, device will disable features which cannot
> +be migrated. Thus migratable cap can impose limitations on a VF so let the user decide.
> +
> +Example of LM with migratable function configuration:
> +- Get migratable capability of the VF device::
> +
> + $ devlink port show pci/0000:06:00.0/2
> + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
> + function:
> + hw_addr 00:00:00:00:00:00 migratable disable
> +
> +- Set migratable capability of the VF device::
> +
> + $ devlink port function set pci/0000:06:00.0/2 migratable enable
> +
> + $ devlink port show pci/0000:06:00.0/2
> + pci/0000:06:00.0/2: type eth netdev enp6s0pf0vf1 flavour pcivf pfnum 0 vfnum 1
> + function:
> + hw_addr 00:00:00:00:00:00 migratable enable
> +
> +- Bind VF to VFIO driver with migration support::
> +
> + $ echo <pci_id> > /sys/bus/pci/devices/0000:08:00.0/driver/unbind
> + $ echo mlx5_vfio_pci > /sys/bus/pci/devices/0000:08:00.0/driver_override
> + $ echo <pci_id> > /sys/bus/pci/devices/0000:08:00.0/driver/bind
> +
> +Attach VF to the VM.
> +Start the VM.
> +Perform live migration.
> +
> Subfunction
> ============
>
> diff --git a/include/net/devlink.h b/include/net/devlink.h
> index ce4c65d2f2e7..0f376a28b9c4 100644
> --- a/include/net/devlink.h
> +++ b/include/net/devlink.h
> @@ -1469,6 +1469,27 @@ struct devlink_ops {
> */
> int (*port_fn_roce_set)(struct devlink_port *devlink_port,
> bool enable, struct netlink_ext_ack *extack);
> + /**
> + * @port_fn_migratable_get: Port function's migratable get function.
> + *
> + * Query migratable state of a function managed by the devlink port.
> + * Return -EOPNOTSUPP if port function migratable handling is not
> + * supported.
> + */
> + int (*port_fn_migratable_get)(struct devlink_port *devlink_port,
> + bool *is_enable,
> + struct netlink_ext_ack *extack);
> + /**
> + * @port_fn_migratable_set: Port function's migratable set function.
> + *
> + * Enable/Disable migratable state of a function managed by the devlink
> + * port.
> + * Return -EOPNOTSUPP if port function migratable handling is not
> + * supported.
> + */
> + int (*port_fn_migratable_set)(struct devlink_port *devlink_port,
> + bool enable,
> + struct netlink_ext_ack *extack);
> /**
> * port_new() - Add a new port function of a specified flavor
> * @devlink: Devlink instance
> diff --git a/include/uapi/linux/devlink.h b/include/uapi/linux/devlink.h
> index 6cc2925bd478..3782d4219ac9 100644
> --- a/include/uapi/linux/devlink.h
> +++ b/include/uapi/linux/devlink.h
> @@ -660,12 +660,15 @@ enum devlink_resource_unit {
>
> enum devlink_port_fn_attr_cap {
> DEVLINK_PORT_FN_ATTR_CAP_ROCE_BIT,
> + DEVLINK_PORT_FN_ATTR_CAP_MIGRATABLE_BIT,
>
> /* Add new caps above */
> __DEVLINK_PORT_FN_ATTR_CAPS_MAX,
> };
>
> #define DEVLINK_PORT_FN_CAP_ROCE _BITUL(DEVLINK_PORT_FN_ATTR_CAP_ROCE_BIT)
> +#define DEVLINK_PORT_FN_CAP_MIGRATABLE \
> + _BITUL(DEVLINK_PORT_FN_ATTR_CAP_MIGRATABLE_BIT)
>
> enum devlink_port_function_attr {
> DEVLINK_PORT_FUNCTION_ATTR_UNSPEC,
> diff --git a/net/core/devlink.c b/net/core/devlink.c
> index 8c0ad52431c5..ab40ebcb4aea 100644
> --- a/net/core/devlink.c
> +++ b/net/core/devlink.c
> @@ -715,6 +715,29 @@ static int devlink_port_fn_roce_fill(const struct devlink_ops *ops,
> return 0;
> }
>
> +static int devlink_port_fn_migratable_fill(const struct devlink_ops *ops,
> + struct devlink_port *devlink_port,
> + struct nla_bitfield32 *caps,
> + struct netlink_ext_ack *extack)
> +{
> + bool is_enable;
> + int err;
> +
> + if (!ops->port_fn_migratable_get ||
> + devlink_port->attrs.flavour != DEVLINK_PORT_FLAVOUR_PCI_VF)
> + return 0;
> +
> + err = ops->port_fn_migratable_get(devlink_port, &is_enable, extack);
> + if (err) {
> + if (err == -EOPNOTSUPP)
> + return 0;
> + return err;
> + }
> +
> + devlink_port_fn_cap_fill(caps, DEVLINK_PORT_FN_CAP_MIGRATABLE, is_enable);
> + return 0;
> +}
> +
> static int devlink_port_fn_caps_fill(const struct devlink_ops *ops,
> struct devlink_port *devlink_port,
> struct sk_buff *msg,
> @@ -728,6 +751,10 @@ static int devlink_port_fn_caps_fill(const struct devlink_ops *ops,
> if (err)
> return err;
>
> + err = devlink_port_fn_migratable_fill(ops, devlink_port, &caps, extack);
> + if (err)
> + return err;
> +
> if (!caps.selector)
> return 0;
> err = nla_put_bitfield32(msg, DEVLINK_PORT_FN_ATTR_CAPS, caps.value,
> @@ -1322,6 +1349,15 @@ static int devlink_port_fn_state_fill(const struct devlink_ops *ops,
> return 0;
> }
>
> +static int
> +devlink_port_fn_mig_set(struct devlink_port *devlink_port, bool enable,
> + struct netlink_ext_ack *extack)
> +{
> + const struct devlink_ops *ops = devlink_port->devlink->ops;
> +
> + return ops->port_fn_migratable_set(devlink_port, enable, extack);
> +}
> +
> static int
> devlink_port_fn_roce_set(struct devlink_port *devlink_port, bool enable,
> struct netlink_ext_ack *extack)
> @@ -1348,6 +1384,13 @@ static int devlink_port_fn_caps_set(struct devlink_port *devlink_port,
> if (err)
> return err;
> }
> + if (caps.selector & DEVLINK_PORT_FN_CAP_MIGRATABLE) {
> + err = devlink_port_fn_mig_set(devlink_port, caps_value &
> + DEVLINK_PORT_FN_CAP_MIGRATABLE,
> + extack);
> + if (err)
> + return err;
> + }
> return 0;
> }
>
> @@ -1769,6 +1812,18 @@ static int devlink_port_function_validate(struct devlink_port *devlink_port,
> "Port doesn't support RoCE function attribute");
> return -EOPNOTSUPP;
> }
> + if (caps.selector & DEVLINK_PORT_FN_CAP_MIGRATABLE) {
> + if (!ops->port_fn_migratable_set) {
> + NL_SET_ERR_MSG_ATTR(extack, attr,
> + "Port doesn't support migratable function attribute");
> + return -EOPNOTSUPP;
> + }
> + if (devlink_port->attrs.flavour != DEVLINK_PORT_FLAVOUR_PCI_VF) {
> + NL_SET_ERR_MSG_ATTR(extack, attr,
> + "migratable function attribute supported for VFs only");
> + return -EOPNOTSUPP;
> + }
> + }
> }
> return 0;
> }
> --
> 2.38.1
>
Powered by blists - more mailing lists