linux-kernel - [PATCH vhost v2 0/7] vdpa/mlx5: Optimze MKEY operations

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240830105838.2666587-2-dtatulea@nvidia.com>
Date: Fri, 30 Aug 2024 13:58:31 +0300
From: Dragos Tatulea <dtatulea@...dia.com>
To: "Michael S . Tsirkin" <mst@...hat.com>, Jason Wang <jasowang@...hat.com>,
	Eugenio Perez Martin <eperezma@...hat.com>, Si-Wei Liu
	<si-wei.liu@...cle.com>, <virtualization@...ts.linux.dev>, Gal Pressman
	<gal@...dia.com>
CC: Dragos Tatulea <dtatulea@...dia.com>, <kvm@...r.kernel.org>,
	<linux-kernel@...r.kernel.org>, Parav Pandit <parav@...dia.com>, Xuan Zhuo
	<xuanzhuo@...ux.alibaba.com>
Subject: [PATCH vhost v2 0/7] vdpa/mlx5: Optimze MKEY operations

This series improves the time of .set_map() operations by parallelizing
the MKEY creation and deletion for direct MKEYs. Looking at the top
level MKEY creation/deletion functions, the following improvement can be
seen:

|-------------------+-------------|
| operation         | improvement |
|-------------------+-------------|
| create_user_mr()  | 3-5x        |
| destroy_user_mr() | 8x          |
|-------------------+-------------|

The last part of the series introduces lazy MKEY deletion which
postpones the MKEY deletion to a later point in a workqueue.

As this series and the previous ones were targeting live migration,
we can also observe improvements on this front:

|-------------------+------------------+------------------|
| Stage             | Downtime #1 (ms) | Downtime #2 (ms) |
|-------------------+------------------+------------------|
| Baseline          | 3140             | 3630             |
| Parallel MKEY ops | 1200             | 2000             |
| Deferred deletion | 1014             | 1253             |
|-------------------+------------------+------------------|

Test configuration: 256 GB VM, 32 CPUs x 2 threads per core, 4 x mlx5
vDPA devices x 32 VQs (16 VQPs)

This series must be applied on top of the parallel VQ suspend/resume
series [0].

[0] https://lore.kernel.org/all/20240816090159.1967650-1-dtatulea@nvidia.com/

---
v2:
- Swapped flex array usage for plain zero length array in first patch.
- Updated code to use Scope-Based Cleanup Helpers where appropriate
  (only second patch).
- Added macro define for MTT alignment in first patch.
- Improved commit messages/comments based on review comments.
- Removed extra newlines.
---

Dragos Tatulea (7):
  vdpa/mlx5: Create direct MKEYs in parallel
  vdpa/mlx5: Delete direct MKEYs in parallel
  vdpa/mlx5: Rename function
  vdpa/mlx5: Extract mr members in own resource struct
  vdpa/mlx5: Rename mr_mtx -> lock
  vdpa/mlx5: Introduce init/destroy for MR resources
  vdpa/mlx5: Postpone MR deletion

 drivers/vdpa/mlx5/core/mlx5_vdpa.h |  25 ++-
 drivers/vdpa/mlx5/core/mr.c        | 288 +++++++++++++++++++++++++----
 drivers/vdpa/mlx5/core/resources.c |   3 -
 drivers/vdpa/mlx5/net/mlx5_vnet.c  |  53 +++---
 4 files changed, 296 insertions(+), 73 deletions(-)

-- 
2.45.1