linux-kernel - [PATCH v4 0/3] Enable P2PDMA in Userspace RDMA

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20240708165714.3401377-1-martin.oliveira@eideticom.com>
Date: Mon,  8 Jul 2024 10:57:11 -0600
From: Martin Oliveira <martin.oliveira@...eticom.com>
To: linux-kernel@...r.kernel.org,
	linux-mm@...ck.org,
	linux-rdma@...r.kernel.org
Cc: Andrew Morton <akpm@...ux-foundation.org>,
	Artemy Kovalyov <artemyko@...dia.com>,
	Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
	Jason Gunthorpe <jgg@...pe.ca>,
	Leon Romanovsky <leon@...nel.org>,
	Logan Gunthorpe <logang@...tatee.com>,
	Martin Oliveira <martin.oliveira@...eticom.com>,
	Michael Guralnik <michaelgur@...dia.com>,
	Mike Marciniszyn <mike.marciniszyn@...el.com>,
	Shiraz Saleem <shiraz.saleem@...el.com>,
	Tejun Heo <tj@...nel.org>,
	John Hubbard <jhubbard@...dia.com>,
	Dan Williams <dan.j.williams@...el.com>,
	David Sloan <david.sloan@...eticom.com>
Subject: [PATCH v4 0/3] Enable P2PDMA in Userspace RDMA

This version of this patch series fixes the unhandled WARN issue from
v3. Everything else remains the same. Thanks to everyone who provided
reviews and feedback!

Martin

Original cover letter:

This patch series enables P2PDMA memory to be used in userspace RDMA
transfers. With this series, P2PDMA memory mmaped into userspace (ie.
only NVMe CMBs, at the moment) can then be used with ibv_reg_mr() (or
similar) interfaces. This can be tested by passing a sysfs p2pmem
allocator to the --mmap flag of the perftest tools.

This requires addressing two issues:

* Stop exporting kernfs VMAs with page_mkwrite, which is incompatible
with FOLL_LONGTERM and is redudant since the default fault code has the
same behavior as kernfs_vma_page_mkwrite() (i.e., call
file_update_time()).

* Remove the restriction on FOLL_LONGTREM with FOLL_PCI_P2PDMA which was
initially put in place due to excessive caution with assuming P2PDMA
would have similar problems to fsdax with unmap_mapping_range(). Seeing
P2PDMA only uses unmap_mapping_range() on device unbind and immediately
waits for all page reference counts to go to zero after calling it, it
is actually believed to be safe from reuse and user access faults. See
[1] for more discussion.

This was tested using a Mellanox ConnectX-6 SmartNIC (MT28908 Family),
using the mlx5_core driver, as well as an NVMe CMB.

Thanks,
Martin

[1]: https://lore.kernel.org/linux-mm/87cypuvh2i.fsf@nvdebian.thelocal/T/

--

Changes in v4:
  - Actually handle the WARN if someone sets ->page_mkwrite

Changes in v3:
  - Change to WARN_ON() if an implementaion of kernfs sets
    .page_mkwrite() (Suggested by Christoph)
  - Removed fast-gup patch

Changes in v2:
  - Remove page_mkwrite() for all kernfs, instead of creating a
    different vm_ops for p2pdma.

Martin Oliveira (3):
  kernfs: remove page_mkwrite() from vm_operations_struct
  mm/gup: allow FOLL_LONGTERM & FOLL_PCI_P2PDMA
  RDMA/umem: add support for P2P RDMA

 drivers/infiniband/core/umem.c |  3 +++
 fs/kernfs/file.c               | 25 ++-----------------------
 mm/gup.c                       |  5 -----
 3 files changed, 5 insertions(+), 28 deletions(-)

base-commit: 22a40d14b572deb80c0648557f4bd502d7e83826
-- 
2.34.1

Martin Oliveira (3):
  kernfs: remove page_mkwrite() from vm_operations_struct
  mm/gup: allow FOLL_LONGTERM & FOLL_PCI_P2PDMA
  RDMA/umem: add support for P2P RDMA

 drivers/infiniband/core/umem.c |  3 +++
 fs/kernfs/file.c               | 40 ++++++++++------------------------
 mm/gup.c                       |  5 -----
 3 files changed, 14 insertions(+), 34 deletions(-)

base-commit: 256abd8e550ce977b728be79a74e1729438b4948
-- 
2.34.1