| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20251110202844.GL15456@unreal>
Date: Mon, 10 Nov 2025 22:28:44 +0200
From: Leon Romanovsky <leon@...nel.org>
To: Alex Williamson <alex@...zbot.org>
Cc: Bjorn Helgaas <bhelgaas@...gle.com>,
Logan Gunthorpe <logang@...tatee.com>, Jens Axboe <axboe@...nel.dk>,
Robin Murphy <robin.murphy@....com>, Joerg Roedel <joro@...tes.org>,
Will Deacon <will@...nel.org>,
Marek Szyprowski <m.szyprowski@...sung.com>,
Jason Gunthorpe <jgg@...pe.ca>,
Andrew Morton <akpm@...ux-foundation.org>,
Jonathan Corbet <corbet@....net>,
Sumit Semwal <sumit.semwal@...aro.org>,
Christian König <christian.koenig@....com>,
Kees Cook <kees@...nel.org>,
"Gustavo A. R. Silva" <gustavoars@...nel.org>,
Ankit Agrawal <ankita@...dia.com>,
Yishai Hadas <yishaih@...dia.com>,
Shameer Kolothum <skolothumtho@...dia.com>,
Kevin Tian <kevin.tian@...el.com>,
Krishnakant Jaju <kjaju@...dia.com>, Matt Ochs <mochs@...dia.com>,
linux-pci@...r.kernel.org, linux-kernel@...r.kernel.org,
linux-block@...r.kernel.org, iommu@...ts.linux.dev,
linux-mm@...ck.org, linux-doc@...r.kernel.org,
linux-media@...r.kernel.org, dri-devel@...ts.freedesktop.org,
linaro-mm-sig@...ts.linaro.org, kvm@...r.kernel.org,
linux-hardening@...r.kernel.org, Alex Mastro <amastro@...com>,
Nicolin Chen <nicolinc@...dia.com>
Subject: Re: [PATCH v7 11/11] vfio/nvgrace: Support get_dmabuf_phys
On Mon, Nov 10, 2025 at 01:05:34PM -0700, Alex Williamson wrote:
> On Thu, 6 Nov 2025 16:16:56 +0200
> Leon Romanovsky <leon@...nel.org> wrote:
>
> > From: Jason Gunthorpe <jgg@...dia.com>
> >
> > Call vfio_pci_core_fill_phys_vec() with the proper physical ranges for the
> > synthetic BAR 2 and BAR 4 regions. Otherwise use the normal flow based on
> > the PCI bar.
> >
> > This demonstrates a DMABUF that follows the region info report to only
> > allow mapping parts of the region that are mmapable. Since the BAR is
> > power of two sized and the "CXL" region is just page aligned the there can
> > be a padding region at the end that is not mmaped or passed into the
> > DMABUF.
> >
> > The "CXL" ranges that are remapped into BAR 2 and BAR 4 areas are not PCI
> > MMIO, they actually run over the CXL-like coherent interconnect and for
> > the purposes of DMA behave identically to DRAM. We don't try to model this
> > distinction between true PCI BAR memory that takes a real PCI path and the
> > "CXL" memory that takes a different path in the p2p framework for now.
> >
> > Signed-off-by: Jason Gunthorpe <jgg@...dia.com>
> > Tested-by: Alex Mastro <amastro@...com>
> > Tested-by: Nicolin Chen <nicolinc@...dia.com>
> > Signed-off-by: Leon Romanovsky <leonro@...dia.com>
> > ---
> > drivers/vfio/pci/nvgrace-gpu/main.c | 56 +++++++++++++++++++++++++++++++++++++
> > 1 file changed, 56 insertions(+)
> >
> > diff --git a/drivers/vfio/pci/nvgrace-gpu/main.c b/drivers/vfio/pci/nvgrace-gpu/main.c
> > index e346392b72f6..7d7ab2c84018 100644
> > --- a/drivers/vfio/pci/nvgrace-gpu/main.c
> > +++ b/drivers/vfio/pci/nvgrace-gpu/main.c
> > @@ -7,6 +7,7 @@
> > #include <linux/vfio_pci_core.h>
> > #include <linux/delay.h>
> > #include <linux/jiffies.h>
> > +#include <linux/pci-p2pdma.h>
> >
> > /*
> > * The device memory usable to the workloads running in the VM is cached
> > @@ -683,6 +684,54 @@ nvgrace_gpu_write(struct vfio_device *core_vdev,
> > return vfio_pci_core_write(core_vdev, buf, count, ppos);
> > }
> >
> > +static int nvgrace_get_dmabuf_phys(struct vfio_pci_core_device *core_vdev,
> > + struct p2pdma_provider **provider,
> > + unsigned int region_index,
> > + struct dma_buf_phys_vec *phys_vec,
> > + struct vfio_region_dma_range *dma_ranges,
> > + size_t nr_ranges)
> > +{
> > + struct nvgrace_gpu_pci_core_device *nvdev = container_of(
> > + core_vdev, struct nvgrace_gpu_pci_core_device, core_device);
> > + struct pci_dev *pdev = core_vdev->pdev;
> > +
> > + if (nvdev->resmem.memlength && region_index == RESMEM_REGION_INDEX) {
> > + /*
> > + * The P2P properties of the non-BAR memory is the same as the
> > + * BAR memory, so just use the provider for index 0. Someday
> > + * when CXL gets P2P support we could create CXLish providers
> > + * for the non-BAR memory.
> > + */
> > + *provider = pcim_p2pdma_provider(pdev, 0);
> > + if (!*provider)
> > + return -EINVAL;
> > + return vfio_pci_core_fill_phys_vec(phys_vec, dma_ranges,
> > + nr_ranges,
> > + nvdev->resmem.memphys,
> > + nvdev->resmem.memlength);
> > + } else if (region_index == USEMEM_REGION_INDEX) {
> > + /*
> > + * This is actually cachable memory and isn't treated as P2P in
> > + * the chip. For now we have no way to push cachable memory
> > + * through everything and the Grace HW doesn't care what caching
> > + * attribute is programmed into the SMMU. So use BAR 0.
> > + */
> > + *provider = pcim_p2pdma_provider(pdev, 0);
> > + if (!*provider)
> > + return -EINVAL;
> > + return vfio_pci_core_fill_phys_vec(phys_vec, dma_ranges,
> > + nr_ranges,
> > + nvdev->usemem.memphys,
> > + nvdev->usemem.memlength);
> > + }
> > + return vfio_pci_core_get_dmabuf_phys(core_vdev, provider, region_index,
> > + phys_vec, dma_ranges, nr_ranges);
> > +}
>
>
> Unless my eyes deceive, we could reduce the redundancy a bit:
>
> struct mem_region *mem_region = NULL;
>
> if (nvdev->resmem.memlength && region_index == RESMEM_REGION_INDEX) {
> /*
> * The P2P properties of the non-BAR memory is the same as the
> * BAR memory, so just use the provider for index 0. Someday
> * when CXL gets P2P support we could create CXLish providers
> * for the non-BAR memory.
> */
> mem_region = &nvdev->resmem;
> } else if (region_index == USEMEM_REGION_INDEX) {
> /*
> * This is actually cachable memory and isn't treated as P2P in
> * the chip. For now we have no way to push cachable memory
> * through everything and the Grace HW doesn't care what caching
> * attribute is programmed into the SMMU. So use BAR 0.
> */
> mem_region = &nvdev->usemem;
> }
>
> if (mem_region) {
> *provider = pcim_p2pdma_provider(pdev, 0);
> if (!*provider)
> return -EINVAL;
> return vfio_pci_core_fill_phys_vec(phys_vec, dma_ranges,
> nr_ranges,
> mem_region->memphys,
> mem_region->memlength);
> }
>
> return vfio_pci_core_get_dmabuf_phys(core_vdev, provider, region_index,
> phys_vec, dma_ranges, nr_ranges);
Yes, this will work too.
Thanks
>
> Thanks,
> Alex
Powered by blists - more mailing lists