linux-kernel - Re: [PATCH v6 2/6] vfio/nvgrace-gpu: Add support for huge pfnmap

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20251125215801.7149bb3c.zhiw@nvidia.com>
Date: Tue, 25 Nov 2025 21:58:01 +0200
From: Zhi Wang <zhiw@...dia.com>
To: <ankita@...dia.com>
CC: <jgg@...pe.ca>, <yishaih@...dia.com>, <skolothumtho@...dia.com>,
	<kevin.tian@...el.com>, <alex@...zbot.org>, <aniketa@...dia.com>,
	<vsethi@...dia.com>, <mochs@...dia.com>, <Yunxiang.Li@....com>,
	<yi.l.liu@...el.com>, <zhangdongdong@...incomputing.com>,
	<avihaih@...dia.com>, <bhelgaas@...gle.com>, <peterx@...hat.com>,
	<pstanner@...hat.com>, <apopple@...dia.com>, <kvm@...r.kernel.org>,
	<linux-kernel@...r.kernel.org>, <cjia@...dia.com>, <kwankhede@...dia.com>,
	<targupta@...dia.com>, <danw@...dia.com>, <dnigam@...dia.com>,
	<kjaju@...dia.com>
Subject: Re: [PATCH v6 2/6] vfio/nvgrace-gpu: Add support for huge pfnmap

On Tue, 25 Nov 2025 17:30:09 +0000
<ankita@...dia.com> wrote:

> From: Ankit Agrawal <ankita@...dia.com>
> 
> NVIDIA's Grace based systems have large device memory. The device
> memory is mapped as VM_PFNMAP in the VMM VMA. The nvgrace-gpu
> module could make use of the huge PFNMAP support added in mm [1].
> 
> To make use of the huge pfnmap support, fault/huge_fault ops
> based mapping mechanism needs to be implemented. Currently nvgrace-gpu
> module relies on remap_pfn_range to do the mapping during VM bootup.
> Replace it to instead rely on fault and use vfio_pci_vmf_insert_pfn
> to setup the mapping.
> 
> Moreover to enable huge pfnmap, nvgrace-gpu module is updated by
> adding huge_fault ops implementation. The implementation establishes
> mapping according to the order request. Note that if the PFN or the
> VMA address is unaligned to the order, the mapping fallbacks to
> the PTE level.
> 
> Link:
> https://lore.kernel.org/all/20240826204353.2228736-1-peterx@redhat.com/
> [1]
> 
> cc: Shameer Kolothum <skolothumtho@...dia.com>
> cc: Alex Williamson <alex@...zbot.org>
> cc: Jason Gunthorpe <jgg@...pe.ca>
> cc: Vikram Sethi <vsethi@...dia.com>
> Signed-off-by: Ankit Agrawal <ankita@...dia.com>
> ---
>  drivers/vfio/pci/nvgrace-gpu/main.c | 84
> +++++++++++++++++++++-------- 1 file changed, 62 insertions(+), 22
> deletions(-)
> 
> diff --git a/drivers/vfio/pci/nvgrace-gpu/main.c
> b/drivers/vfio/pci/nvgrace-gpu/main.c index
> e346392b72f6..8a982310b188 100644 ---
> a/drivers/vfio/pci/nvgrace-gpu/main.c +++
> b/drivers/vfio/pci/nvgrace-gpu/main.c @@ -130,6 +130,62 @@ static
> void nvgrace_gpu_close_device(struct vfio_device *core_vdev)
> vfio_pci_core_close_device(core_vdev); }
>  
> +static unsigned long addr_to_pgoff(struct vm_area_struct *vma,
> +				   unsigned long addr)
> +{
> +	u64 pgoff = vma->vm_pgoff &
> +		((1U << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1);
> +
> +	return ((addr - vma->vm_start) >> PAGE_SHIFT) + pgoff;
> +}
> +
> +static vm_fault_t nvgrace_gpu_vfio_pci_huge_fault(struct vm_fault
> *vmf,
> +						  unsigned int order)
> +{
> +	struct vm_area_struct *vma = vmf->vma;
> +	struct nvgrace_gpu_pci_core_device *nvdev =
> vma->vm_private_data;
> +	struct vfio_pci_core_device *vdev = &nvdev->core_device;
> +	unsigned int index =
> +		vma->vm_pgoff >> (VFIO_PCI_OFFSET_SHIFT -
> PAGE_SHIFT);
> +	vm_fault_t ret = VM_FAULT_SIGBUS;
> +	struct mem_region *memregion;
> +	unsigned long pfn, addr;
> +
> +	memregion = nvgrace_gpu_memregion(index, nvdev);
> +	if (!memregion)
> +		return ret;
> +
> +	addr = vmf->address & ~((PAGE_SIZE << order) - 1);

ALIGN_DOWN(vmf->address, PAGE_SIZE << order).

> +	pfn = PHYS_PFN(memregion->memphys) + addr_to_pgoff(vma,
> addr); +
> +	if (order && (addr < vma->vm_start ||
> +		      addr + (PAGE_SIZE << order) > vma->vm_end ||
> +		      pfn & ((1 << order) - 1)))

!IS_ALIGNED(pfn, 1 << order).

Other parts looks good to me.

Reviewed-by: Zhi Wang <zhiw@...dia.com>