linux-kernel - Re: [PATCH v6 6/6] vfio/nvgrace-gpu: wait for the GPU mem to be ready

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20251125222858.7839ee83.zhiw@nvidia.com>
Date: Tue, 25 Nov 2025 22:28:58 +0200
From: Zhi Wang <zhiw@...dia.com>
To: <ankita@...dia.com>
CC: <jgg@...pe.ca>, <yishaih@...dia.com>, <skolothumtho@...dia.com>,
	<kevin.tian@...el.com>, <alex@...zbot.org>, <aniketa@...dia.com>,
	<vsethi@...dia.com>, <mochs@...dia.com>, <Yunxiang.Li@....com>,
	<yi.l.liu@...el.com>, <zhangdongdong@...incomputing.com>,
	<avihaih@...dia.com>, <bhelgaas@...gle.com>, <peterx@...hat.com>,
	<pstanner@...hat.com>, <apopple@...dia.com>, <kvm@...r.kernel.org>,
	<linux-kernel@...r.kernel.org>, <cjia@...dia.com>, <kwankhede@...dia.com>,
	<targupta@...dia.com>, <danw@...dia.com>, <dnigam@...dia.com>,
	<kjaju@...dia.com>
Subject: Re: [PATCH v6 6/6] vfio/nvgrace-gpu: wait for the GPU mem to be
 ready

On Tue, 25 Nov 2025 17:30:13 +0000
<ankita@...dia.com> wrote:

> From: Ankit Agrawal <ankita@...dia.com>
> 
> Speculative prefetches from CPU to GPU memory until the GPU is
> ready after reset can cause harmless corrected RAS events to
> be logged on Grace systems. It is thus preferred that the
> mapping not be re-established until the GPU is ready post reset.
> 
> The GPU readiness can be checked through BAR0 registers similar
> to the checking at the time of device probe.
> 
> It can take several seconds for the GPU to be ready. So it is
> desirable that the time overlaps as much of the VM startup as
> possible to reduce impact on the VM bootup time. The GPU
> readiness state is thus checked on the first fault/huge_fault
> request or read/write access which amortizes the GPU readiness
> time.
> 

snip

> @@ -179,8 +215,12 @@ static vm_fault_t
> nvgrace_gpu_vfio_pci_huge_fault(struct vm_fault *vmf, pfn & ((1 <<
> order) - 1))) return VM_FAULT_FALLBACK;
>  
> -	scoped_guard(rwsem_read, &vdev->memory_lock)
> +	scoped_guard(rwsem_read, &vdev->memory_lock) {
> +		if (nvgrace_gpu_check_device_ready(nvdev))
> +			return ret;
> +

I would suggest opening the error code if we don't have a "bailing
out without touching the ret" similar to vfio_pci_mmap_huge_fault(),
since this looks unnecessarily confusing.

Please also fix the same in PATCH 2. 

>  		ret = vfio_pci_vmf_insert_pfn(vdev, vmf, pfn, order);
> +	}
>  
>  	dev_dbg_ratelimited(&vdev->pdev->dev,
>  			    "%s order = %d pfn 0x%lx: 0x%x\n",
> @@ -592,9 +632,15 @@ nvgrace_gpu_read_mem(struct