linux-kernel - Re: [PATCH 1/3] migration: update BAR space size

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <dc5c2f1a-181e-4f91-0f8a-ede622b9d1dc@huawei.com>
Date: Tue, 25 Feb 2025 14:00:20 +0800
From: liulongfang <liulongfang@...wei.com>
To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@...wei.com>,
	"alex.williamson@...hat.com" <alex.williamson@...hat.com>, "jgg@...dia.com"
	<jgg@...dia.com>, Jonathan Cameron <jonathan.cameron@...wei.com>
CC: "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linuxarm@...neuler.org" <linuxarm@...neuler.org>
Subject: Re: [PATCH 1/3] migration: update BAR space size

On 2025/2/18 18:20, Shameerali Kolothum Thodi wrote:
> 
> 
>> -----Original Message-----
>> From: liulongfang <liulongfang@...wei.com>
>> Sent: Tuesday, February 18, 2025 2:15 AM
>> To: alex.williamson@...hat.com; jgg@...dia.com; Shameerali Kolothum
>> Thodi <shameerali.kolothum.thodi@...wei.com>; Jonathan Cameron
>> <jonathan.cameron@...wei.com>
>> Cc: kvm@...r.kernel.org; linux-kernel@...r.kernel.org;
>> linuxarm@...neuler.org; liulongfang <liulongfang@...wei.com>
>> Subject: [PATCH 1/3] migration: update BAR space size
>>
>> On the new hardware platform, the live migration configuration region
>> is moved from VF to PF. The VF's own configuration space is
>> restored to the complete 64KB, and there is no need to divide the
>> size of the BAR configuration space equally.
>>
>> Signed-off-by: Longfang Liu <liulongfang@...wei.com>
>> ---
>>  .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c    | 41 +++++++++++++++----
>>  1 file changed, 32 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c
>> b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c
>> index 451c639299eb..599905dbb707 100644
>> --- a/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c
>> +++ b/drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c
>> @@ -1195,6 +1195,33 @@ static struct hisi_qm *hisi_acc_get_pf_qm(struct
>> pci_dev *pdev)
>>  	return !IS_ERR(pf_qm) ? pf_qm : NULL;
>>  }
>>
>> +static size_t hisi_acc_get_resource_len(struct vfio_pci_core_device *vdev,
>> +					unsigned int index)
>> +{
>> +	struct hisi_acc_vf_core_device *hisi_acc_vdev =
>> +			hisi_acc_drvdata(vdev->pdev);
>> +
>> +	/*
>> +	 * ACC VF dev 64KB BAR2 region consists of both functional
>> +	 * register space and migration control register space, each
>> +	 * uses 32KB BAR2 region, on the system with more than 64KB
>> +	 * page size, even if the migration control register space
>> +	 * is written by VM, it will only affects the VF.
>> +	 *
>> +	 * In order to support the live migration function in the
>> +	 * system with a page size above 64KB, the driver needs
>> +	 * to ensure that the VF region size is aligned with the
>> +	 * system page size.
> 
> I didn't get this. Are you referring to kernel with 64K page size? And this is 
> for new hardware or QM_HW_V3 one?
>

This 64KB page refers to the 64KB page size of BAR2.
If the configuration space of the live migration device is not
aligned to the 64KB page (for example, only 32KB on the QM_HW_V3 platform),
when the host kernel page is 64KB, multiple VF devices will fail during VFIO
enumeration.
Therefore, on the new hardware platform, the live migration configuration
space is moved to the PF, which can align the 64KB memory space and solve
this problem.

>> +	 *
>> +	 * On the new hardware platform, the live migration control register
>> +	 * has been moved from VF to PF.
>> +	 */
>> +	if (hisi_acc_vdev->pf_qm->ver == QM_HW_V3)
>> +		return (pci_resource_len(vdev->pdev, index) >> 1);
>> +
>> +	return pci_resource_len(vdev->pdev, index);
>> +}
>> +
>>  static int hisi_acc_pci_rw_access_check(struct vfio_device *core_vdev,
>>  					size_t count, loff_t *ppos,
>>  					size_t *new_count)
>> @@ -1205,8 +1232,9 @@ static int hisi_acc_pci_rw_access_check(struct
>> vfio_device *core_vdev,
>>
>>  	if (index == VFIO_PCI_BAR2_REGION_INDEX) {
>>  		loff_t pos = *ppos & VFIO_PCI_OFFSET_MASK;
>> -		resource_size_t end = pci_resource_len(vdev->pdev, index) /
>> 2;
>> +		resource_size_t end;
>>
>> +		end = hisi_acc_get_resource_len(vdev, index);
>>  		/* Check if access is for migration control region */
>>  		if (pos >= end)
>>  			return -EINVAL;
>> @@ -1227,8 +1255,9 @@ static int hisi_acc_vfio_pci_mmap(struct
>> vfio_device *core_vdev,
>>  	index = vma->vm_pgoff >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT);
>>  	if (index == VFIO_PCI_BAR2_REGION_INDEX) {
>>  		u64 req_len, pgoff, req_start;
>> -		resource_size_t end = pci_resource_len(vdev->pdev, index) /
>> 2;
>> +		resource_size_t end;
>>
>> +		end = PAGE_ALIGN(hisi_acc_get_resource_len(vdev, index));
> 
> So here, the whole BAR2 will be mapped to Guest in case of QM_HW_V3 &&
> 64K kernel as well, right?
>


Here, the length of BAR2 is mapped to the Guest OS according to the actual
situation of the platform. If it is QM_HW_V3, only 32KB is mapped.
Others are mapped 64KB.

Thanks.

> Thanks,
> Shameer
> 
>>  		req_len = vma->vm_end - vma->vm_start;
>>  		pgoff = vma->vm_pgoff &
>>  			((1U << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1);
>> @@ -1275,7 +1304,6 @@ static long hisi_acc_vfio_pci_ioctl(struct
>> vfio_device *core_vdev, unsigned int
>>  	if (cmd == VFIO_DEVICE_GET_REGION_INFO) {
>>  		struct vfio_pci_core_device *vdev =
>>  			container_of(core_vdev, struct vfio_pci_core_device,
>> vdev);
>> -		struct pci_dev *pdev = vdev->pdev;
>>  		struct vfio_region_info info;
>>  		unsigned long minsz;
>>
>> @@ -1290,12 +1318,7 @@ static long hisi_acc_vfio_pci_ioctl(struct
>> vfio_device *core_vdev, unsigned int
>>  		if (info.index == VFIO_PCI_BAR2_REGION_INDEX) {
>>  			info.offset =
>> VFIO_PCI_INDEX_TO_OFFSET(info.index);
>>
>> -			/*
>> -			 * ACC VF dev BAR2 region consists of both
>> functional
>> -			 * register space and migration control register
>> space.
>> -			 * Report only the functional region to Guest.
>> -			 */
>> -			info.size = pci_resource_len(pdev, info.index) / 2;
>> +			info.size = hisi_acc_get_resource_len(vdev,
>> info.index);
>>
>>  			info.flags = VFIO_REGION_INFO_FLAG_READ |
>>  					VFIO_REGION_INFO_FLAG_WRITE |
>> --
>> 2.24.0
> 
> .
>