Date:   Wed, 8 Dec 2021 13:33:22 +0000
From:   Shameerali Kolothum Thodi <shameerali.kolothum.thodi@...wei.com>
To:     "eric.auger@...hat.com" <eric.auger@...hat.com>,
        Zhangfei Gao <zhangfei.gao@...aro.org>,
        "eric.auger.pro@...il.com" <eric.auger.pro@...il.com>,
        "iommu@...ts.linux-foundation.org" <iommu@...ts.linux-foundation.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        "kvmarm@...ts.cs.columbia.edu" <kvmarm@...ts.cs.columbia.edu>,
        "joro@...tes.org" <joro@...tes.org>,
        "will@...nel.org" <will@...nel.org>,
        "robin.murphy@....com" <robin.murphy@....com>,
        "jean-philippe@...aro.org" <jean-philippe@...aro.org>,
        zhukeqian <zhukeqian1@...wei.com>
CC:     "alex.williamson@...hat.com" <alex.williamson@...hat.com>,
        "jacob.jun.pan@...ux.intel.com" <jacob.jun.pan@...ux.intel.com>,
        "yi.l.liu@...el.com" <yi.l.liu@...el.com>,
        "kevin.tian@...el.com" <kevin.tian@...el.com>,
        "ashok.raj@...el.com" <ashok.raj@...el.com>,
        "maz@...nel.org" <maz@...nel.org>,
        "peter.maydell@...aro.org" <peter.maydell@...aro.org>,
        "vivek.gautam@....com" <vivek.gautam@....com>,
        wangxingang <wangxingang5@...wei.com>,
        jiangkunkun <jiangkunkun@...wei.com>,
        yuzenghui <yuzenghui@...wei.com>,
        "nicoleotsuka@...il.com" <nicoleotsuka@...il.com>,
        "chenxiang (M)" <chenxiang66@...ilicon.com>,
        "sumitg@...dia.com" <sumitg@...dia.com>,
        "nicolinc@...dia.com" <nicolinc@...dia.com>,
        "vdumpa@...dia.com" <vdumpa@...dia.com>,
        "zhangfei.gao@...il.com" <zhangfei.gao@...il.com>,
        "vsethi@...dia.com" <vsethi@...dia.com>
Subject: RE: [RFC v16 0/9] SMMUv3 Nested Stage Setup (IOMMU part)



> -----Original Message-----
> From: Eric Auger [mailto:eric.auger@...hat.com]
> Sent: 07 December 2021 11:06
> To: Zhangfei Gao <zhangfei.gao@...aro.org>; eric.auger.pro@...il.com;
> iommu@...ts.linux-foundation.org; linux-kernel@...r.kernel.org;
> kvm@...r.kernel.org; kvmarm@...ts.cs.columbia.edu; joro@...tes.org;
> will@...nel.org; robin.murphy@....com; jean-philippe@...aro.org;
> zhukeqian <zhukeqian1@...wei.com>
> Cc: alex.williamson@...hat.com; jacob.jun.pan@...ux.intel.com;
> yi.l.liu@...el.com; kevin.tian@...el.com; ashok.raj@...el.com;
> maz@...nel.org; peter.maydell@...aro.org; vivek.gautam@....com;
> Shameerali Kolothum Thodi <shameerali.kolothum.thodi@...wei.com>;
> wangxingang <wangxingang5@...wei.com>; jiangkunkun
> <jiangkunkun@...wei.com>; yuzenghui <yuzenghui@...wei.com>;
> nicoleotsuka@...il.com; chenxiang (M) <chenxiang66@...ilicon.com>;
> sumitg@...dia.com; nicolinc@...dia.com; vdumpa@...dia.com;
> zhangfei.gao@...il.com; lushenming@...wei.com; vsethi@...dia.com
> Subject: Re: [RFC v16 0/9] SMMUv3 Nested Stage Setup (IOMMU part)
> 
> Hi Zhangfei,
> 
> On 12/7/21 11:35 AM, Zhangfei Gao wrote:
> >
> >
> > On 2021/12/7 下午6:27, Eric Auger wrote:
> >> Hi Zhangfei,
> >>
> >> On 12/3/21 1:27 PM, Zhangfei Gao wrote:
> >>> Hi, Eric
> >>>
> >>> On 2021/10/27 下午6:44, Eric Auger wrote:
> >>>> This series brings the IOMMU part of HW nested paging support
> >>>> in the SMMUv3.
> >>>>
> >>>> The SMMUv3 driver is adapted to support 2 nested stages.
> >>>>
> >>>> The IOMMU API is extended to convey the guest stage 1
> >>>> configuration and the hook is implemented in the SMMUv3 driver.
> >>>>
> >>>> This allows the guest to own the stage 1 tables and context
> >>>> descriptors (so-called PASID table) while the host owns the
> >>>> stage 2 tables and main configuration structures (STE).
> >>>>
> >>>> This work mainly is provided for test purpose as the upper
> >>>> layer integration is under rework and bound to be based on
> >>>> /dev/iommu instead of VFIO tunneling. In this version we also get
> >>>> rid of the MSI BINDING ioctl, assuming the guest enforces
> >>>> flat mapping of host IOVAs used to bind physical MSI doorbells.
> >>>> In the current QEMU integration this is achieved by exposing
> >>>> RMRs to the guest, using Shameer's series [1]. This approach
> >>>> is RFC as the IORT spec is not really meant to do that
> >>>> (single mapping flag limitation).
> >>>>
> >>>> Best Regards
> >>>>
> >>>> Eric
> >>>>
> >>>> This series (Host) can be found at:
> >>>> https://github.com/eauger/linux/tree/v5.15-rc7-nested-v16
> >>>> This includes a rebased VFIO integration (although not meant
> >>>> to be upstreamed)
> >>>>
> >>>> Guest kernel branch can be found at:
> >>>> https://github.com/eauger/linux/tree/shameer_rmrr_v7
> >>>> featuring [1]
> >>>>
> >>>> QEMU integration (still based on VFIO and exposing RMRs)
> >>>> can be found at:
> >>>>
> https://github.com/eauger/qemu/tree/v6.1.0-rmr-v2-nested_smmuv3_v10
> >>>> (use iommu=nested-smmuv3 ARM virt option)
> >>>>
> >>>> Guest dependency:
> >>>> [1] [PATCH v7 0/9] ACPI/IORT: Support for IORT RMR node
> >>> Thanks a lot for upgrading these patches.
> >>>
> >>> I have basically verified these patches on HiSilicon Kunpeng920.
> >>> And integrated them to these branches.
> >>> https://github.com/Linaro/linux-kernel-uadk/tree/uacce-devel-5.16
> >>> https://github.com/Linaro/qemu/tree/v6.1.0-rmr-v2-nested_smmuv3_v10
> >>>
> >>> Though they are provided for test purpose,
> >>>
> >>> Tested-by: Zhangfei Gao <zhangfei.gao@...aro.org>
> >> Thank you very much. As you mentioned, until we do not have the
> >> /dev/iommu integration this is maintained for testing purpose. The SMMU
> >> changes shouldn't be much impacted though.
> >> The added value of this respin was to propose an MSI binding solution
> >> based on RMRRs which simplify things at kernel level.
> >
> > Current RMRR solution requires uefi enabled,
> > and QEMU_EFI.fd  has to be provided to start qemu.
> >
> > Any plan to support dtb as well, which will be simpler since no need
> > QEMU_EFI.fd anymore.
> Yes the solution is based on ACPI IORT nodes. No clue if some DT
> integration is under work. Shameer?

There was some attempt in the past to create identity mappings using DT.
This is the latest I can find on it,
https://lore.kernel.org/linux-iommu/YTelDHx2REIIvV%2FN@orome.fritz.box/T/

Thanks,
Shameer

Powered by blists - more mailing lists