linux-kernel - RE: [PATCH v15 00/12] SMMUv3 Nested Stage Setup (IOMMU part)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <f6b0bcd156e04d0f958cf79a50ac69e2@huawei.com>
Date:   Wed, 14 Apr 2021 06:56:31 +0000
From:   Shameerali Kolothum Thodi <shameerali.kolothum.thodi@...wei.com>
To:     wangxingang <wangxingang5@...wei.com>,
        Eric Auger <eric.auger@...hat.com>,
        "eric.auger.pro@...il.com" <eric.auger.pro@...il.com>,
        "jean-philippe@...aro.org" <jean-philippe@...aro.org>,
        "iommu@...ts.linux-foundation.org" <iommu@...ts.linux-foundation.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        "kvmarm@...ts.cs.columbia.edu" <kvmarm@...ts.cs.columbia.edu>,
        "will@...nel.org" <will@...nel.org>,
        "maz@...nel.org" <maz@...nel.org>,
        "robin.murphy@....com" <robin.murphy@....com>,
        "joro@...tes.org" <joro@...tes.org>,
        "alex.williamson@...hat.com" <alex.williamson@...hat.com>,
        "tn@...ihalf.com" <tn@...ihalf.com>,
        zhukeqian <zhukeqian1@...wei.com>
CC:     "jacob.jun.pan@...ux.intel.com" <jacob.jun.pan@...ux.intel.com>,
        "yi.l.liu@...el.com" <yi.l.liu@...el.com>,
        "zhangfei.gao@...aro.org" <zhangfei.gao@...aro.org>,
        "zhangfei.gao@...il.com" <zhangfei.gao@...il.com>,
        "vivek.gautam@....com" <vivek.gautam@....com>,
        yuzenghui <yuzenghui@...wei.com>,
        "nicoleotsuka@...il.com" <nicoleotsuka@...il.com>,
        lushenming <lushenming@...wei.com>,
        "vsethi@...dia.com" <vsethi@...dia.com>,
        "chenxiang (M)" <chenxiang66@...ilicon.com>,
        "vdumpa@...dia.com" <vdumpa@...dia.com>,
        jiangkunkun <jiangkunkun@...wei.com>
Subject: RE: [PATCH v15 00/12] SMMUv3 Nested Stage Setup (IOMMU part)



> -----Original Message-----
> From: wangxingang
> Sent: 14 April 2021 03:36
> To: Eric Auger <eric.auger@...hat.com>; eric.auger.pro@...il.com;
> jean-philippe@...aro.org; iommu@...ts.linux-foundation.org;
> linux-kernel@...r.kernel.org; kvm@...r.kernel.org;
> kvmarm@...ts.cs.columbia.edu; will@...nel.org; maz@...nel.org;
> robin.murphy@....com; joro@...tes.org; alex.williamson@...hat.com;
> tn@...ihalf.com; zhukeqian <zhukeqian1@...wei.com>
> Cc: jacob.jun.pan@...ux.intel.com; yi.l.liu@...el.com; zhangfei.gao@...aro.org;
> zhangfei.gao@...il.com; vivek.gautam@....com; Shameerali Kolothum
> Thodi <shameerali.kolothum.thodi@...wei.com>; yuzenghui
> <yuzenghui@...wei.com>; nicoleotsuka@...il.com; lushenming
> <lushenming@...wei.com>; vsethi@...dia.com; chenxiang (M)
> <chenxiang66@...ilicon.com>; vdumpa@...dia.com; jiangkunkun
> <jiangkunkun@...wei.com>
> Subject: Re: [PATCH v15 00/12] SMMUv3 Nested Stage Setup (IOMMU part)
> 
> Hi Eric, Jean-Philippe
> 
> On 2021/4/11 19:12, Eric Auger wrote:
> > SMMUv3 Nested Stage Setup (IOMMU part)
> >
> > This series brings the IOMMU part of HW nested paging support
> > in the SMMUv3. The VFIO part is submitted separately.
> >
> > This is based on Jean-Philippe's
> > [PATCH v14 00/10] iommu: I/O page faults for SMMUv3
> > https://www.spinics.net/lists/arm-kernel/msg886518.html
> > (including the patches that were not pulled for 5.13)
> >
> > The IOMMU API is extended to support 2 new API functionalities:
> > 1) pass the guest stage 1 configuration
> > 2) pass stage 1 MSI bindings
> >
> > Then those capabilities gets implemented in the SMMUv3 driver.
> >
> > The virtualizer passes information through the VFIO user API
> > which cascades them to the iommu subsystem. This allows the guest
> > to own stage 1 tables and context descriptors (so-called PASID
> > table) while the host owns stage 2 tables and main configuration
> > structures (STE).
> >
> > Best Regards
> >
> > Eric
> >
> > This series can be found at:
> > v5.12-rc6-jean-iopf-14-2stage-v15
> > (including the VFIO part in its last version: v13)
> >
> 
> I am testing the performance of an accelerator with/without SVA/vSVA,
> and found there might be some potential performance loss risk for SVA/vSVA.
> 
> I use a Network and computing encryption device (SEC), and send 1MB
> request for 10000 times.
> 
> I trigger mm fault before I send the request, so there should be no iopf.
> 
> Here's what I got:
> 
> physical scenario:
> performance:		SVA:9MB/s  	NOSVA:9MB/s
> tlb_miss: 		SVA:302,651	NOSVA:1,223
> trans_table_walk_access:SVA:302,276	NOSVA:1,237
> 
> VM scenario:
> performance:		vSVA:9MB/s  	NOvSVA:6MB/s  about 30~40% loss
> tlb_miss: 		vSVA:4,423,897	NOvSVA:1,907
> trans_table_walk_access:vSVA:61,928,430	NOvSVA:21,948
> 
> In physical scenario, there's almost no performance loss, but the
> tlb_miss and trans_table_walk_access of stage 1 for SVA is quite high,
> comparing to NOSVA.
> 
> In VM scenario, there's about 30~40% performance loss, this is because
> the two stage tlb_miss and trans_table_walk_access is even higher, and
> impact the performance.
> 
> I compare the procedure of building page table of SVA and NOSVA, and
> found that NOSVA uses 2MB mapping as far as possible, while SVA uses
> only 4KB.
> 
> I retest with huge page, and huge page could solve this problem, the
> performance of SVA/vSVA is almost the same as NOSVA.
> 
> I am wondering do you have any other solution for the performance loss
> of vSVA, or any other method to reduce the tlb_miss/trans_table_walk.

Hi Xingang,

Just curious, do you have DVM enabled on this board or does it use explicit
SMMU TLB invalidations?

Thanks,
Shameer