netdev - Re: [PATCH v11 4/6] iommu/s390: Force ISM devices to use IOMMU_DOMAIN

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <67ec0e36908b7e6d7a6eba642ec76ef87d0d4945.camel@linux.ibm.com>
Date: Wed, 23 Aug 2023 12:53:03 +0200
From: Niklas Schnelle <schnelle@...ux.ibm.com>
To: Robin Murphy <robin.murphy@....com>, Joerg Roedel <joro@...tes.org>,
        Matthew Rosato <mjrosato@...ux.ibm.com>, Will Deacon <will@...nel.org>,
        Wenjia Zhang <wenjia@...ux.ibm.com>, Jason Gunthorpe <jgg@...pe.ca>
Cc: Gerd Bayer <gbayer@...ux.ibm.com>, Julian Ruess <julianr@...ux.ibm.com>,
        Pierre Morel <pmorel@...ux.ibm.com>,
        Alexandra Winter
 <wintera@...ux.ibm.com>,
        Heiko Carstens <hca@...ux.ibm.com>, Vasily Gorbik
 <gor@...ux.ibm.com>,
        Alexander Gordeev <agordeev@...ux.ibm.com>,
        Christian
 Borntraeger <borntraeger@...ux.ibm.com>,
        Sven Schnelle
 <svens@...ux.ibm.com>,
        Suravee Suthikulpanit
 <suravee.suthikulpanit@....com>,
        Hector Martin <marcan@...can.st>, Sven
 Peter <sven@...npeter.dev>,
        Alyssa Rosenzweig <alyssa@...enzweig.io>,
        David
 Woodhouse <dwmw2@...radead.org>,
        Lu Baolu <baolu.lu@...ux.intel.com>, Andy
 Gross <agross@...nel.org>,
        Bjorn Andersson <andersson@...nel.org>,
        Konrad
 Dybcio <konrad.dybcio@...aro.org>,
        Yong Wu <yong.wu@...iatek.com>,
        Matthias
 Brugger <matthias.bgg@...il.com>,
        AngeloGioacchino Del Regno
 <angelogioacchino.delregno@...labora.com>,
        Gerald Schaefer
 <gerald.schaefer@...ux.ibm.com>,
        Orson Zhai <orsonzhai@...il.com>,
        Baolin
 Wang <baolin.wang@...ux.alibaba.com>,
        Chunyan Zhang <zhang.lyra@...il.com>, Chen-Yu Tsai <wens@...e.org>,
        Jernej Skrabec <jernej.skrabec@...il.com>,
        Samuel Holland <samuel@...lland.org>,
        Thierry Reding
 <thierry.reding@...il.com>,
        Krishna Reddy <vdumpa@...dia.com>,
        Jonathan
 Hunter <jonathanh@...dia.com>,
        Jonathan Corbet <corbet@....net>, linux-s390@...r.kernel.org,
        netdev@...r.kernel.org, linux-kernel@...r.kernel.org,
        iommu@...ts.linux.dev, asahi@...ts.linux.dev,
        linux-arm-kernel@...ts.infradead.org, linux-arm-msm@...r.kernel.org,
        linux-mediatek@...ts.infradead.org, linux-sunxi@...ts.linux.dev,
        linux-tegra@...r.kernel.org, linux-doc@...r.kernel.org
Subject: Re: [PATCH v11 4/6] iommu/s390: Force ISM devices to use
 IOMMU_DOMAIN_DMA

On Fri, 2023-08-18 at 20:10 +0100, Robin Murphy wrote:
> On 2023-07-17 12:00, Niklas Schnelle wrote:
> > ISM devices are virtual PCI devices used for cross-LPAR communication.
> > Unlike real PCI devices ISM devices do not use the hardware IOMMU but
> > inspects IOMMU translation tables directly on IOTLB flush (s390 RPCIT
> > instruction).
> > 
> > While ISM devices keep their DMA allocations static and only very rarely
> > DMA unmap at all, For each IOTLB flush that occurs after unmap the ISM
> > devices will inspect the area of the IOVA space indicated by the flush.
> > This means that for the global IOTLB flushes used by the flush queue
> > mechanism the entire IOVA space would be inspected. In principle this
> > would be fine, albeit potentially unnecessarily slow, it turns out
> > however that ISM devices are sensitive to seeing IOVA addresses that are
> > currently in use in the IOVA range being flushed. Seeing such in-use
> > IOVA addresses will cause the ISM device to enter an error state and
> > become unusable.
> > 
> > Fix this by forcing IOMMU_DOMAIN_DMA to be used for ISM devices. This
> > makes sure IOTLB flushes only cover IOVAs that have been unmapped and
> > also restricts the range of the IOTLB flush potentially reducing latency
> > spikes.
> 
> Would it not be simpler to return false for IOMMU_CAP_DEFERRED_FLUSH for 
> these devices?
> 
> Cheers,
> Robin.

Nice idea thank you. This is indeed less code, basically just return
zdev->pft != PCI_FUNC_TYPE_ISM for the IOMMU_CAP_DEFERRED_FLUSH check.
I think it's also semantically more clear in that we don't really care
about the domain type but about not getting deferred flushes.

Thanks,
Niklas

> 
> > Signed-off-by: Niklas Schnelle <schnelle@...ux.ibm.com>
> > ---
> >   drivers/iommu/s390-iommu.c | 10 ++++++++++
> >   1 file changed, 10 insertions(+)
> > 
> > diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c
> > index f6d6c60e5634..020cc538e4c4 100644
> > --- a/drivers/iommu/s390-iommu.c
> > +++ b/drivers/iommu/s390-iommu.c
> > @@ -710,6 +710,15 @@ struct zpci_iommu_ctrs *zpci_get_iommu_ctrs(struct zpci_dev *zdev)
> >   	return &zdev->s390_domain->ctrs;
> >   }
> >   
> > +static int s390_iommu_def_domain_type(struct device *dev)
> > +{
> > +	struct zpci_dev *zdev = to_zpci_dev(dev);
> > +
> > +	if (zdev->pft == PCI_FUNC_TYPE_ISM)
> > +		return IOMMU_DOMAIN_DMA;
> > +	return 0;
> > +}
> > +
> >   int zpci_init_iommu(struct zpci_dev *zdev)
> >   {
> >   	u64 aperture_size;
> > @@ -789,6 +798,7 @@ static const struct iommu_ops s390_iommu_ops = {
> >   	.probe_device = s390_iommu_probe_device,
> >   	.probe_finalize = s390_iommu_probe_finalize,
> >   	.release_device = s390_iommu_release_device,
> > +	.def_domain_type = s390_iommu_def_domain_type,
> >   	.device_group = generic_device_group,
> >   	.pgsize_bitmap = SZ_4K,
> >   	.get_resv_regions = s390_iommu_get_resv_regions,
> > 
>