[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <e1efbbd827e34800bd7fb0ea687645cc6c65e1ab.camel@linux.ibm.com>
Date: Wed, 27 Sep 2023 15:20:07 +0200
From: Niklas Schnelle <schnelle@...ux.ibm.com>
To: Joerg Roedel <joro@...tes.org>
Cc: Jason Gunthorpe <jgg@...pe.ca>,
Matthew Rosato <mjrosato@...ux.ibm.com>,
Will Deacon <will@...nel.org>,
Wenjia Zhang <wenjia@...ux.ibm.com>,
Robin Murphy <robin.murphy@....com>,
Gerd Bayer <gbayer@...ux.ibm.com>,
Julian Ruess <julianr@...ux.ibm.com>,
Pierre Morel <pmorel@...ux.ibm.com>,
Alexandra Winter <wintera@...ux.ibm.com>,
Heiko Carstens <hca@...ux.ibm.com>,
Vasily Gorbik <gor@...ux.ibm.com>,
Alexander Gordeev <agordeev@...ux.ibm.com>,
Christian Borntraeger <borntraeger@...ux.ibm.com>,
Sven Schnelle <svens@...ux.ibm.com>,
Suravee Suthikulpanit <suravee.suthikulpanit@....com>,
Hector Martin <marcan@...can.st>,
Sven Peter <sven@...npeter.dev>,
Alyssa Rosenzweig <alyssa@...enzweig.io>,
David Woodhouse <dwmw2@...radead.org>,
Lu Baolu <baolu.lu@...ux.intel.com>,
Andy Gross <agross@...nel.org>,
Bjorn Andersson <andersson@...nel.org>,
Konrad Dybcio <konrad.dybcio@...aro.org>,
Yong Wu <yong.wu@...iatek.com>,
Matthias Brugger <matthias.bgg@...il.com>,
AngeloGioacchino Del Regno
<angelogioacchino.delregno@...labora.com>,
Gerald Schaefer <gerald.schaefer@...ux.ibm.com>,
Orson Zhai <orsonzhai@...il.com>,
Baolin Wang <baolin.wang@...ux.alibaba.com>,
Chunyan Zhang <zhang.lyra@...il.com>,
Chen-Yu Tsai <wens@...e.org>,
Jernej Skrabec <jernej.skrabec@...il.com>,
Samuel Holland <samuel@...lland.org>,
Thierry Reding <thierry.reding@...il.com>,
Krishna Reddy <vdumpa@...dia.com>,
Jonathan Hunter <jonathanh@...dia.com>,
Jonathan Corbet <corbet@....net>, linux-s390@...r.kernel.org,
netdev@...r.kernel.org, linux-kernel@...r.kernel.org,
iommu@...ts.linux.dev, asahi@...ts.linux.dev,
linux-arm-kernel@...ts.infradead.org,
linux-arm-msm@...r.kernel.org, linux-mediatek@...ts.infradead.org,
linux-sunxi@...ts.linux.dev, linux-tegra@...r.kernel.org,
linux-doc@...r.kernel.org
Subject: Re: [PATCH v12 0/6] iommu/dma: s390 DMA API conversion and
optimized IOTLB flushing
On Wed, 2023-09-27 at 13:24 +0200, Niklas Schnelle wrote:
> On Wed, 2023-09-27 at 11:55 +0200, Joerg Roedel wrote:
> > Hi Niklas,
> >
> > On Wed, Sep 27, 2023 at 10:55:23AM +0200, Niklas Schnelle wrote:
> > > The problem is that something seems to be broken in the iommu/core
> > > branch. Regardless of whether I have my DMA API conversion on top or
> > > with the base iommu/core branch I can not use ConnectX-4 VFs.
> >
> > Have you already tried to bisect the issue in the iommu/core branch?
> > The result might sched some light on the issue.
> >
> > Regards,
> >
> > Joerg
>
> Hi Joerg,
>
> Working on it, somehow I must have messed up earlier. It now looks like
> it might in fact be caused by my DMA API conversion rebase and the
> "s390/pci: Use dma-iommu layer" commit. Maybe there is some interaction
> with Jason's patches that I haven't thought about. So sorry for any
> wrong blame.
>
> Thanks,
> Niklas
Hi,
I tracked the problem down from mlx5_core's alloc_cmd_page() via
dma_alloc_coherent(), ops->alloc, iommu_dma_alloc_remap(), and
__iommu_dma_alloc_noncontiguous() to a failed iommu_dma_alloc_iova().
The allocation here is for 4K so nothing crazy.
On second look I also noticed:
nvme 2007:00:00.0: Using 42-bit DMA addresses
for the NVMe that is working. The problem here seems to be that we set
iommu_dma_forcedac = true in s390_iommu_probe_finalize() because we
have currently have a reserved region over the first 4 GiB anyway so
will always use IOVAs larger than that. That however is too late since
iommu_dma_set_pci_32bit_workaround() is already checked in
__iommu_probe_device() which is called just before ops-
>probe_finalize(). So I moved setting iommu_dma_forcedac = true to
zpci_init_iommu() and that gets rid of the notice for the NVMe but I
still get a failure of iommu_dma_alloc_iova() in
__iommu_dma_alloc_noncontiguous(). So I'll keep digging.
Thanks,
Niklas
Powered by blists - more mailing lists