linux-kernel - Re: [PATCH v1 4/5] iommu/arm-smmu-v3: Keep track of attached ssids

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAKHBV25kgZFdO=w7tDJ6wTSXgY_Dx67ec3A5yTZdFTu_8PTC-g@mail.gmail.com>
Date:   Tue, 23 May 2023 15:57:48 +0800
From:   Michael Shavit <mshavit@...gle.com>
To:     Jean-Philippe Brucker <jean-philippe@...aro.org>
Cc:     Jason Gunthorpe <jgg@...dia.com>, Will Deacon <will@...nel.org>,
        Robin Murphy <robin.murphy@....com>,
        Joerg Roedel <joro@...tes.org>, nicolinc@...dia.com,
        baolu.lu@...ux.intel.com, linux-arm-kernel@...ts.infradead.org,
        iommu@...ts.linux.dev, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v1 4/5] iommu/arm-smmu-v3: Keep track of attached ssids

Oh nice, this is exactly what I was looking for (minus the missing ATC
inv, but that's somewhat easier to reason from code)! Thanks for the
detailed guide Jean!
I finally got around to trying it out and was able to see the page
fault followed by invalidations on this patch-series as it is. This
will be super useful to start refactoring SVA for a v2 of this patch.

On Fri, May 12, 2023 at 3:59 AM Jean-Philippe Brucker
<jean-philippe@...aro.org> wrote:
>
> On Thu, May 11, 2023 at 11:26:48PM +0800, Michael Shavit wrote:
> > > You should be getting rid of mm->pasid in this series as well.
> > >
> > > When each domain keeps track of what STE/CD entries that point to it then
> > > *ALL* invalidation should iterate over the list of pointing entires
> > > and generate the correct invalidation for that pointer.
> > >
> >
> > Completely agree. The arm_smmu_atc_inv_domain_ssid function introduced
> > by this patch is a stopgap to decompose this patch from the SVA
> > refactor that's required to stop using ssid in these calls.
> > I also agree that such a refactoring probably belongs in the same
> > patch series. @Jean-Philippe Brucker and others: is there any way I
> > can about testing or at least exercising the SVA flow without physical
> > hardware that supports SVA?
>
> Yes, there is a model with a simple test device that supports PASID and
> I/O page faults. It's not completely straightforward to setup and the
> driver needs to be rewritten from scratch, but it's the best we have at
> the moment.  I'd like to do something equally useful for QEMU, so we can
> have proper regression tests, but that requires a lot of preliminary work
> to add PASID+PRI to PCI, virtio and IOMMUs.
>
> You'll need a kernel with the driver and a rootfs with the smmute
> tool [1]; the RevC model [2] and a boot-wrapper [3].
>
>   $ ${BOOTWRAPPER}/configure --host=aarch64-linux-gnu
>      --with-dtb=${KERNEL}/arch/arm64/boot/dts/arm/fvp-base-revc.dts
>      --with-kernel-dir=${KERNEL}
>      --with-initrd=${BUILDROOT}/images/rootfs.cpio
>      --with-cmdline="console=ttyAMA0"
>      --enable-psci --enable-gicv3
>   $ make        # produces linux-system.axf
>
> Run the model:
>   $ FVP_Base_RevC-2xAEMvA
>      -C bp.secure_memory=0
>      -C 'pctl.startup=0.*.*.*'
>      -C bp.refcounter.non_arch_start_at_default=1
>      -C cache_state_modelled=0
>      -C bp.vis.disable_visualisation=1
>      -C bp.virtio_net.enabled=1
>      -C bp.virtio_net.hostbridge.userNetPorts=8022=22
>      -C bp.virtio_net.hostbridge.userNetworking=1
>      -C pci.pci_smmuv3.mmu.SMMU_IDR0=135100351
>      -C pci.pci_smmuv3.mmu.SMMU_IDR3=4116
>      -C pci.pci_smmuv3.mmu.SMMU_IDR5=8389749
>      -C cluster0.NUM_CORES=4
>      -C cluster1.NUM_CORES=4
>      -a 'cluster*.cpu*=linux-system.axf'
>
> Then run a job using the tool. The process allocates two buffers and
> passes their VA to the device (via the kernel driver). The device memcpies
> one buffer to the other:
>
>    $ smmute -u mmap
>    ... Success
>
> With smmu and iommu trace events enabled, a trace should contain
> smmu_mm_invalidate and dev_fault/dev_page_response events.
>
> It's not entirely representative of SVA flow, where an assignable device
> interface is mapped into the process and the process launches jobs
> directly without going through the kernel (that would now use
> drivers/misc/uacce), but it does exercise IOMMU SVA: sva_bind(), device
> accessing the process address space with PASID and some IOPFs, which I
> think is what you're looking for. However this model doesn't have a PCI
> test device so you won't be able to test ATC invalidations with PASID.
>
> Other useful tests would be enabling lockdep (some intricate locking
> between IOPF, the driver and mm), killing bound processes (-k), triggering
> invalid accesses to verify TLB invalidations (-f tlb, I think). There is a
> lot more to test, like thp and oom, but I don't have those in this branch.
>
> Thanks,
> Jean
>
> [1] https://jpbrucker.net/git/linux/log/?h=sva/smmute-revc
> [2] https://developer.arm.com/downloads/-/arm-ecosystem-models
> [3] https://git.kernel.org/pub/scm/linux/kernel/git/mark/boot-wrapper-aarch64.git/