linux-kernel - Re: [PATCH v2 3/5] iommu/amd: Introduce Disable IRTE Caching Support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <875e1afa-6bab-bddd-19d7-9f60b47c64f7@oracle.com>
Date:   Mon, 22 May 2023 13:02:19 -0400
From:   Alejandro Jimenez <alejandro.j.jimenez@...cle.com>
To:     Suravee Suthikulpanit <suravee.suthikulpanit@....com>,
        linux-kernel@...r.kernel.org, iommu@...ts.linux.dev
Cc:     joro@...tes.org, joao.m.martins@...cle.com,
        boris.ostrovsky@...cle.com, jon.grimm@....com,
        santosh.shukla@....com, vasant.hegde@....com,
        kishon.vijayabraham@....com
Subject: Re: [PATCH v2 3/5] iommu/amd: Introduce Disable IRTE Caching Support

Hi Suravee,

On 5/18/2023 8:55 PM, Suravee Suthikulpanit wrote:
> An Interrupt Remapping Table (IRT) stores interrupt remapping configuration
> for each device. In a normal operation, the AMD IOMMU caches the table
> to optimize subsequent data accesses. This requires the IOMMU driver to
> invalidate IRT whenever it updates the table. The invalidation process
> includes issuing an INVALIDATE_INTERRUPT_TABLE command following by
> a COMPLETION_WAIT command.
>
> However, there are cases in which the IRT is updated at a high rate.
> For example, for IOMMU AVIC, the IRTE[IsRun] bit is updated on every
> vcpu scheduling (i.e. amd_iommu_update_ga()). On system with large
> amount of vcpus and VFIO PCI pass-through devices, the invalidation
> process could potentially become a performance bottleneck.
>
> Introducing a new kernel boot option:
>
>      amd_iommu=irtcachedis
>
> which disables IRTE caching by setting the IRTCachedis bit in each IOMMU
> Control register, and bypass the IRT invalidation process.
>
> Co-developed-by: Alejandro Jimenez <alejandro.j.jimenez@...cle.com>
> [Awaiting sign-off by Alejandro]
Signed-off-by: Alejandro Jimenez <alejandro.j.jimenez@...cle.com

I sanity tested the kdump mechanism, and confirmed that 
CONTROL_IRTCACHEDIS is set appropriately based on the irtcachedis kernel 
parameter.

Also, I have not observed any errors during multiple rounds of testing 
with large vCPU counts and VFIO passthrough devices under 
interrupt-intensive workload that causes heavy vCPU scheduling activity 
and exercises the relevant code path.

Thank you,
Alejandro
> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@....com>
> ---
>   .../admin-guide/kernel-parameters.txt         |  1 +
>   drivers/iommu/amd/amd_iommu_types.h           |  4 +++
>   drivers/iommu/amd/init.c                      | 36 +++++++++++++++++++
>   3 files changed, 41 insertions(+)
>
>