linux-kernel - Re: [PATCH v2] iommu/arm-smmu-v3: Add a user-configurable tlb_invalidate

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <ZN+n+lwM655TVeJp@Asurada-Nvidia>
Date:   Fri, 18 Aug 2023 10:18:50 -0700
From:   Nicolin Chen <nicolinc@...dia.com>
To:     Will Deacon <will@...nel.org>
CC:     <robin.murphy@....com>, <jgg@...dia.com>, <joro@...tes.org>,
        <jean-philippe@...aro.org>, <apopple@...dia.com>,
        <linux-kernel@...r.kernel.org>,
        <linux-arm-kernel@...ts.infradead.org>, <iommu@...ts.linux.dev>
Subject: Re: [PATCH v2] iommu/arm-smmu-v3: Add a user-configurable
 tlb_invalidate_threshold

On Fri, Aug 18, 2023 at 05:11:19PM +0100, Will Deacon wrote:
 
> On Thu, Aug 17, 2023 at 11:36:18AM -0700, Nicolin Chen wrote:
> > On Wed, Aug 16, 2023 at 01:43:50PM -0700, Nicolin Chen wrote:
> >
> > > When receiving an __arm_smmu_tlb_inv_range() call with a large size, there
> > > could be a long latency at this function call: one part is coming from a
> > > large software overhead in the routine of building commands, and the other
> > > part is coming from CMDQ hardware consuming the large number of commands.
> > > This latency could be significantly large on an SMMU that does not support
> > > range invalidation commands, i.e. no ARM_SMMU_FEAT_RANGE_INV.
> > >
> > > One way to optimize this is to replace a large number of VA invalidation
> > > commands with one single per-asid invalidation command, when the requested
> > > size reaches a threshold. This threshold can be configurable depending on
> > > the SMMU implementaion.
> >
> > I'm rethinking about this size-based threshold, since what really
> > affects the latency is the number of the invalidation commands in
> > the request. So having an npages-based threshold might be optimal,
> > though the idea and implementation would be similar.
> 
> On the CPU side, we just have:
> 
> #define MAX_TLBI_OPS    PTRS_PER_PTE
> 
> in asm/tlbflush.h
> 
> Can we start off with something similar for the SMMU? I'm not massively
> keen on exposing this as a knob to userspace, because I don't think most
> people will have a clue about how to tune it.

Yes! I was hesitating about an arbitrary threshold setup that actually
fits our situation better. Now it makes sense. Thanks for the input!

What I would do was to pick a number based on our test results. Yet, it
seems that 1024 was chosen at the beginning to fix a softlock bug, and
it changed to PTRS_PER_PTE for a better perf, IIUIC. Should we use the
similar setup for SMMU? I found it is stored in data->bits_per_level,
so perhaps pass it back to the driver via "struct io_pgtable_cfg".

Nicolin