Message-ID: <alpine.DEB.2.22.394.2506271219070.862517@ubuntu-linux-20-04-desktop>
Date: Fri, 27 Jun 2025 12:20:37 -0700
From: Stefano Stabellini <stefano.stabellini@....com>
To: Robin Murphy <robin.murphy@....com>
CC: Michal Simek <michal.simek@....com>, Will Deacon <will@...nel.org>,
	"Stabellini, Stefano" <stefano.stabellini@....com>, linux-arm-kernel
	<linux-arm-kernel@...ts.infradead.org>, <iommu@...ts.linux.dev>, "Linux
 Kernel Mailing List" <linux-kernel@...r.kernel.org>, "Sarangi, Anirudha"
	<anirudha.sarangi@....com>
Subject: Re: SMMUv3 interrupt handling via custom logic

Hi Will, Robin,

Long time no see!

On Fri, 27 Jun 2025, Robin Murphy wrote:
> On 2025-06-27 8:19 am, Michal Simek wrote:
> > Hi Will and Robin, (+Stefano, Anirudha)
> > 
> > We are using smmu-v3 in our SOC and I would like to ask you for
> > recommendation how to handle our interrupt cases.
> > 
> > here is description which we are using
> > 
> > smmu: iommu@...00000 {
> >      compatible = "arm,smmu-v3";
> >      reg = <...>;
> >      #iommu-cells = <1>;
> >      interrupt-names = "combined";
> >      interrupts = <0 169 4>;
> > };
> > 
> > but it is missing one important detail which just arise that actually there
> > is additional HW logic which deals with SMMU interrupts separately.
> > There is a secure part (global, cmd, event - gerror, cmdq-sync, eventq in
> > DT)
> > and non secure part (pri, global, cmd, event - priq, gerror, cmdq-sync,
> > eventq in DT).
> > Based on my information all these interrupts should be acked once handled to
> > be able to get another one.
> > The driver itself is able to handle them separately but we didn't create any
> > solution to reach custom HW to do it.
> > 
> > I looked at f935448acf46 ("iommu/arm-smmu-v3: Add workaround for Cavium
> > ThunderX2 erratum #126") which introduced combined IRQs but it looks like
> > that there is no need for additional ACK of that IRQs.
> 
> Per the architecture, SMMU interrupts are logically edge-triggered so there is
> nothing to clear at the SMMU end (the "interrupt status" is implicit in
> whatever condition caused an interrupt to be sent, e.g. the event queue
> becoming non-empty, SMMU_GERROR becoming different from SMMU_GERRORN, etc.)
> 
> If this is an Arm SMMU IP (MMU-600/700/S3) then the physical interrupt outputs
> are most definitely rising-edge. If somone's stuck some interrupt combiner in
> between those and the main interrupt controller, then yes, that interrupt
> combiner really should have its own driver.
> 
> > The HW logic itself is handling secure and non secure settings for SMMU
> > that's why would be the best to avoid directly mapping it in Linux.
> > 
> > One way to go is to create secondary interrupt controller driver
> > a) ioremap one with notice about secure part because we are using SMMU only
> > with NS world
> > b) firmware based to tunnel accesses via SMCs and allow only access to
> > limited amount of registers
> > 
> > The second way is likely create any hooks in the driver to be able to
> > provide additional SOC specific hooks.
> 
> If this thing is munging *all* the SMMU interrupt outputs as I suspect, then
> the big problem with that idea is that "the driver" is at least two separate
> drivers (SMMU and PMU), 3 if it has RAS and you ever want to entertain the
> idea of kernel-first handling.

Yeah... I tend to favor simple solutions when possible and the secondary
interrupt controller driver approach is looking increasingly complex.

In addition to what Robin mentioned, I understand that this email is
directed to linux-arm-kernel and the LKML, so the focus is naturally to
solve the problem for Linux. However, let me point out that this issue
also affects Xen, all hypervisors, and proprietary operating systems.
The "big problem" is even bigger :-(  Complexity will multiply very
quickly.

Do you have any creative ideas for how this could be handled more
simply?  Maybe with help from the firmware (e.g. TF-A)?

Powered by blists - more mailing lists