linux-kernel - Re: [PATCH v2 08/13] x86/irq: Install posted MSI notification handler

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240411103847.57f47a48@jacob-builder>
Date: Thu, 11 Apr 2024 10:38:47 -0700
From: Jacob Pan <jacob.jun.pan@...ux.intel.com>
To: "Tian, Kevin" <kevin.tian@...el.com>
Cc: LKML <linux-kernel@...r.kernel.org>, X86 Kernel <x86@...nel.org>, Peter
 Zijlstra <peterz@...radead.org>, "iommu@...ts.linux.dev"
 <iommu@...ts.linux.dev>, Thomas Gleixner <tglx@...utronix.de>, Lu Baolu
 <baolu.lu@...ux.intel.com>, "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
 "Hansen, Dave" <dave.hansen@...el.com>, Joerg Roedel <joro@...tes.org>, "H.
 Peter Anvin" <hpa@...or.com>, Borislav Petkov <bp@...en8.de>, Ingo Molnar
 <mingo@...hat.com>, "Luse, Paul E" <paul.e.luse@...el.com>, "Williams, Dan
 J" <dan.j.williams@...el.com>, Jens Axboe <axboe@...nel.dk>, "Raj, Ashok"
 <ashok.raj@...el.com>, "maz@...nel.org" <maz@...nel.org>,
 "seanjc@...gle.com" <seanjc@...gle.com>, Robin Murphy
 <robin.murphy@....com>, "jim.harris@...sung.com" <jim.harris@...sung.com>,
 "a.manzanares@...sung.com" <a.manzanares@...sung.com>, Bjorn Helgaas
 <helgaas@...nel.org>, "Zeng, Guang" <guang.zeng@...el.com>,
 "robert.hoo.linux@...il.com" <robert.hoo.linux@...il.com>,
 jacob.jun.pan@...ux.intel.com
Subject: Re: [PATCH v2 08/13] x86/irq: Install posted MSI notification
 handler

Hi Kevin,

On Thu, 11 Apr 2024 07:52:28 +0000, "Tian, Kevin" <kevin.tian@...el.com>
wrote:

> > From: Jacob Pan <jacob.jun.pan@...ux.intel.com>
> > Sent: Saturday, April 6, 2024 6:31 AM
> > 
> > +
> > +/*
> > + * De-multiplexing posted interrupts is on the performance path, the
> > code
> > + * below is written to optimize the cache performance based on the
> > following
> > + * considerations:
> > + * 1.Posted interrupt descriptor (PID) fits in a cache line that is
> > frequently
> > + *   accessed by both CPU and IOMMU.
> > + * 2.During posted MSI processing, the CPU needs to do 64-bit read and
> > xchg
> > + *   for checking and clearing posted interrupt request (PIR), a 256
> > bit field
> > + *   within the PID.
> > + * 3.On the other side, the IOMMU does atomic swaps of the entire PID
> > cache
> > + *   line when posting interrupts and setting control bits.
> > + * 4.The CPU can access the cache line a magnitude faster than the
> > IOMMU.
> > + * 5.Each time the IOMMU does interrupt posting to the PIR will evict
> > the PID
> > + *   cache line. The cache line states after each operation are as
> > follows:
> > + *   CPU		IOMMU			PID Cache line
> > state
> > + *   ---------------------------------------------------------------
> > + *...read64					exclusive
> > + *...lock xchg64				modified
> > + *...			post/atomic swap	invalid
> > + *...-------------------------------------------------------------
> > + *  
> 
> According to VT-d spec: 5.2.3 Interrupt-Posting Hardware Operation:
> 
> "
> - Read contents of the Posted Interrupt Descriptor, claiming exclusive
>   ownership of its hosting cache-line.
>   ...
> - Modify the following descriptor field values atomically:
>   ...
> - Promote the cache-line to be globally observable, so that the
> modifications are visible to other caching agents. Hardware may
> write-back the cache-line anytime after this step.
> "
> 
> sounds that the PID cache line is not evicted after IOMMU posting?
IOMMU follows the same MESI protocol defined in SDM. VT-d spec. also says
"This atomic read-modify-write operation will always snoop processor caches"

So if the PID cache line is in modified state (caused by writing ON bit,
clear PIR, etc.), IOMMU request ownership will evict the cache.

Thanks,

Jacob