Message-ID: <b937cac0-4a48-4991-9958-0c6010887a1a@amd.com>
Date: Mon, 10 Nov 2025 15:25:40 +0530
From: Vasant Hegde <vasant.hegde@....com>
To: Tom Lendacky <thomas.lendacky@....com>, Jason Gunthorpe <jgg@...dia.com>
Cc: Wei Wang <wei.w.wang@...mail.com>, "alex@...zbot.org" <alex@...zbot.org>,
 "suravee.suthikulpanit@....com" <suravee.suthikulpanit@....com>,
 "joro@...tes.org" <joro@...tes.org>,
 "kevin.tian@...el.com" <kevin.tian@...el.com>,
 "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
 "iommu@...ts.linux.dev" <iommu@...ts.linux.dev>,
 Alexey Kardashevskiy <aik@....com>
Subject: Re: [PATCH v2 2/2] vfio/type1: Set IOMMU_MMIO in dma->prot for
 MMIO-backed addresses

On 11/8/2025 1:29 AM, Tom Lendacky wrote:
> On 11/7/25 12:32, Jason Gunthorpe wrote:
>> On Fri, Nov 07, 2025 at 11:56:51AM -0600, Tom Lendacky wrote:
>>
>>> When you are on bare-metal, or in the hypervisor, System Memory Encryption
>>> (SME) deals with the encryption bit set in the page table entries
>>> (including the nested page table entries for guests). 
>>
>> So "decrypted" means something about AMD's unique memory encryption
>> scheme on bare metal but in a CC guest it is a cross arch 'shared with
>> hypervisor' flag?
> 
> Note, that if the encryption bit is not set in the guest, then the host
> encryption key is used if the underlying NPT leaf entry has the encryption
> bit set. In that case, both the host and guest can read the memory, with
> the memory still being encrypted in physical memory.
> 
>>
>> What about CXL memory? What about ZONE_DEVICE coherent memory? Do
>> these get the C bit set too?
> 
> When CXL memory is presented as system memory to the OS it does support
> the encryption bit. So when pages are allocated for the guest, the memory
> pages will be encrypted with the guest key.
> 
> Not sure what you mean by ZONE_DEVICE coherent memory. Is it presented to
> the system as system physical memory that the hypervisor can allocate as
> guest memory?
> 
>>
>> :( :( :(
>>
>>> In the guest (prior to Trusted I/O / TDISP), decrypted (or shared) memory
>>> is used because a device cannot DMA to or from guest memory using the
>>> guest encryption key. So all DMA must go to "decrypted" memory or be
>>> bounce-buffered through "decrypted" memory (SWIOTLB) - basically memory
>>> that does not get encrypted/decrypted using the guest encryption key.
>>
>> Where is the code for this? As I wrote we always do sme_set in the
>> iommu driver, even on guests, even for "decrypted" bounce buffered
>> memory.
>>
>> That sounds like a bug by your explanation?
>>
>> Does this mean vIOMMU has never worked in AMD CC guests?
> 
> I assume by vIOMMU you mean a VMM-emulated IOMMU in the guest. This does
> does not work today with AMD CC guests since it requires the hypervisor to
> read the guest IOMMU buffers in order to emulate the behavior and those
> buffers are encrypted. So there is no vIOMMU support today in AMD CC
> guests.
> 
> There was a patch series submitted a while back to allocate the IOMMU
> buffers in shared memory in order to support a (non-secure) vIOMMU in the
> guest in order to support >255 vCPUs, but that was rejected in favor of
> using kvm-msi-ext-dest-id.
> 
> https://lore.kernel.org/linux-iommu/20240430152430.4245-1-suravee.suthikulpanit@amd.com/
> 
>>
>>> It is not until we get to Trusted I/O / TDISP where devices will be able
>>> to DMA directly to guest encrypted memory and guests will require secure
>>> MMIO addresses which will need the encryption bit set (Alexey can correct
>>> me on the TIO statements if they aren't correct, as he is closer to it all).
>>
>> So in this case we do need to do sme_set on MMIO even though that MMIO
>> is not using the dram encryption key?

Yes. Its mapped to GPA (at least IOMMU VF MMIO BAR, I believe its same for TIO
device as well) and we need to set 'C' bit.

-Vasan



> 
> @Alexey will be able to provide more details on how this works.
> 
> Thanks,
> Tom
> 
>>
>> Jason
> 

Powered by blists - more mailing lists