linux-kernel - Re: [PATCH v8 15/16] x86/virt/tdx: Flush cache in kexec() when TDX is enabled

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <b84232220d03889321248ffb82739c64204cc4af.camel@intel.com>
Date:   Wed, 11 Jan 2023 00:13:45 +0000
From:   "Huang, Kai" <kai.huang@...el.com>
To:     "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        "Hansen, Dave" <dave.hansen@...el.com>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
CC:     "Luck, Tony" <tony.luck@...el.com>,
        "bagasdotme@...il.com" <bagasdotme@...il.com>,
        "ak@...ux.intel.com" <ak@...ux.intel.com>,
        "Wysocki, Rafael J" <rafael.j.wysocki@...el.com>,
        "kirill.shutemov@...ux.intel.com" <kirill.shutemov@...ux.intel.com>,
        "Christopherson,, Sean" <seanjc@...gle.com>,
        "Chatre, Reinette" <reinette.chatre@...el.com>,
        "pbonzini@...hat.com" <pbonzini@...hat.com>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "Yamahata, Isaku" <isaku.yamahata@...el.com>,
        "tglx@...utronix.de" <tglx@...utronix.de>,
        "Shahar, Sagi" <sagis@...gle.com>,
        "imammedo@...hat.com" <imammedo@...hat.com>,
        "Gao, Chao" <chao.gao@...el.com>,
        "Brown, Len" <len.brown@...el.com>,
        "peterz@...radead.org" <peterz@...radead.org>,
        "sathyanarayanan.kuppuswamy@...ux.intel.com" 
        <sathyanarayanan.kuppuswamy@...ux.intel.com>,
        "Huang, Ying" <ying.huang@...el.com>,
        "Williams, Dan J" <dan.j.williams@...el.com>
Subject: Re: [PATCH v8 15/16] x86/virt/tdx: Flush cache in kexec() when TDX is
 enabled

On Tue, 2023-01-10 at 07:27 -0800, Dave Hansen wrote:
> On 1/10/23 03:29, Huang, Kai wrote:
> > On Fri, 2023-01-06 at 16:35 -0800, Dave Hansen wrote:
> > > On 12/8/22 22:52, Kai Huang wrote:
> ...
> > > > However, this implementation doesn't convert TDX private pages back to
> > > > normal in kexec() because of below considerations:
> > > > 
> > > > 1) Neither the kernel nor the TDX module has existing infrastructure to
> > > >    track which pages are TDX private pages.
> > > > 2) The number of TDX private pages can be large, and converting all of
> > > >    them (cache flush + using MOVDIR64B to clear the page) in kexec() can
> > > >    be time consuming.
> > > > 3) The new kernel will almost only use KeyID 0 to access memory.  KeyID
> > > >    0 doesn't support integrity-check, so it's OK.
> > > > 4) The kernel doesn't (and may never) support MKTME.  If any 3rd party
> > > >    kernel ever supports MKTME, it can/should do MOVDIR64B to clear the
> > > >    page with the new MKTME KeyID (just like TDX does) before using it.
> > > 
> > > Yeah, why are we getting all worked up about MKTME when there is not
> > > support?
> > 
> > I am not sure whether we need to consider 3rd party kernel case?
> 
> No, we don't.

Good to know.

> 
> > > The only thing that matters here is dirty cacheline writeback.  There
> > > are two things the kernel needs to do to mitigate that:
> > > 
> > >  1. Stop accessing TDX private memory mappings
> > >   1a. Stop making TDX module calls (uses global private KeyID)
> > >   1b. Stop TDX guests from running (uses per-guest KeyID)
> > >  2. Flush any cachelines from previous private KeyID writes
> > > 
> > > There are a couple of ways we can do #2.  We do *NOT* need to convert
> > > *ANYTHING* back to KeyID 0.  Page conversion doesn't even come into play
> > > in any way as far as I can tell.
> > 
> > May I ask why?  When I was writing this patch I was not sure whether kexec()
> > should give the new kernel a clean slate.  SGX driver doesn't EREMOVE all EPC
> > during kexec() but depends on the new kernel to do that too, but I don't know
> > what's the general guide of supporting kexec().
> 
> Think about it this way: kexec() is modifying persistent (across kexec)
> state to get the system ready for the new kernel.  The caches are
> persistent state.  Devices have persistent state.  Memory state persists
> across kexec().  The memory integrity metadata persists.
> 
> What persistent state does a conversion to KeyID-0 affect?  It resets
> the integrity metadata and the memory contents.
> 
> Kexec leaves memory contents in place and doesn't zero them, so memory
> contents don't matter.  The integrity metadata also doesn't matter
> because the memory will be used as KeyID-0 and that KeyID doesn't read
> the integrity metadata.

Right.  So I guess we just need to call out the new kernel will use memory as
KeyID-0?

> 
> What practical impact does a conversion back to KeyID-0 serve?  What
> persistent state does it affect that matters?

If we can be sure the new kernel will use KeyID-0, then we don't need to
convert.  In the 3) and 4) in my changelog, I actually was trying to convery
this.
  
> 
> > > I think you're also saying that since all CPUs go through this path and
> > > there is no TDX activity between the WBINVD and the native_halt() that
> > > 1a and 1b basically happen for "free" without needing to do theme
> > > explicitly.
> > 
> > Yes.  Should we mention this part in changelog?
> 
> That would be nice.
> 

Will do.