linux-kernel - Re: [PATCH v8] x86/sgx: Maintain encl->refcount for each encl->mm

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YHcPfRqKCx+KXwoJ@google.com>
Date:   Wed, 14 Apr 2021 15:51:25 +0000
From:   Sean Christopherson <seanjc@...gle.com>
To:     Haitao Huang <haitao.huang@...ux.intel.com>
Cc:     linux-sgx@...r.kernel.org, Jarkko Sakkinen <jarkko@...nel.org>,
        dave.hansen@...el.com, Dave Hansen <dave.hansen@...ux.intel.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
        x86@...nel.org, "H. Peter Anvin" <hpa@...or.com>,
        Jethro Beekman <jethro@...tanix.com>,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH v8] x86/sgx: Maintain encl->refcount for each
 encl->mm_list entry

On Tue, Apr 13, 2021, Haitao Huang wrote:
> On Sun, 07 Feb 2021 16:14:01 -0600, Jarkko Sakkinen <jarkko@...nel.org>
> wrote:
> 
> > This has been shown in tests:
> > 
> > [  +0.000008] WARNING: CPU: 3 PID: 7620 at kernel/rcu/srcutree.c:374
> > cleanup_srcu_struct+0xed/0x100
> > 
> > This is essentially a use-after free, although SRCU notices it as
> > an SRCU cleanup in an invalid context.
> > 
> The comments in code around this warning indicate a potential memory leak.
> Not sure how use-after-free come into play. Anyway, this fix seems to work
> for the warning above.
> 
> However, I still have doubts on another potential race. See below.
> 
> 
> > diff --git a/arch/x86/kernel/cpu/sgx/driver.c
> > b/arch/x86/kernel/cpu/sgx/driver.c
> > index f2eac41bb4ff..8ce6d8371cfb 100644
> > --- a/arch/x86/kernel/cpu/sgx/driver.c
> > +++ b/arch/x86/kernel/cpu/sgx/driver.c
> > @@ -72,6 +72,9 @@ static int sgx_release(struct inode *inode, struct
> > file *file)
> >  		synchronize_srcu(&encl->srcu);
> >  		mmu_notifier_unregister(&encl_mm->mmu_notifier, encl_mm->mm);
> >  		kfree(encl_mm);
> 
> Note here you are freeing the encl_mm, outside protection of encl->refcount.
> 
> > +
> > +		/* 'encl_mm' is gone, put encl_mm->encl reference: */
> > +		kref_put(&encl->refcount, sgx_encl_release);
> >  	}
> > 	kref_put(&encl->refcount, sgx_encl_release);
> > diff --git a/arch/x86/kernel/cpu/sgx/encl.c
> > b/arch/x86/kernel/cpu/sgx/encl.c
> > index 20a2dd5ba2b4..7449ef33f081 100644
> > --- a/arch/x86/kernel/cpu/sgx/encl.c
> > +++ b/arch/x86/kernel/cpu/sgx/encl.c
> > @@ -473,6 +473,9 @@ static void sgx_mmu_notifier_free(struct
> > mmu_notifier *mn)
> >  {
> >  	struct sgx_encl_mm *encl_mm = container_of(mn, struct sgx_encl_mm,
> > mmu_notifier);
> > +	/* 'encl_mm' is going away, put encl_mm->encl reference: */
> > +	kref_put(&encl_mm->encl->refcount, sgx_encl_release);
> > +
> >  	kfree(encl_mm);
> 
> Could this access to and kfree of encl_mm possibly be after the
> kfree(encl_mm) noted above?

No, the mmu_notifier_unregister() ensures that all in-progress notifiers complete
before it returns, i.e. SGX's notifier call back is not reachable after it's
unregistered.

> Also is there a reason we do kfree(encl_mm) in notifier_free not directly in
> notifier_release?

Because encl_mm is the anchor to the enclave reference

	/* 'encl_mm' is going away, put encl_mm->encl reference: */
	kref_put(&encl_mm->encl->refcount, sgx_encl_release);

as well as the mmu notifier reference (the mmu_notifier_put(mn) call chain).
Freeing encl_mm immediately would prevent sgx_mmu_notifier_free() from dropping
the enclave reference.  And the mmu notifier reference need to be dropped in
sgx_mmu_notifier_release() because the encl_mm has been taken off encl->mm_list.