linux-kernel - Re: [PATCH v2 1/3] x86/mce: Avoid infinite loop for copy from user recovery

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20210820185945.GA1623421@agluck-desk2.amr.corp.intel.com>
Date:   Fri, 20 Aug 2021 11:59:45 -0700
From:   "Luck, Tony" <tony.luck@...el.com>
To:     Borislav Petkov <bp@...en8.de>
Cc:     Jue Wang <juew@...gle.com>, Ding Hui <dinghui@...gfor.com.cn>,
        naoya.horiguchi@....com, osalvador@...e.de,
        Youquan Song <youquan.song@...el.com>, huangcun@...gfor.com.cn,
        x86@...nel.org, linux-edac@...r.kernel.org, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH v2 1/3] x86/mce: Avoid infinite loop for copy from user
 recovery

On Fri, Aug 20, 2021 at 07:31:43PM +0200, Borislav Petkov wrote:
> On Tue, Aug 17, 2021 at 05:29:40PM -0700, Tony Luck wrote:
> > +	/* Ten is likley overkill. Don't expect more than two faults before task_work() */
> 
> "likely"

Oops.

> 
> > +	if (count > 10)
> > +		mce_panic("Too many machine checks while accessing user data", m, msg);
> 
> Ok, aren't we too nasty here? Why should we panic the whole box even
> with 10 MCEs? It is still user memory...
> 
> IOW, why not:
> 
> 	if (count > 10)
> 		current->mce_kill_me.func = kill_me_now;
> 
> and when we return, that user process dies immediately.

It's the "when we return" part that is the problem here. Logical
trace looks like:

user-syscall:

	kernel does get_user() or copyin(), hits user poison address

		machine check
		sees that this was kernel get_user()/copyin() and
		uses extable to "return" to exception path

	still in kernel, see that get_user() or copyin() failed

	Kernel does another get_user() or copyin() (maybe the first
	was inside a pagefault_disable() region, and kernel is trying
	again to see if the error was a fixable page fault. But that
	wasn't the problem so ...

		machine check
		sees that this was kernel get_user()/copyin() and
		uses extable to "return" to exception path

	still in kernel ... but persistently thinks that just trying again
	might fix it.

		machine check
		sees that this was kernel get_user()/copyin() and
		uses extable to "return" to exception path

	still in kernel ... this time for sure! get_user()

		machine check
		sees that this was kernel get_user()/copyin() and
		uses extable to "return" to exception path

	still in kernel ... but you may see the pattern get_user()

		machine check
		sees that this was kernel get_user()/copyin() and
		uses extable to "return" to exception path

	I'm bored typing this, but the kernel may not ever give up

		machine check
		sees that this was kernel get_user()/copyin() and
		uses extable to "return" to exception path

I.e. the kernel doesn't ever get to call current->mce_kill_me.func()

I do have tests that show as many as 4 consecutive machine checks
before the kernel gives up trying and returns to the user to complete
recovery.

Maybe the message could be clearer?

	mce_panic("Too many consecutive machine checks in kernel while accessing user data", m, msg);

> 
> > +	/* Second or later call, make sure page address matches the one from first call */
> > +	if (count > 1 && (current->mce_addr >> PAGE_SHIFT) != (m->addr >> PAGE_SHIFT))
> > +		mce_panic("Machine checks to different user pages", m, msg);
> 
> Same question here.

Not quite the same answer ... but similar.  We could in theory handle
multiple different machine check addresses by turning the "mce_addr"
field in the task structure into an array and saving each address so
that when the kernel eventually gives up poking at poison and tries
to return to user kill_me_maybe() could loop through them and deal
with each poison page.

I don't think this can happen. Jue Wang suggested that multiple poisoned
pages passed to a single write(2) syscall might trigger this panic (and
because of a bug in my earlier version, he managed to trigger this
"different user pages" panic). But this fixed up version survives the
"Jue test".

-Tony