linux-kernel - Re: Process-shared futexes on hugepages puts the kernel in an infinite loop in 2.6.32.11; is this fixed now?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 16 Apr 2010 22:27:48 +0200
From:	Peter Zijlstra <peterz@...radead.org>
To:	r6144 <rainy6144@...il.com>
Cc:	linux-kernel@...r.kernel.org, Darren Hart <dvhltc@...ibm.com>,
	tglx <tglx@...utronix.de>, Mel Gorman <mel@....ul.ie>,
	Andrea Arcangeli <aarcange@...hat.com>,
	Lee Schermerhorn <lee.schermerhorn@...com>
Subject: Re: Process-shared futexes on hugepages puts the kernel in an
 infinite loop in 2.6.32.11; is this fixed now?

On Fri, 2010-04-16 at 23:45 +0800, r6144 wrote:
> Hello all,
> 
> I'm having an annoying kernel bug regarding huge pages in Fedora 12:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=552257
> 
> Basically I want to use huge pages in a multithreaded number crunching
> program, which happens to use process-shared semaphores (because fftw
> does it).  The futex for the semaphore ends up lying on a huge page, and
> I then get an endless loop in get_futex_key(), apparently because the
> anonymous huge page containing the futex does not have a page->mapping.
> A test case is provided in the above link.
> 
> I reported the bug to Fedora bugzilla months ago, but haven't received
> any feedback yet. 

No, it works much better if you simply mail LKML and CC people who work
on the code in question ;-)

>  The Fedora kernel is based on 2.6.32.11, and a
> cursory glance at the 2.6.34-rc3 source does not yield any relevant
> change.
> 
> So, could anyone tell me if the current mainline kernel might act better
> in this respect, before I get around to compiling it?

Right, so I had a quick chat with Mel, and it appears MAP_PRIVATE
hugetlb pages don't have their page->mapping set.

I guess something like the below might work, but I'd really rather not
add hugetlb knowledge to futex.c. Does anybody else have a better idea?
Maybe create something similar to an anon_vma for hugetlb pages?

---
 kernel/futex.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel/futex.c b/kernel/futex.c
index e7a35f1..b0f1b2d 100644
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -252,7 +252,7 @@ again:
 
 	page = compound_head(page);
 	lock_page(page);
-	if (!page->mapping) {
+	if (!page->mapping && !PageHuge(page)) {
 		unlock_page(page);
 		put_page(page);
 		goto again;
@@ -265,7 +265,7 @@ again:
 	 * it's a read-only handle, it's expected that futexes attach to
 	 * the object not the particular process.
 	 */
-	if (PageAnon(page)) {
+	if (PageAnon(page) || (PageHuge(page) && !page->mapping)) {
 		key->both.offset |= FUT_OFF_MMSHARED; /* ref taken on mm */
 		key->private.mm = mm;
 		key->private.address = address;


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/