linux-kernel - [PATCH] [214/275] x86: Flush TLB if PGD entry is changed in i386 PAE mode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20110330210739.07D5E3E1A05@tassilo.jf.intel.com>
Date:	Wed, 30 Mar 2011 14:07:38 -0700 (PDT)
From:	Andi Kleen <andi@...stfloor.org>
To:	shaohua.li@...el.com, y-goto@...fujitsu.com, ak@...ux.intel.com,
	riel@...hat.com, asit.k.mallick@...el.com,
	torvalds@...ux-foundation.org, akpm@...ux-foundation.org,
	linux-mm@...ck.org, mingo@...e.hu, gregkh@...e.de,
	linux-kernel@...r.kernel.org, stable@...nel.org,
	tim.bird@...sony.com
Subject: [PATCH] [214/275] x86: Flush TLB if PGD entry is changed in i386 PAE mode

2.6.35-longterm review patch.  If anyone has any objections, please let me know.

------------------
From: Shaohua Li <shaohua.li@...el.com>

commit 4981d01eada5354d81c8929d5b2836829ba3df7b upstream.

According to intel CPU manual, every time PGD entry is changed in i386 PAE
mode, we need do a full TLB flush. Current code follows this and there is
comment for this too in the code.

But current code misses the multi-threaded case. A changed page table
might be used by several CPUs, every such CPU should flush TLB. Usually
this isn't a problem, because we prepopulate all PGD entries at process
fork. But when the process does munmap and follows new mmap, this issue
will be triggered.

When it happens, some CPUs keep doing page faults:

  http://marc.info/?l=linux-kernel&m=129915020508238&w=2

Reported-by: Yasunori Goto<y-goto@...fujitsu.com>
Tested-by: Yasunori Goto<y-goto@...fujitsu.com>
Signed-off-by: Andi Kleen <ak@...ux.intel.com>
Reviewed-by: Rik van Riel <riel@...hat.com>
Signed-off-by: Shaohua Li<shaohua.li@...el.com>
Cc: Mallick Asit K <asit.k.mallick@...el.com>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Andrew Morton <akpm@...ux-foundation.org>
Cc: linux-mm <linux-mm@...ck.org>
LKML-Reference: <1300246649.2337.95.camel@...10-conroe>
Signed-off-by: Ingo Molnar <mingo@...e.hu>
Signed-off-by: Greg Kroah-Hartman <gregkh@...e.de>

---
 arch/x86/include/asm/pgtable-3level.h |   11 +++--------
 arch/x86/mm/pgtable.c                 |    3 +--
 2 files changed, 4 insertions(+), 10 deletions(-)

Index: linux-2.6.35.y/arch/x86/include/asm/pgtable-3level.h
===================================================================
--- linux-2.6.35.y.orig/arch/x86/include/asm/pgtable-3level.h	2011-03-29 22:50:32.352430601 -0700
+++ linux-2.6.35.y/arch/x86/include/asm/pgtable-3level.h	2011-03-29 23:03:02.707230900 -0700
@@ -69,8 +69,6 @@
 
 static inline void pud_clear(pud_t *pudp)
 {
-	unsigned long pgd;
-
 	set_pud(pudp, __pud(0));
 
 	/*
@@ -79,13 +77,10 @@
 	 * section 8.1: in PAE mode we explicitly have to flush the
 	 * TLB via cr3 if the top-level pgd is changed...
 	 *
-	 * Make sure the pud entry we're updating is within the
-	 * current pgd to avoid unnecessary TLB flushes.
+	 * Currently all places where pud_clear() is called either have
+	 * flush_tlb_mm() followed or don't need TLB flush (x86_64 code or
+	 * pud_clear_bad()), so we don't need TLB flush here.
 	 */
-	pgd = read_cr3();
-	if (__pa(pudp) >= pgd && __pa(pudp) <
-	    (pgd + sizeof(pgd_t)*PTRS_PER_PGD))
-		write_cr3(pgd);
 }
 
 #ifdef CONFIG_SMP
Index: linux-2.6.35.y/arch/x86/mm/pgtable.c
===================================================================
--- linux-2.6.35.y.orig/arch/x86/mm/pgtable.c	2011-03-29 22:50:32.352430601 -0700
+++ linux-2.6.35.y/arch/x86/mm/pgtable.c	2011-03-29 23:03:02.708230874 -0700
@@ -160,8 +160,7 @@
 	 * section 8.1: in PAE mode we explicitly have to flush the
 	 * TLB via cr3 if the top-level pgd is changed...
 	 */
-	if (mm == current->active_mm)
-		write_cr3(read_cr3());
+	flush_tlb_mm(mm);
 }
 #else  /* !CONFIG_X86_PAE */
 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/