linux-kernel - Re: [PATCH v4 10/10] x86/mm: Try to preserve old TLB entries using PCID

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 5 Jul 2017 14:18:07 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     Andy Lutomirski <luto@...nel.org>
Cc:     x86@...nel.org, linux-kernel@...r.kernel.org,
        Borislav Petkov <bp@...en8.de>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Mel Gorman <mgorman@...e.de>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        Nadav Amit <nadav.amit@...il.com>,
        Rik van Riel <riel@...hat.com>,
        Dave Hansen <dave.hansen@...el.com>,
        Arjan van de Ven <arjan@...ux.intel.com>
Subject: Re: [PATCH v4 10/10] x86/mm: Try to preserve old TLB entries using
 PCID

On Thu, Jun 29, 2017 at 08:53:22AM -0700, Andy Lutomirski wrote:
> @@ -104,18 +140,20 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
>  
>  		/* Resume remote flushes and then read tlb_gen. */
>  		cpumask_set_cpu(cpu, mm_cpumask(next));

Barriers should have a comment... what is being ordered here against
what?

> +		smp_mb__after_atomic();
>  		next_tlb_gen = atomic64_read(&next->context.tlb_gen);
>  
> +		if (this_cpu_read(cpu_tlbstate.ctxs[prev_asid].tlb_gen) <
> +		    next_tlb_gen) {
>  			/*
>  			 * Ideally, we'd have a flush_tlb() variant that
>  			 * takes the known CR3 value as input.  This would
>  			 * be faster on Xen PV and on hypothetical CPUs
>  			 * on which INVPCID is fast.
>  			 */
> +			this_cpu_write(cpu_tlbstate.ctxs[prev_asid].tlb_gen,
>  				       next_tlb_gen);
> +			write_cr3(__pa(next->pgd) | prev_asid);
>  			trace_tlb_flush(TLB_FLUSH_ON_TASK_SWITCH,
>  					TLB_FLUSH_ALL);
>  		}

> @@ -152,14 +190,25 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
>  		 * Start remote flushes and then read tlb_gen.
>  		 */
>  		cpumask_set_cpu(cpu, mm_cpumask(next));
> +		smp_mb__after_atomic();

idem

>  		next_tlb_gen = atomic64_read(&next->context.tlb_gen);
>  
> +		choose_new_asid(next, next_tlb_gen, &new_asid, &need_flush);
>  
> +		if (need_flush) {
> +			this_cpu_write(cpu_tlbstate.ctxs[new_asid].ctx_id, next->context.ctx_id);
> +			this_cpu_write(cpu_tlbstate.ctxs[new_asid].tlb_gen, next_tlb_gen);
> +			write_cr3(__pa(next->pgd) | new_asid);
> +			trace_tlb_flush(TLB_FLUSH_ON_TASK_SWITCH,
> +					TLB_FLUSH_ALL);
> +		} else {
> +			/* The new ASID is already up to date. */
> +			write_cr3(__pa(next->pgd) | new_asid | CR3_NOFLUSH);
> +			trace_tlb_flush(TLB_FLUSH_ON_TASK_SWITCH, 0);
> +		}
> +
> +		this_cpu_write(cpu_tlbstate.loaded_mm, next);
> +		this_cpu_write(cpu_tlbstate.loaded_mm_asid, new_asid);
>  	}
>  
>  	load_mm_cr4(next);