linux-kernel - Re: [PATCH 12/20] mm: Extended batches for generic mmu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 20 Apr 2011 12:40:40 +0200
From:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	Andrea Arcangeli <aarcange@...hat.com>,
	Avi Kivity <avi@...hat.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Rik van Riel <riel@...hat.com>, Ingo Molnar <mingo@...e.hu>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	linux-kernel@...r.kernel.org, linux-arch@...r.kernel.org,
	linux-mm@...ck.org,
	Benjamin Herrenschmidt <benh@...nel.crashing.org>,
	David Miller <davem@...emloft.net>,
	Hugh Dickins <hugh.dickins@...cali.co.uk>,
	Mel Gorman <mel@....ul.ie>, Nick Piggin <npiggin@...nel.dk>,
	Paul McKenney <paulmck@...ux.vnet.ibm.com>,
	Yanmin Zhang <yanmin_zhang@...ux.intel.com>,
	Hugh Dickins <hughd@...gle.com>
Subject: Re: [PATCH 12/20] mm: Extended batches for generic mmu_gather

On Tue, 2011-04-19 at 13:06 -0700, Andrew Morton wrote:
> On Fri, 01 Apr 2011 14:13:10 +0200
> Peter Zijlstra <a.p.zijlstra@...llo.nl> wrote:
> 
> > Instead of using a single batch (the small on-stack, or an allocated
> > page), try and extend the batch every time it runs out and only flush
> > once either the extend fails or we're done.
> 
> why?

To avoid sending extra TLB invalidates.

> > @@ -86,22 +86,48 @@ struct mmu_gather {
> >  #ifdef CONFIG_HAVE_RCU_TABLE_FREE
> >  	struct mmu_table_batch	*batch;
> >  #endif
> > +	unsigned int		need_flush : 1,	/* Did free PTEs */
> > +				fast_mode  : 1; /* No batching   */
> 
> mmu_gather.fast_mode gets modified in several places apparently without
> locking to protect itself.  I don't think that these modifications will
> accidentally trash need_flush, mainly by luck.

The other way around I'd think.

> Please review the concurrency issues here and document them clearly.

Its an on-stack structure, there is no concurrency. /me shall add a
comment.

> > +#ifdef CONFIG_SMP
> > +  #define tlb_fast_mode(tlb) (tlb->fast_mode)
> > +#else
> > +  #define tlb_fast_mode(tlb) 1
> > +#endif
> 
> Mutter.
> 
> Could have been written in C.

Fixed in my last patch uninlining bits

> Will cause a compile error with, for example, tlb_fast_mode(tlb + 1).

Well, that'd actually be a good reason to keep the macro ;-)

> > +static inline int tlb_next_batch(struct mmu_gather *tlb)
> >  {
> > +	struct mmu_gather_batch *batch;
> >  
> > +	batch = tlb->active;
> > +	if (batch->next) {
> > +		tlb->active = batch->next;
> > +		return 1;
> >  	}
> > +
> > +	batch = (void *)__get_free_pages(GFP_NOWAIT | __GFP_NOWARN, 0);
> 
> A comment explaining the gfp_t decision would be useful.

Done

> > +	if (!batch)
> > +		return 0;
> > +
> > +	batch->next = NULL;
> > +	batch->nr   = 0;
> > +	batch->max  = MAX_GATHER_BATCH;
> > +
> > +	tlb->active->next = batch;
> > +	tlb->active = batch;
> > +
> > +	return 1;
> >  }
> >  
> >  /* tlb_gather_mmu
> > @@ -114,16 +140,13 @@ tlb_gather_mmu(struct mmu_gather *tlb, s
> >  {
> >  	tlb->mm = mm;
> >  
> > +	tlb->fullmm     = fullmm;
> > +	tlb->need_flush = 0;
> > +	tlb->fast_mode  = (num_possible_cpus() == 1);
> 
> The changelog didn't tell us why we switched from num_online_cpus() to
> num_possible_cpus().

And that is a very good question... somehow I remember a conversation
with BenH about this, but on second thought that might have been about
his pgtable_free_tlb() optimization (which is somewhat similar).

Let me restore that to num_online_cpus() and maybe do a later patch
removing fast_mode all together as Hugh suggested, since even UP might
have benefit from the batching due to less zone-lock activity on bulk
frees.

> > +	tlb->local.next = NULL;
> > +	tlb->local.nr   = 0;
> > +	tlb->local.max  = ARRAY_SIZE(tlb->__pages);
> > +	tlb->active     = &tlb->local;
> >  
> >  #ifdef CONFIG_HAVE_RCU_TABLE_FREE
> >  	tlb->batch = NULL;
> >
> > ...
> >
> > @@ -177,15 +205,24 @@ tlb_finish_mmu(struct mmu_gather *tlb, u
> > +	batch = tlb->active;
> > +	batch->pages[batch->nr++] = page;
> > +	VM_BUG_ON(batch->nr > batch->max);
> > +	if (batch->nr == batch->max) {
> > +		if (!tlb_next_batch(tlb))
> > +			return 0;
> > +	}
> 
> Moving the VM_BUG_ON() down to after the if() would save a few cycles.

Done.

> > +	return batch->max - batch->nr;
> >  }
> >  
> >  /* tlb_remove_page
> > 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/