Message-ID: <alpine.LNX.2.03.1506042024090.18025@nvidia.com>
Date:	Thu, 4 Jun 2015 20:29:30 -0700
From:	John Hubbard <jhubbard@...dia.com>
To:	Jerome Glisse <j.glisse@...il.com>
CC:	<akpm@...ux-foundation.org>, <linux-kernel@...r.kernel.org>,
	<linux-mm@...ck.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	<joro@...tes.org>, Mel Gorman <mgorman@...e.de>,
	"H. Peter Anvin" <hpa@...or.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Andrea Arcangeli <aarcange@...hat.com>,
	Johannes Weiner <jweiner@...hat.com>,
	Larry Woodman <lwoodman@...hat.com>,
	Rik van Riel <riel@...hat.com>,
	Dave Airlie <airlied@...hat.com>,
	Brendan Conoboy <blc@...hat.com>,
	Joe Donohue <jdonohue@...hat.com>,
	Duncan Poole <dpoole@...dia.com>,
	Sherry Cheung <SCheung@...dia.com>,
	Subhash Gutti <sgutti@...dia.com>,
	Mark Hairgrove <mhairgrove@...dia.com>,
	Lucien Dunning <ldunning@...dia.com>,
	Cameron Buschardt <cabuschardt@...dia.com>,
	Arvind Gopalakrishnan <arvindg@...dia.com>,
	Haggai Eran <haggaie@...lanox.com>,
	Shachar Raindel <raindel@...lanox.com>,
	Liran Liss <liranl@...lanox.com>,
	Roland Dreier <roland@...estorage.com>,
	Ben Sander <ben.sander@....com>,
	Greg Stoner <Greg.Stoner@....com>,
	John Bridgman <John.Bridgman@....com>,
	Michael Mantor <Michael.Mantor@....com>,
	Paul Blinzer <Paul.Blinzer@....com>,
	Laurent Morichetti <Laurent.Morichetti@....com>,
	Alexander Deucher <Alexander.Deucher@....com>,
	Oded Gabbay <Oded.Gabbay@....com>,
	Jérôme Glisse <jglisse@...hat.com>
Subject: Re: [PATCH 02/36] mmu_notifier: keep track of active invalidation
 ranges v3

On Wed, 3 Jun 2015, Jerome Glisse wrote:

> On Tue, Jun 02, 2015 at 02:32:01AM -0700, John Hubbard wrote:
> > On Thu, 21 May 2015, j.glisse@...il.com wrote:
> > 
> > > From: Jérôme Glisse <jglisse@...hat.com>
> > > 
> > > The mmu_notifier_invalidate_range_start() and mmu_notifier_invalidate_range_end()
> > > can be considered as forming an "atomic" section for the cpu page table update
> > > point of view. Between this two function the cpu page table content is unreliable
> > > for the address range being invalidated.
> > > 
> > > Current user such as kvm need to know when they can trust the content of the cpu
> > > page table. This becomes even more important to new users of the mmu_notifier
> > > api (such as HMM or ODP).
> > > 
> > > This patch use a structure define at all call site to invalidate_range_start()
> > > that is added to a list for the duration of the invalidation. It adds two new
> > > helpers to allow querying if a range is being invalidated or to wait for a range
> > > to become valid.
> > > 
> > > For proper synchronization, user must block new range invalidation from inside
> > > there invalidate_range_start() callback, before calling the helper functions.
> > > Otherwise there is no garanty that a new range invalidation will not be added
> > > after the call to the helper function to query for existing range.
> > 
> > Hi Jerome,
> > 
> > Most of this information will make nice block comments for the new helper 
> > routines. I can help tighten up the writing slightly, but first:
> > 
> > Question: in hmm.c's hmm_notifier_invalidate function (looking at the 
> > entire patchset, for a moment), I don't see any blocking of new range 
> > invalidations, even though you point out, above, that this is required. Am 
> > I missing it, and if so, where should I be looking instead?
> 
> This is a 2 sided synchronization:
> 
> - hmm_device_fault_start() will wait for active invalidation that conflict
>   to be done
> - hmm_wait_device_fault() will block new invalidation until
>   active fault that conflict back off.
>


OK. I'll wait until those patches to talk about those, then.
 
> 
> > [...]
> > 
> > > -					   enum mmu_event event)
> > > +					   struct mmu_notifier_range *range)
> > >  
> > >  {
> > >  	struct mmu_notifier *mn;
> > >  	int id;
> > >  
> > > +	spin_lock(&mm->mmu_notifier_mm->lock);
> > > +	list_add_tail(&range->list, &mm->mmu_notifier_mm->ranges);
> > > +	mm->mmu_notifier_mm->nranges++;
> > 
> > 
> > Is this missing a call to wake_up(&mm->mmu_notifier_mm->wait_queue)? If 
> > not, then it would be helpful to explain why that's only required for 
> > nranges--, and not for the nranges++ case. The helper routine is merely 
> > waiting for nranges to *change*, not looking for greater than or less 
> > than.
> 
> This is on purpose, as the waiting side only wait for active invalidation
> to be done ie for mm->mmu_notifier_mm->nranges-- so there is no reasons to
> wake up when a new invalidation is starting. Also the test need to be a not
> equal because other non conflicting range might be added/removed meaning
> that wait might finish even if mm->mmu_notifier_mm->nranges > saved_nranges.
> 


OK, I convinced myself that this works as intended. So I don't see 
anything wrong with this approach.

thanks,
john h

> 
> [...]
> > > +static bool mmu_notifier_range_is_valid_locked(struct mm_struct *mm,
> > > +					       unsigned long start,
> > > +					       unsigned long end)
> > 
> > 
> > This routine is named "_range_is_valid_", but it takes in an implicit 
> > range (start, end), and also a list of ranges (buried in mm), and so it's 
> > a little confusing. I'd like to consider *maybe* changing either the name, 
> > or the args (range* instead of start, end?), or something.
> > 
> > Could you please say a few words about the intent of this routine, to get 
> > us started there?
> 
> It is just the same as mmu_notifier_range_is_valid() but it expects locks
> to be taken. This is for the benefit of mmu_notifier_range_wait_valid()
> which need to test if a range is valid (ie no conflicting invalidation)
> or not. I added a comment to explain this 3 function and to explain how
> the 2 publics helper needs to be use.
> 
> Cheers,
> Jérôme
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@...ck.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@...ck.org"> email@...ck.org </a>
> 

Powered by blists - more mailing lists