Date:	Thu, 12 Jul 2012 22:21:22 +0200
From:	Andrea Arcangeli <aarcange@...hat.com>
To:	Rik van Riel <riel@...hat.com>
Cc:	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	Hillf Danton <dhillf@...il.com>, Dan Smith <danms@...ibm.com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...e.hu>, Paul Turner <pjt@...gle.com>,
	Suresh Siddha <suresh.b.siddha@...el.com>,
	Mike Galbraith <efault@....de>,
	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	Lai Jiangshan <laijs@...fujitsu.com>,
	Bharata B Rao <bharata.rao@...il.com>,
	Lee Schermerhorn <Lee.Schermerhorn@...com>,
	Johannes Weiner <hannes@...xchg.org>,
	Srivatsa Vaddagiri <vatsa@...ux.vnet.ibm.com>,
	Christoph Lameter <cl@...ux.com>,
	Alex Shi <alex.shi@...el.com>,
	Mauricio Faria de Oliveira <mauricfo@...ux.vnet.ibm.com>,
	Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>,
	Don Morris <don.morris@...com>,
	Benjamin Herrenschmidt <benh@...nel.crashing.org>
Subject: Re: [PATCH 40/40] autonuma: shrink the per-page page_autonuma struct
 size

On Mon, Jul 02, 2012 at 03:18:46AM -0400, Rik van Riel wrote:
> On 06/28/2012 08:56 AM, Andrea Arcangeli wrote:
> >  From 32 to 12 bytes, so the AutoNUMA memory footprint is reduced to
> > 0.29% of RAM.
> 
> Still not ideal, however once we get native THP migration working
> it could be practical to switch to a "have a bucket with N
> page_autonuma structures for every N*M pages" approach.
> 
> For example, we could have 4 struct page_autonuma pages, for 32
> memory pages. That would necessitate reintroducing the page pointer
> into struct page_autonuma, but it would reduce memory use by roughly
> a factor 8.
> 
> To get from a struct page to a struct page_autonuma, we would have
> to look at the bucket and check whether one of the page_autonuma
> structs points at us. If none do, we have to claim an available one.
> On migration, we would have to free our page_autonuma struct, which
> would make it available for other pages to use.
> 
> This would complicate the code somewhat, and potentially slow down
> the migration of 4kB pages, but with 2MB pages things could continue
> exactly the way they work today.
> 
> Does this seem reasonably in any way?

Reducing the max lru size loses info too. The thing I dislike is that
knuma_migrated may not migrate the page until a few knuma_scand passed
on large systems (giving a chance to last_nid_set to notice if there's
false sharing and cancel the migration). I conceptually like the
unlimited sized LRU migration list.

The other cons is that it'll increase the complexity even more by
having to deal with dynamic objects instead of an extension of the
struct page.

And the 2bytes for the last_nid information would need to be retained
for every page, unless we drop the last_nid logic which I doubt would
be good.

And the alternative without an hash is not feasible: one could reduce
it to 8bytes per-page (for the pointer to the page_autonuma structure)
plus 2 bytes for last_nid, so 10 bytes per page plus the actual array,
instead of the current 12 bytes per page.

> I also wonder if it would make sense to have this available as a
> generic list type, not autonuma specific but an "item number list"
> include file with corresponding macros.
> 
> It might be useful to have lists with item numbers, instead of
> prev & next pointers, in other places in the kernel.
> 
> Besides, introducing this list type separately could make things
> easier to review.

Macros bypassing type checking usually aren't recommended and
certainly it's more readable as it is now. But this can always be done
later if needed.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists