linux-kernel - Re: [PATCH 35/35] autonuma: page

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20120529171407.GH21339@redhat.com>
Date:	Tue, 29 May 2012 19:14:07 +0200
From:	Andrea Arcangeli <aarcange@...hat.com>
To:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
Cc:	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	Hillf Danton <dhillf@...il.com>, Dan Smith <danms@...ibm.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	Ingo Molnar <mingo@...e.hu>, Paul Turner <pjt@...gle.com>,
	Suresh Siddha <suresh.b.siddha@...el.com>,
	Mike Galbraith <efault@....de>,
	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	Lai Jiangshan <laijs@...fujitsu.com>,
	Bharata B Rao <bharata.rao@...il.com>,
	Lee Schermerhorn <Lee.Schermerhorn@...com>,
	Rik van Riel <riel@...hat.com>,
	Johannes Weiner <hannes@...xchg.org>,
	Srivatsa Vaddagiri <vatsa@...ux.vnet.ibm.com>,
	Christoph Lameter <cl@...ux.com>
Subject: Re: [PATCH 35/35] autonuma: page_autonuma

On Tue, May 29, 2012 at 06:44:15PM +0200, Peter Zijlstra wrote:
> On Fri, 2012-05-25 at 19:02 +0200, Andrea Arcangeli wrote:
> > Move the AutoNUMA per page information from the "struct page" to a
> > separate page_autonuma data structure allocated in the memsection
> > (with sparsemem) or in the pgdat (with flatmem).
> > 
> > This is done to avoid growing the size of the "struct page" and the
> > page_autonuma data is only allocated if the kernel has been booted on
> > real NUMA hardware (or if noautonuma is passed as parameter to the
> > kernel).
> > 
> 
> Argh, please fold this change back into the series proper. If you want
> to keep it.. as it is its not really an improvement IMO, see below.

The whole objective of this patch is to avoid allocating the
page_autonuma structures when the kernel is booted on not NUMA
hardware.

It's not an improvement when booting the kernel on NUMA hardware
that's for sure.

I didn't merge it with the previous because this was the most
experimental recent change, so I wanted bisectability here. When
something goes wrong here, the kernel won't boot, so unless you use
kvm with gdbstub it's a little tricky to debug (indeed I debugged it
with gdbstub, there it's trivial).

> > +struct page_autonuma {
> > +       /*
> > +        * FIXME: move to pgdat section along with the memcg and allocate
> > +        * at runtime only in presence of a numa system.
> > +        */
> > +       /*
> > +        * To modify autonuma_last_nid lockless the architecture,
> > +        * needs SMP atomic granularity < sizeof(long), not all archs
> > +        * have that, notably some alpha. Archs without that requires
> > +        * autonuma_last_nid to be a long.
> > +        */
> 
> Looking at arch/alpha/include/asm/xchg.h it looks to have that just
> fine, so maybe we simply don't support SMP on those early Alphas that
> had that weirdness.

I agree we should never risk that.

> This makes a shadow page frame of 32 bytes per page, or ~0.8% of memory.
> This isn't in fact an improvement.
> 
> The suggestion done by Rik was to have something like a sqrt(nr_pages)
> (?) scaled array of such things containing the list_head and page
> pointer -- and leave the two nids in the regular page frame. Although I
> think you've got to fight the memcg people over that last word in struct
> page.
> 
> That places a limit on the amount of pages that can be in migration
> concurrently, but also greatly reduces the memory overhead.

Yes, however for the last_nid I'd still need it for every page (and if
I allocate it dynamic I still first need to find a way to remove the
struct page pointer).

I thought to add a pointer in the memsection (or maybe to use a vmemmap
so that it won't even require a pointer in every memsection). I've to
check a few more things before I allow_the autonuma->page translation
without a page pointer, notably to verify the boot time allocations
points won't just allocate power of two blocks of memory (they
shouldn't but I didn't verify).

This is clearly a move in the right direction to avoid the memory
overhead when not booted on NUMA hardware, and I don't think there's
anything fundamental that prevents us remove the page pointer from the
page_autonuma structure, and to later experiment with a limited size
array of async migration structures.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/