Message-Id: <20090702144415.8B21.E1E9C6FF@jp.fujitsu.com>
Date:	Thu, 02 Jul 2009 14:59:19 +0900
From:	Yasunori Goto <y-goto@...fujitsu.com>
To:	yakui <yakui.zhao@...el.com>, "Li, Shaohua" <shaohua.li@...el.com>
Cc:	Christoph Lameter <cl@...ux-foundation.org>,
	"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"mel@....ul.ie" <mel@....ul.ie>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
Subject: Re: + memory-hotplug-alloc-page-from-other-node-in-memory-online.patch added to -mm tree

> On Thu, 02 Jul 2009 09:11:13 +0800
> yakui <yakui.zhao@...el.com> wrote:
> 
> > On Thu, 2009-07-02 at 01:22 +0800, Christoph Lameter wrote:
> > > On Wed, 1 Jul 2009, yakui wrote:
> > > 
> > > > If we can't allocate memory from other node when there is no memory on
> > > > this node, we will have to do something like the bootmem allocator.
> > > > After the memory page is added to the system memory, we will have to
> > > > free the memory space used by the memory allocator. At the same time we
> > > > will have to assure that the hot-plugged memory exists physically.
> > > 
> > > The bootmem allocator must stick around it seems. Its more like a node
> > > bootstrap allocator then.
> > > 
> > > Maybe we can generalize that. The bootstrap allocator may only need to be
> > > able boot one node (which simplifies design). During system bringup only
> > > the boot node is brought up.
> > > 
> > > Then the other nodes are hotplugged later all in turn using the bootstrap
> > > allocator for their node setup?
> > Your idea looks fragrant. But it seems that it is difficult to realize.
> > In the boot phase the bootmem allocator is initialized. And after the
> > page buddy mechanism is enabled, the memory space used by bootmem
> > allocator will be freed.
> > 
> > If we also do the similar thing for the hotplugged node, how and when to
> > free the memory space used by the bootstrap allocator? It seems that we
> > will have to wait before all the memory sections are onlined for this
> > hotplugged node. And before all the memory sections are onlined, the
> > bootstrap allocator and buddy page allocator will co-exist.
> > 
> 
> When I was an eager developper of memory hotplug, I planned that.
> A special page allocater which works from allocating pgdat until memmap setup.
> But there were problems.
> example)
>   1. We wanted to reuse bootmem.c but it was difficult.
>   2. IBM guys uses 16MB section. Then, they cannot allocate local pgdat/memmap
>     as other platform which have larger section size.
>   3. At memory hotplug, "memory section which includes pgdat for a node should be
>      removed after all other sections on the node are removed"
>      There is the same problem to memmap.
> 
> Because current memory hotplug works sane and above problem was too complicated for
> me, I stopped. But there are more NUMAs than we implemented memory hotplug initially.
> I hope someone fixes this mis-allocation problem.
> 
> IIUC, "3" is the worst problem. It creates dependency among memory.

I made tiny basic functions to make it 1 or 2 years ago.
get_page_bootmem() record section/node id or counting up
how many other pages use it. It would be used for dependency
checking when removing memory.
I was going to make new allocator with those information.
(put_page_bootmem() is to free them.)

However, I don't enough time for memory hotplug now, 
and they are just redundant functions now.
If someone create new allocator (and unifying bootmem allocator),
I'm very glad. :-)


Bye.


-- 
Yasunori Goto 


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists