linux-kernel - Re: [PATCH RFC] hotplug-memory: refactor online_pages to separate zone growth from page onlining

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1206981741.31896.51.camel@nimitz.home.sr71.net>
Date:	Mon, 31 Mar 2008 09:42:21 -0700
From:	Dave Hansen <dave@...ux.vnet.ibm.com>
To:	Jeremy Fitzhardinge <jeremy@...p.org>
Cc:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	Yasunori Goto <y-goto@...fujitsu.com>,
	Christoph Lameter <clameter@....com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Anthony Liguori <anthony@...emonkey.ws>,
	Mel Gorman <mel@....ul.ie>
Subject: Re: [PATCH RFC] hotplug-memory: refactor online_pages to separate
	zone growth from page onlining


On Sat, 2008-03-29 at 16:53 -0700, Jeremy Fitzhardinge wrote:
> Dave Hansen wrote:
> > On Fri, 2008-03-28 at 19:08 -0700, Jeremy Fitzhardinge wrote:
> >   
> >> My big remaining problem is how to disable the sysfs interface for this 
> >> memory.  I need to prevent any onlining via /sys/device/system/memory.
> >>     
> >
> > I've been thinking about this some more, and I wish that you wouldn't
> > just throw this interface away or completely disable it.
> 
> I had no intention of globally disabling it.  I just need to disable it 
> for my use case.

Right, but by disabling it for your case, you have given up all of the
testing that others have done on it.  Let's try and see if we can get
the interface to work for you.

> > To me, it sounds like the only different thing that you want is to make
> > sure that only partial sections are onlined.  So, shall we work with the
> > existing interfaces to online partial sections, or will we just disable
> > it entirely when we see Xen?
> >   
> 
> Well, yes and no.
> 
> For the current balloon driver, it doesn't make much sense.  It would 
> add a fair amount of complexity without any real gain.  It's currently 
> based around alloc_page/free_page.  When it wants to shrink the domain 
> and give memory back to the host, it allocates pages, adds the page 
> structures to a ballooned pages list, and strips off the backing memory 
> and gives it to the host.  Growing the domain is the converse: it gets 
> pages from the host, pulls page structures off the list, binds them 
> together and frees them back to the kernel.  If it runs out of ballooned 
> page structures, it hotplugs in some memory to add more.

How does this deal with things like present_pages in the zones?  Does
the total ram just grow with each hot-add, or does it grow on a per-page
basis from the ballooning?


> That said, if (partial-)sections were much smaller - say 2-4 meg - and 
> page migration/defrag worked reliably, then we could probably do without 
> the balloon driver and do it all in terms of memory hot plug/unplug.  
> That would give us a general mechanism which could either be driven from 
> userspace, and/or have in-kernel Xen/kvm/s390/etc policy modules.  Aside 
> from small sections, the only additional requirement would be an online 
> hook which can actually attach backing memory to the pages being 
> onlined, rather than just assuming an underlying DIMM as current code does.

Even with 1MB sections and a flat sparsemem map, you're only looking at
~500k of overhead for the sparsemem storage.  Less if you use vmemmap.  

-- Dave

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/