linux-kernel - Re: [Part3 PATCH v2 0/4] Support hot-remove local pagetable pages.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130619100042.GA4545@dhcp-192-168-178-175.profitbricks.localdomain>
Date:	Wed, 19 Jun 2013 12:00:42 +0200
From:	Vasilis Liaskovitis <vasilis.liaskovitis@...fitbricks.com>
To:	Tang Chen <tangchen@...fujitsu.com>
Cc:	yinghai@...nel.org, tglx@...utronix.de, mingo@...e.hu,
	hpa@...or.com, akpm@...ux-foundation.org, tj@...nel.org,
	trenn@...e.de, jiang.liu@...wei.com, wency@...fujitsu.com,
	laijs@...fujitsu.com, isimatu.yasuaki@...fujitsu.com,
	mgorman@...e.de, minchan@...nel.org, mina86@...a86.com,
	gong.chen@...ux.intel.com, lwoodman@...hat.com, riel@...hat.com,
	jweiner@...hat.com, prarit@...hat.com, x86@...nel.org,
	linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org
Subject: Re: [Part3 PATCH v2 0/4] Support hot-remove local pagetable pages.

Hi Tang,
On Wed, Jun 19, 2013 at 03:29:06PM +0800, Tang Chen wrote:
> Hi Vasilis, Yinghai,
> 
> On 06/19/2013 01:05 AM, Vasilis Liaskovitis wrote:
> ......
> >
> >This could be a design problem of part3: if we allow local pagetable memory
> >to not be offlined but allow the offlining to return successfully, then
> >hot-remove is going to succeed. But the direct mapped pagetable pages are still
> >mapped in the kernel. The hot-removed memblocks will suddenly disappear (think
> >physical DIMMs getting disabled in real hardware, or in a VM case the
> >corresponding guest memory getting freed from the emulator e.g. qemu/kvm). The
> >system can crash as a result.
> >
> 
> Yes. Since the pagetable pages is only allocated to local node, a node may
> have more than one device, hot-remove only one memory device could be
> problematic.
> 
> But I think it will work if we hot-remove a whole node. I should have
> mentioned it. And sorry for the not fully test.

ok, the crash I saw was also for the partial node removal.

> I think allocating pagetable pages to local device will resolve this
> problem.

ok. Yes, you mentioned this approach before I think.

> And need to restructure this patch-set.
> 
> >I think these local pagetables do need to be unmapped from kernel, offlined and
> >removed somehow - otherwise hot-remove should fail. Could they be migrated
> >alternatively e.g. to node 0 memory?  But Iiuc direct mapped pages cannot be
> >migrated, correct?
> 
> I think we have unmapped the local pagetables. in functions
> free_pud/pmd/pte_table(), we cleared pud, pmd, and pte. We just didn't
> free the pagetable pages to buddy.

ok, thanks for explaining.

> 
> But when we are not hot-removing the whole node, it is still problematic.
> This is true, and it is my design problem.
> 
> >
> >What is the original reason for local node pagetable allocation with regards
> >to memory hotplug? I assume we want to have hotplugged nodes use only their local
> >memory, so that there are no inter-node memory dependencies for hot-add/remove.
> >Are there other reasons that I am missing?
> 
> I think the original reason to do local node pagetable is to improve
> performance.
> Using local pagetable, vmemmap and so on will be faster.
> 
> But actually I think there is no particular reason to implement
> memory hot-remove
> and local node pagetable at the same time. And before this
> patch-set, I also
> suggested once that implement memory hot-remove first, and then
> improve it to
> local pagetable. But Yinghai has done the local pagetable work in
> has patches (part1).
> And my work is based on his patches. So I just did it.
> 
> But obviously it is more complicated than I thought.
> 
> And now, it seems tj has some more thinking on part1.
> 
> So how about the following plan:
> 1. Implement arranging hotpluggable memory with SRAT first, without
> local pagetable.
>    (The main work in part2. And of course, need some patches in part1.)

agreed (and yes, several patches from part1 will be needed to do the early srat
parsing here)

> 2. Do the local device pagetable work, not local node.
> 3. Improve memory hotplug to support local device pagetable.

ok, I 'll think about these as well, and help out.

thanks,

- Vasilis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/