| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <51088CBA.4070602@cn.fujitsu.com>
Date: Wed, 30 Jan 2013 11:00:10 +0800
From: Tang Chen <tangchen@...fujitsu.com>
To: Simon Jeons <simon.jeons@...il.com>
CC: akpm@...ux-foundation.org, rientjes@...gle.com,
len.brown@...el.com, benh@...nel.crashing.org, paulus@...ba.org,
cl@...ux.com, minchan.kim@...il.com,
kosaki.motohiro@...fujitsu.com, isimatu.yasuaki@...fujitsu.com,
wujianguo@...wei.com, wency@...fujitsu.com, hpa@...or.com,
linfeng@...fujitsu.com, laijs@...fujitsu.com, mgorman@...e.de,
yinghai@...nel.org, glommer@...allels.com, x86@...nel.org,
linux-mm@...ck.org, linux-kernel@...r.kernel.org,
linuxppc-dev@...ts.ozlabs.org, linux-acpi@...r.kernel.org,
linux-s390@...r.kernel.org, linux-sh@...r.kernel.org,
linux-ia64@...r.kernel.org, cmetcalf@...era.com,
sparclinux@...r.kernel.org
Subject: Re: [PATCH v6 00/15] memory-hotplug: hot-remove physical memory
On 01/30/2013 10:48 AM, Simon Jeons wrote:
> On Wed, 2013-01-30 at 10:32 +0800, Tang Chen wrote:
>> On 01/29/2013 08:52 PM, Simon Jeons wrote:
>>> Hi Tang,
>>>
>>> On Wed, 2013-01-09 at 17:32 +0800, Tang Chen wrote:
>>>> Here is the physical memory hot-remove patch-set based on 3.8rc-2.
>>
>> Hi Simon,
>>
>> I'll summarize all the info and answer you later. :)
>>
>> Thanks for asking. :)
>
> Thanks Tang, IIRC, there's qemu feature support memory hot-add/remove
> emulation if we don't have machine which supports memory hot-add/remove
> to test. Is that qemu feature merged? Otherwise where can I get that
> patchset?
Hi Simon,
There are patches to support hot-add/remove in qemu, but they are not
merged yet.
You can get the latest patches here:
http://lists.nongnu.org/archive/html/qemu-devel/2012-12/msg02693.html
BTY, it is unstable and full of problems, and you need to compile your
own seabios too.
Thanks. :)
>
>>
>>>
>>> Some questions ask you, not has relationship with this patchset, but is
>>> memory hotplug stuff.
>>>
>>> 1. In function node_states_check_changes_online:
>>>
>>> comments:
>>> * If we don't have HIGHMEM nor movable node,
>>> * node_states[N_NORMAL_MEMORY] contains nodes which have zones of
>>> * 0...ZONE_MOVABLE, set zone_last to ZONE_MOVABLE.
>>>
>>> How to understand it? Why we don't have HIGHMEM nor movable node and
>>> node_staes[N_NORMAL_MEMORY] contains 0...ZONE_MOVABLE, IIUC,
>>> N_NORMAL_MEMORY only means the node has regular memory.
>>>
>>> * If we don't have movable node, node_states[N_NORMAL_MEMORY]
>>> * contains nodes which have zones of 0...ZONE_MOVABLE,
>>> * set zone_last to ZONE_MOVABLE.
>>>
>>> How to understand?
>>>
>>> 2. In function move_pfn_range_left, why end<= z2->zone_start_pfn is not
>>> correct? The comments said that must include/overlap, why?
>>>
>>> 3. In function online_pages, the normal case(w/o online_kenrel,
>>> online_movable), why not check if the new zone is overlap with adjacent
>>> zones?
>>>
>>> 4. Could you summarize the difference implementation between hot-add and
>>> logic-add, hot-remove and logic-remove?
>>>
>>>
>>>>
>>>> This patch-set aims to implement physical memory hot-removing.
>>>>
>>>> The patches can free/remove the following things:
>>>>
>>>> - /sys/firmware/memmap/X/{end, start, type} : [PATCH 4/15]
>>>> - memmap of sparse-vmemmap : [PATCH 6,7,8,10/15]
>>>> - page table of removed memory : [RFC PATCH 7,8,10/15]
>>>> - node and related sysfs files : [RFC PATCH 13-15/15]
>>>>
>>>>
>>>> Existing problem:
>>>> If CONFIG_MEMCG is selected, we will allocate memory to store page cgroup
>>>> when we online pages.
>>>>
>>>> For example: there is a memory device on node 1. The address range
>>>> is [1G, 1.5G). You will find 4 new directories memory8, memory9, memory10,
>>>> and memory11 under the directory /sys/devices/system/memory/.
>>>>
>>>> If CONFIG_MEMCG is selected, when we online memory8, the memory stored page
>>>> cgroup is not provided by this memory device. But when we online memory9, the
>>>> memory stored page cgroup may be provided by memory8. So we can't offline
>>>> memory8 now. We should offline the memory in the reversed order.
>>>>
>>>> When the memory device is hotremoved, we will auto offline memory provided
>>>> by this memory device. But we don't know which memory is onlined first, so
>>>> offlining memory may fail.
>>>>
>>>> In patch1, we provide a solution which is not good enough:
>>>> Iterate twice to offline the memory.
>>>> 1st iterate: offline every non primary memory block.
>>>> 2nd iterate: offline primary (i.e. first added) memory block.
>>>>
>>>> And a new idea from Wen Congyang<wency@...fujitsu.com> is:
>>>> allocate the memory from the memory block they are describing.
>>>>
>>>> But we are not sure if it is OK to do so because there is not existing API
>>>> to do so, and we need to move page_cgroup memory allocation from MEM_GOING_ONLINE
>>>> to MEM_ONLINE. And also, it may interfere the hugepage.
>>>>
>>>>
>>>>
>>>> How to test this patchset?
>>>> 1. apply this patchset and build the kernel. MEMORY_HOTPLUG, MEMORY_HOTREMOVE,
>>>> ACPI_HOTPLUG_MEMORY must be selected.
>>>> 2. load the module acpi_memhotplug
>>>> 3. hotplug the memory device(it depends on your hardware)
>>>> You will see the memory device under the directory /sys/bus/acpi/devices/.
>>>> Its name is PNP0C80:XX.
>>>> 4. online/offline pages provided by this memory device
>>>> You can write online/offline to /sys/devices/system/memory/memoryX/state to
>>>> online/offline pages provided by this memory device
>>>> 5. hotremove the memory device
>>>> You can hotremove the memory device by the hardware, or writing 1 to
>>>> /sys/bus/acpi/devices/PNP0C80:XX/eject.
>>>
>>> Is there a similar knode to hot-add the memory device?
>>>
>>>>
>>>>
>>>> Note: if the memory provided by the memory device is used by the kernel, it
>>>> can't be offlined. It is not a bug.
>>>>
>>>>
>>>> Changelogs from v5 to v6:
>>>> Patch3: Add some more comments to explain memory hot-remove.
>>>> Patch4: Remove bootmem member in struct firmware_map_entry.
>>>> Patch6: Repeatedly register bootmem pages when using hugepage.
>>>> Patch8: Repeatedly free bootmem pages when using hugepage.
>>>> Patch14: Don't free pgdat when offlining a node, just reset it to 0.
>>>> Patch15: New patch, pgdat is not freed in patch14, so don't allocate a new
>>>> one when online a node.
>>>>
>>>> Changelogs from v4 to v5:
>>>> Patch7: new patch, move pgdat_resize_lock into sparse_remove_one_section() to
>>>> avoid disabling irq because we need flush tlb when free pagetables.
>>>> Patch8: new patch, pick up some common APIs that are used to free direct mapping
>>>> and vmemmap pagetables.
>>>> Patch9: free direct mapping pagetables on x86_64 arch.
>>>> Patch10: free vmemmap pagetables.
>>>> Patch11: since freeing memmap with vmemmap has been implemented, the config
>>>> macro CONFIG_SPARSEMEM_VMEMMAP when defining __remove_section() is
>>>> no longer needed.
>>>> Patch13: no need to modify acpi_memory_disable_device() since it was removed,
>>>> and add nid parameter when calling remove_memory().
>>>>
>>>> Changelogs from v3 to v4:
>>>> Patch7: remove unused codes.
>>>> Patch8: fix nr_pages that is passed to free_map_bootmem()
>>>>
>>>> Changelogs from v2 to v3:
>>>> Patch9: call sync_global_pgds() if pgd is changed
>>>> Patch10: fix a problem int the patch
>>>>
>>>> Changelogs from v1 to v2:
>>>> Patch1: new patch, offline memory twice. 1st iterate: offline every non primary
>>>> memory block. 2nd iterate: offline primary (i.e. first added) memory
>>>> block.
>>>>
>>>> Patch3: new patch, no logical change, just remove reduntant codes.
>>>>
>>>> Patch9: merge the patch from wujianguo into this patch. flush tlb on all cpu
>>>> after the pagetable is changed.
>>>>
>>>> Patch12: new patch, free node_data when a node is offlined.
>>>>
>>>>
>>>> Tang Chen (6):
>>>> memory-hotplug: move pgdat_resize_lock into
>>>> sparse_remove_one_section()
>>>> memory-hotplug: remove page table of x86_64 architecture
>>>> memory-hotplug: remove memmap of sparse-vmemmap
>>>> memory-hotplug: Integrated __remove_section() of
>>>> CONFIG_SPARSEMEM_VMEMMAP.
>>>> memory-hotplug: remove sysfs file of node
>>>> memory-hotplug: Do not allocate pdgat if it was not freed when
>>>> offline.
>>>>
>>>> Wen Congyang (5):
>>>> memory-hotplug: try to offline the memory twice to avoid dependence
>>>> memory-hotplug: remove redundant codes
>>>> memory-hotplug: introduce new function arch_remove_memory() for
>>>> removing page table depends on architecture
>>>> memory-hotplug: Common APIs to support page tables hot-remove
>>>> memory-hotplug: free node_data when a node is offlined
>>>>
>>>> Yasuaki Ishimatsu (4):
>>>> memory-hotplug: check whether all memory blocks are offlined or not
>>>> when removing memory
>>>> memory-hotplug: remove /sys/firmware/memmap/X sysfs
>>>> memory-hotplug: implement register_page_bootmem_info_section of
>>>> sparse-vmemmap
>>>> memory-hotplug: memory_hotplug: clear zone when removing the memory
>>>>
>>>> arch/arm64/mm/mmu.c | 3 +
>>>> arch/ia64/mm/discontig.c | 10 +
>>>> arch/ia64/mm/init.c | 18 ++
>>>> arch/powerpc/mm/init_64.c | 10 +
>>>> arch/powerpc/mm/mem.c | 12 +
>>>> arch/s390/mm/init.c | 12 +
>>>> arch/s390/mm/vmem.c | 10 +
>>>> arch/sh/mm/init.c | 17 ++
>>>> arch/sparc/mm/init_64.c | 10 +
>>>> arch/tile/mm/init.c | 8 +
>>>> arch/x86/include/asm/pgtable_types.h | 1 +
>>>> arch/x86/mm/init_32.c | 12 +
>>>> arch/x86/mm/init_64.c | 390 +++++++++++++++++++++++++++++
>>>> arch/x86/mm/pageattr.c | 47 ++--
>>>> drivers/acpi/acpi_memhotplug.c | 8 +-
>>>> drivers/base/memory.c | 6 +
>>>> drivers/firmware/memmap.c | 96 +++++++-
>>>> include/linux/bootmem.h | 1 +
>>>> include/linux/firmware-map.h | 6 +
>>>> include/linux/memory_hotplug.h | 15 +-
>>>> include/linux/mm.h | 4 +-
>>>> mm/memory_hotplug.c | 459 +++++++++++++++++++++++++++++++---
>>>> mm/sparse.c | 8 +-
>>>> 23 files changed, 1094 insertions(+), 69 deletions(-)
>>>>
>>>> --
>>>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>>>> the body to majordomo@...ck.org. For more info on Linux MM,
>>>> see: http://www.linux-mm.org/ .
>>>> Don't email:<a href=mailto:"dont@...ck.org"> email@...ck.org</a>
>>>
>>>
>>>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists