| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Mon, 02 Sep 2013 09:03:18 +0800
From: Tang Chen <tangchen@...fujitsu.com>
To: rjw@...k.pl, lenb@...nel.org, tglx@...utronix.de, mingo@...e.hu,
hpa@...or.com, akpm@...ux-foundation.org, tj@...nel.org,
trenn@...e.de, yinghai@...nel.org, jiang.liu@...wei.com,
wency@...fujitsu.com, laijs@...fujitsu.com,
isimatu.yasuaki@...fujitsu.com, izumi.taku@...fujitsu.com,
mgorman@...e.de, minchan@...nel.org, mina86@...a86.com,
gong.chen@...ux.intel.com, vasilis.liaskovitis@...fitbricks.com,
lwoodman@...hat.com, riel@...hat.com, jweiner@...hat.com,
prarit@...hat.com, zhangyanfei@...fujitsu.com
CC: x86@...nel.org, linux-doc@...r.kernel.org,
linux-kernel@...r.kernel.org, linux-mm@...ck.org,
linux-acpi@...r.kernel.org
Subject: Re: [PATCH 00/11] x86, memblock: Allocate memory near kernel image
before SRAT parsed.
Hi guys,
Any comment to this patch-set ? And shall agree on using this solution
suggested by Tejun ?
Thanks.
On 08/27/2013 05:37 PM, Tang Chen wrote:
> This patch-set is based on tj's suggestion, and not fully tested.
> Just for review and discussion.
>
>
> [Problem]
>
> The current Linux cannot migrate pages used by the kerenl because
> of the kernel direct mapping. In Linux kernel space, va = pa + PAGE_OFFSET.
> When the pa is changed, we cannot simply update the pagetable and
> keep the va unmodified. So the kernel pages are not migratable.
>
> There are also some other issues will cause the kernel pages not migratable.
> For example, the physical address may be cached somewhere and will be used.
> It is not to update all the caches.
>
> When doing memory hotplug in Linux, we first migrate all the pages in one
> memory device somewhere else, and then remove the device. But if pages are
> used by the kernel, they are not migratable. As a result, memory used by
> the kernel cannot be hot-removed.
>
> Modifying the kernel direct mapping mechanism is too difficult to do. And
> it may cause the kernel performance down and unstable. So we use the following
> way to do memory hotplug.
>
>
> [What we are doing]
>
> In Linux, memory in one numa node is divided into several zones. One of the
> zones is ZONE_MOVABLE, which the kernel won't use.
>
> In order to implement memory hotplug in Linux, we are going to arrange all
> hotpluggable memory in ZONE_MOVABLE so that the kernel won't use these memory.
> To do this, we need ACPI's help.
>
> In ACPI, SRAT(System Resource Affinity Table) contains NUMA info. The memory
> affinities in SRAT record every memory range in the system, and also, flags
> specifying if the memory range is hotpluggable.
> (Please refer to ACPI spec 5.0 5.2.16)
>
> With the help of SRAT, we have to do the following two things to achieve our
> goal:
>
> 1. When doing memory hot-add, allow the users arranging hotpluggable as
> ZONE_MOVABLE.
> (This has been done by the MOVABLE_NODE functionality in Linux.)
>
> 2. when the system is booting, prevent bootmem allocator from allocating
> hotpluggable memory for the kernel before the memory initialization
> finishes.
>
> The problem 2 is the key problem we are going to solve. But before solving it,
> we need some preparation. Please see below.
>
>
> [Preparation]
>
> Bootloader has to load the kernel image into memory. And this memory must be
> unhotpluggable. We cannot prevent this anyway. So in a memory hotplug system,
> we can assume any node the kernel resides in is not hotpluggable.
>
> Before SRAT is parsed, we don't know which memory ranges are hotpluggable. But
> memblock has already started to work. In the current kernel, memblock allocates
> the following memory before SRAT is parsed:
>
> setup_arch()
> |->memblock_x86_fill() /* memblock is ready */
> |......
> |->early_reserve_e820_mpc_new() /* allocate memory under 1MB */
> |->reserve_real_mode() /* allocate memory under 1MB */
> |->init_mem_mapping() /* allocate page tables, about 2MB to map 1GB memory */
> |->dma_contiguous_reserve() /* specified by user, should be low */
> |->setup_log_buf() /* specified by user, several mega bytes */
> |->relocate_initrd() /* could be large, but will be freed after boot, should reorder */
> |->acpi_initrd_override() /* several mega bytes */
> |->reserve_crashkernel() /* could be large, should reorder */
> |......
> |->initmem_init() /* Parse SRAT */
>
> According to Tejun's advice, before SRAT is parsed, we should try our best to
> allocate memory near the kernel image. Since the whole node the kernel resides
> in won't be hotpluggable, and for a modern server, a node may have at least 16GB
> memory, allocating several mega bytes memory around the kernel image won't cross
> to hotpluggable memory.
>
>
> [About this patch-set]
>
> So this patch-set does the following:
>
> 1. Make memblock be able to allocate memory from low address to high address.
> Also introduce low limit to prevent memblock allocating memory too low.
>
> 2. Improve init_mem_mapping() to support allocate page tables from low address
> to high address.
>
> 3. Introduce "movablenode" boot option to enable and disable this functionality.
>
> PS: Reordering of relocate_initrd() and reserve_crashkernel() has not been done
> yet. acpi_initrd_override() needs to access initrd with virtual address. So
> relocate_initrd() must be done before acpi_initrd_override().
>
>
> Tang Chen (11):
> memblock: Rename current_limit to current_limit_high in memblock.
> memblock: Rename memblock_set_current_limit() to
> memblock_set_current_limit_high().
> memblock: Introduce lowest limit in memblock.
> memblock: Introduce memblock_set_current_limit_low() to set lower
> limit of memblock.
> memblock: Introduce allocation order to memblock.
> memblock: Improve memblock to support allocation from lower address.
> x86, memblock: Set lowest limit for memblock_alloc_base_nid().
> x86, acpi, memblock: Use __memblock_alloc_base() in
> acpi_initrd_override()
> mem-hotplug: Introduce movablenode boot option to {en|dis}able using
> SRAT.
> x86, mem-hotplug: Support initialize page tables from low to high.
> x86, mem_hotplug: Allocate memory near kernel image before SRAT is
> parsed.
>
> Documentation/kernel-parameters.txt | 15 ++++
> arch/arm/mm/mmu.c | 2 +-
> arch/arm64/mm/mmu.c | 4 +-
> arch/microblaze/mm/init.c | 2 +-
> arch/powerpc/mm/40x_mmu.c | 4 +-
> arch/powerpc/mm/44x_mmu.c | 2 +-
> arch/powerpc/mm/fsl_booke_mmu.c | 4 +-
> arch/powerpc/mm/hash_utils_64.c | 4 +-
> arch/powerpc/mm/init_32.c | 4 +-
> arch/powerpc/mm/ppc_mmu_32.c | 4 +-
> arch/powerpc/mm/tlb_nohash.c | 4 +-
> arch/unicore32/mm/mmu.c | 2 +-
> arch/x86/kernel/setup.c | 41 ++++++++++-
> arch/x86/mm/init.c | 119 ++++++++++++++++++++++++--------
> drivers/acpi/osl.c | 4 +-
> include/linux/memblock.h | 33 ++++++++--
> include/linux/memory_hotplug.h | 5 ++
> mm/memblock.c | 131 +++++++++++++++++++++++++++++-----
> mm/memory_hotplug.c | 9 +++
> mm/nobootmem.c | 4 +-
> 20 files changed, 320 insertions(+), 77 deletions(-)
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@...ck.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email:<a href=mailto:"dont@...ck.org"> email@...ck.org</a>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists