| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <D76KWXMCPPKW.32HH5CW3DQW8P@nvidia.com>
Date: Sun, 19 Jan 2025 22:27:01 -0500
From: "Zi Yan" <ziy@...dia.com>
To: "Hyeonggon Yoo" <hyeonggon.yoo@...com>
Cc: <kernel_team@...ynix.com>, <42.hyeyoo@...il.com>, "Bruno Faccini"
<bfaccini@...dia.com>, "linux-kernel@...r.kernel.org"
<linux-kernel@...r.kernel.org>, "linux-mm@...ck.org" <linux-mm@...ck.org>,
"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>, "rppt@...nel.org"
<rppt@...nel.org>, "david@...hat.com" <david@...hat.com>,
"jhubbard@...dia.com" <jhubbard@...dia.com>, "mrusiniak@...dia.com"
<mrusiniak@...dia.com>
Subject: Re: [fake numa not working] [PATCH 1/1] mm/fake-numa: allow later
numa node hotplug
On Sun Jan 19, 2025 at 9:37 PM EST, Hyeonggon Yoo wrote:
>
>
> On 1/20/2025 11:16 AM, Zi Yan wrote:
> > On Sun Jan 19, 2025 at 7:37 PM EST, Hyeonggon Yoo wrote:
> >>
> >>
> >> On 1/18/2025 4:09 AM, Zi Yan wrote:
> >>> On 17 Jan 2025, at 0:00, Hyeonggon Yoo wrote:
> >>>
> >>>> On 1/6/2025 9:06 PM, Bruno Faccini wrote:
> >>>>> Current fake-numa implementation prevents new Numa
> >>>>> nodes to be later hot-plugged by drivers.
> >>>>> A common symptom of this limitation is the
> >>>>> "node <X> was absent from the node_possible_map"
> >>>>> message by associated warning in mm/memory_hotplug.c:
> >>>>> add_memory_resource().
> >>>>> This comes from the lack of remapping in both
> >>>>> pxm_to_node_map[] and node_to_pxm_map[] tables
> >>>>> to take fake-numa nodes into account and thus
> >>>>> triggers collisions with original and physical nodes
> >>>>> only-mapping that had been determined from BIOS tables.
> >>>>> This patch fixes this by doing the necessary node-ids
> >>>>> translation in both pxm_to_node_map[]/node_to_pxm_map[]
> >>>>> tables.
> >>>>> node_distance[] table has also been fixed accordingly.
> >>>>>
> >>>>> Signed-off-by: Bruno Faccini <bfaccini@...dia.com>
> >>>>
> >>>> Hi Bruno,
> >>>>
> >>>> This commit causes WARN() and disables fakenuma
> >>>> when I pass "numa=fake=4U" on my QEMU setup.
> >>>>
> >>>> Attaching WARN() log and git bisect log to help debugging.
> >>>
> >>> Is your VM getting 4 NUMA nodes at the end?
> >>
> >> No, it only gets node 0 at the end.
> >
> > Got it.
> >
> >>
> >>> Can you also share your QEMU command line and your kernel config?
> >>
> >> The config is attached (fakenuma.config)
> >>
> >> QEMU command line:
> >> $ qemu-system-x86_64 \
> >> -cpu host \
> >> -enable-kvm \
> >> -smp $(nproc) \
> >> -m 32G \
> >> -nographic \
> >> -kernel arch/x86/boot/bzImage \
> >> -initrd ../../rootfs.cpio.gz \
> >> -nic user,model=virtio-net-pci \
> >> -append "console=ttyS0 numa=fake=4U"
> >>
> >>> This can help Bruno reproduce the issue locally. At least, I
> >>> cannot reproduce it on mm-everything-2025-01-16-23-18.
> >>
> >> Still reproduced on mm-everything-2025-01-18-04-55 with my setup.
> >>
> >>> I also
> >>> notice that my VM does not have “No NUMA configuration found”
> >>> (in your log) in its kernel log and that might explain
> >>> why I could not reproduce it.
> >>
> >> Did you do NUMA configuration (-numa in QEMU) for your VM?
> >> I didn't, but it's still expected to work 'cause it's fake numa.
> >
> > No. It seems that the "No NUMA configuration found" is caused by your
> > qemu, where dummy_numa_init() is used (maybe ACPI SRAT is not
> > populated?)
>
> FYI, Yes. ACPI SRAT is not populated.
>
> $ qemu-system-x86_64 --version
> QEMU emulator version 6.2.0 (Debian 1:6.2+dfsg-2ubuntu6.24)
> Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
Basically, fix_pxm_node_maps() breaks fakenuma when it cannot fix the
pxm_to_node_map() due to ACPI SRAT is missing. IMHO, it should let
fakenuma proceed with a warning. The diff below fixed the issue
locally, but Bruno can tell if I miss anything.
diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c
index 59fffe34c9d0..18d04f054e93 100644
--- a/drivers/acpi/numa/srat.c
+++ b/drivers/acpi/numa/srat.c
@@ -95,9 +95,12 @@ int __init fix_pxm_node_maps(int max_nid)
int i, j, index = -1, count = 0;
nodemask_t nodes_to_enable;
- if (numa_off || srat_disabled())
+ if (numa_off)
return -1;
+ if (srat_disabled())
+ return 0;
+
/* find fake nodes PXM mapping */
for (i = 0; i < MAX_NUMNODES; i++) {
if (node_to_pxm_map[i] != PXM_INVAL) {
@@ -117,9 +120,9 @@ int __init fix_pxm_node_maps(int max_nid)
}
}
}
- if (WARN(index != max_nid, "%d max nid when expected %d\n",
- index, max_nid))
- return -1;
+ /* No srat data */
+ if (index != max_nid)
+ return 0;
nodes_clear(nodes_to_enable);
>
> > I used the patch below to emulate the situation and
> > reproduced the issue. Since dummy_numa_init() is used when there is
> > no underlying NUMA architecture, NUMA initialization fails, or NUMA
> > is disabled on the command line (the last does not apply here),
> > it should be properly handled. I will let Bruno look into it.
> >
> > Thank you for the info.
> >
> > diff --git a/arch/x86/mm/numa.c b/arch/x86/mm/numa.c
> > index 64e5cdb2460a..70dfd9bfb23f 100644
> > --- a/arch/x86/mm/numa.c
> > +++ b/arch/x86/mm/numa.c
> > @@ -223,7 +223,7 @@ static int __init dummy_numa_init(void)
> > */
> > void __init x86_numa_init(void)
> > {
> > - if (!numa_off) {
> > + if (0 && !numa_off) {
> > #ifdef CONFIG_ACPI_NUMA
> > if (!numa_init(x86_acpi_numa_init))
> > return;
> >
> >>>>
> >>>> [WARN() splat]
> >>>>
> >>>> [ 0.009676] No NUMA configuration found
> >>>> [ 0.009677] Faking a node at [mem 0x0000000000000000-0x000000083fffffff]
> >>>> [ 0.009705] Fake node size 16895MB too small, increasing to 16896MB
> >>>> [ 0.009708] Faking node 0 at [mem 0x0000000000001000-0x0000000420000fff] (16896MB)
> >>>> [ 0.009711] Faking node 1 at [mem 0x0000000420001000-0x000000083fffffff] (16895MB)
> >>>> [ 0.009738] ------------[ cut here ]------------
> >>>> [ 0.009739] -1 max nid when expected 1
> >>>> [ 0.009760] WARNING: CPU: 0 PID: 0 at drivers/acpi/numa/srat.c:120 fix_pxm_node_maps+0x577/0x810
> >>>> [ 0.009766] Modules linked in:
> >>>> [ 0.009769] CPU: 0 UID: 0 PID: 0 Comm: swapper Not tainted 6.13.0-rc6+ #31
> >>>> [ 0.009771] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
> >>>> [ 0.009772] RIP: 0010:fix_pxm_node_maps+0x577/0x810
> >>>> [ 0.009774] Code: 8b 85 40 ff ff ff 44 8b 8d 44 ff ff ff 8b 85 48 ff ff ff e9 83 fb ff ff 44 89 f2 44 89 ce 48 c7 c7 aa 2f 84 8d e8 b9 71 01 fe <0f> 0b 41 b8 ff ff ff ff e9 ba fb ff ff 48 c7 c7 a0 d8 d1 8d 44 89
> >>>> [ 0.009776] RSP: 0000:ffffffff8da03be0 EFLAGS: 00010046 ORIG_RAX: 0000000000000000
> >>>> [ 0.009778] RAX: 0000000000000000 RBX: 0000000000000400 RCX: 0000000000000000
> >>>> [ 0.009779] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
> >>>> [ 0.009780] RBP: ffffffff8da03ca8 R08: 0000000000000000 R09: 0000000000000000
> >>>> [ 0.009781] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000001
> >>>> [ 0.009782] R13: 00000000000007ff R14: 0000000000000001 R15: 0000000000000000
> >>>> [ 0.009783] FS: 0000000000000000(0000) GS:ffffffff8de8c000(0000) knlGS:0000000000000000
> >>>> [ 0.009784] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> >>>> [ 0.009785] CR2: ffff90e4bde01000 CR3: 00000001bd03e000 CR4: 00000000000200b0
> >>>> [ 0.009788] Call Trace:
> >>>> [ 0.009790] <TASK>
> >>>> [ 0.009792] ? show_regs+0x71/0x90
> >>>> [ 0.009797] ? __warn+0x8d/0x150
> >>>> [ 0.009799] ? fix_pxm_node_maps+0x577/0x810
> >>>> [ 0.009802] ? report_bug+0x1ab/0x1c0
> >>>> [ 0.009805] ? fixup_exception+0x27/0x380
> >>>> [ 0.009809] ? early_fixup_exception+0xa2/0xf0
> >>>> [ 0.009813] ? do_early_exception+0x28/0x90
> >>>> [ 0.009816] ? early_idt_handler_common+0x2f/0x3a
> >>>> [ 0.009819] ? fix_pxm_node_maps+0x577/0x810
> >>>> [ 0.009822] ? numa_cleanup_meminfo+0x8c/0x5b0
> >>>> [ 0.009828] numa_emulation+0x4e3/0xad0
> >>>> [ 0.009830] ? __pfx_dummy_numa_init+0x10/0x10
> >>>> [ 0.009833] ? __pfx_dummy_numa_init+0x10/0x10
> >>>> [ 0.009836] numa_memblks_init+0x10c/0x2c0
> >>>> [ 0.009838] ? __pfx_dummy_numa_init+0x10/0x10
> >>>> [ 0.009840] numa_init+0x61/0x3e0
> >>>> [ 0.009842] ? topology_register_boot_apic+0x2a/0x40
> >>>> [ 0.009846] x86_numa_init+0x65/0x80
> >>>> [ 0.009848] initmem_init+0xe/0x20
> >>>> [ 0.009850] setup_arch+0x9d8/0xfa0
> >>>> [ 0.009852] ? _printk+0x58/0x90
> >>>> [ 0.009855] start_kernel+0x5f/0xb50
> >>>> [ 0.009860] x86_64_start_reservations+0x18/0x30
> >>>> [ 0.009863] x86_64_start_kernel+0xc0/0x110
> >>>> [ 0.009864] ? setup_ghcb+0xe/0x140
> >>>> [ 0.009867] common_startup_64+0x13e/0x141
> >>>> [ 0.009871] </TASK>
> >>>> [ 0.009871] ---[ end trace 0000000000000000 ]---
> >>>>
> >>>> [Git bisect log]
> >>>>
> >>>> # bad: [f378252a2168c2fbf8fc08b635061e5f6748c1f2] kasan: sw_tags: use str_on_off() helper in kasan_init_sw_tags()
> >>>> git bisect bad f378252a2168c2fbf8fc08b635061e5f6748c1f2
> >>>> # good: [cbc5dde0a461240046e8a41c43d7c3b76d5db952] fs/proc: fix softlockup in __read_vmcore (part 2)
> >>>> git bisect good cbc5dde0a461240046e8a41c43d7c3b76d5db952
> >>>> # good: [fb73203263021f2ee9dd54d280b1c543d10acd76] mm: move common part of pagetable_*_ctor to helper
> >>>> git bisect good fb73203263021f2ee9dd54d280b1c543d10acd76
> >>>> # bad: [8fd966c9310f14d5bbe653cf087853582170aad6] mm, swap: remove old allocation path for HDD
> >>>> git bisect bad 8fd966c9310f14d5bbe653cf087853582170aad6
> >>>> # bad: [8646971f02651146554233d63fdd721f5f060973] mm: shmem: skip swapcache for swapin of synchronous swap device
> >>>> git bisect bad 8646971f02651146554233d63fdd721f5f060973
> >>>> # good: [b8ba614eaa8dd31702c61e7df7231b1b18b99259] mm/damon/paddr: report filter-passed bytes back for DAMOS_STAT action
> >>>> git bisect good b8ba614eaa8dd31702c61e7df7231b1b18b99259
> >>>> # good: [fd0935b8e9e8c2edf05a3d0ffa09d0aa3e9cf2dd] Docs/ABI/damon: document per-region DAMOS filter-passed bytes stat file
> >>>> git bisect good fd0935b8e9e8c2edf05a3d0ffa09d0aa3e9cf2dd
> >>>> # good: [3ffd3cd7e2e8793d3b5fbb83c03668c47ab6d599] selftests/damon: remove tests for DAMON debugfs interface
> >>>> git bisect good 3ffd3cd7e2e8793d3b5fbb83c03668c47ab6d599
> >>>> # good: [bf9f93f0b7bff8af780330f5094775a39caf3612] mm/damon: remove DAMON debugfs interface
> >>>> git bisect good bf9f93f0b7bff8af780330f5094775a39caf3612
> >>>> # bad: [1746744be6ff271e29345a5be4cc144aa33b10ab] mm/memmap: prevent double scanning of memmap by kmemleak
> >>>> git bisect bad 1746744be6ff271e29345a5be4cc144aa33b10ab
> >>>> # bad: [ca19a5ab8756ef78c34d02d2b8c266a9cc7bf57d] mm/fake-numa: allow later numa node hotplug
> >>>> git bisect bad ca19a5ab8756ef78c34d02d2b8c266a9cc7bf57d
> >>>> # first bad commit: [ca19a5ab8756ef78c34d02d2b8c266a9cc7bf57d] mm/fake-numa: allow later numa node hotplug
> >>>>
> >>>>> ---
> >>>>> drivers/acpi/numa/srat.c | 86 ++++++++++++++++++++++++++++++++++++
> >>>>> include/acpi/acpi_numa.h | 5 +++
> >>>>> include/linux/numa_memblks.h | 3 ++
> >>>>> mm/numa_emulation.c | 45 ++++++++++++++++---
> >>>>> mm/numa_memblks.c | 2 +-
> >>>>> 5 files changed, 133 insertions(+), 8 deletions(-)
> >>>>>
> >>>>> diff --git a/drivers/acpi/numa/srat.c b/drivers/acpi/numa/srat.c
> >>>>> index bec0dcd1f9c3..59fffe34c9d0 100644
> >>>>> --- a/drivers/acpi/numa/srat.c
> >>>>> +++ b/drivers/acpi/numa/srat.c
> >>>>> @@ -81,6 +81,92 @@ int acpi_map_pxm_to_node(int pxm)
> >>>>> }
> >>>>> EXPORT_SYMBOL(acpi_map_pxm_to_node);
> >>>>> +#ifdef CONFIG_NUMA_EMU
> >>>>> +/*
> >>>>> + * Take max_nid - 1 fake-numa nodes into account in both
> >>>>> + * pxm_to_node_map()/node_to_pxm_map[] tables.
> >>>>> + */
> >>>>> +int __init fix_pxm_node_maps(int max_nid)
> >>>>> +{
> >>>>> + static int pxm_to_node_map_copy[MAX_PXM_DOMAINS] __initdata
> >>>>> + = { [0 ... MAX_PXM_DOMAINS - 1] = NUMA_NO_NODE };
> >>>>> + static int node_to_pxm_map_copy[MAX_NUMNODES] __initdata
> >>>>> + = { [0 ... MAX_NUMNODES - 1] = PXM_INVAL };
> >>>>> + int i, j, index = -1, count = 0;
> >>>>> + nodemask_t nodes_to_enable;
> >>>>> +
> >>>>> + if (numa_off || srat_disabled())
> >>>>> + return -1;
> >>>>> +
> >>>>> + /* find fake nodes PXM mapping */
> >>>>> + for (i = 0; i < MAX_NUMNODES; i++) {
> >>>>> + if (node_to_pxm_map[i] != PXM_INVAL) {
> >>>>> + for (j = 0; j <= max_nid; j++) {
> >>>>> + if ((emu_nid_to_phys[j] == i) &&
> >>>>> + WARN(node_to_pxm_map_copy[j] != PXM_INVAL,
> >>>>> + "Node %d is already binded to PXM %d\n",
> >>>>> + j, node_to_pxm_map_copy[j]))
> >>>>> + return -1;
> >>>>> + if (emu_nid_to_phys[j] == i) {
> >>>>> + node_to_pxm_map_copy[j] =
> >>>>> + node_to_pxm_map[i];
> >>>>> + if (j > index)
> >>>>> + index = j;
> >>>>> + count++;
> >>>>> + }
> >>>>> + }
> >>>>> + }
> >>>>> + }
> >>>>> + if (WARN(index != max_nid, "%d max nid when expected %d\n",
> >>>>> + index, max_nid))
> >>>>> + return -1;
> >>>>> +
> >>>>> + nodes_clear(nodes_to_enable);
> >>>>> +
> >>>>> + /* map phys nodes not used for fake nodes */
> >>>>> + for (i = 0; i < MAX_NUMNODES; i++) {
> >>>>> + if (node_to_pxm_map[i] != PXM_INVAL) {
> >>>>> + for (j = 0; j <= max_nid; j++)
> >>>>> + if (emu_nid_to_phys[j] == i)
> >>>>> + break;
> >>>>> + /* fake nodes PXM mapping has been done */
> >>>>> + if (j <= max_nid)
> >>>>> + continue;
> >>>>> + /* find first hole */
> >>>>> + for (j = 0;
> >>>>> + j < MAX_NUMNODES &&
> >>>>> + node_to_pxm_map_copy[j] != PXM_INVAL;
> >>>>> + j++)
> >>>>> + ;
> >>>>> + if (WARN(j == MAX_NUMNODES,
> >>>>> + "Number of nodes exceeds MAX_NUMNODES\n"))
> >>>>> + return -1;
> >>>>> + node_to_pxm_map_copy[j] = node_to_pxm_map[i];
> >>>>> + node_set(j, nodes_to_enable);
> >>>>> + count++;
> >>>>> + }
> >>>>> + }
> >>>>> +
> >>>>> + /* creating reverse mapping in pxm_to_node_map[] */
> >>>>> + for (i = 0; i < MAX_NUMNODES; i++)
> >>>>> + if (node_to_pxm_map_copy[i] != PXM_INVAL &&
> >>>>> + pxm_to_node_map_copy[node_to_pxm_map_copy[i]] == NUMA_NO_NODE)
> >>>>> + pxm_to_node_map_copy[node_to_pxm_map_copy[i]] = i;
> >>>>> +
> >>>>> + /* overwrite with new mapping */
> >>>>> + for (i = 0; i < MAX_NUMNODES; i++) {
> >>>>> + node_to_pxm_map[i] = node_to_pxm_map_copy[i];
> >>>>> + pxm_to_node_map[i] = pxm_to_node_map_copy[i];
> >>>>> + }
> >>>>> +
> >>>>> + /* enable other nodes found in PXM for hotplug */
> >>>>> + nodes_or(numa_nodes_parsed, nodes_to_enable, numa_nodes_parsed);
> >>>>> +
> >>>>> + pr_debug("found %d total number of nodes\n", count);
> >>>>> + return 0;
> >>>>> +}
> >>>>> +#endif
> >>>>> +
> >>>>> static void __init
> >>>>> acpi_table_print_srat_entry(struct acpi_subtable_header *header)
> >>>>> {
> >>>>> diff --git a/include/acpi/acpi_numa.h b/include/acpi/acpi_numa.h
> >>>>> index b5f594754a9e..99b960bd473c 100644
> >>>>> --- a/include/acpi/acpi_numa.h
> >>>>> +++ b/include/acpi/acpi_numa.h
> >>>>> @@ -17,11 +17,16 @@ extern int node_to_pxm(int);
> >>>>> extern int acpi_map_pxm_to_node(int);
> >>>>> extern unsigned char acpi_srat_revision;
> >>>>> extern void disable_srat(void);
> >>>>> +extern int fix_pxm_node_maps(int max_nid);
> >>>>> extern void bad_srat(void);
> >>>>> extern int srat_disabled(void);
> >>>>> #else /* CONFIG_ACPI_NUMA */
> >>>>> +static inline int fix_pxm_node_maps(int max_nid)
> >>>>> +{
> >>>>> + return 0;
> >>>>> +}
> >>>>> static inline void disable_srat(void)
> >>>>> {
> >>>>> }
> >>>>> diff --git a/include/linux/numa_memblks.h b/include/linux/numa_memblks.h
> >>>>> index cfad6ce7e1bd..dd85613cdd86 100644
> >>>>> --- a/include/linux/numa_memblks.h
> >>>>> +++ b/include/linux/numa_memblks.h
> >>>>> @@ -29,7 +29,10 @@ int __init numa_cleanup_meminfo(struct numa_meminfo *mi);
> >>>>> int __init numa_memblks_init(int (*init_func)(void),
> >>>>> bool memblock_force_top_down);
> >>>>> +extern int numa_distance_cnt;
> >>>>> +
> >>>>> #ifdef CONFIG_NUMA_EMU
> >>>>> +extern int emu_nid_to_phys[MAX_NUMNODES];
> >>>>> int numa_emu_cmdline(char *str);
> >>>>> void __init numa_emu_update_cpu_to_node(int *emu_nid_to_phys,
> >>>>> unsigned int nr_emu_nids);
> >>>>> diff --git a/mm/numa_emulation.c b/mm/numa_emulation.c
> >>>>> index 031fb9961bf7..9d55679d99ce 100644
> >>>>> --- a/mm/numa_emulation.c
> >>>>> +++ b/mm/numa_emulation.c
> >>>>> @@ -8,11 +8,12 @@
> >>>>> #include <linux/memblock.h>
> >>>>> #include <linux/numa_memblks.h>
> >>>>> #include <asm/numa.h>
> >>>>> +#include <acpi/acpi_numa.h>
> >>>>> #define FAKE_NODE_MIN_SIZE ((u64)32 << 20)
> >>>>> #define FAKE_NODE_MIN_HASH_MASK (~(FAKE_NODE_MIN_SIZE - 1UL))
> >>>>> -static int emu_nid_to_phys[MAX_NUMNODES];
> >>>>> +int emu_nid_to_phys[MAX_NUMNODES];
> >>>>> static char *emu_cmdline __initdata;
> >>>>> int __init numa_emu_cmdline(char *str)
> >>>>> @@ -379,6 +380,7 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
> >>>>> size_t phys_size = numa_dist_cnt * numa_dist_cnt * sizeof(phys_dist[0]);
> >>>>> int max_emu_nid, dfl_phys_nid;
> >>>>> int i, j, ret;
> >>>>> + nodemask_t physnode_mask = numa_nodes_parsed;
> >>>>> if (!emu_cmdline)
> >>>>> goto no_emu;
> >>>>> @@ -395,7 +397,6 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
> >>>>> * split the system RAM into N fake nodes.
> >>>>> */
> >>>>> if (strchr(emu_cmdline, 'U')) {
> >>>>> - nodemask_t physnode_mask = numa_nodes_parsed;
> >>>>> unsigned long n;
> >>>>> int nid = 0;
> >>>>> @@ -465,9 +466,6 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
> >>>>> */
> >>>>> max_emu_nid = setup_emu2phys_nid(&dfl_phys_nid);
> >>>>> - /* commit */
> >>>>> - *numa_meminfo = ei;
> >>>>> -
> >>>>> /* Make sure numa_nodes_parsed only contains emulated nodes */
> >>>>> nodes_clear(numa_nodes_parsed);
> >>>>> for (i = 0; i < ARRAY_SIZE(ei.blk); i++)
> >>>>> @@ -475,10 +473,21 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
> >>>>> ei.blk[i].nid != NUMA_NO_NODE)
> >>>>> node_set(ei.blk[i].nid, numa_nodes_parsed);
> >>>>> - numa_emu_update_cpu_to_node(emu_nid_to_phys, ARRAY_SIZE(emu_nid_to_phys));
> >>>>> + /* fix pxm_to_node_map[] and node_to_pxm_map[] to avoid collision
> >>>>> + * with faked numa nodes, particularly during later memory hotplug
> >>>>> + * handling, and also update numa_nodes_parsed accordingly.
> >>>>> + */
> >>>>> + ret = fix_pxm_node_maps(max_emu_nid);
> >>>>> + if (ret < 0)
> >>>>> + goto no_emu;
> >>>>> +
> >>>>> + /* commit */
> >>>>> + *numa_meminfo = ei;
> >>>>> +
> >>>>> + numa_emu_update_cpu_to_node(emu_nid_to_phys, max_emu_nid + 1);
> >>>>> /* make sure all emulated nodes are mapped to a physical node */
> >>>>> - for (i = 0; i < ARRAY_SIZE(emu_nid_to_phys); i++)
> >>>>> + for (i = 0; i < max_emu_nid + 1; i++)
> >>>>> if (emu_nid_to_phys[i] == NUMA_NO_NODE)
> >>>>> emu_nid_to_phys[i] = dfl_phys_nid;
> >>>>> @@ -501,12 +510,34 @@ void __init numa_emulation(struct numa_meminfo *numa_meminfo, int numa_dist_cnt)
> >>>>> numa_set_distance(i, j, dist);
> >>>>> }
> >>>>> }
> >>>>> + for (i = 0; i < numa_distance_cnt; i++) {
> >>>>> + for (j = 0; j < numa_distance_cnt; j++) {
> >>>>> + int physi, physj;
> >>>>> + u8 dist;
> >>>>> +
> >>>>> + /* distance between fake nodes is already ok */
> >>>>> + if (emu_nid_to_phys[i] != NUMA_NO_NODE &&
> >>>>> + emu_nid_to_phys[j] != NUMA_NO_NODE)
> >>>>> + continue;
> >>>>> + if (emu_nid_to_phys[i] != NUMA_NO_NODE)
> >>>>> + physi = emu_nid_to_phys[i];
> >>>>> + else
> >>>>> + physi = i - max_emu_nid;
> >>>>> + if (emu_nid_to_phys[j] != NUMA_NO_NODE)
> >>>>> + physj = emu_nid_to_phys[j];
> >>>>> + else
> >>>>> + physj = j - max_emu_nid;
> >>>>> + dist = phys_dist[physi * numa_dist_cnt + physj];
> >>>>> + numa_set_distance(i, j, dist);
> >>>>> + }
> >>>>> + }
> >>>>> /* free the copied physical distance table */
> >>>>> memblock_free(phys_dist, phys_size);
> >>>>> return;
> >>>>> no_emu:
> >>>>> + numa_nodes_parsed = physnode_mask;
> >>>>> /* No emulation. Build identity emu_nid_to_phys[] for numa_add_cpu() */
> >>>>> for (i = 0; i < ARRAY_SIZE(emu_nid_to_phys); i++)
> >>>>> emu_nid_to_phys[i] = i;
> >>>>> diff --git a/mm/numa_memblks.c b/mm/numa_memblks.c
> >>>>> index a3877e9bc878..ff4054f4334d 100644
> >>>>> --- a/mm/numa_memblks.c
> >>>>> +++ b/mm/numa_memblks.c
> >>>>> @@ -7,7 +7,7 @@
> >>>>> #include <linux/numa.h>
> >>>>> #include <linux/numa_memblks.h>
> >>>>> -static int numa_distance_cnt;
> >>>>> +int numa_distance_cnt;
> >>>>> static u8 *numa_distance;
> >>>>> nodemask_t numa_nodes_parsed __initdata;
> >>>
> >>>
> >>> Best Regards,
> >>> Yan, Zi
> >>>
> >
> >
> >
> >
--
Best Regards,
Yan, Zi
Powered by blists - more mailing lists