linux-kernel - Re: [PATCH v3] alloc_tag: add per-NUMA node stats

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CALCePG0Q8vAEzP+5ASzY-ytnN_eph8F=-F3MQya5MaYcwMNMBg@mail.gmail.com>
Date: Wed, 23 Jul 2025 17:37:56 -0700
From: Casey Chen <cachen@...estorage.com>
To: David Wang <00107082@....com>
Cc: akpm@...ux-foundation.org, surenb@...gle.com, kent.overstreet@...ux.dev, 
	corbet@....net, dennis@...nel.org, tj@...nel.org, cl@...two.org, 
	vbabka@...e.cz, mhocko@...e.com, jackmanb@...gle.com, hannes@...xchg.org, 
	ziy@...dia.com, rientjes@...gle.com, roman.gushchin@...ux.dev, 
	harry.yoo@...cle.com, linux-mm@...ck.org, linux-kernel@...r.kernel.org, 
	linux-doc@...r.kernel.org, yzhong@...estorage.com, souravpanda@...gle.com
Subject: Re: [PATCH v3] alloc_tag: add per-NUMA node stats

Thanks David.
The calling site doesn't detect NUMA imbalance by itself. We know its
imbalance from per-NUMA stats. I would think a software developer
makes global memory scheme based on information provided by per-NUMA
stats. For example, the system has several major consumers of memory,
most of them request memory from NUMA node 0, leading to imbalance. By
detecting the imbalance, we let some of them request memory from NUMA
node 1, by hard-coding.

"What if the numa nodes usage are almost balanced globally, but
strangely unbalance locally for some calling site."
I didn't see such an issue so I have no idea about this.

I wonder if Sourav Panda or David Rientjes could provide us with some
examples. Thanks

On Thu, Jul 10, 2025 at 9:16 PM David Wang <00107082@....com> wrote:
>
>
> At 2025-07-11 08:42:05, "Casey Chen" <cachen@...estorage.com> wrote:
> >Hi All,
> >
> >Thanks for reviewing my previous patches. I am replying some comments
> >in our previous discussion
> >https://lore.kernel.org/all/CAJuCfpHhSUhxer-6MP3503w6520YLfgBTGp7Q9Qm9kgN4TNsfw@mail.gmail.com/T/#u
> >
> >Most people care about the motivations and usages of this feature.
> >Internally, we used to have systems having asymmetric memory to NUMA
> >nodes. Node 0 uses a lot of memory but node 1 is pretty empty.
> >Requests to allocate memory on node 0 always fail. With this patch, we
> >can find the imbalance and optimize the memory usage. Also, David
> >Rientjes and Sourav Panda provide their scenarios in which this patch
> >would be very useful. It is easy to turn on an off so I think it is
> >nice to have, enabling more scenarios in the future.
> >
> >Andrew / Kent,
> >* I agree with Kent on using for_each_possible_cpu rather than
> >for_each_online_cpu, considering CPU online/offline.
> >* When failing to allocate counters for in-kernel alloc_tag, panic()
> >is better than WARN(), eventually the kernel would panic at invalid
> >memory access.
> >* percpu stats would bloat data structures quite a bit.
> >
> >David Wang,
> >I don't really understand what is 'granularity of calling sites'. If
> >NUMA imbalance is found, the calling site could request memory
> >allocation from different nodes. Other factors can affect NUMA
> >balance, those information can be implemented in a different patch.
>
> I think my concern mostly due to my lack of knowledge and experience of NUMA,
> but I still wondering what action to take when " the calling site could request memory
> allocation from different nodes", does the calling site needs to detect numa unbalance at runtime
>  or it should change to hard coded numa node?
>
> By 'granularity of calling sites', i meant to emphasize that information is local per calling site,
> not global.  What if the numa nodes usage are almost balanced globally, but strangely unbalance locally for some calling site.
>
> "what adjustment the calling site would make to solve numa unbalance" is the *big* question to me
>
> Thanks
> David
>
> >
> >Thanks,
> >Casey
> >