linux-kernel - Re: [PATCH v5 6/8] x86/resctrl: Sub NUMA Cluster detection and enable

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <66d91674-fc2d-638e-ff92-7d3515e3aa08@intel.com>
Date:   Thu, 28 Sep 2023 11:44:45 -0700
From:   Reinette Chatre <reinette.chatre@...el.com>
To:     Tony Luck <tony.luck@...el.com>
CC:     Fenghua Yu <fenghua.yu@...el.com>,
        Peter Newman <peternewman@...gle.com>,
        Jonathan Corbet <corbet@....net>,
        Shuah Khan <skhan@...uxfoundation.org>, <x86@...nel.org>,
        Shaopeng Tan <tan.shaopeng@...itsu.com>,
        James Morse <james.morse@....com>,
        Jamie Iles <quic_jiles@...cinc.com>,
        Babu Moger <babu.moger@....com>,
        Randy Dunlap <rdunlap@...radead.org>,
        <linux-kernel@...r.kernel.org>, <linux-doc@...r.kernel.org>,
        <patches@...ts.linux.dev>
Subject: Re: [PATCH v5 6/8] x86/resctrl: Sub NUMA Cluster detection and enable

Hi Tony,

On 9/28/2023 11:12 AM, Tony Luck wrote:
> On Mon, Sep 25, 2023 at 04:29:44PM -0700, Reinette Chatre wrote:
>> On 8/29/2023 4:44 PM, Tony Luck wrote:

>>> +	if (cpumask_first(topology_core_cpumask(cpu)) != cpu)
>>> +		return;
>>> +
>>
>> This is an area I am not familiar with. The above code seems
>> to assume that CPUs are onlined in a particular numerical
>> order. For example, if I understand correctly, if CPUs
>> are onlined from higher number to lower number then
>> the above code may end up running on every CPU online.
> 
> This sent me on a voyage of exploration into early Linux
> bringup. There's a CONFIG_HOTPLUG_PARALLEL option to bring
> CPUs up in parallel. I have it set on my kernel, but I still
> see the "announce_cpu()" console messages show up in monotonic
> increasing order:
> 
> [    3.148423] smpboot: x86: Booting SMP configuration:
> [    3.148940] .... node  #0, CPUs:          #1   #2   #3   #4   #5   #6 #7 and so on
> 
> But, without solving this mystery, I realized this doesn't matter.
> Whatever order the CPUs come online was all completed long before
> resctrl is initialized:
> 
> 	late_initcall(resctrl_late_init);
> 
> So the order that resctrl sees CPUs is dependent on the callbacks
> from the registration with cpuhp_setup_state(). That happens with:
> 
>         /*
>          * Try to call the startup callback for each present cpu
>          * depending on the hotplug state of the cpu.
>          */
>         for_each_present_cpu(cpu) {
> 
> which is going to call in increasing numerical order as the bitmap
> of present CPUs is traversed.
> 
> If someone changed this, the only ill effect on the code I'm
> adding would be to set the MSR multiple times (which is
> inefficient, but won't break anything).
> 
> 

Thank you very much for investigating this.


Reinette