linux-kernel - RE: [PATCH] cpu-topology: warn if NUMA configurations conflicts with lower layer

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <678F3D1BB717D949B966B68EAEB446ED340AFCA0@dggemm526-mbx.china.huawei.com>
Date:   Fri, 3 Jan 2020 04:24:04 +0000
From:   "Zengtao (B)" <prime.zeng@...ilicon.com>
To:     Valentin Schneider <valentin.schneider@....com>,
        Sudeep Holla <sudeep.holla@....com>
CC:     Linuxarm <linuxarm@...wei.com>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        "Rafael J. Wysocki" <rafael@...nel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "Morten Rasmussen" <morten.rasmussen@....com>
Subject: RE: [PATCH] cpu-topology: warn if NUMA configurations conflicts
 with lower layer

> -----Original Message-----
> From: Valentin Schneider [mailto:valentin.schneider@....com]
> Sent: Thursday, January 02, 2020 9:22 PM
> To: Zengtao (B); Sudeep Holla
> Cc: Linuxarm; Greg Kroah-Hartman; Rafael J. Wysocki;
> linux-kernel@...r.kernel.org; Morten Rasmussen
> Subject: Re: [PATCH] cpu-topology: warn if NUMA configurations conflicts
> with lower layer
> 
> On 02/01/2020 12:47, Zengtao (B) wrote:
> >>
> >> As I said, wrong configurations need to be detected when generating
> >> DT/ACPI if possible. The above will print warning on systems with
> NUMA
> >> within package.
> >>
> >> NUMA:  0-7, 8-15
> >> core_siblings:   0-15
> >>
> >> The above is the example where the die has 16 CPUs and 2 NUMA
> nodes
> >> within a package, your change throws error to the above config which is
> >> wrong.
> >>
> > From your example, the core 7 and core 8 has got different LLC but the
> same Low
> > Level cache?
> 
> AFAIA what matters here is memory controllers, less so LLCs. Cores within
> a single die could have private LLCs and separate memory controllers, or
> shared LLC and separate memory controllers.
> 
> > From schedule view of point, lower level sched domain should be a subset
> of higher
> > Level sched domain.
> >
> 
> Right, and that is checked when you have sched_debug on the cmdline
> (or write 1 to /sys/kernel/debug/sched_debug & regenerate the sched
> domains)
> 

No, here I think you don't get my issue, please try to understand my example
First:.

*************************************
NUMA:         0-2,  3-7
core_siblings:    0-3,  4-7
*************************************
When we are building the sched domain, per the current code:
(1) For core 3
 MC sched domain fallbacks to 3~7
 DIE sched domain is 3~7
(2) For core 4:
 MC sched domain is 4~7
 DIE sched domain is 3~7

When we are build sched groups for the MC level:
(1). core3's sched groups chain is built like as: 3->4->5->6->7->3 
(2). core4's sched groups chain is built like as: 4->5->6->7->4 
so after (2), 
core3's sched groups is overlapped, and it's not a chain any more.
In the afterwards usecase of core3's sched groups, deadloop happens.

And it's difficult for the scheduler to find out such errors,
that is why I think a warning is necessary here.

> Now, I don't know how this plays out for the numa-in-package topologies
> like
> the one suggested by Sudeep. x86 and AMD had to play some games to
> get
> numa-in-package topologies working, see e.g.
> 
>   cebf15eb09a2 ("x86, sched: Add new topology for multi-NUMA-node
> CPUs")
> 
> perhaps you need to "lie" here and ensure sub-NUMA sched domain levels
> are
> a subset of the NUMA levels, i.e. lie for core_siblings. I might go and
> play with this to see how it behaves.