linux-kernel - Re: [PATCH] sched/topology: Use Identity node only if required

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20180831112618.GA10911@linux.vnet.ibm.com>
Date:   Fri, 31 Aug 2018 16:56:18 +0530
From:   Srikar Dronamraju <srikar@...ux.vnet.ibm.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     Ingo Molnar <mingo@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Mel Gorman <mgorman@...hsingularity.net>,
        Rik van Riel <riel@...riel.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Michael Ellerman <mpe@...erman.id.au>,
        Heiko Carstens <heiko.carstens@...ibm.com>,
        Suravee Suthikulpanit <suravee.suthikulpanit@....com>,
        linuxppc-dev <linuxppc-dev@...ts.ozlabs.org>,
        Andre Wild <wild@...ux.vnet.ibm.com>,
        Benjamin Herrenschmidt <benh@....ibm.com>
Subject: Re: [PATCH] sched/topology: Use Identity node only if required

* Peter Zijlstra <peterz@...radead.org> [2018-08-31 12:41:15]:

> On Fri, Aug 31, 2018 at 03:22:48AM -0700, Srikar Dronamraju wrote:
> 
> > At boot: Before topology update.
> 
> How does that work; you do SMP bringup _before_ you know the topology !?
> 

If you look at the other mail that I sent, the system boots to its regular
state with a certain topology. The hypervisor might detect and push topology
updates after the system has been booted and initialized.  This topology
update can happen much much later after boot. We boot with a particular
topology and a later point of time, the topology update event occurs.


> > After topology update.
> > 
> > For CPU 0
> > domain-0: span=0-7 level=SMT
> >  groups: 0:{ span=0 }, 1:{ span=1 }, 2:{ span=2 }, 3:{ span=3 }, 4:{ span=4 }, 5:{ span=5 }, 6:{ span=6 }, 7:{ span=7 }
> >  domain-1: span=0-7,32-39,64-71,96-103,128-135,160-167,192-199,224-231,256-263,288-295 level=DIE
> >   groups: 0:{ span=0-7 cap=8192 }, 32:{ span=32-39 cap=8192 }, 64:{ span=64-71 cap=8192 }, 96:{ span=96-103 cap=8192 }, 128:{ span=128-135 cap=8192 }, 160:{ span=160-167 cap=8192 }, 192:{ span=192-199 cap=8192 }, 224:{ span=224-231 cap=8192 }, 256:{ span=256-263 cap=8192 }, 288:{ span=288-295 cap=8192 }
> >   domain-2: span=0-303 level=NODE
> >    groups: 0:{ span=0-7,32-39,64-71,96-103,128-135,160-167,192-199,224-231,256-263,288-295 cap=81920 }, 8:{ span=8-15,40-47,72-79,104-111,136-143,168-175,200-207,232-239,264-271,296-303 cap=81920 }, 16:{ span=16-23,48-55,80-87,112-119,144-151,176-183,208-215,240-247,272-279 cap=73728 }, 24:{ span=24-31,56-63,88-95,120-127,152-159,184-191,216-223,248-255,280-287 cap=73728 }
> > 

> > For CPU 9
> >  domain-0: span=8-15 level=SMT
> >   groups: 9:{ span=9 }, 10:{ span=10 }, 11:{ span=11 }, 12:{ span=12 }, 13:{ span=13 }, 14:{ span=14 }, 15:{ span=15 }, 8:{ span=8 }
> >   domain-1: span=8-15,40-47,72-79,104-111,136-143,168-175,200-207,232-239,264-271,296-303 level=DIE
> >    groups: 8:{ span=8-15 cap=8192 }, 40:{ span=40-47 cap=8192 }, 72:{ span=72-79 cap=8192 }, 104:{ span=104-111 cap=8192 }, 136:{ span=136-143 cap=8192 }, 168:{ span=168-175 cap=8192 }, 200:{ span=200-207 cap=8192 }, 232:{ span=232-239 cap=8192 }, 264:{ span=264-271 cap=8192 }, 296:{ span=296-303 cap=8192 }
> >    domain-2: span=8-15,40-47,72-79,104-111,136-143,168-175,200-207,232-239,264-271,296-303 level=NODE
> >     groups: 8:{ span=8-15,40-47,72-79,104-111,136-143,168-175,200-207,232-239,264-271,296-303 cap=81920 }
> >     domain-3: span=0-303 level=NUMA
> >      groups: 8:{ span=8-15,40-47,72-79,104-111,136-143,168-175,200-207,232-239,264-271,296-303 cap=81920 }, 16:{ span=16-23,48-55,80-87,112-119,144-151,176-183,208-215,240-247,272-279 cap=73728 }, 24:{ span=24-31,56-63,88-95,120-127,152-159,184-191,216-223,248-255,280-287 cap=73728 }
> > ERROR: groups don't span domain->span
> 
> This is all very confused... and does not include the error we saw
> earlier.

> 
> CPU 0 has: SMT, DIE, NODE
> CPU 8 has: SMT, DIE, NODE, NUMA
> 

This was the same in my previous posting too. Before the topology update
happened, all the cpus would be in SMT, DIE. The topology updates can be
disabled using a kernel parameter topology_updates=off. Its documented under
https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html as

      topology_updates= [KNL, PPC, NUMA] Format: {off} Specify if the kernel
      should ignore (off) topology updates sent by the hypervisor to this
      LPAR.

and is not something new in powerpc.

> Something is completely buggered in your topology setup.
>