linux-kernel - RE: [PATCH] sched: dynamic config sd

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <SL2PR06MB3082DBB58303601F73FB77D3BD119@SL2PR06MB3082.apcprd06.prod.outlook.com>
Date:   Wed, 16 Mar 2022 02:46:06 +0000
From:   王擎 <wangqing@...o.com>
To:     Dietmar Eggemann <dietmar.eggemann@....com>,
        Russell King <linux@...linux.org.uk>,
        Catalin Marinas <catalin.marinas@....com>,
        Will Deacon <will@...nel.org>,
        Paul Walmsley <paul.walmsley@...ive.com>,
        Palmer Dabbelt <palmer@...belt.com>,
        Albert Ou <aou@...s.berkeley.edu>,
        Sudeep Holla <sudeep.holla@....com>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        "Rafael J. Wysocki" <rafael@...nel.org>,
        Ingo Molnar <mingo@...hat.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Juri Lelli <juri.lelli@...hat.com>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        Ben Segall <bsegall@...gle.com>, Mel Gorman <mgorman@...e.de>,
        Daniel Bristot de Oliveira <bristot@...hat.com>,
        "linux-arm-kernel@...ts.infradead.org" 
        <linux-arm-kernel@...ts.infradead.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-riscv@...ts.infradead.org" <linux-riscv@...ts.infradead.org>
Subject: RE: [PATCH] sched: dynamic config sd_flags if described in DT


>(1) Can you share more information about your CPU topology?
>
>I guess it is a single DSU (DynamIQ Shared Unit) ARMv9 system with 8
>CPUs? So L3 spans over [CPU0..CPU7].
>
>You also mentioned complexes. Am I right in assuming that [CPU0..CPU3]
>are Cortex-A510 cores where each 2 CPUs share a complex?
>
>What kind of uarch are the CPUs in [CPU4..CPU7]? Are they Cortex-A510's
>as well? I'm not sure after reading your email:

Yes, Android systems are currently used default_domain with wrong sd_flags, 
take Qualcomm SM8450 as an example, the CPU and cache topology(1+3+4):
|                           DSU                            |
|           cluster0         |       cluster1     |cluster2|
| core0  core1  core2  core3 |  core4 core5 core6 | core7  |
|   complex0  |   complex1   |  ------------------------   |
|   L2 cache  |   L2 cache   |   L2  |  L2 |  L2  |   L2   |
|                         L3 cache                         |

The sched domain now:
DIE[0-7]  (no SD_SHARE_PKG_RESOURCES)
MC[0-3][4-6][7] (SD_SHARE_PKG_RESOURCES)

The sched domain should be:
DIE[0-7]  (SD_SHARE_PKG_RESOURCES)
MC[0-3][4-6][7] (no SD_SHARE_PKG_RESOURCES)
*CLS[0-1][2-3](SD_SHARE_PKG_RESOURCES)

>https://lkml.kernel.org/r/SL2PR06MB30828CF9FF2879AFC9DC53D2BD0C9@SL2PR06MB3082.apcprd06.prod.outlook.com
>
>You might run into the issue that individual CPUs of your system see a
>different SD hierarchy in case that [CPU4..CPU7] aren't Cortex-A510's,
>i.e. CPUs not sharing complexes.
>
>(2) Related to your MC Sched Domain (SD) layer:
>
>If you have a single DSU ARMv9 system, then in Linux kernel mainline you
>shouldn't have sub-clustering of [CPU0..CPU3] and [CPU4...CPU7].
>
>I.e. the cpu-map entry in your dts file should only list cores, not
>clusters.

But in fact we will, as mentioned above.
>
>I know that in Android the cluster entries are used to sub-group
>different uarch CPUs in an asymmetric CPU capacity system (a.k.a. Arm
>DynamIQ and Phantom domains) but this is eclipsing the true L3 (LLC)
>information and is not "supported" (in the sense of "used") in mainline.
>
>But I have a hard time to see what [CPU0..CPU3] or [CPU4..CPU7] are
>shareing in your system.

They share L3 cache, but no share L2 which only shared within complex.
>
>(3) Why do you want this different SD hierarchy?
>
>I assume in mainline your system will have a single SD which is MC (w/o
>the Phantom domain approach from Android).
>
>You mentioned cpus_share_cache(). Or is it the extra SD level which
>changes the behaviour of CFS load-balancing? I'm just wondering since
>EAS wouldn't be affected here. I'm sure I can understand this better
>once we know more about your CPU topology.

What I want to do is :
1.Config the right sd_llc to sd, better to get it dynamically from DT
2.Benefit from the shared cache(L2) of the complex
i.e. when we look for sibling idle CPU, prior select the L2 shared CPU(inner complex) if L3 is all shared.

Thanks,
Wang