linux-kernel - Re: Scheduling tasks on idle cpu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <CAKfTPtDFZ+Kom5=2Eo8OPaw0v3d0fnxBHgWYLinGp3jHnx0BhA@mail.gmail.com>
Date:   Fri, 15 Apr 2022 09:15:57 +0200
From:   Vincent Guittot <vincent.guittot@...aro.org>
To:     David Laight <David.Laight@...lab.com>
Cc:     Qais Yousef <qais.yousef@....com>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        "mingo@...hat.com" <mingo@...hat.com>,
        "peterz@...radead.org" <peterz@...radead.org>,
        "juri.lelli@...hat.com" <juri.lelli@...hat.com>,
        "rostedt@...dmis.org" <rostedt@...dmis.org>,
        "bsegall@...gle.com" <bsegall@...gle.com>,
        "mgorman@...e.de" <mgorman@...e.de>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "parth@...ux.ibm.com" <parth@...ux.ibm.com>,
        "chris.hyser@...cle.com" <chris.hyser@...cle.com>,
        "pkondeti@...eaurora.org" <pkondeti@...eaurora.org>,
        "Valentin.Schneider@....com" <Valentin.Schneider@....com>,
        "patrick.bellasi@...bug.net" <patrick.bellasi@...bug.net>,
        "pjt@...gle.com" <pjt@...gle.com>, "pavel@....cz" <pavel@....cz>,
        "tj@...nel.org" <tj@...nel.org>,
        "qperret@...gle.com" <qperret@...gle.com>,
        "tim.c.chen@...ux.intel.com" <tim.c.chen@...ux.intel.com>,
        Wei Wang <wvw@...gle.com>
Subject: Re: Scheduling tasks on idle cpu

On Thu, 14 Apr 2022 at 16:14, David Laight <David.Laight@...lab.com> wrote:
>
> From: Vincent Guittot
> > Sent: 14 April 2022 11:17
> ...
> > > > David can rerun is use case after disabling sched_feat(SIS_PROP)
> > >
> > > How would I do that?
> >
> > echo NO_SIS_PROP > /sys/kernel/debug/sched/features
>
> That may not be in the kernel I'm using.
>
> # cat /sys/kernel/debug/sched_features
> GENTLE_FAIR_SLEEPERS START_DEBIT NO_NEXT_BUDDY LAST_BUDDY CACHE_HOT_BUDDY WAKEUP_PREEMPTION ARCH_POWER NO_HRTICK NO_DOUBLE_TICK LB_BIAS NONTASK_POWER TTWU_QUEUE RT_RUNTIME_SHARE NO_LB_MIN NUMA NUMA_FAVOUR_HIGHER NO_NUMA_RESIST_LOWER

SIS_PROP has been normally added in v4.13  so I wonder which kernel
version you are using
Before SIS_PROP, the policy was either to loop all cpu or none
depending of avg_idle and avg_cost

>
> I've been looking at another ftrace output.
> The scheduler does like migrating the process to the current cpu.
> I have seen it migrate from one idle cpu to another idle cpu.
> I've not seen it migrate from an idle cpu to the current cpu.
>   (But I've not looked hard.)
>
> These are all the migrates:
>     TiNG task:12-1005  [026] d... 1111081.796560: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=11 dest_cpu=16
>     TiNG task:31-1026  [005] d... 1111081.836556: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=16 dest_cpu=11
>     TiNG task:28-1023  [033] d... 1111081.856589: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=11 dest_cpu=33
>     TiNG task:11-1004  [013] d... 1111081.856606: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=33 dest_cpu=13
>     TiNG task:19-1012  [002] d... 1111081.896564: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=13 dest_cpu=10
>     TiNG task:26-1019  [008] d... 1111081.956551: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=10 dest_cpu=18
>     TiNG task:34-1029  [001] d... 1111082.016527: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=18 dest_cpu=1
>     TiNG task:20-1013  [021] d... 1111082.016589: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=1 dest_cpu=21
>     TiNG task:32-1027  [000] d... 1111082.036455: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=21 dest_cpu=2
>     TiNG task:15-1008  [006] d... 1111082.056539: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=2 dest_cpu=14
>     TiNG task:34-1029  [001] d... 1111082.076536: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=14 dest_cpu=1
>     TiNG task:21-1014  [004] d... 1111082.076589: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=1 dest_cpu=4
>     TiNG task:11-1004  [013] d... 1111082.096526: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=4 dest_cpu=13
>     TiNG task:28-1023  [033] d... 1111082.096584: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=13 dest_cpu=33
>     TiNG task:25-1018  [029] d... 1111082.116549: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=33 dest_cpu=11
>     TiNG task:27-1020  [032] d... 1111082.176519: sched_migrate_task: comm=RTP sockets pid=990 prio=120 orig_cpu=11 dest_cpu=16
> There are a couple of places where there 2 wakeups before the schedule.
> The scheduler definitely doesn't like waking up a process on an even cpu from on odd one.

odd and even cpus don't belong to the same llc and we only migrate in
the llc at wakeup

> But there are also the 13->33 and 1->21 ones.
>
>         David
>
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
> Registration No: 1397386 (Wales)