| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <2fba04b0-e55e-41f4-8b7a-723734fe1ad2@arm.com> Date: Thu, 2 May 2024 11:26:14 +0100 From: Luis Machado <luis.machado@....com> To: Peter Zijlstra <peterz@...radead.org> Cc: mingo@...hat.com, juri.lelli@...hat.com, vincent.guittot@...aro.org, dietmar.eggemann@....com, rostedt@...dmis.org, bsegall@...gle.com, mgorman@...e.de, bristot@...hat.com, vschneid@...hat.com, linux-kernel@...r.kernel.org, kprateek.nayak@....com, wuyun.abel@...edance.com, tglx@...utronix.de, efault@....de, nd <nd@....com>, John Stultz <jstultz@...gle.com>, Hongyan.Xia2@....com Subject: Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue On 4/29/24 15:33, Luis Machado wrote: > Hi Peter, > > On 4/26/24 10:32, Peter Zijlstra wrote: >> On Thu, Apr 25, 2024 at 01:49:49PM +0200, Peter Zijlstra wrote: >>> On Thu, Apr 25, 2024 at 12:42:20PM +0200, Peter Zijlstra wrote: >>> >>>>> I wonder if the delayed dequeue logic is having an unwanted effect on the calculation of >>>>> utilization/load of the runqueue and, as a consequence, we're scheduling things to run on >>>>> higher OPP's in the big cores, leading to poor decisions for energy efficiency. >>>> >>>> Notably util_est_update() gets delayed. Given we don't actually do an >>>> enqueue when a delayed task gets woken, it didn't seem to make sense to >>>> update that sooner. >>> >>> The PELT runnable values will be inflated because of delayed dequeue. >>> cpu_util() uses those in the @boost case, and as such this can indeed >>> affect things. >>> >>> This can also slightly affect the cgroup case, but since the delay goes >>> away as contention goes away, and the cgroup case must already assume >>> worst case overlap, this seems limited. >>> >>> /me goes ponder things moar. >> >> First order approximation of a fix would be something like the totally >> untested below I suppose... > > I gave this a try on the Pixel 6, and I noticed some improvement (see below), but not > enough to bring it back to the original levels. > > (1) m6.6-stock - Basic EEVDF with wakeup preemption fix (baseline) > (2) m6.6-eevdf-complete: m6.6-stock plus this series. > (3) m6.6-eevdf-complete-no-delay-dequeue: (2) + NO_DELAY_DEQUEUE > (4) m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero: (2) + NO_DELAY_DEQUEUE + NO_DELAY_ZERO > (5) m6.6-eevdf-complete-no-delay-zero: (2) + NO_DELAY_ZERO > (6) m6.6-eevdf-complete-pelt-fix: (2) + the proposed load_avg update patch. > > I included (3), (4) and (5) to exercise the impact of disabling the individual > scheduler features. > > > Energy use. > > +------------+------------------------------------------------------+-----------+ > | cluster | tag | perc_diff | > +------------+------------------------------------------------------+-----------+ > | CPU | m6.6-stock | 0.0% | > | CPU-Big | m6.6-stock | 0.0% | > | CPU-Little | m6.6-stock | 0.0% | > | CPU-Mid | m6.6-stock | 0.0% | > | GPU | m6.6-stock | 0.0% | > | Total | m6.6-stock | 0.0% | > | CPU | m6.6-eevdf-complete | 114.51% | > | CPU-Big | m6.6-eevdf-complete | 90.75% | > | CPU-Little | m6.6-eevdf-complete | 98.74% | > | CPU-Mid | m6.6-eevdf-complete | 213.9% | > | GPU | m6.6-eevdf-complete | -7.04% | > | Total | m6.6-eevdf-complete | 100.92% | > | CPU | m6.6-eevdf-complete-no-delay-dequeue | 117.77% | > | CPU-Big | m6.6-eevdf-complete-no-delay-dequeue | 113.79% | > | CPU-Little | m6.6-eevdf-complete-no-delay-dequeue | 97.47% | > | CPU-Mid | m6.6-eevdf-complete-no-delay-dequeue | 189.0% | > | GPU | m6.6-eevdf-complete-no-delay-dequeue | -6.74% | > | Total | m6.6-eevdf-complete-no-delay-dequeue | 103.84% | > | CPU | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | 120.45% | > | CPU-Big | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | 113.65% | > | CPU-Little | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | 99.04% | > | CPU-Mid | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | 201.14% | > | GPU | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | -5.37% | > | Total | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | 106.38% | > | CPU | m6.6-eevdf-complete-no-delay-zero | 119.05% | > | CPU-Big | m6.6-eevdf-complete-no-delay-zero | 107.55% | > | CPU-Little | m6.6-eevdf-complete-no-delay-zero | 98.66% | > | CPU-Mid | m6.6-eevdf-complete-no-delay-zero | 206.58% | > | GPU | m6.6-eevdf-complete-no-delay-zero | -5.25% | > | Total | m6.6-eevdf-complete-no-delay-zero | 105.14% | > | CPU | m6.6-eevdf-complete-pelt-fix | 105.56% | > | CPU-Big | m6.6-eevdf-complete-pelt-fix | 100.45% | > | CPU-Little | m6.6-eevdf-complete-pelt-fix | 94.4% | > | CPU-Mid | m6.6-eevdf-complete-pelt-fix | 150.94% | > | GPU | m6.6-eevdf-complete-pelt-fix | -3.96% | > | Total | m6.6-eevdf-complete-pelt-fix | 93.31% | > +------------+------------------------------------------------------+-----------+ > > Utilization and load levels. > > +---------+------------------------------------------------------+----------+-----------+ > | cluster | tag | variable | perc_diff | > +---------+------------------------------------------------------+----------+-----------+ > | little | m6.6-stock | load | 0.0% | > | little | m6.6-stock | util | 0.0% | > | little | m6.6-eevdf-complete | load | 29.56% | > | little | m6.6-eevdf-complete | util | 55.4% | > | little | m6.6-eevdf-complete-no-delay-dequeue | load | 42.89% | > | little | m6.6-eevdf-complete-no-delay-dequeue | util | 69.47% | > | little | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | load | 51.05% | > | little | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | util | 76.55% | > | little | m6.6-eevdf-complete-no-delay-zero | load | 34.51% | > | little | m6.6-eevdf-complete-no-delay-zero | util | 72.53% | > | little | m6.6-eevdf-complete-pelt-fix | load | 29.96% | > | little | m6.6-eevdf-complete-pelt-fix | util | 59.82% | > | mid | m6.6-stock | load | 0.0% | > | mid | m6.6-stock | util | 0.0% | > | mid | m6.6-eevdf-complete | load | 29.37% | > | mid | m6.6-eevdf-complete | util | 75.22% | > | mid | m6.6-eevdf-complete-no-delay-dequeue | load | 36.4% | > | mid | m6.6-eevdf-complete-no-delay-dequeue | util | 80.28% | > | mid | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | load | 30.35% | > | mid | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | util | 90.2% | > | mid | m6.6-eevdf-complete-no-delay-zero | load | 37.83% | > | mid | m6.6-eevdf-complete-no-delay-zero | util | 93.79% | > | mid | m6.6-eevdf-complete-pelt-fix | load | 33.57% | > | mid | m6.6-eevdf-complete-pelt-fix | util | 67.83% | > | big | m6.6-stock | load | 0.0% | > | big | m6.6-stock | util | 0.0% | > | big | m6.6-eevdf-complete | load | 97.39% | > | big | m6.6-eevdf-complete | util | 12.63% | > | big | m6.6-eevdf-complete-no-delay-dequeue | load | 139.69% | > | big | m6.6-eevdf-complete-no-delay-dequeue | util | 22.58% | > | big | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | load | 125.36% | > | big | m6.6-eevdf-complete-no-delay-dequeue-no-delay-zero | util | 23.15% | > | big | m6.6-eevdf-complete-no-delay-zero | load | 128.56% | > | big | m6.6-eevdf-complete-no-delay-zero | util | 25.03% | > | big | m6.6-eevdf-complete-pelt-fix | load | 130.73% | > | big | m6.6-eevdf-complete-pelt-fix | util | 17.52% | > +---------+------------------------------------------------------+----------+-----------+ Going through the code, my understanding is that the util_est functions seem to be getting called correctly, and in the right order. That is, we first util_est_enqueue, then util_est_dequeue and finally util_est_update. So the stats *should* be correct. On dequeuing (dequeue_task_fair), we immediately call util_est_dequeue, even for the case of a DEQUEUE_DELAYED task, since we're no longer going to run the dequeue_delayed task for now, even though it is still in the rq. We delay the util_est_update of dequeue_delayed tasks until a later time in dequeue_entities. Eventually the dequeue_delayed task will have its lag zeroed when it becomes eligible again, (requeue_delayed_entity) while still being in the rq. It will then get dequeued/enqueued (requeued), and marked as a non-dequeue-delayed task. Next time we attempt to enqueue such a task (enqueue_task_fair), it will skip the ENQUEUE_DELAYED block and call util_est_enqueue. Still, something seems to be signalling that util/load is high, and causing migration to the big cores. Maybe we're not decaying the util/load properly at some point, and inflated numbers start to happen. I'll continue investigating.
Powered by blists - more mailing lists