[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20250722070600.3267819-1-jstultz@google.com>
Date: Tue, 22 Jul 2025 07:05:46 +0000
From: John Stultz <jstultz@...gle.com>
To: LKML <linux-kernel@...r.kernel.org>
Cc: John Stultz <jstultz@...gle.com>, Joel Fernandes <joelagnelf@...dia.com>,
Qais Yousef <qyousef@...alina.io>, Ingo Molnar <mingo@...hat.com>,
Peter Zijlstra <peterz@...radead.org>, Juri Lelli <juri.lelli@...hat.com>,
Vincent Guittot <vincent.guittot@...aro.org>, Dietmar Eggemann <dietmar.eggemann@....com>,
Valentin Schneider <vschneid@...hat.com>, Steven Rostedt <rostedt@...dmis.org>,
Ben Segall <bsegall@...gle.com>, Zimuzo Ezeozue <zezeozue@...gle.com>, Mel Gorman <mgorman@...e.de>,
Will Deacon <will@...nel.org>, Waiman Long <longman@...hat.com>, Boqun Feng <boqun.feng@...il.com>,
"Paul E. McKenney" <paulmck@...nel.org>, Metin Kaya <Metin.Kaya@....com>,
Xuewen Yan <xuewen.yan94@...il.com>, K Prateek Nayak <kprateek.nayak@....com>,
Thomas Gleixner <tglx@...utronix.de>, Daniel Lezcano <daniel.lezcano@...aro.org>,
Suleiman Souhlal <suleiman@...gle.com>, kuyo chang <kuyo.chang@...iatek.com>, hupu <hupu.gm@...il.com>,
kernel-team@...roid.com
Subject: [RFC][PATCH v20 0/6] Donor Migration for Proxy Execution (v20)
Hey All,
As Peter just queued the Single-RQ portion of the Proxy
Execution series, I wanted to start getting some initial review
feedback for the next chunk of the series: Donor Migration
v20 is not very different from v19 of the whole series that
I’ve shared previously, I’ve only rebased it upon Peter’s
sched/core branch, dropping the queued changes, resolving
trivial conflicts and making some small tweaks to drop
CONFIG_SMP conditionals that have been removed in -tip tree,
along with a few minor cleanups.
I’m trying to submit this larger work in smallish digestible
pieces, so in this portion of the series, I’m only submitting
for review and consideration the logic that allows us to do
donor(blocked waiter) migration, allowing us to proxy-execute
lock owners that might be on other cpu runqueues. This requires
some additional changes to locking and extra state tracking to
ensure we don’t accidentally run a migrated donor on a cpu it
isn’t affined to, as well as some extra handling to deal with
balance callback state that needs to be reset when we decide to
pick a different task after doing donor migration.
I’d love to get some initial feedback on any place where these
patches are confusing, or could use additional clarifications.
Also you can find the full proxy-exec series here:
https://github.com/johnstultz-work/linux-dev/commits/proxy-exec-v20-peterz-sched-core/
https://github.com/johnstultz-work/linux-dev.git proxy-exec-v20-peterz-sched-core
Issues still to address with the full series:
* There’s a new quirk from recent changes for dl_server that
is causing the ksched_football test in the full series to hang
at boot. I’ve bisected and reverted the change for now, but I
need to better understand what’s going wrong.
* I spent some more time thinking about Peter’s suggestion to
avoid using the blocked_on_state == BO_WAKING check to protect
against running proxy-migrated tasks on cpus out of their
affinity mask. His suggestion to just dequeue the task prior
to the wakeup in the unlock-wakeup path is more elegant, but
this would be insufficient to protect from other wakeup paths
that don’t dequeue. I’m still thinking if there is a clean way
around this, but I’ve not yet found it.
* Need to sort out what is needed for sched_ext to be ok with
proxy-execution enabled.
* K Prateek Nayak did some testing about a bit over a year ago
with an earlier version of the series and saw ~3-5%
regressions in some cases. Need to re-evaluate this with the
proxy-migration avoidance optimization Suleiman suggested now
implemented.
* The chain migration functionality needs further iterations and
better validation to ensure it truly maintains the RT/DL load
balancing invariants (despite this being broken in vanilla
upstream with RT_PUSH_IPI currently)
I’d really appreciate any feedback or review thoughts on the
full series as well. I’m trying to keep the chunks small,
reviewable and iteratively testable, but if you have any
suggestions on how to improve the series, I’m all ears.
Credit/Disclaimer:
--------------------
As always, this Proxy Execution series has a long history with
lots of developers that deserve credit:
First described in a paper[1] by Watkins, Straub, Niehaus, then
from patches from Peter Zijlstra, extended with lots of work by
Juri Lelli, Valentin Schneider, and Connor O'Brien. (and thank
you to Steven Rostedt for providing additional details here!)
So again, many thanks to those above, as all the credit for this
series really is due to them - while the mistakes are likely
mine.
Thanks so much!
-john
[1] https://static.lwn.net/images/conf/rtlws11/papers/proc/p38.pdf
Cc: Joel Fernandes <joelagnelf@...dia.com>
Cc: Qais Yousef <qyousef@...alina.io>
Cc: Ingo Molnar <mingo@...hat.com>
Cc: Peter Zijlstra <peterz@...radead.org>
Cc: Juri Lelli <juri.lelli@...hat.com>
Cc: Vincent Guittot <vincent.guittot@...aro.org>
Cc: Dietmar Eggemann <dietmar.eggemann@....com>
Cc: Valentin Schneider <vschneid@...hat.com>
Cc: Steven Rostedt <rostedt@...dmis.org>
Cc: Ben Segall <bsegall@...gle.com>
Cc: Zimuzo Ezeozue <zezeozue@...gle.com>
Cc: Mel Gorman <mgorman@...e.de>
Cc: Will Deacon <will@...nel.org>
Cc: Waiman Long <longman@...hat.com>
Cc: Boqun Feng <boqun.feng@...il.com>
Cc: "Paul E. McKenney" <paulmck@...nel.org>
Cc: Metin Kaya <Metin.Kaya@....com>
Cc: Xuewen Yan <xuewen.yan94@...il.com>
Cc: K Prateek Nayak <kprateek.nayak@....com>
Cc: Thomas Gleixner <tglx@...utronix.de>
Cc: Daniel Lezcano <daniel.lezcano@...aro.org>
Cc: Suleiman Souhlal <suleiman@...gle.com>
Cc: kuyo chang <kuyo.chang@...iatek.com>
Cc: hupu <hupu.gm@...il.com>
Cc: kernel-team@...roid.com
John Stultz (5):
locking: Add task::blocked_lock to serialize blocked_on state
kernel/locking: Add blocked_on_state to provide necessary tri-state
for return migration
sched: Add logic to zap balance callbacks if we pick again
sched: Handle blocked-waiter migration (and return migration)
sched: Migrate whole chain in proxy_migrate_task()
Peter Zijlstra (1):
sched: Add blocked_donor link to task for smarter mutex handoffs
include/linux/sched.h | 107 ++++++++-----
init/init_task.c | 4 +
kernel/fork.c | 4 +
kernel/locking/mutex.c | 80 +++++++--
kernel/locking/ww_mutex.h | 17 +-
kernel/sched/core.c | 329 +++++++++++++++++++++++++++++++++++---
kernel/sched/fair.c | 3 +-
kernel/sched/sched.h | 2 +-
8 files changed, 459 insertions(+), 87 deletions(-)
--
2.50.0.727.gbf7dc18ff4-goog
Powered by blists - more mailing lists