Message-ID: <Zqj8zRdEIt4D_fOK@jlelli-thinkpadt14gen4.remote.csb>
Date: Tue, 30 Jul 2024 16:46:37 +0200
From: Juri Lelli <juri.lelli@...hat.com>
To: John Stultz <jstultz@...gle.com>
Cc: LKML <linux-kernel@...r.kernel.org>, Joel Fernandes <joelaf@...gle.com>,
	Qais Yousef <qyousef@...alina.io>, Ingo Molnar <mingo@...hat.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Vincent Guittot <vincent.guittot@...aro.org>,
	Dietmar Eggemann <dietmar.eggemann@....com>,
	Valentin Schneider <vschneid@...hat.com>,
	Steven Rostedt <rostedt@...dmis.org>,
	Ben Segall <bsegall@...gle.com>,
	Zimuzo Ezeozue <zezeozue@...gle.com>,
	Youssef Esmat <youssefesmat@...gle.com>,
	Mel Gorman <mgorman@...e.de>, Will Deacon <will@...nel.org>,
	Waiman Long <longman@...hat.com>, Boqun Feng <boqun.feng@...il.com>,
	"Paul E. McKenney" <paulmck@...nel.org>,
	Metin Kaya <Metin.Kaya@....com>,
	Xuewen Yan <xuewen.yan94@...il.com>,
	K Prateek Nayak <kprateek.nayak@....com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Daniel Lezcano <daniel.lezcano@...aro.org>, kernel-team@...roid.com,
	Luca Abeni <luca.abeni@...tannapisa.it>
Subject: Re: [PATCH v11 0/7] Preparatory changes for Proxy Execution v11

Hi John,

On 09/07/24 13:31, John Stultz wrote:
> Hey All,
> 
> I wanted to send out v11 of the preparatory patches for Proxy
> Execution - an approach for a generalized form of priority
> inheritance. Here again, I’m only submitting the early /
> preparatory changes for review, in the hope that we can move
> these more straightforward patches along and then iteratively
> move through the more interesting patches in the Proxy Execution
> series. That said, I’ve not gotten a ton of feedback with this
> approach, so I’m open to other suggestions.

I'd actually have some additional thoughts on what we discussed at
OSPM24. Hope it's OK if I use this cover letter as a starting point to
possibly discuss that further. Please don't hesitate to tell if you
would rather prefer we have that discussion separately after we agreed
on this first split of the series (I don't think - or I just hope -
whatever we decide about the migration logic will need changes in this
set).

...

> Issues still to address with the full series:

...

> * The chain migration functionality needs further iterations and
>   better validation to ensure it truly maintains the RT/DL load
>   balancing invariants (despite this being broken in vanilla
>   upstream with RT_PUSH_IPI currently)
> * At OSPM, Juri Lelli and the (very very sadly) late Daniel
>   Bristot de Oliveira raised the point that Proxy Exec may not
>   actually be generalizable for SCHED_DEADLINE tasks, as one
>   cannot always correctly donate the resources of the waiter to
>   an owner on a different cpu. If one was to reverse the
>   proxy-migration direction, migrating the owner to the waiter
>   cpu, this would preserve the SCHED_DEADLINE bandwidth
>   calculations, but would break down if the owner's cpu affinity
>   disallowed it. To my understanding this constraint seems to
>   make most forms of priority inheritance infeasible with
>   SCHED_DEADLINE, but I’ll have to leave that to the
>   folks/academics who know it well. After talking with Juri, my
>   current plan is just to special case find_proxy_task() to not
>   proxy with SCHED_DEADLINE (falling back to the current behavior
>   where we deactivate the waiting task). But SCHED_NORMAL waiter
>   tasks would still be able to benefit from Proxy Exec.

So, I've been discussing this a bit with Luca (now cc-ed), Tommaso and
Enrico (which I think you met at OSPM24 and/or at some previous
editions). Please consider that I am essentially thinking out loud, so
I'm pretty sure I'm missing details and possibly be just wrong, but
tl;dr it looks like we could somewhat reconcile the current
implementation (i.e. donors move to owners CPU) to what SCHED_DEADLINE
proxy execution theory (M-BWI [1]) wants if we maybe try to only migrate
the top-waiter (donor, one task) to the owner's CPU, possibly swapping
that with the next highest priority task enqueued on the owner's CPU so
that global invariants are respected. In this case we would leave other
potential donors on their CPUs and either ignore them when picking tasks
for execution or do slightly more fancy things for DEADLINE (can do that
at a later stage, but we would need to consume runtime of DEADLINE
entities even if the owner is running some place else, let's try to
ignore this detail for now I suggest).

Not sure if it makes any sense at all to you/others, but here it is. :)
Hope we can consider the alternative and discuss about it. I actually
wonder if it wouldn't also simplify blocking chains management a bit (no
need to migrate chains around anymore), but I'd guess it might
complicate local scheduling "a bit".

Please let me know what you think and/or if you would like to leave this
for a later stage.

Best,
Juri

1 - https://retis.santannapisa.it/~tommaso/publications/ECRTS-2010.pdf

Powered by blists - more mailing lists