linux-kernel - Re: [PATCHSET v6] sched: Implement BPF extensible scheduler class

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Mon, 24 Jun 2024 08:11:37 -1000
From: Tejun Heo <tj@...nel.org>
To: Chris Mason <clm@...a.com>
Cc: Thomas Gleixner <tglx@...utronix.de>,
	Linus Torvalds <torvalds@...ux-foundation.org>, mingo@...hat.com,
	peterz@...radead.org, juri.lelli@...hat.com,
	vincent.guittot@...aro.org, dietmar.eggemann@....com,
	rostedt@...dmis.org, bsegall@...gle.com, mgorman@...e.de,
	bristot@...hat.com, vschneid@...hat.com, ast@...nel.org,
	daniel@...earbox.net, andrii@...nel.org, martin.lau@...nel.org,
	joshdon@...gle.com, brho@...gle.com, pjt@...gle.com,
	derkling@...gle.com, haoluo@...gle.com, dvernet@...a.com,
	dschatzberg@...a.com, dskarlat@...cmu.edu, riel@...riel.com,
	changwoo@...lia.com, himadrics@...ia.fr, memxor@...il.com,
	andrea.righi@...onical.com, joel@...lfernandes.org,
	linux-kernel@...r.kernel.org, bpf@...r.kernel.org,
	kernel-team@...a.com
Subject: Re: [PATCHSET v6] sched: Implement BPF extensible scheduler class

Hello,

On Mon, Jun 24, 2024 at 12:42:01PM -0400, Chris Mason wrote:
...
> >     - How is this supposed to work with different applications requiring
> >       different sched_ext schedulers?
> 
> I'll let Tejun pitch in on this one.

Long term, the tentative plan is to support a hierarchy of schedulers where
the intermediate schedulers are responsible for granting CPUs to leaf
schedulers which are responsible for scheduling tasks. Barret Rhoden has a
framework called flux on top of ghost which already implements this albeit
with compile time composition. Nothing is set in stone yet but it's likely
that I'll follow what Barret is doing in many parts.

Taking a step back, because sched_ext currently supports a single
system-wide scheduler, many of the techniques that the current crop of
schedulers are playing with are pretty generic, at least to a class of
problems - e.g. gaming.

Even for scx_layered which is the least generic in a sense, while we it's
also used for a really specific ML setup internally too, what it's more
widely used for is experimenting with things like soft affinity where e.g.
workloads that are not latency sensitive are grouped into few hot running
CPUs which are dynamically scaled to keep caches cleaner (and other effects
too) for the latency sensitive parts of the system. Dan Schatzberg is trying
to generalize that with scx_mitosis.

So, the summry is that there are plans to support a tree of schedulers but
we're currently mostly focusing on more generic single scheduler
experiments.

Thanks.

-- 
tejun