Message-ID: <aBv2AG-VbZ4gcWpi@pavilion.home>
Date: Thu, 8 May 2025 02:08:32 +0200
From: Frederic Weisbecker <frederic@...nel.org>
To: Xi Wang <xii@...gle.com>
Cc: Tejun Heo <tj@...nel.org>, linux-kernel@...r.kernel.org,
	cgroups@...r.kernel.org, Ingo Molnar <mingo@...hat.com>,
	Peter Zijlstra <peterz@...radead.org>,
	Juri Lelli <juri.lelli@...hat.com>,
	Vincent Guittot <vincent.guittot@...aro.org>,
	Dietmar Eggemann <dietmar.eggemann@....com>,
	Steven Rostedt <rostedt@...dmis.org>,
	Ben Segall <bsegall@...gle.com>,
	David Rientjes <rientjes@...gle.com>, Mel Gorman <mgorman@...e.de>,
	Valentin Schneider <vschneid@...hat.com>,
	Waiman Long <longman@...hat.com>,
	Johannes Weiner <hannes@...xchg.org>,
	Michal Koutný <mkoutny@...e.com>,
	Vlastimil Babka <vbabka@...e.cz>,
	Dan Carpenter <dan.carpenter@...aro.org>,
	Chen Yu <yu.c.chen@...el.com>, Kees Cook <kees@...nel.org>,
	Yu-Chun Lin <eleanor15x@...il.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Mickaël Salaün <mic@...ikod.net>,
	jiangshanlai@...il.com
Subject: Re: [RFC/PATCH] sched: Support moving kthreads into cpuset cgroups

Le Wed, May 07, 2025 at 01:07:16PM -0700, Xi Wang a écrit :
> On Wed, May 7, 2025 at 10:36 AM Tejun Heo <tj@...nel.org> wrote:
> >
> > Hello,
> >
> > On Wed, May 07, 2025 at 10:23:24AM -0700, Xi Wang wrote:
> > > Overall I think your arguments depend on kernel and application threads are
> > > significantly different for cpu affinity management, but there isn't enough
> > > evidence for it. If cpuset is a bad idea for kernel threads it's probably not
> > > a good idea for user threads either. Maybe we should just remove cpuset from
> > > kernel and let applications threads go with boot time global variables and
> > > set their own cpu affinities.
> >
> > I can't tell whether you're making a good faith argument. Even if you are,
> > you're making one bold claim without much substance and then jumping to the
> > other extreme based on that. This isn't a productive way to discuss these
> > things.
> >
> > Thanks.
> >
> > --
> > tejun
> 
> Yes this is still serious technical discussion. Frederic made several "we can't
> have b because we already have / are working on a" statements which were not
> very actionable. Deducing to a particular case is a quick way to simplify.

I referred to a particular case (isolation) because you said this is your
usecase. You still haven't explained us why the current affinity management for
kthreads doesn't work for you.

> I'd prefer to focus more on higher level technical tradeoffs.
> 
> Overall compartmentalization limits resource (cpu) sharing which limits
> overcommit thus efficiency.
> cpumask restrictions are not ideal but sometimes
> necessary. Dynamically configurable cpumasks are better than statically
> reserved cpus.

For which usecase?

> I do think the cgroup tree structure sometimes helps and we don't have to use
> it for all cases.

Also kernel threads are special beasts, even some !PF_NO_SETAFFINTIY kthreads
have actual affinity preferences. If they can go through cpusets, this must be
dealt with. And admins will need to know about those affinity preferences for
each kthreads.

Also do we want to be able to expose all the cgroup limits to kthreads? Even
if only cpusets is allowed to have kthreads, does cpusets.mems make
sense to be exposed for example?

If your issue is ever resolved through cpusets, this will have to be maintained
forever with all those subtleties in mind.

I tend to think that CPU isolation is a very straightforward cpusets usecase:
no load balancing, NULL domains and tasks usually don't compete much for the
CPU since the point is to not be disturbed anyway.

And NULL domains already exclude kernel threads, dynamically. So please give
us a compelling reason for doing this.

Thanks.

-- 
Frederic Weisbecker
SUSE Labs

Powered by blists - more mailing lists