linux-kernel - Re: [PATCH v2 09/10] sched/ext: Relinquish DL server reservations when not needed

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <97260b2e-3f27-4eb2-a66f-86868a44c58a@nvidia.com>
Date: Tue, 3 Jun 2025 00:29:29 -0400
From: Joel Fernandes <joelagnelf@...dia.com>
To: linux-kernel@...r.kernel.org, Tejun Heo <tj@...nel.org>,
 David Vernet <void@...ifault.com>, Andrea Righi <arighi@...dia.com>,
 Changwoo Min <changwoo@...lia.com>, Ingo Molnar <mingo@...hat.com>,
 Peter Zijlstra <peterz@...radead.org>, Juri Lelli <juri.lelli@...hat.com>,
 Vincent Guittot <vincent.guittot@...aro.org>,
 Dietmar Eggemann <dietmar.eggemann@....com>,
 Steven Rostedt <rostedt@...dmis.org>, Ben Segall <bsegall@...gle.com>,
 Mel Gorman <mgorman@...e.de>, Valentin Schneider <vschneid@...hat.com>
Subject: Re: [PATCH v2 09/10] sched/ext: Relinquish DL server reservations
 when not needed



On 6/2/2025 2:01 PM, Joel Fernandes wrote:
> I tested loading a test SCX program and verifying the bandwidth both
> before and after applying the patch:
> 
> Without patch:
> Before loading scx:
>   .dl_bw->total_bw               : 1887408
> After unloading scx:
>   .dl_bw->total_bw               : 3774816
> 
> After patch:
> Before loading scx:
>   .dl_bw->total_bw               : 1887408
> After unloading scx:
>   .dl_bw->total_bw               : 1887408
> 
> Signed-off-by: Joel Fernandes <joelagnelf@...dia.com>
> ---
>  kernel/sched/ext.c | 44 ++++++++++++++++++++++++++++++++++++++++----
>  1 file changed, 40 insertions(+), 4 deletions(-)
> 
> diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c
> index 52f98c3944ed..c938a19cd44f 100644
> --- a/kernel/sched/ext.c
> +++ b/kernel/sched/ext.c
> @@ -4784,13 +4784,28 @@ static void scx_ops_disable_workfn(struct kthread_work *work)
>  	scx_task_iter_stop(&sti);
>  	percpu_up_write(&scx_fork_rwsem);
>  
> -	/*
> -	 * Invalidate all the rq clocks to prevent getting outdated
> -	 * rq clocks from a previous scx scheduler.
> -	 */
>  	for_each_possible_cpu(cpu) {
>  		struct rq *rq = cpu_rq(cpu);
> +		struct rq_flags rf;
> +
> +		/*
> +		 * Invalidate all the rq clocks to prevent getting outdated
> +		 * rq clocks from a previous scx scheduler.
> +		 */
>  		scx_rq_clock_invalidate(rq);
> +
> +		/*
> +		 * We are unloading the sched_ext scheduler, we do not need its
> +		 * DL server bandwidth anymore, remove it for all CPUs. Whenever
> +		 * the first SCX task is enqueued (when scx is re-loaded), its DL
> +		 * server bandwidth will be re-initialized.
> +		 */
> +		rq_lock(rq, &rf);
> +		if (dl_server_active(&rq->ext_server)) {
> +			dl_server_stop(&rq->ext_server);
> +		}
> +		dl_server_remove_params(&rq->ext_server);
> +		rq_unlock(rq, &rf);

I am sorry - this bit also needs to use the irq-disable locking:
		rq_lock_irqsave(rq, &rf);
                ...
		rq_unlock_irqrestore(rq, &rf);

Otherwise the sched_ext self-test hangs. I have squashed it in my tree for next
revision.

thanks,

 - Joel


>  	}
>  
>  	/* no task is on scx, turn off all the switches and flush in-progress calls */
> @@ -5547,6 +5562,27 @@ static int scx_ops_enable(struct sched_ext_ops *ops, struct bpf_link *link)
>  		check_class_changed(task_rq(p), p, old_class, p->prio);
>  	}
>  	scx_task_iter_stop(&sti);
> +
> +	if (scx_switching_all) {
> +		for_each_possible_cpu(cpu) {
> +			struct rq *rq = cpu_rq(cpu);
> +			struct rq_flags rf;
> +
> +			/*
> +			 * We are switching all fair tasks to the sched_ext scheduler,
> +			 * we do not need fair server's DL bandwidth anymore, remove it
> +			 * for all CPUs. Whenever the first CFS task is enqueued (when
> +			 * scx is unloaded), the fair server's DL bandwidth will be
> +			 * re-initialized.
> +			 */
> +			rq_lock_irqsave(rq, &rf);
> +			if (dl_server_active(&rq->fair_server))
> +				dl_server_stop(&rq->fair_server);
> +			dl_server_remove_params(&rq->fair_server);
> +			rq_unlock_irqrestore(rq, &rf);
> +		}
> +	}
> +
>  	percpu_up_write(&scx_fork_rwsem);
>  
>  	scx_ops_bypass(false);