linux-kernel - Re: [RFCv3 PATCH 48/48] sched: Disable energy-unfriendly nohz kicks

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 20 Feb 2015 19:26:38 +0000
From:	Dietmar Eggemann <dietmar.eggemann@....com>
To:	Morten Rasmussen <morten.rasmussen@....com>,
	"peterz@...radead.org" <peterz@...radead.org>,
	"mingo@...hat.com" <mingo@...hat.com>
CC:	"vincent.guittot@...aro.org" <vincent.guittot@...aro.org>,
	"yuyang.du@...el.com" <yuyang.du@...el.com>,
	"preeti@...ux.vnet.ibm.com" <preeti@...ux.vnet.ibm.com>,
	"mturquette@...aro.org" <mturquette@...aro.org>,
	"nico@...aro.org" <nico@...aro.org>,
	"rjw@...ysocki.net" <rjw@...ysocki.net>,
	Juri Lelli <Juri.Lelli@....com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [RFCv3 PATCH 48/48] sched: Disable energy-unfriendly nohz kicks

Hi Morten,

On 04/02/15 18:31, Morten Rasmussen wrote:
> With energy-aware scheduling enabled nohz_kick_needed() generates many
> nohz idle-balance kicks which lead to nothing when multiple tasks get
> packed on a single cpu to save energy. This causes unnecessary wake-ups
> and hence wastes energy. Make these conditions depend on !energy_aware()
> for now until the energy-aware nohz story gets sorted out.
> 
> cc: Ingo Molnar <mingo@...hat.com>
> cc: Peter Zijlstra <peterz@...radead.org>
> 
> Signed-off-by: Morten Rasmussen <morten.rasmussen@....com>
> ---
>  kernel/sched/fair.c | 7 +++++--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 1c248f8..cfe65ae 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -8195,6 +8195,8 @@ static void nohz_idle_balance(struct rq *this_rq, enum cpu_idle_type idle)
>  	clear_bit(NOHZ_BALANCE_KICK, nohz_flags(this_cpu));
>  }
>  
> +static int cpu_overutilized(int cpu, struct sched_domain *sd);
> +
>  /*
>   * Current heuristic for kicking the idle load balancer in the presence
>   * of an idle cpu in the system.
> @@ -8234,12 +8236,13 @@ static inline bool nohz_kick_needed(struct rq *rq)
>  	if (time_before(now, nohz.next_balance))
>  		return false;
>  
> -	if (rq->nr_running >= 2)
> +	sd = rcu_dereference(rq->sd);
> +	if (rq->nr_running >= 2 && (!energy_aware() || cpu_overutilized(cpu, sd)))
>  		return true;

CONFIG_PROVE_RCU checking revealed this one:

[    3.814454] ===============================
[    3.826989] [ INFO: suspicious RCU usage. ]
[    3.839526] 3.19.0-rc7+ #10 Not tainted
[    3.851018] -------------------------------
[    3.863554] kernel/sched/fair.c:8239 suspicious
rcu_dereference_check() usage!
[    3.885216]
[    3.885216] other info that might help us debug this:
[    3.885216]
[    3.909236]
[    3.909236] rcu_scheduler_active = 1, debug_locks = 1
[    3.928817] no locks held by kthreadd/437.

The RCU read-side critical section has to be extended to incorporate
this sd = rcu_dereference(rq->sd):

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index cfe65aec3237..145360ee6e4a 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -8236,11 +8236,13 @@ static inline bool nohz_kick_needed(struct rq *rq)
        if (time_before(now, nohz.next_balance))
                return false;

+       rcu_read_lock();
        sd = rcu_dereference(rq->sd);
-       if (rq->nr_running >= 2 && (!energy_aware() ||
cpu_overutilized(cpu, sd)))
-               return true;
+       if (rq->nr_running >= 2 && (!energy_aware() ||
cpu_overutilized(cpu, sd))) {
+               kick = true;
+               goto unlock;
+       }

-       rcu_read_lock();
        sd = rcu_dereference(per_cpu(sd_busy, cpu));
        if (sd && !energy_aware()) {
                sgc = sd->groups->sgc;

-- Dietmar

>  
>  	rcu_read_lock();
>  	sd = rcu_dereference(per_cpu(sd_busy, cpu));
> -	if (sd) {
> +	if (sd && !energy_aware()) {
>  		sgc = sd->groups->sgc;
>  		nr_busy = atomic_read(&sgc->nr_busy_cpus);
>  
> 


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/