linux-kernel - Re: [PATCH v2] sched/fair: Remove the cost of a redundant cpumask_next_wrap in select_idle

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAGsJ_4zpN98_J2aRHyqz4XvSzP+0ngVu2k=ufn9JQNMwe7zZjw@mail.gmail.com>
Date:   Thu, 25 Nov 2021 01:02:00 +1300
From:   Barry Song <21cnbao@...il.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     Ingo Molnar <mingo@...hat.com>, Juri Lelli <juri.lelli@...hat.com>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        Ben Segall <bsegall@...gle.com>, Mel Gorman <mgorman@...e.de>,
        Daniel Bristot de Oliveira <bristot@...hat.com>,
        Barry Song <song.bao.hua@...ilicon.com>
Subject: Re: [PATCH v2] sched/fair: Remove the cost of a redundant
 cpumask_next_wrap in select_idle_cpu

On Thu, Nov 25, 2021 at 12:57 AM Barry Song <21cnbao@...il.com> wrote:
>
> On Thu, Nov 25, 2021 at 12:49 AM Barry Song <21cnbao@...il.com> wrote:
> >
> > On Thu, Nov 25, 2021 at 12:13 AM Peter Zijlstra <peterz@...radead.org> wrote:
> > >
> > > On Wed, Nov 24, 2021 at 05:15:46PM +0800, Barry Song wrote:
> > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > > > index 6e476f6..8cd23f1 100644
> > > > --- a/kernel/sched/fair.c
> > > > +++ b/kernel/sched/fair.c
> > > > @@ -6278,6 +6278,7 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
> > > >               time = cpu_clock(this);
> > > >       }
> > > >
> > > > +     --nr;
> > > >       for_each_cpu_wrap(cpu, cpus, target + 1) {
> > > >               if (has_idle_core) {
> > > >                       i = select_idle_core(p, cpu, cpus, &idle_cpu);
> > > > @@ -6285,11 +6286,11 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, bool
> > > >                               return i;
> > > >
> > > >               } else {
> > > > -                     if (!--nr)
> > > > -                             return -1;
> > > >                       idle_cpu = __select_idle_cpu(cpu, p);
> > > >                       if ((unsigned int)idle_cpu < nr_cpumask_bits)
> > > >                               break;
> > > > +                     if (!--nr)
> > > > +                             return -1;
> > > >               }
> > > >       }
> > >
> > > This way nr can never be 1 for a single iteration -- it current isn't,
> > > but that's besides the point.
> >
> > Yep. nr=1 seems to be a corner case.
> > if nr=1, the original code will return -1 directly without scanning
> > any cpu. but the new code will scan
> > one  time because I haven't checked if(!--nr)  and return before
> > for_each_cpu_wrap(). so this changes
> > the behaviour for this corner case.
> >
> > but if i change "--nr" to "nr--", this new code will scan nr  times
> > rather than nr-1, this changes the behaviour
> > for all cases besides nr!=1. The original code was looping nr times
> > but scanning nr-1 times only.
> >
> > so you prefer here the codes should scan nr times and change the
> > scanning amount from nr-1
> > to nr?
>
> Let me make it clearer. if nr=5, the original code will  loop 5 times,
> but in the 5th loop, it returns directly, so  __select_idle_cpu is
> only done 4 times.
>
> if nr=1, the original code will  loop 1 time, but in the 1st loop,
> it returns directly, so  __select_idle_cpu is  done 0 times.

this is also why in the first version of patch, i did this:
                span_avg = sd->span_weight * avg_idle;
                if (span_avg > 4*avg_cost)
-                       nr = div_u64(span_avg, avg_cost);
+                       nr = div_u64(span_avg, avg_cost) - 1;
                else
-                       nr = 4;
+                       nr = 3;

because we are actually scanning 3 times or div_u64(span_avg, avg_cost) - 1
times but not 4 times or div_u64(span_avg, avg_cost) times.

this is not confusing at all. the only thing which is confusing is the original
code.

>
> if i change the code to if(!nr--), while nr=5, the new code will
> do  __select_idle_cpu() 5 times rather than 4 times in the
> original code.
>
> but of course the new code changes the  __select_idle_cpu
> from zero to one time for the corner case nr==1.
> >
> > Thanks
> > Barry