linux-kernel - Re: [RFC PATCH v1] sched/fair: limit load balance redo times at the same sched

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 25 Jan 2021 14:40:35 +0000
From:   Mel Gorman <mgorman@...hsingularity.net>
To:     "Li, Aubrey" <aubrey.li@...ux.intel.com>
Cc:     Aubrey Li <aubrey.li@...el.com>, mingo@...hat.com,
        peterz@...radead.org, juri.lelli@...hat.com,
        vincent.guittot@...aro.org, dietmar.eggemann@....com,
        rostedt@...dmis.org, bsegall@...gle.com, bristot@...hat.com,
        linux-kernel@...r.kernel.org, Andi Kleen <ak@...ux.intel.com>,
        Tim Chen <tim.c.chen@...ux.intel.com>,
        Srinivas Pandruvada <srinivas.pandruvada@...ux.intel.com>,
        "Rafael J . Wysocki" <rafael.j.wysocki@...el.com>
Subject: Re: [RFC PATCH v1] sched/fair: limit load balance redo times at the
 same sched_domain level

On Mon, Jan 25, 2021 at 09:53:28PM +0800, Li, Aubrey wrote:
> On 2021/1/25 17:06, Mel Gorman wrote:
> > On Mon, Jan 25, 2021 at 02:02:58PM +0800, Aubrey Li wrote:
> >> A long-tail load balance cost is observed on the newly idle path,
> >> this is caused by a race window between the first nr_running check
> >> of the busiest runqueue and its nr_running recheck in detach_tasks.
> >>
> >> Before the busiest runqueue is locked, the tasks on the busiest
> >> runqueue could be pulled by other CPUs and nr_running of the busiest
> >> runqueu becomes 1, this causes detach_tasks breaks with LBF_ALL_PINNED
> >> flag set, and triggers load_balance redo at the same sched_domain level.
> >>
> >> In order to find the new busiest sched_group and CPU, load balance will
> >> recompute and update the various load statistics, which eventually leads
> >> to the long-tail load balance cost.
> >>
> >> This patch introduces a variable(sched_nr_lb_redo) to limit load balance
> >> redo times, combined with sysctl_sched_nr_migrate, the max load balance
> >> cost is reduced from 100+ us to 70+ us, measured on a 4s x86 system with
> >> 192 logical CPUs.
> >>
> >> Cc: Andi Kleen <ak@...ux.intel.com>
> >> Cc: Tim Chen <tim.c.chen@...ux.intel.com>
> >> Cc: Srinivas Pandruvada <srinivas.pandruvada@...ux.intel.com>
> >> Cc: Rafael J. Wysocki <rafael.j.wysocki@...el.com>
> >> Signed-off-by: Aubrey Li <aubrey.li@...ux.intel.com>
> > 
> > If redo_max is a constant, why is it not a #define instead of increasing
> > the size of lb_env?
> > 
> 
> I followed the existing variable sched_nr_migrate_break, I think this might
> be a tunable as well.
> 

I don't think it is, the tunable is sched_nr_migrate and it's not clear
to me at all why sched_nr_migrate_break is not also a #define. It just
happens that sched_nr_migrate == sched_nr_migrate_break by default.

-- 
Mel Gorman
SUSE Labs