linux-kernel - Re: [PATCH] sched: Fix numabalancing to work with isolated cpus

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170406092329.52zzkxlkvitnh44c@hirez.programming.kicks-ass.net>
Date:   Thu, 6 Apr 2017 11:23:29 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     Srikar Dronamraju <srikar@...ux.vnet.ibm.com>,
        Ingo Molnar <mingo@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Mel Gorman <mgorman@...hsingularity.net>,
        Rik van Riel <riel@...hat.com>
Subject: Re: [PATCH] sched: Fix numabalancing to work with isolated cpus

On Thu, Apr 06, 2017 at 09:34:36AM +0200, Michal Hocko wrote:
> On Thu 06-04-17 12:49:50, Srikar Dronamraju wrote:

> > Similar example that I gave in my reply to Mel.
> > 
> > Lets consider 2 node, 24 core with 12 cores in each node.
> > Cores 0-11 in Node 1 and cores 12-23 in the other node.
> > Lets also disable smt/hyperthreading, enable isolcpus from core
> > 6-11,12-17.  Lets run 48 thread ebizzy workload and give it a cpu list
> > of say 11,12-17 using taskset.
> > 
> > Now all the 48 ebizzy threads will only run on core 11. It will never
> > spread to other cores even in the same node(or in the same node/but
> > isolated cpus) or to the different nodes. i.e even if numabalancing is
> > running or not, even if my fix is around or not, all threads will be
> > confined to core 11, even though the cpus_allowed is 11,12-17.

Argh, why such a convoluted example :-(

> Isn't that a bug in isolcpus implementation? It is certainly an
> unexpected behavior I would say. Is this documented anywhere?

Without engaging the brain too much to decipher the example; it does
look right. isolcpus will have no balancing.

> Isn't sched_setaffinity the only way how to actually make it possible to
> run on isolcpus?

Think so.. Personally I hate isolcpus and never use it.

> I would really like to see it confirmed by the scheduler maintainers and
> documented properly as well. What you are claiming here is rather
> surprising to my understanding of what isolcpus acutally is.

isolcpus gets you a set of fully partitioned CPUs. What's surprising
about that?