linux-kernel - Re: [PATCH 00/13] Reconcile NUMA balancing decisions with the load balancer v6

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 15 May 2020 14:51:28 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     Mel Gorman <mgorman@...hsingularity.net>
Cc:     Jirka Hladky <jhladky@...hat.com>, Phil Auld <pauld@...hat.com>,
        Ingo Molnar <mingo@...nel.org>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Juri Lelli <juri.lelli@...hat.com>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Ben Segall <bsegall@...gle.com>,
        Valentin Schneider <valentin.schneider@....com>,
        Hillf Danton <hdanton@...a.com>,
        LKML <linux-kernel@...r.kernel.org>,
        Douglas Shakshober <dshaks@...hat.com>,
        Waiman Long <longman@...hat.com>,
        Joe Mario <jmario@...hat.com>, Bill Gray <bgray@...hat.com>
Subject: Re: [PATCH 00/13] Reconcile NUMA balancing decisions with the load
 balancer v6

On Fri, May 15, 2020 at 01:22:31PM +0100, Mel Gorman wrote:
> On Fri, May 15, 2020 at 01:28:15PM +0200, Peter Zijlstra wrote:

> > > +			if (val & _TIF_POLLING_NRFLAG)
> > > +				goto activate;
> > 
> > I'm completely confused... the result here is that if you're polling you
> > do _NOT_ queue on the wake_list, but instead immediately enqueue.
> > 
> > (which kinda makes sense, since if the remote CPU is idle, it doesn't
> > have these lines in its cache anyway)
> > 
> 
> Crap, I rushed this and severely confused myself about what is going

Hehe, and here I though I was confused :-)

> on. It is definitely the case that flipping this check does not give
> any benefit. The patch shows a benefit but I'm failing to understand
> exactly why. How I ended up here was perf indicating a lot of time spent
> on smp_cond_load_acquire() which made me look closely at ttwu_remote()
> and looking at function graphs to compare the different types of wakeups
> and their timings.

So the raisin we did this remote wakeup thing in the first place was
that Oracle was having very heavy rq->lock cache-line contention. By
farming off the enqueue to the CPU that was going to run the task
anyway, the rq->lock (and the other runqueue structure lines) could stay
in the CPU that was using them (hard). Less cacheline ping-pong, more
win.

The observation here is that if a CPU is idle, it's rq will not be
contended.