linux-kernel - Re: [PATCH 00/27] Latest numa/core release, v16

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20121121224653.GA4164@gmail.com>
Date:	Wed, 21 Nov 2012 23:46:53 +0100
From:	Ingo Molnar <mingo@...nel.org>
To:	Linus Torvalds <torvalds@...ux-foundation.org>
Cc:	David Rientjes <rientjes@...gle.com>, Mel Gorman <mgorman@...e.de>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	linux-mm <linux-mm@...ck.org>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Paul Turner <pjt@...gle.com>,
	Lee Schermerhorn <Lee.Schermerhorn@...com>,
	Christoph Lameter <cl@...ux.com>,
	Rik van Riel <riel@...hat.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Andrea Arcangeli <aarcange@...hat.com>,
	Thomas Gleixner <tglx@...utronix.de>,
	Johannes Weiner <hannes@...xchg.org>,
	Hugh Dickins <hughd@...gle.com>
Subject: Re: [PATCH 00/27] Latest numa/core release, v16


* Linus Torvalds <torvalds@...ux-foundation.org> wrote:

> On Wed, Nov 21, 2012 at 7:10 AM, Ingo Molnar <mingo@...nel.org> wrote:
> >
> > Because scalability slowdowns are often non-linear.
> 
> Only if you hold locks or have other non-cpu-private activity.
> 
> Which the vsyscall code really shouldn't have.

Yeah, the faults accessing any sort of thread shared cache line 
was my main thinking - the vsyscall faults are so hidden, and 
David's transaction score was so low that I could not exclude 
some extremely high page fault rate (which would not get 
reported by anything other than a strange blip on the profile). 
I was thinking of a hundred thousand vsyscall page faults per 
second as a possibility - SPECjbb measures time for every 
transaction.

So this was just a "maybe-that-has-an-effect" blind theory of 
mine - and David's testing did not confirm it so we know it was 
a bad idea.

I basically wanted to see a profile from David that looked as 
flat as mine - that would have excluded a handful of unknown 
unknowns.

> That said, it might be worth removing the 
> "prefetchw(&mm->mmap_sem)" from the VM fault path. Partly 
> because software prefetches have never ever worked on any 
> reasonable hardware, and partly because it could seriously 
> screw up things like the vsyscall stuff.

Yeah, I was wondering about that one too ...

> I think we only turn prefetchw into an actual prefetch 
> instruction on 3DNOW hardware. Which is the *old* AMD chips. I 
> don't think even the Athlon does that.
> 
> Anyway, it might be interesting to see a instruction-level 
> annotated profile of do_page_fault() or whatever

Yes.

> > So with CONFIG_NUMA_BALANCING=y we are taking a higher page 
> > fault rate, in exchange for a speedup.
> 
> The thing is, so is autonuma.
> 
> And autonuma doesn't show any of these problems. [...]

AutoNUMA regresses on this workload, at least on my box:

                         v3.7  AutoNUMA   |  numa/core-v16    [ vs. v3.7]
                        -----  --------   |  -------------    -----------
                                          |
  [ SPECjbb transactions/sec ]            |
  [ higher is better         ]            |
                                          |
  SPECjbb single-1x32    524k     507k    |       638k           +21.7%

It regresses by 3.3% over mainline. [I have not measured a 
THP-disabled number for AutoNUMA.]

Maybe it does not regress on David's box - I have just 
re-checked all of David's mails and AFAICS he has not reported 
AutoNUMA SPECjbb performance.

> Why are you ignoring that fact?

I'm not :-(

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/