linux-kernel - Re: [LKP] [lkp] [mm] 5c0a85fad9: unixbench.score -6.3% regression

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160613125248.GA30109@black.fi.intel.com>
Date:	Mon, 13 Jun 2016 15:52:48 +0300
From:	"Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>
To:	Linus Torvalds <torvalds@...ux-foundation.org>
Cc:	"Huang, Ying" <ying.huang@...el.com>,
	Rik van Riel <riel@...hat.com>,
	Michal Hocko <mhocko@...e.com>,
	LKML <linux-kernel@...r.kernel.org>,
	Michal Hocko <mhocko@...nel.org>,
	Minchan Kim <minchan@...nel.org>,
	Vinayak Menon <vinmenon@...eaurora.org>,
	Mel Gorman <mgorman@...e.de>,
	Andrew Morton <akpm@...ux-foundation.org>, LKP <lkp@...org>,
	Dave Hansen <dave.hansen@...ux.intel.com>
Subject: Re: [LKP] [lkp] [mm] 5c0a85fad9: unixbench.score -6.3% regression

On Sat, Jun 11, 2016 at 06:02:57PM -0700, Linus Torvalds wrote:
> On Sat, Jun 11, 2016 at 5:49 PM, Huang, Ying <ying.huang@...el.com> wrote:
> >
> > From perf profile, the time spent in page_fault and its children
> > functions are almost same (7.85% vs 7.81%).  So the time spent in page
> > fault and page table operation itself doesn't changed much.  So, you
> > mean CPU may be slower to load the page table entry to TLB if accessed
> > bit is not set?
> 
> So the CPU does take a microfault internally when it needs to set the
> accessed/dirty bit. It's not architecturally visible, but you can see
> it when you do timing loops.
> 
> I've timed it at over a thousand cycles on at least some CPU's, but
> that's still peanuts compared to a real page fault. It shouldn't be
> *that* noticeable, ie no way it's a 6% regression on its own.

Looks like setting accessed bit is the problem.

Withouth mkold:

Score: 1952.9

  Performance counter stats for './Run shell8 -c 1' (3 runs):
 
    468,562,316,621      cycles:u                                                      ( +-  0.02% )
      4,596,299,472      dtlb_load_misses_walk_duration:u                                     ( +-  0.07% )
      5,245,488,559      itlb_misses_walk_duration:u                                     ( +-  0.10% )
 
      189.336404566 seconds time elapsed                                          ( +-  0.01% )

With mkold:

Score: 1885.5

  Performance counter stats for './Run shell8 -c 1' (3 runs):
 
    503,185,676,256      cycles:u                                                      ( +-  0.06% )
      8,137,007,894      dtlb_load_misses_walk_duration:u                                     ( +-  0.85% )
      7,220,632,283      itlb_misses_walk_duration:u                                     ( +-  1.40% )
 
      189.363223499 seconds time elapsed                                          ( +-  0.01% )

We spend 36% more time in page walk only, about 1% of total userspace time.
Combining this with page walk footprint on caches, I guess we can get to
this 3.5% score difference I see.

I'm not sure if there's anything we can do to solve the issue without
screwing relacim logic again. :(

-- 
 Kirill A. Shutemov