linux-kernel - Re: Unhelpful caching decisions, possibly related to active/inactive sizing

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 12 Feb 2016 20:35:53 +0100
From:	Andres Freund <andres@...razel.de>
To:	Rik van Riel <riel@...hat.com>
Cc:	Johannes Weiner <hannes@...xchg.org>, linux-mm@...ck.org,
	linux-kernel@...r.kernel.org, Vlastimil Babka <vbabka@...e.cz>
Subject: Re: Unhelpful caching decisions, possibly related to active/inactive
 sizing

On 2016-02-12 13:46:53 +0100, Andres Freund wrote:
> I'm wondering why pages that are repeatedly written to, in units above
> the page size, are promoted to the active list? I mean if there never
> are any reads or re-dirtying an already-dirty page, what's the benefit
> of moving that page onto the active list?

We chatted about this on IRC and you proposed testing this by removing
FGP_ACCESSED in grab_cache_page_write_begin.  I ran tests with that,
after removing the aforementioned code to issue posix_fadvise(DONTNEED)
in postgres.

base (4.5-rc2+10)
        latency average = 3.079 ms
        latency stddev = 8.269 ms
        tps = 10384.545914 (including connections establishing)
        tps = 10384.866341 (excluding connections establishing)

inactive/active patch:
        latency average = 2.931 ms
        latency stddev = 7.683 ms
        tps = 10908.905039 (including connections establishing)
        tps = 10909.256946 (excluding connections establishing)

inactive/active patch + no FGP_ACCESSED in grab_cache_page_write_begin:
        latency average = 2.806 ms
        latency stddev = 7.871 ms
        tps = 11392.893213 (including connections establishing)
        tps = 11393.839826 (excluding connections establishing)

Here the active/inactive lists didn't change as much as I hoped. A bit
of reading made it apparent that the workingset logic in
add_to_page_cache_lru() defated that attempt, by moving an previously
discarded page directly into the active list. I added a variant of
add_to_page_cache_lru() that accepts fgp_flags and only does the
workingset check if FGP_ACCESSED is set. That results in:

inactive/active patch + no FGP_ACCESSED in grab_cache_page_write_begin * add_to_page_cache_lru:
        latency average: 2.292 ms
        latency stddev: 6.487 ms
        tps = 13940.530898 (including connections establishing)
        tps = 13941.774874 (excluding connections establishing)

that's only slightly worse than doing explicit posix_fadvise(DONTNEED)
calls... Pretty good.

To make an actually usable patch out of this it seems we'd have to add a
'partial' argument to grab_cache_page_write_begin(), so writes to parts
of a page still cause the pages to be marked active.  Is it preferrable
to change all callers of grab_cache_page_write_begin and
add_to_page_cache_lru or make them into wrapper functions, and call the
real deal when it matters?

I do think that that's a reasonable algorithmic change, but nonetheless
its obviously possible that such changes regress some workloads. What's
the policy around testing such things?

Greetings,

Andres Freund