linux-kernel - Re: [RFC 1/2] deactive invalidated pages

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 23 Nov 2010 18:07:18 +0900
From:	Minchan Kim <minchan.kim@...il.com>
To:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	linux-mm <linux-mm@...ck.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Rik van Riel <riel@...hat.com>,
	Johannes Weiner <hannes@...xchg.org>,
	Nick Piggin <npiggin@...nel.dk>
Subject: Re: [RFC 1/2] deactive invalidated pages

On Tue, Nov 23, 2010 at 6:05 PM, Minchan Kim <minchan.kim@...il.com> wrote:
> On Tue, Nov 23, 2010 at 6:02 PM, KOSAKI Motohiro
> <kosaki.motohiro@...fujitsu.com> wrote:
>>> On Tue, Nov 23, 2010 at 5:01 PM, KOSAKI Motohiro
>>> <kosaki.motohiro@...fujitsu.com> wrote:
>>> >> Hi KOSAKI,
>>> >>
>>> >> 2010/11/23 KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>:
>>> >> >> By Other approach, app developer uses POSIX_FADV_DONTNEED.
>>> >> >> But it has a problem. If kernel meets page is writing
>>> >> >> during invalidate_mapping_pages, it can't work.
>>> >> >> It is very hard for application programmer to use it.
>>> >> >> Because they always have to sync data before calling
>>> >> >> fadivse(..POSIX_FADV_DONTNEED) to make sure the pages could
>>> >> >> be discardable. At last, they can't use deferred write of kernel
>>> >> >> so that they could see performance loss.
>>> >> >> (http://insights.oetiker.ch/linux/fadvise.html)
>>> >> >
>>> >> > If rsync use the above url patch, we don't need your patch.
>>> >> > fdatasync() + POSIX_FADV_DONTNEED should work fine.
>>> >>
>>> >> It works well. But it needs always fdatasync before calling fadvise.
>>> >> For small file, it hurt performance since we can't use the deferred write.
>>> >
>>> > I doubt rsync need to call fdatasync. Why?
>>> >
>>> > If rsync continue to do following loop, some POSIX_FADV_DONTNEED
>>> > may not drop some dirty pages. But they can be dropped at next loop's
>>> > POSIX_FADV_DONTNEED. Then, It doesn't make serious issue.
>>> >
>>> > 1) read
>>> > 2) write
>>> > 3) POSIX_FADV_DONTNEED
>>> > 4) goto 1
>>>
>>> fadvise need pair (offset and len).
>>> if the pair in next turn is different with one's previous turn, it
>>> couldn't be dropped.
>>
>> invalidate_mapping_pages() are using pagevec_lookup() and pagevec_lookup()
>> are using radix tree lookup. Then, Even if rsync always use [0, inf) pair, I don't think
>> it makes much slowdown.
>>
>
> I mean fdatasync causes slowdown, not fadvise.
> if you fadvise(don't need) without fdatasync, you could loss the data.

Oops sorry. Not lose data.
Frequent fdatasync call doesn't use buffered write so that it causes
slowdown than using deferred write.

> --
> Kind regards,
> Minchan Kim
>



-- 
Kind regards,
Minchan Kim
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/