linux-kernel - Re: [RFC 1/2] deactive invalidated pages

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87mxp09mm2.fsf@gmail.com>
Date:	Tue, 23 Nov 2010 08:48:53 -0500
From:	Ben Gamari <bgamari@...il.com>
To:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>
Cc:	kosaki.motohiro@...fujitsu.com,
	Minchan Kim <minchan.kim@...il.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	linux-mm <linux-mm@...ck.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Rik van Riel <riel@...hat.com>,
	Johannes Weiner <hannes@...xchg.org>,
	Nick Piggin <npiggin@...nel.dk>
Subject: Re: [RFC 1/2] deactive invalidated pages

On Tue, 23 Nov 2010 16:16:55 +0900 (JST), KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com> wrote:
> > On Sun, 21 Nov 2010 23:30:23 +0900, Minchan Kim <minchan.kim@...il.com> wrote:
> > > 
> > > Ben, Remain thing is to modify rsync and use
> > > fadvise(POSIX_FADV_DONTNEED). Could you test it?
> > 
> > Thanks a ton for the patch. Looks good. Testing as we speak.
> 
For the record, this was a little premature. As I spoke the kernel was
building but I still haven't had a chance to take any data. Any
suggestions for how to determine the effect (or hopefully lack thereof)
of rsync on the system's working set?

> If possible, can you please post your rsync patch and your testcase
> (or your rsync option + system memory size info + data size info)?
> 
Patch coming right up.

The original test case is a backup script for my home directory. rsync
is invoked with,

rsync --archive --update --progress --delete --delete-excluded
--exclude-from=~/.backup/exclude --log-file=~/.backup/rsync.log -e ssh
/home/ben ben@...erver:/mnt/backup/current

My home directory is 120 GB with typical delta sizes of tens of
megabytes between backups (although sometimes deltas can be gigabytes,
after which the server has severe interactivity issues). The server is
unfortunately quite memory constrained with only 1.5GB of memory (old
inherited hardware). Given the size of my typical deltas, I'm worried
that even simply walking the directory hierarchy might be enough to push
out my working set.

Looking at the rsync access pattern with strace it seems that it does
a very good job of avoid duplicate reads which is good news for these
patches.

Cheers,

- Ben


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/