linux-kernel - Re: [PATCH 2/2] [RFC] fadvise: Add _VOLATILE,_ISVOLATILE, and

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20120217162714.09250710@notabene.brown>
Date:	Fri, 17 Feb 2012 16:27:14 +1100
From:	NeilBrown <neilb@...e.de>
To:	Dave Chinner <david@...morbit.com>
Cc:	John Stultz <john.stultz@...aro.org>, linux-kernel@...r.kernel.org,
	Andrew Morton <akpm@...ux-foundation.org>,
	Android Kernel Team <kernel-team@...roid.com>,
	Robert Love <rlove@...gle.com>, Mel Gorman <mel@....ul.ie>,
	Hugh Dickins <hughd@...gle.com>,
	Dave Hansen <dave@...ux.vnet.ibm.com>,
	Rik van Riel <riel@...hat.com>
Subject: Re: [PATCH 2/2] [RFC] fadvise: Add _VOLATILE,_ISVOLATILE, and
 _NONVOLATILE flags

On Fri, 17 Feb 2012 15:45:57 +1100 Dave Chinner <david@...morbit.com> wrote:

> On Wed, Feb 15, 2012 at 12:37:50PM +1100, NeilBrown wrote:
> > On Tue, 14 Feb 2012 16:29:10 -0800 John Stultz <john.stultz@...aro.org> wrote:
> > 
> > > But I'm open to other ideas and arguments.
> > 
> > I didn't notice the original patch, but found it at
> > https://lwn.net/Articles/468837/
> > and had a look.
> > 
> > My first comment is -ENODOC.  A bit background always helps, so let me try to
> > construct that:
> > 
> >  The goal is to allow applications to interact with the kernel's cache
> >  management infrastructure.  In particular an application can say "this
> >  memory contains data that might be useful in the future, but can be
> >  reconstructed if necessary, and it is cheaper to reconstruct it than to read
> >  it back from disk, so don't bother writing it out".
> > 
> >  The proposed mechanism - at a high level - is for user-space to be able to
> >  say "This memory is volatile" and then later "this memory is no longer
> >  volatile".  If the content of the memory is still available the second
> >  request succeeds.  If not, it fails.. Well, actually it succeeds but reports
> >  that some content has been lost. (not sure what happens then - can the app do
> >  a binary search to find which pages it still has or something).
> > 
> >  (technically we should probably include the cost to reconstruct the page,
> >  which the kernel measures as 'seeks' but maybe that isn't necessary).
> > 
> >  This is implemented by using files in a 'tmpfs' filesystem.  These file
> >  support three new flags to fadvise:
> > 
> >  POSIX_FADV_VOLATILE - this marks a range of pages as 'volatile'.  They may be
> >         removed from the page cache as needed, even if they are not 'clean'.
> >  POSIX_FADV_NONVOLATILE - this marks a range of pages as non-volatile.
> >         If any pages in the range were previously volatile but have since been
> >         removed, then a status is returned reporting this.
> >  POSIX_FADV_ISVOLATILE - this does not actually give any advice to the kernel
> >         but rather asks a question: Are any of these pages volatile?
> 
> What about for files that aren't on tmpfs? the fadvise() interface
> is not tmpfs specific, and given that everyone is talking about
> volatility of page cache pages, I fail to see what is tmpfs specific
> about this proposal.

It seems I was looking at an earlier version of the patch which only seemed
to affect tmpfs file.  I see now that the latest version can affect all
filesystems.

> 
> So what are the semantics that are supposed to apply to a file that
> is on a filesystem with stable storage that is cached in the page
> cache?

This is my question too.  Does this make any sense at all for a
storage-backed filesystem?

If I understand the current code (which is by no means certain), then
there is nothing concrete that stops volatile pages from being written back
to storage.  Whether they are or not would be the result of a race between
the 'volatile_shrinker' purging them, and the VM cleaning them.

Given that the volatile_shrinker sets 'seeks = DEFAULT_SEEKS * 4', I would
guess that the VM would get to the pages before the shrinker, but that is
mostly just a guess.

If this really what we want?

Certainly having this clarified in Documents/volatile.txt would help a lot :-)

Thanks,
NeilBrown


Download attachment "signature.asc" of type "application/pgp-signature" (829 bytes)