linux-kernel - Re: [PATCH v3 0/2] fadvise: support POSIX_FADV

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 27 Jun 2011 14:39:46 +0200
From:	Andrea Righi <andrea@...terlinux.com>
To:	Pádraig Brady <P@...igBrady.com>
Cc:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	akpm@...ux-foundation.org, minchan.kim@...il.com, riel@...hat.com,
	peterz@...radead.org, hannes@...xchg.org,
	kamezawa.hiroyu@...fujitsu.com, aarcange@...hat.com,
	hughd@...gle.com, jamesjer@...terlinux.com, marcus@...ehost.com,
	matt@...ehost.com, tytso@....edu, shaohua.li@...el.com,
	linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v3 0/2] fadvise: support POSIX_FADV_NOREUSE

On Mon, Jun 27, 2011 at 12:53:53PM +0100, Pádraig Brady wrote:
> On 27/06/11 11:29, Andrea Righi wrote:
> > The actual problem I think is that apps expect that DONTNEED can be used
> > to drop cache, but this is not written anywhere in the POSIX standard.
> > 
> > I would also like to have both functionalities: 1) be sure to drop page
> > cache pages (now there's only a system-wide knob to do this:
> > /proc/sys/vm/drop_caches), 2) give an advice to the kernel that I will
> > not reuse some pages in the future.
> > 
> > The standard can only provide 2). If we also want 1) at the file
> > granularity, I think we'd need to introduce something linux specific to
> > avoid having portability problems.
> 
> True, though Linux is the reference for posix_fadvise() implementations,
> given its lack of support on other platforms.
> 
> So just to summarize for _my_ reference.
> You're changing DONTNEED to mean "drop if !PageActive()".
> I.E. according to http://linux-mm.org/PageReplacementDesign
> "drop if files only accessed once".

Drop if pages were only accessed once, they're not mapped by any other
process and they're not unevictable.

> 
> This will mean that there is no way currently to
> remove a particular file from the cache on linux.

Correct. There's not a way to do this for a single file (except running
POSIX_FADV_DONTNEED twice...).

> Hopefully that won't affect any of:
> http://codesearch.google.com/#search/&q=POSIX_FADV_DONTNEED
> 
> Ideally I'd like cache functions for:
>  DROP, ADD, ADD if space¹
> which could correspond to:
>  DONTNEED, WILLNEED, NOREUSE
> but what we're going for are these somewhat overlapping functions:
>  DROP if used once², ADD, ADD if space

IIUC, NOREUSE means "the application will use this range of the file
once". It's something that we do _before_ accessing the file.  And the
kernel needs to remember the ranges of NOREUSE data for each file, so
that page cache can be immediately dropped after the data has been
accessed (if possible).

-Andrea

> 
> cheers,
> Pádraig.
> 
> ¹ Not implemented yet.
> 
> ² Hopefully there are no access patterns a single
> process can do to make a PageActive as that would
> probably not be desired in relation to "Drop if used once"
> functionality.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/