linux-kernel - Re: read()/readv() only from page cache

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20140905154829.GA22956@infradead.org>
Date:	Fri, 5 Sep 2014 08:48:29 -0700
From:	Christoph Hellwig <hch@...radead.org>
To:	Mel Gorman <mgorman@...e.de>
Cc:	Milosz Tanski <milosz@...in.com>,
	LKML <linux-kernel@...r.kernel.org>,
	Volker Lendecke <Volker.Lendecke@...Net.DE>,
	Tejun Heo <tj@...nel.org>, linux-aio@...ck.org
Subject: Re: read()/readv() only from page cache

On Fri, Sep 05, 2014 at 12:09:27PM +0100, Mel Gorman wrote:
> I suggest you look at the recent fincore debate. It did not progress much
> the last time because the author wanted to push a lot of functionality in
> there where as reviewers felt it should start simple.  The simple case is
> likely a good fit for what you want. The primary downside is that it would
> be race-prone in memory pressure situations as the page could be reclaimed
> between the fincore check and the read but I expect that your application
> is already avoiding reclaim activity.

I've actually experimentally hacked up O_NONBLOCK support for regular
files so that it only returns data from the page cache, and not
otherwise.  Volker promised to test it with Samba, but we never made
any progress on it, and just last week a customer told me they would
have liked to use it if it was available.

Note that we might want to also avoid blocking on locks, and I have some
vague memory that we shouldn't actually implement O_NONBLOCK on regular
files due to compatibility options but would have to use a new flag
instead.

Note that mincor/fincore would not help for the usual use case where you
have a non blocking event main loop and want to offload actual blocking
I/O to helper threads, as you it returns information that can be stale
any time.

One further consideration would be to finally implement real buffered
I/O in kernel space by something like the above and offloading to
workqueues in kernelspace.  I think our workqueues now are way better
than any possible user thread pool, although we'd need to find a way to
temporarily tie the work threads to a user address space.

View attachment "support-O_NONBLOCK-reads" of type "text/plain" (2926 bytes)