linux-kernel - Re: [PATCH v7 0/5] vfs: Non-blockling buffered fs read (page cache only)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CANP1eJE1ztbfFbxUgStMcPCp7ajGjOZLHHpdG+rN9eLW6YjgYA@mail.gmail.com>
Date:	Mon, 30 Mar 2015 18:35:39 -0400
From:	Milosz Tanski <milosz@...in.com>
To:	Andrew Morton <akpm@...ux-foundation.org>
Cc:	Jeremy Allison <jra@...ba.org>,
	Christoph Hellwig <hch@...radead.org>,
	LKML <linux-kernel@...r.kernel.org>,
	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	"linux-aio@...ck.org" <linux-aio@...ck.org>,
	Mel Gorman <mgorman@...e.de>,
	Volker Lendecke <Volker.Lendecke@...net.de>,
	Tejun Heo <tj@...nel.org>, Jeff Moyer <jmoyer@...hat.com>,
	"Theodore Ts'o" <tytso@....edu>, Al Viro <viro@...iv.linux.org.uk>,
	Linux API <linux-api@...r.kernel.org>,
	Michael Kerrisk <mtk.manpages@...il.com>,
	linux-arch@...r.kernel.org, Dave Chinner <david@...morbit.com>
Subject: Re: [PATCH v7 0/5] vfs: Non-blockling buffered fs read (page cache only)

On Mon, Mar 30, 2015 at 4:37 PM, Andrew Morton
<akpm@...ux-foundation.org> wrote:
> On Mon, 30 Mar 2015 13:32:27 -0700 Jeremy Allison <jra@...ba.org> wrote:
>
>> On Mon, Mar 30, 2015 at 01:26:25PM -0700, Andrew Morton wrote:
>> >
>> > cons:
>> >
>> > d) fincore() is more expensive
>> >
>> > e) fincore() will very occasionally block
>>
>> The above is the killer for Samba. If fincore
>> returns true but when we schedule the pread
>> we block, we're hosed.
>>
>> Once we block, we're done serving clients on the main
>> thread until this returns. That can cause unpredictable
>> response times which can cause client timeouts.
>>
>> A fincore+pread solution that blocks is simply unsafe
>> to use for us. We'll have to stay with the threadpool :-(.
>
> Finally.  Thanks ;)
>
> This implies that the samba main thread also has to avoid any memory
> allocations both direct and within syscall and pagefault - those will
> occasionally exhibit similar worse-case latency. Is this done now?
>
>

It's entirely possible to have an application with a low / semi static
working set, and leave lots of free memory for the kernel especially
for the page cache. For example the Google want to minimize malloc().
So in tcmalloc() they grab large chunks and rarely release it to back
to the OS, in fact old version never shrank it. So you can entirely
avoid stalls in malloc() for many workloads.

-- 
Milosz Tanski
CTO
16 East 34th Street, 15th floor
New York, NY 10016

p: 646-253-9055
e: milosz@...in.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/