linux-kernel - Re: [PATCH 3/3] [RFC] tmpfs: Add FALLOC_FL_MARK_VOLATILE/UNMARK

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4FC94F61.20305@linaro.org>
Date:	Fri, 01 Jun 2012 16:25:21 -0700
From:	John Stultz <john.stultz@...aro.org>
To:	KOSAKI Motohiro <kosaki.motohiro@...il.com>
CC:	LKML <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Android Kernel Team <kernel-team@...roid.com>,
	Robert Love <rlove@...gle.com>, Mel Gorman <mel@....ul.ie>,
	Hugh Dickins <hughd@...gle.com>,
	Dave Hansen <dave@...ux.vnet.ibm.com>,
	Rik van Riel <riel@...hat.com>,
	Dmitry Adamushko <dmitry.adamushko@...il.com>,
	Dave Chinner <david@...morbit.com>, Neil Brown <neilb@...e.de>,
	Andrea Righi <andrea@...terlinux.com>,
	"Aneesh Kumar K.V" <aneesh.kumar@...ux.vnet.ibm.com>,
	Taras Glek <tgek@...illa.com>, Mike Hommey <mh@...ndium.org>,
	Jan Kara <jack@...e.cz>
Subject: Re: [PATCH 3/3] [RFC] tmpfs: Add FALLOC_FL_MARK_VOLATILE/UNMARK_VOLATILE
 handlers

On 06/01/2012 03:34 PM, KOSAKI Motohiro wrote:
> (6/1/12 5:44 PM), John Stultz wrote:
>> On 06/01/2012 02:37 PM, KOSAKI Motohiro wrote:
>>> (6/1/12 5:03 PM), John Stultz wrote:
>>>> On 06/01/2012 01:17 PM, KOSAKI Motohiro wrote:
>>>>> I like this patch concept. This is cleaner than userland
>>>>> notification quirk. But I don't like you use shrinker. Because of,
>>>>> after applying this patch, normal page reclaim path can still make
>>>>> swap out. this is undesirable. 
>>>> Any recommendations for alternative approaches? What should I be hooking
>>>> into in order to get notified that tmpfs should drop volatile pages?
>>> I thought to modify shmem_write_page(). But other way is also ok to me.
>> So initially the patch used shmem_write_page(), purging ranges if a page
>> was to be swapped (and just dropping it instead). The problem there is
>> that if there's a large range that is very active, we might purge the
>> entire range just because it contains one rarely used page. This is why
>> the LRU list for unpurged volatile ranges is useful.
> ???
> But, volatile marking order is not related to access frequency.

Correct.

>  Why do you
> bother more inaccurate one? At least, pageout() should affect lru order
> of volatile ranges?

Not sure I'm following you here.

The key point is we want volatile ranges to be purged in the order they
were marked volatile.
If we use the page lru via shmem_writeout to trigger range purging, we
wouldn't necessarily get this desired behavior.

That said, Dave's idea is to still use a volatile range LRU, but to free
it via shmem_writeout. This allows us to purge volatile pages before
swapping out pages. I'll be sending a modified patchset out shortly that
does this, hopefully it helps makes this idea clear.

>> However, Dave Hansen just suggested to me on irc the idea of if we're
>> swapping any pages, we might want to just purge a volatile range
>> instead. This allows us to keep the unpurged LRU range list, but just
>> uses write_page as the flag for needing to free memory.
> Can you please elaborate more? I don't understand what's different 
> "just dropping it instead" and "just purge a volatile range instead".
So in the first implementation, on writeout we checked if the page was
in a volatile range, and if so we dropped the page (just unlocking the
page) and marked the range as purged instead of swapping the page out.
This was non-optimal since the entire range was marked purged, but other
volatile pages in that range would not be dropped until writeout was
called on them.

My next implementation purged the entire range (via
shmem_truncate_range) if we did a writeout on a page in that range. This
was better, but still left us open to purging recently marked volatile
ranges if only a single page in that range had not been accessed in awhile.

That's when I added the LRU tracking at the volatile range level (which
reverted back to the behavior ashmem has always used), and have been
using that model sense.

Hopefully this clarifies things. My apologies if I don't always use the
correct terminology, as I'm still a newbie when it comes to VM code.

thanks
-john

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/