linux-kernel - Re: [PATCH, RFC 00/16] Transparent huge page cache

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite for Android: free password hash cracker in your pocket
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date:	Fri, 05 Apr 2013 09:24:32 +0800
From:	Ric Mason <ric.masonn@...il.com>
To:	Hugh Dickins <hughd@...gle.com>
CC:	"Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
	Andrea Arcangeli <aarcange@...hat.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Al Viro <viro@...iv.linux.org.uk>,
	Wu Fengguang <fengguang.wu@...el.com>, Jan Kara <jack@...e.cz>,
	Mel Gorman <mgorman@...e.de>, linux-mm@...ck.org,
	Andi Kleen <ak@...ux.intel.com>,
	Matthew Wilcox <matthew.r.wilcox@...el.com>,
	"Kirill A. Shutemov" <kirill@...temov.name>,
	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH, RFC 00/16] Transparent huge page cache

Hi Hugh,
On 01/29/2013 01:03 PM, Hugh Dickins wrote:
> On Mon, 28 Jan 2013, Kirill A. Shutemov wrote:
>> From: "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>
>>
>> Here's first steps towards huge pages in page cache.
>>
>> The intend of the work is get code ready to enable transparent huge page
>> cache for the most simple fs -- ramfs.
>>
>> It's not yet near feature-complete. It only provides basic infrastructure.
>> At the moment we can read, write and truncate file on ramfs with huge pages in
>> page cache. The most interesting part, mmap(), is not yet there. For now
>> we split huge page on mmap() attempt.
>>
>> I can't say that I see whole picture. I'm not sure if I understand locking
>> model around split_huge_page(). Probably, not.
>> Andrea, could you check if it looks correct?
>>
>> Next steps (not necessary in this order):
>>   - mmap();
>>   - migration (?);
>>   - collapse;
>>   - stats, knobs, etc.;
>>   - tmpfs/shmem enabling;
>>   - ...
>>
>> Kirill A. Shutemov (16):
>>    block: implement add_bdi_stat()
>>    mm: implement zero_huge_user_segment and friends
>>    mm: drop actor argument of do_generic_file_read()
>>    radix-tree: implement preload for multiple contiguous elements
>>    thp, mm: basic defines for transparent huge page cache
>>    thp, mm: rewrite add_to_page_cache_locked() to support huge pages
>>    thp, mm: rewrite delete_from_page_cache() to support huge pages
>>    thp, mm: locking tail page is a bug
>>    thp, mm: handle tail pages in page_cache_get_speculative()
>>    thp, mm: implement grab_cache_huge_page_write_begin()
>>    thp, mm: naive support of thp in generic read/write routines
>>    thp, libfs: initial support of thp in
>>      simple_read/write_begin/write_end
>>    thp: handle file pages in split_huge_page()
>>    thp, mm: truncate support for transparent huge page cache
>>    thp, mm: split huge page on mmap file page
>>    ramfs: enable transparent huge page cache
>>
>>   fs/libfs.c                  |   54 +++++++++---
>>   fs/ramfs/inode.c            |    6 +-
>>   include/linux/backing-dev.h |   10 +++
>>   include/linux/huge_mm.h     |    8 ++
>>   include/linux/mm.h          |   15 ++++
>>   include/linux/pagemap.h     |   14 ++-
>>   include/linux/radix-tree.h  |    3 +
>>   lib/radix-tree.c            |   32 +++++--
>>   mm/filemap.c                |  204 +++++++++++++++++++++++++++++++++++--------
>>   mm/huge_memory.c            |   62 +++++++++++--
>>   mm/memory.c                 |   22 +++++
>>   mm/truncate.c               |   12 +++
>>   12 files changed, 375 insertions(+), 67 deletions(-)
> Interesting.
>
> I was starting to think about Transparent Huge Pagecache a few
> months ago, but then got washed away by incoming waves as usual.
>
> Certainly I don't have a line of code to show for it; but my first
> impression of your patches is that we have very different ideas of
> where to start.
>
> Perhaps that's good complementarity, or perhaps I'll disagree with
> your approach.  I'll be taking a look at yours in the coming days,
> and trying to summon back up my own ideas to summarize them for you.
>
> Perhaps I was naive to imagine it, but I did intend to start out
> generically, independent of filesystem; but content to narrow down
> on tmpfs alone where it gets hard to support the others (writeback
> springs to mind).  khugepaged would be migrating little pages into
> huge pages, where it saw that the mmaps of the file would benefit
> (and for testing I would hack mmap alignment choice to favour it).
>
> I had arrived at a conviction that the first thing to change was
> the way that tail pages of a THP are refcounted, that it had been a
> mistake to use the compound page method of holding the THP together.
> But I'll have to enter a trance now to recall the arguments ;)

One offline question, do you have any idea hugetlbfs pages support swapping?

>
> Hugh
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@...ck.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@...ck.org"> email@...ck.org </a>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/