linux-kernel - Re: [PATCH, RFC 00/16] Transparent huge page cache

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.LNX.2.00.1301282041280.27186@eggly.anvils>
Date:	Mon, 28 Jan 2013 21:03:41 -0800 (PST)
From:	Hugh Dickins <hughd@...gle.com>
To:	"Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>
cc:	Andrea Arcangeli <aarcange@...hat.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Al Viro <viro@...iv.linux.org.uk>,
	Wu Fengguang <fengguang.wu@...el.com>, Jan Kara <jack@...e.cz>,
	Mel Gorman <mgorman@...e.de>, linux-mm@...ck.org,
	Andi Kleen <ak@...ux.intel.com>,
	Matthew Wilcox <matthew.r.wilcox@...el.com>,
	"Kirill A. Shutemov" <kirill@...temov.name>,
	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH, RFC 00/16] Transparent huge page cache

On Mon, 28 Jan 2013, Kirill A. Shutemov wrote:
> From: "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>
> 
> Here's first steps towards huge pages in page cache.
> 
> The intend of the work is get code ready to enable transparent huge page
> cache for the most simple fs -- ramfs.
> 
> It's not yet near feature-complete. It only provides basic infrastructure.
> At the moment we can read, write and truncate file on ramfs with huge pages in
> page cache. The most interesting part, mmap(), is not yet there. For now
> we split huge page on mmap() attempt.
> 
> I can't say that I see whole picture. I'm not sure if I understand locking
> model around split_huge_page(). Probably, not.
> Andrea, could you check if it looks correct?
> 
> Next steps (not necessary in this order):
>  - mmap();
>  - migration (?);
>  - collapse;
>  - stats, knobs, etc.;
>  - tmpfs/shmem enabling;
>  - ...
> 
> Kirill A. Shutemov (16):
>   block: implement add_bdi_stat()
>   mm: implement zero_huge_user_segment and friends
>   mm: drop actor argument of do_generic_file_read()
>   radix-tree: implement preload for multiple contiguous elements
>   thp, mm: basic defines for transparent huge page cache
>   thp, mm: rewrite add_to_page_cache_locked() to support huge pages
>   thp, mm: rewrite delete_from_page_cache() to support huge pages
>   thp, mm: locking tail page is a bug
>   thp, mm: handle tail pages in page_cache_get_speculative()
>   thp, mm: implement grab_cache_huge_page_write_begin()
>   thp, mm: naive support of thp in generic read/write routines
>   thp, libfs: initial support of thp in
>     simple_read/write_begin/write_end
>   thp: handle file pages in split_huge_page()
>   thp, mm: truncate support for transparent huge page cache
>   thp, mm: split huge page on mmap file page
>   ramfs: enable transparent huge page cache
> 
>  fs/libfs.c                  |   54 +++++++++---
>  fs/ramfs/inode.c            |    6 +-
>  include/linux/backing-dev.h |   10 +++
>  include/linux/huge_mm.h     |    8 ++
>  include/linux/mm.h          |   15 ++++
>  include/linux/pagemap.h     |   14 ++-
>  include/linux/radix-tree.h  |    3 +
>  lib/radix-tree.c            |   32 +++++--
>  mm/filemap.c                |  204 +++++++++++++++++++++++++++++++++++--------
>  mm/huge_memory.c            |   62 +++++++++++--
>  mm/memory.c                 |   22 +++++
>  mm/truncate.c               |   12 +++
>  12 files changed, 375 insertions(+), 67 deletions(-)

Interesting.

I was starting to think about Transparent Huge Pagecache a few
months ago, but then got washed away by incoming waves as usual.

Certainly I don't have a line of code to show for it; but my first
impression of your patches is that we have very different ideas of
where to start.

Perhaps that's good complementarity, or perhaps I'll disagree with
your approach.  I'll be taking a look at yours in the coming days,
and trying to summon back up my own ideas to summarize them for you.

Perhaps I was naive to imagine it, but I did intend to start out
generically, independent of filesystem; but content to narrow down
on tmpfs alone where it gets hard to support the others (writeback
springs to mind).  khugepaged would be migrating little pages into
huge pages, where it saw that the mmaps of the file would benefit
(and for testing I would hack mmap alignment choice to favour it).

I had arrived at a conviction that the first thing to change was
the way that tail pages of a THP are refcounted, that it had been a
mistake to use the compound page method of holding the THP together.
But I'll have to enter a trance now to recall the arguments ;)

Hugh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/