linux-kernel - Re: [RFC PATCH 0/4] (Take 2): transcendent memory ("tmem") for Linux

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 13 Jul 2009 17:01:12 -0400
From:	Chris Mason <chris.mason@...cle.com>
To:	Anthony Liguori <anthony@...emonkey.ws>
Cc:	Avi Kivity <avi@...hat.com>,
	Dan Magenheimer <dan.magenheimer@...cle.com>,
	Rik van Riel <riel@...hat.com>, linux-kernel@...r.kernel.org,
	npiggin@...e.de, akpm@...l.org, jeremy@...p.org,
	xen-devel@...ts.xensource.com, tmem-devel@....oracle.com,
	alan@...rguk.ukuu.org.uk, linux-mm@...ck.org,
	kurt.hackel@...cle.com, Rusty Russell <rusty@...tcorp.com.au>,
	dave.mccracken@...cle.com, Marcelo Tosatti <mtosatti@...hat.com>,
	sunil.mushran@...cle.com, Schwidefsky <schwidefsky@...ibm.com>,
	Balbir Singh <balbir@...ux.vnet.ibm.com>
Subject: Re: [RFC PATCH 0/4] (Take 2): transcendent memory ("tmem") for
	Linux

On Mon, Jul 13, 2009 at 03:38:45PM -0500, Anthony Liguori wrote:
> Chris Mason wrote:
>> This depends on the extent to which tmem is integrated into the VM.  For
>> filesystem usage, the hooks are relatively simple because we already
>> have a lot of code sharing in this area.  Basically tmem is concerned
>> with when we free a clean page and when the contents of a particular
>> offset in the file are no longer valid.
>>   
>
> But filesystem usage is perhaps the least interesting part of tmem.
>
> The VMM already knows which pages in the guest are the result of disk IO  
> (it's the one that put it there, afterall).  It also knows when those  
> pages have been invalidated (or it can tell based on write-faulting).
>
> The VMM also knows when the disk IO has been rerequested by tracking  
> previous requests.  It can keep the old IO requests cached in memory and  
> use that to satisfy re-reads as long as the memory isn't needed for  
> something else.  Basically, we have tmem today with kvm and we use it by  
> default by using the host page cache to do I/O caching (via  
> cache=writethrough).

I'll definitely grant that caching with writethough adds more caching,
but it does need trim support before it is similar to tmem.  The caching
is transparent to the guest, but it is also transparent to qemu, and so
it is harder to manage and size (or even get a stat for how big it
currently is).

>
> The difference between our "tmem" is that instead of providing an  
> interface where the guest explicitly says, "I'm throwing away this  
> memory, I may need it later", and then asking again for it, the guest  
> throws away the page and then we can later satisfy the disk I/O request  
> that results from re-requesting the page instantaneously.
>
> This transparent approach is far superior too because it enables  
> transparent sharing across multiple guests.  This works well for CoW  
> images and would work really well if we had a file system capable of  
> block-level deduplification... :-)

Grin, I'm afraid that even if someone were to jump in and write the
perfect cow based filesystem and then find a willing contributor to code
up a dedup implementation, each cow image would be a different file
and so it would have its own address space.

Dedup and COW are an easy way to have hints about which pages are
supposed to be have the same contents, but they would have to go with
some other duplicate page sharing scheme.

-chris

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/