linux-kernel - Re: [ANNOUNCE] Native Linux KVM tool v2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4DF93010.1040006@codemonkey.ws>
Date:	Wed, 15 Jun 2011 17:20:00 -0500
From:	Anthony Liguori <anthony@...emonkey.ws>
To:	Alexander Graf <agraf@...e.de>
CC:	Prasad Joshi <prasadjoshi124@...il.com>,
	Pekka Enberg <penberg@...nel.org>, Avi Kivity <avi@...hat.com>,
	linux-kernel@...r.kernel.org, kvm@...r.kernel.org,
	Andrew Morton <akpm@...ux-foundation.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Ingo Molnar <mingo@...e.hu>,
	Sasha Levin <levinsasha928@...il.com>,
	Cyrill Gorcunov <gorcunov@...il.com>,
	Asias He <asias.hejun@...il.com>,
	Jens Axboe <jaxboe@...ionio.com>
Subject: Re: [ANNOUNCE] Native Linux KVM tool v2

On 06/15/2011 05:07 PM, Alexander Graf wrote:
>
> On 16.06.2011, at 00:04, Anthony Liguori wrote:
>
>> On 06/15/2011 03:13 PM, Prasad Joshi wrote:
>>> On Wed, Jun 15, 2011 at 6:10 PM, Pekka Enberg<penberg@...nel.org>   wrote:
>>>> On Wed, Jun 15, 2011 at 7:30 PM, Avi Kivity<avi@...hat.com>   wrote:
>>>>> On 06/15/2011 06:53 PM, Pekka Enberg wrote:
>>>>>>
>>>>>> - Fast QCOW2 image read-write support beating Qemu in fio benchmarks. See
>>>>>> the
>>>>>>    following URL for test result details: https://gist.github.com/1026888
>>>>>
>>>>> This is surprising.  How is qemu invoked?
>>>>
>>>> Prasad will have the details. Please note that the above are with Qemu
>>>> defaults which doesn't use virtio. The results with virtio are little
>>>> better but still in favor of tools/kvm.
>>>>
>>>
>>> The qcow2 image used for testing was copied on to /dev/shm to avoid
>>> the disk delays in performance measurement.
>>>
>>> QEMU was invoked with following parameters
>>>
>>> $ qemu-system-x86_64 -hda<disk image on hard disk>   -hdb
>>> /dev/shm/test.qcow2 -m 1024M
>>
>> Looking more closely at native KVM tools, you would need to use the following invocation to have an apples-to-apples comparison:
>>
>> qemu-system-x86_64 -drive file=/dev/shm/test.qcow2,cache=writeback,if=virtio
>
> Wouldn't this still be using threaded AIO mode? I thought KVM tools used native AIO?

Nope.  The relevant code is:

> 	/* blk device ?*/
> 	disk		= blkdev__probe(filename, &st);
> 	if (disk)
> 		return disk;
>
> 	fd		= open(filename, readonly ? O_RDONLY : O_RDWR);
> 	if (fd < 0)
> 		return NULL;
>
> 	/* qcow image ?*/
> 	disk		= qcow_probe(fd, readonly);
> 	if (disk)
> 		return disk;
>
> 	/* raw image ?*/
> 	disk		= raw_image__probe(fd, &st, readonly);
> 	if (disk)
> 		return disk;

It uses a synchronous I/O model similar to qcow2 in QEMU with what I 
assume is a global lock that's outside of the actual implementation.

I think it lacks some of the caching that Kevin's added recently though 
so I assume that if QEMU was run with cache=writeback, it would probably 
do quite a bit better than native KVM tool.

It also turns out that while they have the infrastructure to deal with 
FLUSH, they don't implement it for qcow2 :-/

So even if the guest does an fsync(), it native KVM tool will never 
actually sync the data to disk...

That's probably why it's fast, it doesn't preserve data integrity :(

Regards,

Anthony Liguori

>
>
> Alex
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/