linux-kernel - Re: [PATCH v1 0/9] block & aio: kernel aio and loop mq conversion

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CACVXFVNGaZ0C1cA5fq0UnEBDPPMpBDqF2PxNk4TAOOCGL3jh5g@mail.gmail.com>
Date:	Fri, 15 Aug 2014 20:59:56 +0800
From:	Ming Lei <ming.lei@...onical.com>
To:	Jens Axboe <axboe@...nel.dk>
Cc:	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Dave Kleikamp <dave.kleikamp@...cle.com>,
	Zach Brown <zab@...bo.net>, Benjamin LaHaise <bcrl@...ck.org>,
	Christoph Hellwig <hch@...radead.org>,
	Kent Overstreet <kmo@...erainc.com>,
	"open list:AIO" <linux-aio@...ck.org>,
	Linux FS Devel <linux-fsdevel@...r.kernel.org>,
	Dave Chinner <david@...morbit.com>
Subject: Re: [PATCH v1 0/9] block & aio: kernel aio and loop mq conversion

On Fri, Aug 15, 2014 at 12:53 AM, Jens Axboe <axboe@...nel.dk> wrote:
> On 08/14/2014 09:50 AM, Ming Lei wrote:
>> Hi,
>>
>> The 1st two patches introduce kernel AIO support, most of
>> is borrowed from Dave's work last year, and thanks to ITER_BVEC,
>> it is much simper to implement kernel AIO now. AIO model
>> is quite suitable for implementing kernel stuff, and can
>> help improve both throughput and CPU utilization. Lots of
>> kernel components should benefit from it, such as:
>>       - loop driver,
>>       - all kinds of kernel I/O target driver(SCSI, USB storage
>>       or UAS, ...)
>>       - kernel socket users might benefit from it too if socket
>>       AIO is mature
>>
>> The following 6 patches convert current loop driver into blk-mq:
>>       - loop's scalability gets improved much
>>       - loop driver gets quite simplified, and the conversion can
>>       be throught as cleanup too
>>
>> The 9th patch uses kernel AIO with O_DIRECT to improve loop's
>> performance in single job situation, and avoid double cache
>> issue for loop driver too.
>>
>> With the change, loop block's performance can be doubled in my
>> fio test(randread, single job, libaio). If more fio jobs are used,
>> the throughput can be improved much more because of blk-mq.
>>
>> Given loop is used quite widely, especially in VM environment,
>> also the change is quite small, hope it can be merged finally.
>
> Looks pretty damn good to me, even the kernel aio bits are nice and
> simple. Do you have any numbers on loop performance before and after the
> blk-mq conversion? Just curious.

Your concern is right, previous I thought that mq conversion wouldn't improve
throughput, but I did ignore workqueue's concurrency management, it
turns out blk-mq conversion can improvment throughput close to 10 times in
my test(loop over virtio-blk which is backed by one image on SSD).  It is like
POSIX style AIO after mq conversion thanks to workqueue, and I need to
update the performance data in V2.

Actually kernel AIO needn't such high concurrency.

Thanks,
--
Ming Lei
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/