netdev - Re: provide vhost thread per virtqueue for forwarding scenario

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 22 May 2013 17:59:03 +0800
From:	Zang Hongyong <zanghongyong@...wei.com>
To:	"Michael S. Tsirkin" <mst@...hat.com>, <jasowang@...hat.com>
CC:	Qinchuanyu <qinchuanyu@...wei.com>,
	"rusty@...tcorp.com.au" <rusty@...tcorp.com.au>,
	"nab@...ux-iscsi.org" <nab@...ux-iscsi.org>,
	"(netdev@...r.kernel.org)" <netdev@...r.kernel.org>,
	"(kvm@...r.kernel.org)" <kvm@...r.kernel.org>,
	"Zhangjie (HZ)" <zhang.zhangjie@...wei.com>
Subject: Re: provide vhost thread per virtqueue for forwarding scenario

On 2013/5/20 15:43, Michael S. Tsirkin wrote:
> On Mon, May 20, 2013 at 02:11:19AM +0000, Qinchuanyu wrote:
>> Vhost thread provide both tx and rx ability for virtio-net.
>> In the forwarding scenarios, tx and rx share the vhost thread, and throughput is limited by single thread.
>>
>> So I did a patch for provide vhost thread per virtqueue, not per vhost_net.
>>
>> Of course, multi-queue virtio-net is final solution, but it require new version of virtio-net working in guest.
>> If you have to work with suse10,11, redhat 5.x as guest, and want to improve the forward throughput,
>> using vhost thread per queue seems to be the only solution.
> Why is it? If multi-queue works well for you, just update the drivers in
> the guests that you care about. Guest driver backport is not so hard.
>
> In my testing, performance of thread per vq varies: some workloads might
> gain throughput but you get more IPIs and more scheduling overhead, so
> you waste more host CPU per byte. As you create more VMs, this stops
> being a win.
>
>> I did the test with kernel 3.0.27 and qemu-1.4.0, guest is suse11-sp2, and then two vhost thread provide
>> double tx/rx forwarding performance than signal vhost thread.
>> The virtqueue of vhost_blk is 1, so it still use one vhost thread without change.
>>
>> Is there something wrong in this solution? If not, I would list patch later.
>>
>> Best regards
>> King
> Yes, I don't think we want to create threads even more aggressively
> in all cases. I'm worried about scalability as it is.
> I think we should explore a flexible approach, use a thread pool
> (for example, a wq) to share threads between virtqueues,
> switch to a separate thread only if there's free CPU and existing
> threads are busy. Hopefully share threads between vhost instances too.
On Xen platform, network backend pv driver model has evolved to this 
way. Netbacks from all DomUs share a thread pool,
and thread number eaqual to cpu core number.
Is there any plan for kvm paltform?
>


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html