[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <4F4322E2.6010308@redhat.com>
Date: Tue, 21 Feb 2012 12:51:46 +0800
From: Jason Wang <jasowang@...hat.com>
To: "Michael S. Tsirkin" <mst@...hat.com>
CC: Anthony Liguori <aliguori@...ibm.com>, netdev@...r.kernel.org,
Tom Lendacky <toml@...ibm.com>,
Cristian Viana <vianac@...ibm.com>
Subject: Re: [PATCH 1/2] vhost: allow multiple workers threads
On 02/19/2012 10:41 PM, Michael S. Tsirkin wrote:
> On Fri, Feb 17, 2012 at 05:02:05PM -0600, Anthony Liguori wrote:
>> > This patch allows vhost to have multiple worker threads for devices such as
>> > virtio-net which may have multiple virtqueues.
>> >
>> > Since virtqueues are a lockless ring queue, in an ideal world data is being
>> > produced by the producer as fast as data is being consumed by the consumer.
>> > These loops will continue to consume data until none is left.
>> >
>> > vhost currently multiplexes the consumer side of the queue on a single thread
>> > by attempting to read from the queue until everything is read or it cannot
>> > process anymore. This means that activity on one queue may stall another queue.
> There's actually an attempt to address this: look up
> VHOST_NET_WEIGHT in the code. I take it, this isn't effective?
>
>> > This is exacerbated when using any form of polling to read from the queues (as
>> > we'll introduce in the next patch). By spawning a thread per-virtqueue, this
>> > is addressed.
>> >
>> > The only problem with this patch right now is how the wake up of the threads is
>> > done. It's essentially a broadcast and we have seen lock contention as a
>> > result.
> On which lock?
>
>> > We've tried some approaches to signal a single thread but I'm not
>> > confident that that code is correct yet so I'm only sending the broadcast
>> > version.
> Yes, that looks like an obvious question.
>
>> > Here are some performance results from this change. There's a modest
>> > improvement with stream although a fair bit of variability too.
>> >
>> > With RR, there's pretty significant improvements as the instance rate drives up.
> Interesting. This was actually tested at one time and we saw
> a significant performance improvement from using
> a single thread especially with a single
> stream in the guest. Profiling indicated that
> with a single thread we get too many context
> switches between TX and RX, since guest networking
> tends to run TX and RX processing on the same
> guest VCPU.
>
> Maybe we were wrong or maybe this went away
> for some reason. I'll see if this can be reproduced.
>
I've tried a similar test in Jan. The test uses one dedicated vhost
thread to handle tx requests and another one for rx. Test result shows
much degradation as the both of the #exits and #irq are increased. There
are some differences as I test between local host and guest, and the
guest does not have very recent virtio changes ( unlocked kick and
exposing index immediately ). I would try the recent kernel.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists