linux-kernel - kvm virtio ethernet ring on guest side over high throughput (packet per second)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAMrG31x0H1OjtNfdcXq2EqZ20XPHCX8qvrQSSuMqR=hsttYmXQ@mail.gmail.com>
Date:	Tue, 21 Jan 2014 16:06:05 -0200
From:	Alejandro Comisario <alejandro.comisario@...cadolibre.com>
To:	kvm@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: kvm virtio ethernet ring on guest side over high throughput (packet
 per second)

Hi guys, we had in the past when using physical servers, several
throughput issues regarding the throughput of our APIS, in our case we
measure this with packets per seconds, since we dont have that much
bandwidth (Mb/s) since our apis respond lots of packets very small
ones (maximum response of 3.5k and avg response of 1.5k), when we
where using this physical servers, when we reach throughput capacity
(due to clients tiemouts) we touched the ethernet ring configuration
and we made the problem dissapear.

Today with kvm and over 10k virtual instances, when we want to
increase the throughput of KVM instances, we bumped with the fact that
when using virtio on guests, we have a max configuration of the ring
of 256 TX/RX, and from the host side the atached vnet has a txqueuelen
of 500.

What i want to know is, how can i tune the guest to support more
packets per seccond if i know that's my bottleneck?

* does virtio exposes more packets to configure in the virtual ethernet's ring ?
* does the use of vhost_net helps me with increasing packets per
second and not only bandwidth?

does anyone has to struggle with this before and knows where i can look into ?
there's LOOOOOOOOOOOOOOOTS of information about networking performance
tuning of kvm, but nothing related to increase throughput in pps
capacity.

This is a couple of configurations that we are having right now on the
compute nodes:

* 2x1Gb bonded interfaces (want to know the more than 20 models we are
using, just ask for it)
* Multi queue interfaces, pined via irq to different cores
* Linux bridges,  no VLAN, no open-vswitch
* ubuntu 12.04 kernel 3.2.0-[40-48]

any help will be incredibly apreciated !!

thank you.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/