Message-ID: <468E724F.9070505@bull.net>
Date:	Fri, 06 Jul 2007 18:48:15 +0200
From:	Benjamin Thery <benjamin.thery@...l.net>
To:	Linux Containers <containers@...ts.osdl.org>
Cc:	netdev@...r.kernel.org, ebiederm@...ssion.com,
	Daniel Lezcano <dlezcano@...ibm.com>,
	Patrick McHardy <kaber@...sh.net>
Subject: L2 network namespaces + macvlan performances

Following a discussion we had at OLS concerning L2 network namespace
performances and how the new macvlan driver could potentially improve
them, I've ported the macvlan patchset on top of Eric's net namespace
patchset on 2.6.22-rc4-mm2.

A little bit of history:

Some months ago, when we ran some performance tests (using netperf)
on net namespace, we observed the following things:

Using 'etun', the virtual ethernet tunnel driver, and IP routes
from inside a network namespace,

- The throughput is the same as the "normal" case(*)
   (* normal case: no namespace, using physical adapters).
   No regression. Good.

- But the CPU load increases a lot. Bad.
   The reasons are:
	- All checksums are done in software. No hardware offloading.
	- Every TCP packets going through the etun devices are
	  duplicated in ip_forward() before we decrease the ttl.
           (packets are routed between both ends of etun)

We also made some testing with bridges, and obtained the same results:
	CPU load increase:
	- No hardware offloading
	- Packets are duplicated somewhere in the bridge+netfilter
           code (can't remember where right now)


This time, I've replaced the etun interface by the new macvlan,
which should benefits from the hardware offloading capabilities of the
physical adapter and suppress the forwarding stuff.

My test setup is:

   Host A                    Host B
  ______________            ___________
|  _________   |          |           |
| | Netns 1 |  |          |           |
| |         |  |          |           |
| | macvlan0|  |          |           |
| |___|_____|  |          |           |
|     |        |          |           |
|_____|________|          |___________|
       | eth0 (192.168.0.2)     | eth0 (192.168.0.1)
       |                        |
-----------------------------------------
macvlan0 (192.168.0.3)

- netperf runs on host A
- netserver runs on host B
- Adapters speed is 1GB/s

On this setup I ran the following netperf tests: TCP_STREAM, 
TCP_MAERTS, TCP_RR, UDP_STREAM, UDP_RR.

Between the "normal" case and the "net namespace + macvlan" case, 
results are  about the same for both the throughput and the local CPU 
load for the following test types: TCP_MAERTS, TCP_RR, UDP_STREAM, UDP_RR.

macvlan looks like a very good candidate for network namespace in 
these cases.

But, with the TCP_STREAM test, I observed the CPU load is about the
same (that's what we wanted) but the throughput decreases by about 5%:
from 850MB/s down to 810MB/s.
I haven't investigated yet why the throughput decrease in the case.
Does it come from my setup, from macvlan additional treatments, other? 
I don't know yet

Attached to this email you'll find the raw netperf outputs for the 
three cases:

- netperf through a physical adapter, no namespace:
	netperf-results-2.6.22-rc4-mm2-netns1-vanilla.txt	
- netperf through etun, inside a namespace:
	netperf-results-2.6.22-rc4-mm2-netns1-using-etun.txt	
- netperf through macvlan, inside a namespace:
	netperf-results-2.6.22-rc4-mm2-netns1-using-macvlan.txt


macvlan looks promising.

Regards,
Benjamin

-- 
B e n j a m i n   T h e r y  - BULL/DT/Open Software R&D

    http://www.bull.com

View attachment "netperf-results-2.6.22-rc4-mm2-netns1-vanilla.txt" of type "text/plain" (3094 bytes)

View attachment "netperf-results-2.6.22-rc4-mm2-netns1-using-etun.txt" of type "text/plain" (3095 bytes)

View attachment "netperf-results-2.6.22-rc4-mm2-netns1-using-macvlan.txt" of type "text/plain" (3102 bytes)

Powered by blists - more mailing lists