netdev - Constantly varying download rate with a complex xen networking setup, why?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20090615075313.GA22819@rivendell>
Date:	Mon, 15 Jun 2009 09:53:13 +0200
From:	Raphael Hertzog <raphael@...za.com>
To:	netdev@...r.kernel.org
Subject: Constantly varying download rate with a complex xen networking
	setup, why?

[ Please put me in CC of your answers ]

Hello,

I have a weird problem and I'm not sure if it comes from linux or from
Xen. Thus I'd like your opinion and maybe your help so that I can
do a proper bugreport where it belongs. I can do supplementary tests on
request of course.

I probably should try to reproduce it with kvm and a newer kernel but the
hardware where this got tested doesn't have the CPU instructions for
virtualization. If you don't have any idea of what can be wrong in this
setup, I will try to arrange that test but otherwise I would like your
feedback before I do more tests.

(Note the full description below has also been submitted to
http://serverfault.com/questions/22219/)

The network configuration in my Xen setup:
- the dom0 has 3 network cards (eth0, eth1, eth2), 3 brigdes (xenbrE,
  xenbrI, xenbrD) and each brigde integrates the corresponding network
  card. Only xenbrD has an IP address configured (192.168.78.2, a private
  LAN) so that it can discuss with all domU.
- there's a domU that is a firewall/router and it also contains 3 virtual
  cards (eth0, eth1, eth2). It does masquerading for traffic going out on
  eth0 (the external interface which is part of xenbrE).
  # grep vif /etc/xen/xm.slis 
  vif = [ 'mac=00:16:3e:14:85:11, bridge=xenbrE', 'mac=00:16:3e:14:85:12, bridge=xenbrI', 'mac=00:16:3e:14:85:13, bridge=xenbrD' ]

My problem is that when I download a big file from the internet by HTTP in
the dom0, the download rate is not stable. It goes up progressively and then
stalls for a few seconds, and restart again going up progressively (and
all this in loop until the download is complete). During the stalls, it looks
like all networking is blocked on the machine (noticed on interactive SSH
sessions).

    dom0                             │domU
         wget                        │
           ↕                         │
    eth2↔xenbrD(192.168.78.2)↔vif2.2←┼→eth2(192.168.78.1/24)
                                     │   ↕ masquerading
    eth0↔xenbrE↔vif2.0←——————————————┼→eth0(192.168.1.20/24)
     ↕
    internet

If I do the same download but uses a (non-caching) HTTP proxy that runs in
the firewall domU, the download rate is stable at its maximum value.

How can I avoid this problem?

I suspect it's a bug in the networking stack but I would like assistance
to diagnose it more precisely (and maybe find a work-around).

This is a Debian Etch system with Xen 3.2 and the 2.6.26-xen-686 kernel of
Debian Lenny (backports). The bridges are created with
/etc/network/interfaces:

    auto lo
    iface lo inet loopback

    auto xenbrE
    iface xenbrE inet manual
            bridge_ports eth0
            bridge_maxwait 0

    auto xenbrI
    iface xenbrI inet manual
            bridge_ports eth1
            bridge_maxwait 0

    auto xenbrD
    iface xenbrD inet static
            address 192.168.78.2
            netmask 255.255.255.0
            gateway 192.168.78.1
            bridge_ports eth2
            bridge_maxwait 0

The xend configuration is not complicated:

    # grep '^(' /etc/xen/xend-config.sxp 
    (network-script network-dummy)
    (vif-script vif-bridge)
    (dom0-min-mem 150)
    (dom0-cpus 0)
    (vncpasswd '')

And the only routing in dom0 redirects to the domU via xenbrD:

    # route -n
    Kernel IP routing table
    Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
    192.168.78.0    0.0.0.0         255.255.255.0   U     0      0        0 xenbrD
    0.0.0.0         192.168.78.1    0.0.0.0         UG    0      0        0 xenbrD

In the domU, the only iptables configuration done is `iptables -t nat -A
POSTROUTING -s 192.168.78.0/24 -o eth0 -j MASQUERADE`.

Thank you for your help.
-- 
Raphaël Hertzog -+- http://www.ouaza.com

Freexian : des développeurs Debian au service des entreprises
http://www.freexian.com
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html