netdev - Re: bond + tc regression ?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 05 May 2009 18:31:47 +0200
From:	Eric Dumazet <dada1@...mosbay.com>
To:	Vladimir Ivashchenko <hazard@...ncoudi.com>
CC:	netdev@...r.kernel.org
Subject: Re: bond + tc regression ?

Vladimir Ivashchenko a écrit :
> Hi,
> 
> I have a traffic policing setup running on Linux, serving about 800 mbps
> of traffic. Due to the traffic growth I decided to employ network
> interface bonding to scale over a single GigE.
> 
> The Sun X4150 server has 2xIntel E5450 QuadCore CPUs and a total of four
> built-in e1000e interfaces, which I grouped into two bond interfaces.
> 
> With kernel 2.6.23.1, everything works fine, but the system locked up
> after a few days.
> 
> With kernel 2.6.28.7/2.6.29.1, I get 10-20% packet loss. I get packet loss as
> soon as I put a classful qdisc, even prio, without even having any
> classes or filters. TC prio statistics report lots of drops, around 10k
> per sec. With exactly the same setup on 2.6.23, the number of drops is
> only 50 per sec.
> 
> On both kernels, the system is running with at least 70% idle CPU.
> The network interrupts are distributed accross the cores.

You should not distribute interrupts, but bound a NIC to one CPU
> 
> I thought it was a e1000e driver issue, but tweaking e1000e ring buffers
> didn't help. I tried using e1000 on 2.6.28 by adding necessary PCI IDs,
> I tried running on a different server with bnx cards, I tried disabling
> NO_HZ and HRTICK, but still I have the same problem.
> 
> However, if I don't utilize bond, but just apply rules on normal ethX
> interfaces, there is no packet loss with 2.6.28/29. 
> 
> So, the problem appears only when I use 2.6.28/29 + bond + classful tc
> combination. 
> 
> Any ideas ?
> 

Yes, we need much more information :)
Is it a forwarding setup only ?

cat /proc/interrupts
cat /proc/net/bonding/bond0
cat /proc/net/bonding/bond1
tc -s -d qdisc
mpstat -P ALL 10
ifconfig -a

and so on ...


--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html