netdev - under-performing bonded interfaces

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CANj2EbeMRzicixHwrKpCQ3mMqQ1qbyjbRCxpW+CmVRREAhyeRQ@mail.gmail.com>
Date:	Wed, 16 Nov 2011 18:44:43 -0500
From:	Simon Chen <simonchennj@...il.com>
To:	netdev@...r.kernel.org
Subject: under-performing bonded interfaces

Hello,

I am bonding two 10G interfaces (ixgbe driver) under Debian 6.0.2. The
bonded interface for some reason can only achieve 12Gbps aggregated
throughput. If a single NIC is used, I can get close to 10Gbps.

I've tried different bonding modes (balance-xor, 802.3ad, balance-alb,
balance-tlb), and different xmit hashing policy (layer2, layer2+3,
layer3+4). I've increased all types of kernel parameters for TCP. MTU
on the physical and bonded interface is set to 8000 and 9000. The MTU
on the switch is 9200+.

Instead of nperf (a single server), I also tried my own TCP sender and
receivers.

All those done, still only 12Gbps... How can I really achieve close to 20Gbps?

(I also tried cutting loose the switch in between, and also getting
12G, so not an issue with the switch.)

Thanks.
-Simon
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html