netdev - Re: under-performing bonded interfaces

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CANj2Ebd_6YrPV333mAk2QCE2Mu05rtM_EcHr5d9bKnNaHxHoLQ@mail.gmail.com>
Date:	Wed, 21 Dec 2011 21:28:45 -0500
From:	Simon Chen <simonchennj@...il.com>
To:	Ben Hutchings <bhutchings@...arflare.com>
Cc:	Ben Greear <greearb@...delatech.com>, netdev@...r.kernel.org
Subject: Re: under-performing bonded interfaces

Just a bit more info...
I have 24 cores, and the interrupts for each 10G NIC are distributed
to all 24 cores...
I am using the most recent 3.7.17 driver ixgbe driver from Intel.
layer3+4 xmit policy.

-Simon

On Wed, Dec 21, 2011 at 8:26 PM, Simon Chen <simonchennj@...il.com> wrote:
> Hi folks,
>
> I added an Intel X520 card to both the sender and receiver... Now I
> have two 10G ports on a PCIe 2.0 x8 slot (5Gx8), so the bandwidth of
> the PCI bus shouldn't be the bottleneck.
>
> Now the throughput test gives me around 16Gbps in aggregate. Any ideas
> how I can push closer to 20G? I don't quite understand where the
> bottleneck is now.
>
> Thanks.
> -Simon
>
> On Wed, Nov 16, 2011 at 9:51 PM, Ben Hutchings
> <bhutchings@...arflare.com> wrote:
>> On Wed, 2011-11-16 at 20:38 -0500, Simon Chen wrote:
>>> Thanks, Ben. That's good discovery...
>>>
>>> Are you saying that both 10G NICs are on the same PCIe x4 slot, so
>>> that they're subject to the 12G throughput bottleneck?
>>
>> I assumed you were using 2 ports on the same board, i.e. the same slot.
>> If you were using 1 port each of 2 boards then I would have expected
>> them both to be usable at full speed.  So far as I can remember, PCIe
>> bridges are usually set up so there isn't contention for bandwidth
>> between slots.
>>
>> Ben.
>>
>> --
>> Ben Hutchings, Staff Engineer, Solarflare
>> Not speaking for my employer; that's the marketing department's job.
>> They asked us to note that Solarflare product names are trademarked.
>>
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html