linux-kernel - Re: Bad network performance over 2Gbps

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20080416042742.GA25188@1wt.eu>
Date:	Wed, 16 Apr 2008 06:27:42 +0200
From:	Willy Tarreau <w@....eu>
To:	Anton Titov <a.titov@...t.bg>
Cc:	"Kok, Auke" <auke-jan.h.kok@...el.com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	linux-net@...r.kernel.org,
	Jesse Brandeburg <jesse.brandeburg@...el.com>
Subject: Re: Bad network performance over 2Gbps

On Wed, Apr 16, 2008 at 01:36:36AM +0300, Anton Titov wrote:
> On Tue, 2008-04-15 at 13:40 -0700, Kok, Auke wrote:
> > Willy Tarreau wrote:
> > 1) turn the in-kernel IRQBALANCE option off !
> Actually it may be already removed. I remember it being under "Processor
> type and features" and I currently cannot find it there for x86_64
> 
> > 2) use either the userspace `irqbalance` daemon or
> > 3) set smp_affinity manually
> 
> I tried echoing 3 (assuming that CPU0 and CPU1 will share their cache,
> as advised in other mails) into smp_affinity of all ethX interrupts and
> no positive result was observed.

But have you disabled irqbalance before doing this ? (you must reboot
and pass "noirqbalance" on the command line for this).

Also, if you are running on quad-core intel CPUs, I'm told that they're
simply two standard dual-core CPUs in the same case, so there is no
shared cache between any core. You should try to assign all irqs to
CPU0 for a test. It *must* make a difference, in either direction.

> I will try disabling NAPI and limiting e1000 interrupts tomorrow.

I found the parameter name I was speaking about : InterruptThrottleRate.
Beware it's an array with one entry per NIC, so you have to set as many
values as you have NICs. I have always observed huge performance boosts
when using the tunables the driver provides.

Willy

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/