linux-kernel - Re: IRQ balancing on a router

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 3 Oct 2008 06:38:57 -0700
From:	Arjan van de Ven <arjan@...ux.intel.com>
To:	Jan Kasprzak <kas@...muni.cz>
Cc:	linux-kernel@...r.kernel.org
Subject: Re: IRQ balancing on a router

On Fri, 3 Oct 2008 15:21:17 +0200
Jan Kasprzak <kas@...muni.cz> wrote:

> 	Hello,
> 
> I have a dual-CPU router/firewall with five gigabit NICs. Recently I
> have found that irqbalance (0.55 from Fedora 9/x86_64) gives a
> suboptimal IRQ to CPU mapping on this box:
> 
> 	During traffic spikes, it assings two NICs to one CPU, and the
> other three to the second CPU. However, this does not account for
> the fact that packets coming from the uplink interface are way more
> expensive to handle than the rest of the traffic: most iptables rules
> apply to the packets received from the uplink interface. The result is
> that the CPU which receives IRQs for the uplink interface
> is 100 % busy (softirq mostly), while the other one is 90% idle.
> 
> 	Setting the IRQ mapping by hand (uplink to one CPU, all the
> other NICs to the other CPU) makes a well balanced system (both CPUs
> 30-60 % busy). I am not sure whether my configuration is too special,
> but it might be worth trying to make irqbalance daemon cope also with
> this usage pattern.
> 

one of the hard cases for irqbalance is that irqbalance doesn't have a
way to find out the actual cpu time spend in the handlers. For
networking it makes an estimate just based on the number of packets
(which is better than nothing)... but that breaks down if you have an
non-symmetry in CPU costs per packet like you have.

The good news is that irqthreads at least have the potential to solve
this "lack of information"; if not, we could consider doing a form of
microaccounting for irq handlers....


-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/