linux-kernel - Re: IRQ balancing on a router

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20081003142941.GC3167@redhat.com>
Date:	Fri, 3 Oct 2008 10:29:41 -0400
From:	Jason Baron <jbaron@...hat.com>
To:	Arjan van de Ven <arjan@...ux.intel.com>
Cc:	Jan Kasprzak <kas@...muni.cz>, linux-kernel@...r.kernel.org
Subject: Re: IRQ balancing on a router

On Fri, Oct 03, 2008 at 06:38:57AM -0700, Arjan van de Ven wrote:
> > 	Hello,
> > 
> > I have a dual-CPU router/firewall with five gigabit NICs. Recently I
> > have found that irqbalance (0.55 from Fedora 9/x86_64) gives a
> > suboptimal IRQ to CPU mapping on this box:
> > 
> > 	During traffic spikes, it assings two NICs to one CPU, and the
> > other three to the second CPU. However, this does not account for
> > the fact that packets coming from the uplink interface are way more
> > expensive to handle than the rest of the traffic: most iptables rules
> > apply to the packets received from the uplink interface. The result is
> > that the CPU which receives IRQs for the uplink interface
> > is 100 % busy (softirq mostly), while the other one is 90% idle.
> > 
> > 	Setting the IRQ mapping by hand (uplink to one CPU, all the
> > other NICs to the other CPU) makes a well balanced system (both CPUs
> > 30-60 % busy). I am not sure whether my configuration is too special,
> > but it might be worth trying to make irqbalance daemon cope also with
> > this usage pattern.
> > 
> 
> one of the hard cases for irqbalance is that irqbalance doesn't have a
> way to find out the actual cpu time spend in the handlers. For
> networking it makes an estimate just based on the number of packets
> (which is better than nothing)... but that breaks down if you have an
> non-symmetry in CPU costs per packet like you have.
> 
> The good news is that irqthreads at least have the potential to solve
> this "lack of information"; if not, we could consider doing a form of
> microaccounting for irq handlers....
> 
> 

perhaps, this could be addressed using tracepoints. The currently
proposed ones are at the beginning and end of 'handle_IRQ_event()'. See:
http://marc.info/?l=linux-kernel&m=121616099830280&w=2

thanks,

-Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/