linux-kernel - Re: IRQ / SoftIRQ CPU time accounting broken by 457533a7d3402d1d91fbc125c8bd1bd16dcd3cd4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20090428163004.46733752.akpm@linux-foundation.org>
Date:	Tue, 28 Apr 2009 16:30:04 -0700
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Andrew Gallatin <gallatin@...i.com>
Cc:	linux-kernel@...r.kernel.org, rick.jones2@...com, brice@...i.com,
	Paul Mackerras <paulus@...ba.org>,
	Benjamin Herrenschmidt <benh@...nel.crashing.org>,
	Martin Schwidefsky <schwidefsky@...ibm.com>,
	Ingo Molnar <mingo@...e.hu>
Subject: Re: IRQ / SoftIRQ CPU time accounting broken by
 457533a7d3402d1d91fbc125c8bd1bd16dcd3cd4

(cc's added)

On Thu, 23 Apr 2009 10:19:38 -0400
Andrew Gallatin <gallatin@...i.com> wrote:

> When running netperf for some 10GbE tests, I noticed
> that  IRQ and SOFTIRQ CPU time is no longer reported for an
> otherwise idle CPU on recent kernels, at least for x86_64.
> 
> If I take a 2-CPU system, and bind the NIC IRQ to CPU0, and
> bind the user-space netserver daemon to CPU1, the problem
> is obvious when blasting 10Gb/s of traffic at it.  I see
> no CPU used for irq or softirq on CPU0, even though it is
> handling 13K interrupts/sec:
> 
> 
> 
> % mpstat -P 0 1
> Linux 2.6.30-rc1 (venice)       04/22/09
> 
> 11:25:25     CPU   %user   %nice %system %iowait    %irq   %soft   %idle 
>     intr/s
> 11:25:26       0    0.00    0.00    0.00    0.00    0.00    0.00  100.00 
>   13248.00
> 11:25:27       0    0.00    0.00    0.00    0.00    0.00    0.00  100.00 
>   13280.00
> 
> Common sense tells me that is wrong, and oprofile verifies there is
> a lot happening on CPU0.  Further, when I run a cpu-soaker in
> usermode bound to CPU0, I start to see irq, softirq, etc,
> being correctly identified:
> 
> 11:28:02     CPU   %user   %nice %system %iowait    %irq   %soft   %idle 
>     intr/s
> 11:28:03       0   45.10    0.00    0.00    0.00    1.96   52.94    0.00 
>   13019.61
> 11:28:04       0   46.46    0.00    0.00    0.00    2.02   51.52    0.00 
>   13414.14
> 
> The problem is observable, but much less obvious when using a more
> common, e1000 1GbE NIC (15% softirq is missing, rather than 50%).
> 
> I spent a few hours git-bisecting until I finally got here:
> 
> % git-bisect bad
> Bisecting: 0 revisions left to test after this
> [457533a7d3402d1d91fbc125c8bd1bd16dcd3cd4] fix scaled & unscaled cputime 
> accounting
> 
> I have neither CONFIG_NO_HZ, CONFIG_VIRT_CPU_ACCOUNTING, or XEN configured.
> 

Thanks for doing the bisection.

457533a7d3402d1d91fbc125c8bd1bd16dcd3cd4 was merged late last year, so
this regression has been around for a while.

We might have fixed it in more recent kernels - stranger things have
happened ;)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/