netdev - NAPI wait before enabling irq's [was Re: [Cbe-oss-dev] Spider DMA wrongness]

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20061214205432.GA4329@austin.ibm.com>
Date:	Thu, 14 Dec 2006 14:54:32 -0600
From:	linas@...tin.ibm.com (Linas Vepstas)
To:	Benjamin Herrenschmidt <benh@...nel.crashing.org>
Cc:	James K Lewis <jklewis@...ibm.com>, Arnd Bergmann <arnd@...db.de>,
	"cbe-oss-dev@...abs.org" <cbe-oss-dev@...abs.org>,
	cbe-oss-dev-bounces+jklewis=us.ibm.com@...abs.org,
	Christoph Hellwig <hch@....de>, netdev@...r.kernel.org
Subject: NAPI wait before enabling irq's [was Re: [Cbe-oss-dev] Spider DMA wrongness]

On Wed, Nov 08, 2006 at 07:38:12AM +1100, Benjamin Herrenschmidt wrote:
> 
> What about Linas patches to do interrupt mitigation with NAPI polling ?
> That didn't end up working ?

It seems to be "working as designed", which is different than "working
as naively expected".

For large packets: 
-- a packet comes in
-- rx interrupt generated
-- rx interrupts turned off
-- tcp poll function runs, receives packet
-- completes all work before next packet has arrived, 
   so interupts are turned back on.
-- go to start

This results in a high number of interrupts, and a high cpu usage.
We were able to prove that napi works by stalling in the poll function
just long enough to allow the next packet to arrive.  In this case, 
napi works great, and number of irqs is vastly reduced. 

Unfortunately, I could not figure out any simple way of turning this
into acceptable code.  I can't just wait a little bit before turning 
on interrupts.  Some network apps, such as netpipe, want to receive 
something before sending the next thing. Without the interrupt, the
packet just sits there, and the OS doesn't realize (until milliseconds
later) that there's a packet that can be handled.  This is a variant
of the so-called "rotting packet" discussed in the napi docs.

What is needed is for the tcp stack to wait for 
  1500Bytes / (1Gbit/sec) = 12 microsecs
and then poll again. If there are *still* no new packets, then and
only then do we re-enable interrupts. This would require a new napi.
Presuming the network stack folks find it even remotely acceptable.

--linas

-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html