netdev - Re: Strange latency spikes/TX network stalls on Sun Fire X4150(x86) and e1000e

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Wed, 6 Jun 2012 11:46:46 -0700
From:	Stephen Hemminger <shemminger@...tta.com>
To:	David Miller <davem@...emloft.net>
Cc:	therbert@...gle.com, shimoda.hiroaki@...il.com,
	jesse.brandeburg@...el.com, eric.dumazet@...il.com,
	denys@...p.net.lb, netdev@...r.kernel.org,
	e1000-devel@...ts.sourceforge.net, jeffrey.t.kirsher@...el.com
Subject: Re: Strange latency spikes/TX network stalls on Sun Fire X4150(x86)
 and e1000e

On Wed, 06 Jun 2012 11:23:32 -0700 (PDT)
David Miller <davem@...emloft.net> wrote:

> From: Tom Herbert <therbert@...gle.com>
> Date: Wed, 6 Jun 2012 11:21:40 -0700
> 
> > I'm not exactly sure what the exact effect of WTHRESH is here.  Does
> > the device coalesce 5 completions regardless of size?  Would the
> > problem be avoided if bql limit_min were MTU, or could same issue be
> > hit with larger that 64 byte packets?
> 
> The problem is that no TX completions are signalled happen until at
> least WTHRESH are pending.
> 
> BQL is the least of the problems generated by this kind of behavior.
> 
> All drivers must TX complete in a small, finite, amount of time so
> it is absolutely illegal to have the behavior that WRTHRESH > 1
> gives.

The TX completion is also controlled by the programming of the corresponding
interrupt moderation register (EITR).  It makes sense to hold off a little
bit to try and reduce the TX completion interrupt load. 

Intel manual..

Descriptors are written back in one of three cases:
• TXDCTL[n].WTHRESH = 0b and a descriptor which has RS set is ready to be written back
• The corresponding EITR counter has reached zero
• TXDCTL[n].WTHRESH > 0b and TXDCTL[n].WTHRESH descriptors have accumulated

For the first condition, write-backs are immediate. This is the default operation and is backward
compatible with previous device implementations.
The other two conditions are only valid if descriptor bursting is enabled (Section 8.12.13). In the
second condition, the EITR counter is used to force timely write-back of descriptors. The first packet
after timer initialization starts the timer. Timer expiration flushes any accumulated descriptors and sets
an interrupt event (TXDW).
For the final condition, if TXDCTL[n].WTHRESH descriptors are ready for write-back, the write-back is
performed.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html