netdev - Re: igb and bnx2: "NETDEV WATCHDOG: transmit queue timed out" when skb has huge linear buffer

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <52F35CC1.60401@citrix.com>
Date:	Thu, 6 Feb 2014 09:58:25 +0000
From:	Zoltan Kiss <zoltan.kiss@...rix.com>
To:	Andrew Cooper <andrew.cooper3@...rix.com>,
	Michael Chan <mchan@...adcom.com>
CC:	Jeff Kirsher <jeffrey.t.kirsher@...el.com>,
	Jesse Brandeburg <jesse.brandeburg@...el.com>,
	Bruce Allan <bruce.w.allan@...el.com>,
	Carolyn Wyborny <carolyn.wyborny@...el.com>,
	Don Skidmore <donald.c.skidmore@...el.com>,
	Greg Rose <gregory.v.rose@...el.com>,
	Peter P Waskiewicz Jr <peter.p.waskiewicz.jr@...el.com>,
	Alex Duyck <alexander.h.duyck@...el.com>,
	John Ronciak <john.ronciak@...el.com>,
	Tushar Dave <tushar.n.dave@...el.com>,
	Akeem G Abodunrin <akeem.g.abodunrin@...el.com>,
	"David S. Miller" <davem@...emloft.net>,
	<e1000-devel@...ts.sourceforge.net>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	<linux-kernel@...r.kernel.org>,
	"xen-devel@...ts.xenproject.org" <xen-devel@...ts.xenproject.org>
Subject: Re: igb and bnx2: "NETDEV WATCHDOG: transmit queue timed out" when
 skb has huge linear buffer

On 05/02/14 20:43, Andrew Cooper wrote:
> On 05/02/2014 20:23, Zoltan Kiss wrote:
>> On 04/02/14 19:47, Michael Chan wrote:
>>> On Fri, 2014-01-31 at 14:29 +0100, Zoltan Kiss wrote:
>>>> [ 5417.275472] WARNING: at net/sched/sch_generic.c:255
>>>> dev_watchdog+0x156/0x1f0()
>>>> [ 5417.275474] NETDEV WATCHDOG: eth1 (bnx2): transmit queue 2 timed out
>>>
>>> The dump shows an internal IRQ pending on MSIX vector 2 which matches
>>> the the queue number that is timing out.  I don't know what happened to
>>> the MSIX and why the driver is not seeing it.  Do you see an IRQ error
>>> message from the kernel a few seconds before the tx timeout message?
>>
>> I haven't seen any IRQ related error message. Note, this is on Xen
>> 4.3.1. Now I have new results with a reworked version of the patch,
>> unfortunately it still has this issue. Here is a bnx2 dump, lspci
>> output and some Xen debug output (MSI and interrupt bindings, I have
>> more if needed).
>
> You need debug-keys 'Q' as well to map between the PCI devices and Xen IRQs
>
> ~Andrew
>

I could have it after reboot:

(XEN) [2014-02-06 09:44:34] 0000:02:00.0 - dom 0   - MSIs < 64 65 66 67 
68 69 >

So the relevant MSI informations:

(XEN) [2014-02-05 20:15:20]  MSI-X   64 vec=d7  fixed  edge   assert 
phys    cpu dest=00000022 mask=1/0/0
(XEN) [2014-02-05 20:15:20]  MSI-X   65 vec=ba  fixed  edge   assert 
phys    cpu dest=00000000 mask=1/0/0
(XEN) [2014-02-05 20:15:20]  MSI-X   66 vec=92  fixed  edge   assert 
phys    cpu dest=00000022 mask=1/0/0
(XEN) [2014-02-05 20:15:20]  MSI-X   67 vec=3a  fixed  edge   assert 
phys    cpu dest=00000021 mask=1/0/0
(XEN) [2014-02-05 20:15:20]  MSI-X   68 vec=b8  fixed  edge   assert 
phys    cpu dest=00000022 mask=1/0/0
(XEN) [2014-02-05 20:15:20]  MSI-X   69 vec=2a  fixed  edge   assert 
phys    cpu dest=00000020 mask=1/1/1
...
(XEN) [2014-02-05 20:15:22]    IRQ:  64 affinity:00000004 vec:d7 
type=PCI-MSI/-X      status=00000030 in-flight=0 domain-list=0:304(---),
(XEN) [2014-02-05 20:15:22]    IRQ:  65 affinity:00000100 vec:ba 
type=PCI-MSI/-X      status=00000010 in-flight=0 domain-list=0:303(---),
(XEN) [2014-02-05 20:15:22]    IRQ:  66 affinity:00000004 vec:92 
type=PCI-MSI/-X      status=00000010 in-flight=0 domain-list=0:302(---),
(XEN) [2014-02-05 20:15:22]    IRQ:  67 affinity:00000002 vec:3a 
type=PCI-MSI/-X      status=00000010 in-flight=0 domain-list=0:301(---),
(XEN) [2014-02-05 20:15:22]    IRQ:  68 affinity:00000004 vec:b8 
type=PCI-MSI/-X      status=00000030 in-flight=0 domain-list=0:300(---),
(XEN) [2014-02-05 20:15:22]    IRQ:  69 affinity:00000001 vec:2a 
type=PCI-MSI/-X      status=00000002 mapped, unbound


Zoli
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html