netdev - Re: sky2 hangs without any messages

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <6278d2220707121429o27a53a6cqd68b647957203626@mail.gmail.com>
Date:	Thu, 12 Jul 2007 22:29:50 +0100
From:	"Daniel J Blueman" <daniel.blueman@...il.com>
To:	"Stephen Hemminger" <shemminger@...ux-foundation.org>
Cc:	"Linux Netdev" <netdev@...r.kernel.org>
Subject: Re: sky2 hangs without any messages

On 12/07/07, Stephen Hemminger <shemminger@...ux-foundation.org> wrote:
> On Wed, 11 Jul 2007 23:55:29 +0100
> "Daniel J Blueman" <daniel.blueman@...il.com> wrote:
>
> > > >
> > > > Please try again with post 2.6.22 git version (1.16)?
> > >
> > > Reproduced with 2.6.22 w/ sky2 1.16 from git. We observe this
> > > characteristic failure on the NFS server (always around 2-3GB of
> > > transmit):
> > >
> > > $ ifconfig lan0
> > > lan0      Link encap:Ethernet  HWaddr 00:03:2D:05:9C:27
> > >           inet addr:192.168.0.250  Bcast:192.168.0.255  Mask:255.255.255.0
> > >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> > >           RX packets:24007220 errors:1 dropped:1 overruns:0 frame:1
> > >           TX packets:13886495 errors:0 dropped:0 overruns:0 carrier:0
> > >           collisions:0 txqueuelen:1000
> > >           RX bytes:171026170 (163.1 MiB)  TX bytes:2262910580 (2.1 GiB)
> > >           Interrupt:16
> > >
> > > I'll rebuild with debugfs and grab the debug you've exported.
> >
> > In quiescent state [1] and in failure state [2]. This time, 2 framing
> > failures [3]; took 3.6GB of transmit to hit the window.
>
> Since the IRQ workaround has a timeout of 100ms. I observed cases where
> the TCP connection dropped (because of lost packets), but the network device
> then recovered.  Can you ping the other side after it hangs?  Or reconnect?
>
> Ifconfig lumps a bunch of different errors together so it can confuse the issue.
> Preference is for:
>         ip -s -s link show eth0
> or
>         grep -v '^0' /sys/class/net/eth0/statistics/*
>
> If the framing error does reproduce with the hang, perhaps the chip needs some
> receive flush logic to recover. Receive errors normally put a message in syslog
> output, did you look there?
>
> > Daniel
> >
> > --- [1]
> >
> > # cat sky2/lan0
> > IRQ src=0 mask=c000001d control=0
> > Status ring (empty)
> > Tx ring pending=191...191 report=191 done=191
> >
> > Rx ring hw get=956 put=61 last=1023
> >
> > --- [2]
> >
> > # cat sky2/lan0
> > IRQ src=0 mask=c000001d control=0
> > Status ring (empty)
> > Tx ring pending=251...251 report=251 done=251
> >
> > Rx ring hw get=1020 put=160 last=1023
> >
> > --- [3]
> >
> > $ ifconfig lan0
> > lan0      Link encap:Ethernet  HWaddr 00:03:2D:05:9C:27
> >           inet addr:192.168.0.250  Bcast:192.168.0.255  Mask:255.255.255.0
> >           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> >           RX packets:13304841 errors:1 dropped:1 overruns:0 frame:2
> >           TX packets:7493765 errors:0 dropped:0 overruns:0 carrier:0
> >           collisions:0 txqueuelen:1000
> >           RX bytes:232720755 (221.9 MiB)  TX bytes:3964088142 (3.6 GiB)
> >           Interrupt:16
>
> You aren't hung because of lost IRQ. When than happens the debugfs output will have
> a bunch of Tx packets stuck (not cleaned up), and Status messages, and receive packets.

I'll grab the above info when I next get chance.

The vendor driver recovery process may be worthwhile taking a look at;
I guess you've seen the code near the bottom of skge.c (under 'case
SK_DRV_RECOVER')? The driver kicks the chip with
SK_PNMI_EVT_XMAC_RESET and calls SkYuk2RestartRxBmu - perhaps
something like this sequence is needed for a more targetted approach?

I'll also confirm if your driver is detecting the hang I'm seeing, or not.

Daniel
-- 
Daniel J Blueman
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html