| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <063D6719AE5E284EB5DD2968C1650D6D1724FA96@AcuExch.aculab.com> Date: Thu, 29 May 2014 09:03:47 +0000 From: David Laight <David.Laight@...LAB.COM> To: 'Vlad Yasevich' <vyasevich@...il.com>, "netdev@...r.kernel.org" <netdev@...r.kernel.org> Subject: RE: SCTP seems to lose its socket state. From: Vlad Yasevich > On 05/27/2014 11:10 AM, David Laight wrote: > > I've been looking at an ethernet trace from one of our customers. > > They seem to have got an SCTP socket into a rather confused state. > > > > There seem to be a significant number of transmit ethernet frames > > that don't read the far end. > > This shouldn't cause a real problem, but we end up with the following: > > This trace was taken on the linux system: > > > > 39964 0.304473 -> SCTP INIT > > 39965 0.292669 <- SCTP INIT (I think this has an invalid checksum) > > 39968 0.467935 <- SCTP INIT > > 39969 0.000093 -> SCTP INIT_ACK > > 39970 0.003947 <- SCTP COOKIE_ECHO > > 39971 0.000072 -> SCTP COOKIE_ACK > > 39972 0.000337 -> M3UA ASPUP > > 39979 0.809659 <- SCTP COOKIE_ECHO > > cookie_ack was dropped for some reason? Most likely, as I said a moderate number of ethernet transmit are not being received by the far end. I think this is a 'real' network and they are 'real' discards somewhere between the two systems. Other parts of the trace show all sorts of retransmissions. > > 39980 0.000058 -> SCTP COOKIE_ACK > > shutdown() called here - seems to be ignored > > 39983 0.949471 <- SCTP COOKIE_ECHO > > Cookie timer fired and resent the cookie_echo. > > > 39984 0.000053 -> SCTP COOKIE_ACK > > 39986 0.730072 -> M3UA ASPUP Same TSN as above > > 40002 0.270589 -> M3UA ASPUP Same TSN as above > > Hmm.. look like more retransmissions. > > > 40008 3.689088 <- SCTP HEARTBEAT > > This probably means that cookie_ack was finally accepted and > we are not heart-beating... > > output of 'cat /proc/net/sctp/assocs' might help. If the local > is running a recent enough kernel, then turning on dynamic debug > in sctp will also help. This is a customer system, I think it is RHEL6 (2.6.32-358) so running a newer kernel is a little tricky to arrange. We have asked to double check the kernel version, it might be RHEL5 (2.6.18 + patches) - which is likely to have a lot of bugs. > > 40009 0.000027 -> SCTP HEARTBEAT_ACK > > 40014 0.261152 <- SCTP HEARTBEAT > > 40015 0.000033 -> SCTP HEARTBEAT_ACK > > 40026 0.123048 <- SCTP HEARTBEAT > > 40027 0.000030 -> SCTP HEARTBEAT_ACK > > 40036 1.615048 -> M3UA ASPUP Same TSN as above > > > > There are no signs of any SACKs for the ASPUP, I think they have the > > correct TSN (the same value as in the INIT_ACK). > > Make sure that verification tags match what was negotiated in > init/init_ack, and the SSN starts at 0. The tags match, SSN is zero. I think the lack of SACK for the data is a bug in the remote system. It shouldn't stop the outward disconnect. > > No signs of any shutdowns or aborts from either system. > > > > What's strange is that some frames are simply not accepted. > Are the nics by any chance ixgbe that has checksum offload and > the checksums are corrupt for some reason? The wireshark trace does show zero for all the transmitted SCTP checksums. But the system works fine for hours, so I don't really believe that is causing the packets to be lost. I think the packet loss is caused by serious network congestion. Not the least of the problems is that it seems to take a system reboot to revover. David -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists