netdev - Re: mlx4_en SIOCSHWTSTAMP cycles the link

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20131217195448.GB8693@sbohrermbp13-local.rgmadvisors.com>
Date:	Tue, 17 Dec 2013 13:54:48 -0600
From:	Shawn Bohrer <shawn.bohrer@...il.com>
To:	Richard Cochran <richardcochran@...il.com>
Cc:	Hadar Hen Zion <hadarh@...lanox.com>,
	Or Gerlitz <ogerlitz@...lanox.com>,
	Amir Vadai <amirv@...lanox.com>, netdev@...r.kernel.org,
	tomk@...advisors.com, Shawn Bohrer <sbohrer@...advisors.com>
Subject: Re: mlx4_en SIOCSHWTSTAMP cycles the link

On Tue, Dec 17, 2013 at 12:26:08PM +0100, Richard Cochran wrote:
> On Mon, Dec 16, 2013 at 05:34:15PM -0600, Shawn Bohrer wrote:
> > On Thu, Dec 12, 2013 at 09:28:15AM -0600, Shawn Bohrer wrote:
> > > Below is v2.  It compiles, and boots, but linuxptp doesn't work yet
> > > and I haven't looked into why.  I think linuxptp is actually
> > > complaining about SIOCSHWTSTAMP not working as expected, but again I
> > > haven't really even started debugging yet.
> > 
> > So performing a SIOCSHWTSTAMP on the mlx4_en driver cycles the link:
> > 
> > [ 9463.081330] mlx4_en: p2p1: Changing Time Stamp configuration
> > [ 9463.084637] mlx4_en: p2p1:   frag:0 - size:1526 prefix:0 align:0 stride:1536
> > [ 9463.155533] mlx4_en: p2p1: Link Down
> > [ 9466.471169] mlx4_en: p2p1: Link Up
> > 
> > This breaks linuxptp (ptp4l) since it causes all of the already open
> > sockets to silently loose their multicast joins.  Thus ptp4l sits
> > there waiting for multicast PTP packets that aren't going to come.
> 
> The ptp4l program should notice this and recover. This happens when
> catching the transmission error on the next sync or delay request. (Or
> in slave only E2E mode the sockets are closed and reopened after every
> announce interval). If this isn't happening, please post to the
> linuxptp-users list the details of your setup.

ptp4l does notice that it isn't receiving packets so it closes its
sockets, opens new ones, and issues the ioctl again, which cycles the
link and the loop repeats.  The real problem is every time you issue
the ioctl the link cycles, which is pointless when the device settings
haven't changed.  I have a patch to mlx4_en that I'll send in a minute
which changes the driver to only cycle the link if the timestamp
options have changed which fixes the issue for me in ptp4l.
 
> > I hacked around this in ptp4l by performing the ioctl once with the
> > hwstamp_ctl program from linuxptp and skipping the ioctl in ptp4l,
> > which works, but quite frankly this sucks.  I'd love to know if anyone
> > has any better ideas on how to make SIOCSHWTSTAMP usable.
> > 
> > After hacking around the SIOCSHWTSTAMP issue I've noticed that ptp4l
> > synchronizes the PHC, however there must be another bug somewhere
> > because it creates a 35 second offset from the PTP master.
> 
> This is the current TAI-UTC offset. Maybe your grand master is serving
> UTC instead of the PTP timescale?

Ah, thanks for point this out.  I just looked over the ptp4l and
phc2sys man page and noticed that it is documented that ptp4l uses the
PTP time scale when run in hardware timestamping mode.  Taking this
into account it appears my mlx4_en PHC driver is working fine so I'll
resend that patch as well.

--
Shawn
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html