Message-ID: <87ft6fulkj.fsf@nanos.tec.linutronix.de>
Date:   Fri, 16 Oct 2020 01:16:28 +0200
From:   Thomas Gleixner <tglx@...utronix.de>
To:     "Meisinger\, Andreas" <andreas.meisinger@...mens.com>,
        "vinicius.gomes\@intel.com" <vinicius.gomes@...el.com>,
        "Geva\, Erez" <erez.geva.ext@...mens.com>,
        "linux-kernel\@vger.kernel.org" <linux-kernel@...r.kernel.org>,
        "netdev\@vger.kernel.org" <netdev@...r.kernel.org>,
        "xiyou.wangcong\@gmail.com" <xiyou.wangcong@...il.com>,
        "davem\@davemloft.net" <davem@...emloft.net>,
        "kuba\@kernel.org" <kuba@...nel.org>,
        "jhs\@mojatatu.com" <jhs@...atatu.com>,
        "jiri\@resnulli.us" <jiri@...nulli.us>,
        "avagin\@gmail.com" <avagin@...il.com>,
        "0x7f454c46\@gmail.com" <0x7f454c46@...il.com>,
        "ebiederm\@xmission.com" <ebiederm@...ssion.com>,
        "mingo\@kernel.org" <mingo@...nel.org>,
        "john.stultz\@linaro.org" <john.stultz@...aro.org>,
        "mkubecek\@suse.cz" <mkubecek@...e.cz>,
        "oleg\@redhat.com" <oleg@...hat.com>,
        "peterz\@infradead.org" <peterz@...radead.org>,
        "richardcochran\@gmail.com" <richardcochran@...il.com>,
        "sboyd\@kernel.org" <sboyd@...nel.org>,
        "vdronov\@redhat.com" <vdronov@...hat.com>,
        "bigeasy\@linutronix.de" <bigeasy@...utronix.de>,
        "frederic\@kernel.org" <frederic@...nel.org>,
        "edumazet\@google.com" <edumazet@...gle.com>
Cc:     "jesus.sanchez-palencia\@intel.com" 
        <jesus.sanchez-palencia@...el.com>,
        "vedang.patel\@intel.com" <vedang.patel@...el.com>,
        "Sudler\, Simon" <simon.sudler@...mens.com>,
        "Bucher\, Andreas" <andreas.bucher@...mens.com>,
        "henning.schild\@siemens.com" <henning.schild@...mens.com>,
        "jan.kiszka\@siemens.com" <jan.kiszka@...mens.com>,
        "Zirkler\, Andreas" <andreas.zirkler@...mens.com>,
        "Sakic\, Ermin" <ermin.sakic@...mens.com>,
        "anninh.nguyen\@siemens.com" <anninh.nguyen@...mens.com>,
        "Saenger\, Michael" <michael.saenger@...mens.com>,
        "Maehringer\, Bernd" <bernd.maehringer@...mens.com>,
        "gisela.greinert\@siemens.com" <gisela.greinert@...mens.com>,
        "Geva\, Erez" <erez.geva.ext@...mens.com>,
        "ErezGeva2\@gmail.com" <ErezGeva2@...il.com>,
        "guenter.steindl\@siemens.com" <guenter.steindl@...mens.com>
Subject: Re: [PATCH 0/7] TC-ETF support PTP clocks series

Andreas,

On Wed, Oct 14 2020 at 09:12, Andreas Meisinger wrote:
> Sorry about the wrong format/style of my last mail, hope I did get it
> right this time.

No problem. Yes this looks better. The only thing which could be
improved is that your mail client fails to add

 In-Reply-To: <messageid>
 References: <msgid1> <msgid2> ...

headers and instead has the MS lookout specific

Thread-Topic: [PATCH 0/7] TC-ETF support PTP clocks series
Thread-Index: AdaiB8a+x+RhfhtwSZ+NKfvRdyiJkw==

headers. If you look at the lore archive you see the effect:

  https://lore.kernel.org/r/AM0PR10MB3073F9694ECAD4F612A86FA7FA050@AM0PR10MB3073.EURPRD10.PROD.OUTLOOK.COM

and that happens to all mail clients which use threading based on the
standard headers. There is config knob in lookout to enable them.

> Let me first point at an important thing because we did have
> discussions here about it too. As of the manpages Linux CLOCK_TAI
> seems to be defined as an nonsettable clock which does have the same
> behaviour as of international atomic time TAI. Yet if it's nonsettable
> it can't be linked or synchronized to the value of International
> Atomic Time?
>
> On the other hand there seems to be code in place to change the
> CLOCK_TAI offset thus making it basically sort of "setable"?

It obviously needs to be modifiable in some way, otherwise
synchronization to a master clock via PTP would not work at all.

But it cannot be set in the way of settimeofday()/clock_settime() like
CLOCK_REALTIME.

>> The user space daemon which does the correlation between these PTP
>> domains and TAI is required in any case, so the magic clock
>> TAI_PRIVATE is not having any advantage.

> I think a userspace daemon handling the translation information
> between different clocks would be fine. I think it's not really that
> relevant who exactly does apply the translation. Kernel level might be
> a little bit more precise but I guess it'd depend on other factors
> too.

Not really. The kernel still provides the timestamp pairs/triplets in the
best way the underlying hardware provides it. Some hardware can even
provide hardware assistet pairs of ART and PTP clock.

> Yet all translation based approaches have in common, setting a clock,
> renders translations done in past invalid. It would be required to fix
> old translated values along with setting the clock. I assume we
> couldn't distinguish between "translated" values and genuine values
> after translation, so fixing them might not be possible at all.

CLOCK_TAI is not really set after the initial sychronization. It's
frequency corrected without causing jumps. PTP daemon uses a PLL based
algorithm for that.

Of course this adjustment has side effects for translation.

> In our usecase we do have a clock for network operation which can't be
> set. We can guarantee this because we are able to define the network
> not being operational when the defined time is not available 😉.
> Having this defined the remaining option would be the target clock to
> be set. As of your suggestion that would be CLOCK_TAI. So at this
> point "setable" or "nonsettable" would become important. Here
> "setable" would introduce a dependency between the clocks. Being
> independent from clocks outside of our control was exactly the reason
> to introduce the separate network clock. To me this means if CLOCK_TAI
> could be changed by anything it couldn't be the base clock for our
> usecase if it can't it might be a solution.

It's under your control as system designer how you operate CLOCK_TAI.

If you use the PTP daemon then it will sync CLOCK_TAI to the PTP clock
of your choice. If you don't have PTP at all then you can use NTP to
sync to a time server, which is less accurate. You can use PPS or just
use nothing.

The kernel does not care which non-standard base time or frequency you
chose.

Applications which communicate over network might care if the other side
uses a differnet time universe. Log files which start at 1971 might be
interesting to analyse against the log file of your other system which
starts in 2020.

>> Depending on the frequency drift between CLOCK_TAI and clock PTP/$N
>> the timer expiry might be slightly inaccurate, but surely not more
>> inaccurate than if that conversion is done purely in user space.
>>
>> The self rearming posix timers would work too, but the self rearming
>> is based on CLOCK_TAI, so rounding errors and drift would be
>> accumulative. So I'd rather stay away from them.
>
> As of the time ranges typically used in tsn networks the drift error
> for single shot timers most likely isn't a big deal. But as you
> suggest I'd stay away from long running timers as well rearming ones
> too, I guess they'd be mostly unusable.

Depends. It's a matter of hardware properties, system requirements,
application/system designers decisions. So what you consider unusable
for your system might be perfectly fine for others.

If we add support for this type of correlation then of course these
things need to be documented.

> Right now there's only one timestamp in CLOCK_TAI format which is used
> by ETF as well as by network card thus causing trouble if time is not
> same there.
>
> If we'd add an (optional) second timestamp to SKB which would have to
> be set to network card time we could avoid the conversion error. As we
> do know which network card will be used for sending the SKB we
> wouldn't need a clock identifier for the second timestamp.  For
> situations where the application is not aware of the network cards
> timespace it wouldn't provide the second timestamp. In these
> situations it'd behave identical to your suggestion. Here the
> CLOCK_TAI timestamp would be translated to network card time based on
> the information of the userspace daemon.

That would work as long as the target PTP clock is correlated because
the TAI timestamp is still required to make all of this work.

There are probably quite some dragons lurking left and right if we go
there, but it looks like a possible option.

Thanks,

        tglx

Powered by blists - more mailing lists