Chrony with hardware timestamping

Introduction
On a local-area network, Network Time Protocol can achieve time-synchronisation accuracy of in the order of 1 millisecond. While more than enough if you get your reference time from another, remote NTP server, using it in conjunction with a more precise time source such as a Precision Time Protocol (PTP) master or a satellite-navigation (e.g. GPS) clock results in a loss of several orders of magnitude of accuracy. One alternative would of course be to use such a time source directly but that is not always possible, for instance due to the cost and complexity of equipping servers with individual GNSS receivers or due to older network infrastructure not handling PTP traffic correctly. Under such circumstances, recent versions of Chrony offer an additional solution in the form of using hardware clocks available on many modern network-interface cards to improve accuracy of NTP itself.

Chrony with hardware timestamping
WARNING: It is assumed here that you have already configured PTP time synchronisation on systems which will use it, using e.g. the ptp4l daemon from the package net-misc/linuxptp. Chrony itself does not speak PTP so if your NIC clock is not accurate, it will be marked as a false ticker at best or actually degrade your synchronisation accuracy at worst.

Host List
This guide uses an example network that has several NTP peers: * parry, topshelf - Internet-facing servers whose NIC clocks are synchronised with a PTP master over a dedicated network link, running chrony. All of their Ethernet devices can support hardware timestamping * mater, mike - internal servers with NICs supporting hardware timestamping, running chrony * thufir - internal embedded server running a different implementation of NTP which does not support timestamping as well as an unnamed PTP master whose configuration is beyond the scope of this guide.

Kernel support
Since we are dealing with hardware clocks here it is necessary to enable support for them in the kernel. As of mid-2020, all the gentoo-sources kernels available in the tree (4.4 and newer) can be built PTP-clock support (CONFIG_PTP_1588_CLOCK). Note that the terminology is a bit misleading - enabling this only allows access to the NIC clocks, it has nothing to do with PTP time synchronisation except that ptp4l also needs to have this enabled.

In addition to the PTP-clock support itself, you might want to check the configuration options available for your NIC drivers - for some (e.g. Cadence MACB/GEM, macb) hardware timestamping has to be explicitly enabled, for some others (e.g. Intel PRO/1000 PCIe, e1000e) there are switches for additional timestamping-related features. Alternatively, you are building a kernel for a KVM guest you can enable CONFIG_PTP_1588_CLOCK_KVM.

After rebooting to the new kernel you can verify that you can now access the NIC clocks by emerging sys-apps/ethtool and running ethtool -T. Example for an interface with all the required features:

You can also confirm that '/dev/ptp2', the device node associated with this interface according to the "PTP Hardware Clock" line, exists and has got sane permissions. Note on the receive filter modes: some NICs can only timestamp PTP packets, i.e. have no HWTSTAMP_FILTER_ALL mode. Such interfaces will be of limited use with NTP because chrony will be unable to hardware-timestamp received packets.

This is what you would see instead if your device doesn't have PTP-clock support:

Chrony config on the Internet-facing, PTP-synchronised servers
topshelf and parry have multiple network adapters and are directly connected to the Internet, with firewalls keeping out the unwashed masses. Their chrony config will differ from the other hosts because they will talk to upstream ntp.org and gentoo.org ntp servers at a higher stratum, and also because they will use their respective PTP-synchronised NIC clocks as reference time sources. The local home network is the private 192.168.1.0 class C subnet.

The first part of the file uses the public pool of ntp.org and gentoo ntp servers for our upstream sources. The iburst option allows topshelf and parry to have an initially larger amount of ntp transactions pack and forth with the upstream servers until the accurate time gets settled upon. However that may not necessarily happen since the initstepslew directive will take effect first. That directive will quickly slew the clock into shape upon startup if the time is off by 30 seconds or more.

The prefer option on these two designates them to be treated with higher priority amongst all of the peers. Thus either parry or topshelf will show up as the reference clock when you issue a tracking command to chronyc from other machine on this network.

The xleave option causes chrony to send transmit timestamps captured after actual transmissions, which helps to reduce jitter and improve accuracy. It supports both hardware and software timestamping, however in case of peer connections it must be supported by and enabled on both ends.

The refclock directive has the host use the NIC clock 3 (which on topshelf is the clock associated with the NIC facing the PTP master) as an additional reference source. It will show up as the PHC0 source in the first line of a chronyc sources. As previously mentioned, chrony will make no attempt to actually synchronise this clock with any other source.

The hwtimestamp * directive tells chrony to enable hardware timestamping on any of the NICs it may use that support it, to timestamp incoming and outgoing NTP packets.

Chrony config on a local peer
For the local peers, we have commented out the ntp.org and gentoo.org server pools since they will be relying on topshelf and parry. For thufir, we have also removed all the xleave options and commented out the refclock directive since it lacks PTP support on its NIC.

chrony with hardware timestamping in action
This is the output of a chronyc sources and tracking report on tube after the newly configured chrony has been pushed out to all of the hosts and has been running for a while. topshelf shows with "=*" since it is the reference clock. parry shows "=+" since it is regarded highly enough to be considered in the clock adjustment combining algorithm. The other peers show as "=-" since they are not currently being regarded as sources for adjustments. The PHC0 shows as "=X" since it is a hardware reference associated with the /dev/ptp0 clock on the eth0 NIC interface.

{{RootCmd|chronyc|output= chrony version 3.3 Copyright (C) 1997-2003, 2007, 2009-2018 Richard P. Curnow and others chrony comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. See the GNU General Public License version 2 for details.

chronyc> sources 210 Number of sources = 8 MS Name/IP address        Stratum Poll Reach LastRx Last sample

=
================================================================== =+ parry.example.com            3   0   365     3    -48ns[  -48ns] +/-   15ms =* topshelf.example.com.>       2   0   377     7   -390ns[ -438ns] +/-   15ms =- thufir.example.com           3   0   165    10    +49us[  +49us] +/-   15ms =- mater.example.com            3   0   377     0  -4842ns[-4842ns] +/-   15ms =- figo.example.com             3   0   375     3   -459ns[ -459ns] +/-   15ms =- mike.example.com             3   0   157     2  +7333ns[+7333ns] +/-   14ms =? tube.example.com             0   0     0     -     +0ns[   +0ns] +/-    0ns chronyc> tracking Reference ID   : C0A80B21 (topshelf.example.com.1.168.192.in-addr.arpa) Stratum        : 3 Ref time (UTC) : Thu Mar 21 20:57:24 2019 System time    : 0.000000522 seconds fast of NTP time Last offset    : +0.000000025 seconds RMS offset     : 0.000001429 seconds Frequency      : 0.321 ppm fast Residual freq  : -0.000 ppm Skew           : 0.007 ppm Root delay     : 0.025426105 seconds Root dispersion : 0.002194689 seconds Update interval : 5.4 seconds Leap status    : Normal chronyc> }}
 * 1) x PHC0                         0   4   377    17   +37.4s[ +37.4s] +/-  901ns