Chrony with hardware timestamping

Introduction
On a local-area network, Network Time Protocol can achieve time-synchronization accuracy of in the order of 1 millisecond. While more than enough if getting reference time from another, remote NTP server, using it in conjunction with a more precise time source such as a Precision Time Protocol (PTP) (wikipedia) master or a satellite-navigation (e.g. GPS) clock results in a loss of several orders of magnitude of accuracy. One alternative would of course be to use such a time source directly but that is not always possible, for instance due to the cost and complexity of equipping servers with individual GNSS receivers or due to older network infrastructure not handling PTP traffic correctly. Under such circumstances, recent versions of Chrony offer an additional solution in the form of using hardware clocks available on many modern network-interface cards to improve accuracy of NTP itself.

Host list
This guide uses an example network that has several NTP peers:

as well as an unnamed PTP master whose configuration is beyond the scope of this guide.
 * parry, topshelf - Internet-facing servers whose NIC clocks are synchronized with a PTP master over a dedicated network link, running chrony. All of their Ethernet devices can support hardware timestamping
 * mater, mike - internal servers with NICs supporting hardware timestamping, running chrony
 * thufir - internal embedded server running a different implementation of NTP which does not support timestamping

Kernel support
When dealing with hardware clocks, it is necessary to enable support for them in the kernel. As of mid-2020, all the gentoo-sources kernels available in the tree (4.4 and newer) can be built with PTP-clock support (CONFIG_PTP_1588_CLOCK). Note that the terminology is a bit misleading - enabling this only allows access to the NIC clocks, it has nothing to do with PTP time synchronization except that ptp4l also needs to have this enabled.

In addition to the PTP-clock support itself, check the configuration options available for the system's NIC drivers - for some (e.g. Cadence MACB/GEM, macb) hardware timestamping has to be explicitly enabled, for some others (e.g. Intel PRO/1000 PCIe, e1000e) there are switches for additional timestamping-related features. Alternatively, when building a kernel for a KVM guest, enable CONFIG_PTP_1588_CLOCK_KVM. Last but not least, set up network PHY device support for the line of adapters if not yet completed.

After rebooting to the new kernel verify access to the NIC clocks by emerging and running. Example for an interface with all the required features:

Confirm that, the device node associated with this interface according to the "PTP Hardware Clock" line, exists and has got sane permissions. Note on the receive filter modes: some NICs can only timestamp PTP packets, i.e. have no HWTSTAMP_FILTER_ALL mode. Such interfaces will be of limited use with NTP because chrony will be unable to hardware-timestamp received packets.

If the device doesn't have PTP-clock support, the output looks like:

Chrony config on the Internet-facing, PTP-synchronized servers
topshelf and parry have multiple network adapters and are directly connected to the Internet, with firewalls keeping out the unwashed masses. Their chrony config will differ from the other hosts because they will talk to upstream ntp.org and gentoo.org ntp servers at a higher stratum, and also because they will use their respective PTP-synchronized NIC clocks as reference time sources. The local home network is the private 192.168.1.0 class C subnet.

The first part of the file uses the public pool of ntp.org and gentoo ntp servers for our upstream sources. The iburst option allows topshelf and parry to have an initially larger amount of ntp transactions pack and forth with the upstream servers until the accurate time gets settled upon. However that may not necessarily happen since the initstepslew directive will take effect first. That directive will quickly slew the clock into shape upon startup if the time is off by 30 seconds or more.

The prefer option on these two designates them to be treated with higher priority amongst all of the peers. Thus either parry or topshelf will show up as the reference clock when issuing a tracking command to chronyc from other machine on this network.

The xleave option causes chrony to send transmit timestamps captured after actual transmissions, which helps to reduce jitter and improve accuracy. It supports both hardware and software timestamping, and is harmless to enable for server connections even if the server does not support this mode. On the other hand, in case of peer connections interleaved mode must be supported by and enabled on both ends.

The refclock directive has the host use the NIC clock 3 (which on topshelf is the clock associated with the NIC facing the PTP master) as an additional reference source. It will show up as the PHC0 source in the first line of a chronyc sources. As previously mentioned, chrony will make no attempt to actually synchronize this clock with any other source.

The hwtimestamp * directive tells chrony to enable hardware timestamping on all the NICs it may use that support it, to timestamp incoming and outgoing NTP packets.

Chrony config on a local peer
The local peers rely on topshelf and parry to provide NTP time so external sources have been removed. There is also no refclock line because the NIC clocks are not PTP-synchronized.

chrony with hardware timestamping in action
This is the output of a 'chronyc sources' and tracking report on mike after the newly configured chrony has been pushed out to all of the hosts and has been running for a while. topshelf shows with "=*" since it is the reference clock. parry shows "=+" since it is regarded highly enough to be considered in the clock adjustment combining algorithm. The other peers show as "=-" since they are not currently being regarded as sources for adjustments.

{{RootCmd|chronyc|output= chrony version 3.3 Copyright (C) 1997-2003, 2007, 2009-2018 Richard P. Curnow and others chrony comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions. See the GNU General Public License version 2 for details. chronyc> sources 210 Number of sources = 4 MS Name/IP address        Stratum Poll Reach LastRx Last sample

=
================================================================== =+ parry.example.com            3   0   365     3    -48ns[  -48ns] +/-   15ms =* topshelf.example.com.>       2   0   377     7   -390ns[ -438ns] +/-   15ms =- thufir.example.com           3   0   165    10    +49us[  +49us] +/-   15ms =- mater.example.com            3   0   377     0  -4842ns[-4842ns] +/-   15ms chronyc> tracking Reference ID   : C0A80B21 (topshelf.example.com.1.168.192.in-addr.arpa) Stratum        : 3 Ref time (UTC) : Thu Mar 21 20:57:24 2019 System time    : 0.000000522 seconds fast of NTP time Last offset    : +0.000000025 seconds RMS offset     : 0.000001429 seconds Frequency      : 0.321 ppm fast Residual freq  : -0.000 ppm Skew           : 0.007 ppm Root delay     : 0.025426105 seconds Root dispersion : 0.002194689 seconds Update interval : 5.4 seconds Leap status    : Normal chronyc> }}