• Announce packet:
  • Sent by the NIC assuming the Grand master role.
• Sync packet
  • Sent by the Grand master clock.
  • The time when this packet is transmitted is captured.
• Follow up packet
  • Sent by the Grandmaster to inform the slave clocks as to when in Grandmaster time was the sync packet was sent.
  • Essentially, puts the sync packet’s timestamp in this packet and transmits
  • This packet is used by the slave to correct clock difference up to seconds’ level.
• Delay request packet
  • Sent by the slave clock.
  • The timestamp of this packets’ exit is captured by the slave.
  • These packets are to correct nanoseconds level clock differences due to path delay between the Grandmaster and the slave.
• Delay response packet
  • Sent by the Grandmaster.
  • Contains the time at which the Grandmaster received the Delay request packet.

Now that we have laid the groundwork, Let’s see what endpoints (function pointers/ APIs) Linux provides us with to implement, for the NICs driver.

The ptp_clock_info struct holds the function pointers to the APIs used by the slave clock for PTP flow as described above, listed as follows:

• adjtime()
  • Is called in the first few sync cycles to accommodate for the difference in time up to seconds’ level.
• adjfreq()
  • Is called to narrow down time up to nanoseconds level by adjusting the clock sampling frequency in proportions to the physical Oscillator frequency, depending on the required nanosecond accuracy.

skb_shared_hwtstamps structure defined in Linux is supposed to be filled with the total time in nanoseconds (ie, seconds * 10⁹ + nanoseconds) of the snapshot taken by the hardware for the transmitted or received packet. The snapshots taken by the hardware are usually available in registers or Descriptors. This is the point at which timestamps are handed over to the stack for PTP related calculations.

When ptp4l receives HW timestamps for the packets it generated or received, It makes a call to the adjtime() API, where we bring in the hardware clocks to sync at about seconds range. This usually happens after 2 sync cycles. The master offset value seen Figure 1 is passed as an argument to the adjtime() API. Once the clocks are in sync up-to seconds, the ptp4l keeps calling the adjfreq() to keep the clocks in sync in the nanoseconds range. The frequency offset in the image above is passed as a parameter to the adjfreq() call. If the NICs are connected back-to-back a master offset of less than 100ns can be seen for a prolonged duration (agin depends on the required nanoseconds accuracy).

The ptp4l showed an out-of-sync-master slave with maxed-out frequency offset. In our case, it was a bug in the conversion of time in seconds and nanoseconds register, to complete nanoseconds only. Usually, hardware maintains two separate registers or descriptor fields of 32 bits each. The stack provides a 64-bit nanoseconds field for the overall timestamp. If the conversion is wrong, we would be misinforming the Linux stack. In this case, the adjtime() called would not have corrected the slave clock to the master clock up-to seconds level at all. This forces the adjfreq() to max out on the slave clock frequency offset, yet not syncing time. This usually gives out a timeout error on the ptp4l application log.

We could have corrected this sooner if we had approached the problem from a timestamp handover point rather than from a device perspective.

Hence, beginning the debug at the point where timestamps are handed over to the Linux stack is a good starting point in the debugging process. One can also trace out the PTP packets and their timestamps through tools like Wireshark if the problem did not lie in the timestamp handover. If the problem still persists, it should be looked for in the adjfreq() API and device-specific details.