NTP not able to sync time, shows "No association IDs returned" in ntpq output

Written by Claudio Kuenzler - 1 comments

Published on September 27th 2023 - Listed in Linux

---

After monitoring reported an increasing system time offset on a Ubuntu Linux server, the NTP daemon on that Ubuntu machine was investigated. But let's go into the details.

Time not in sync

Obviously the first thing to do is to verify there is an actual time difference. And this could be proven quite quickly, by comparing the time to another (time-synchronized) machine.

Obviously the date command was executed at the exact same time, using the Terminator Terminal's broadcast feature. The difference is obvious, the Ubuntu server (on top) runs 32s in the future. 

NTP: No association ID's returned

A quick verification showed that the NTP daemon was started and listened on (several interfaces) port 123. Let's verify the offset with NTP peers:

root@ubuntu:~# ntpq -p
No association ID's returned

OK, that's not normal. A list of NTP peers should show up instead. These peers are then used to find the "correct time" to sync to.

Maybe a restart would do it?

root@ubuntu:~# systemctl restart ntp

root@ubuntu:~# systemctl status ntp
- ntp.service - Network Time Service
     Loaded: loaded (/lib/systemd/system/ntp.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2023-09-27 13:42:15 CEST; 5s ago
       Docs: man:ntpd(8)
    Process: 3383293 ExecStart=/usr/lib/ntp/ntp-systemd-wrapper (code=exited, status=0/SUCCESS)
   Main PID: 3383313 (ntpd)
      Tasks: 2 (limit: 9448)
     Memory: 1.5M
     CGroup: /system.slice/ntp.service
             |-3383313 /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 111:115

Sep 27 13:42:15 ubuntu ntpd[3383313]: Listening on routing socket on fd #78 for interface updates
Sep 27 13:42:15 ubuntu ntpd[3383313]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
Sep 27 13:42:15 ubuntu ntpd[3383313]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized

According to the status output, the NTP service was correctly started, listens correctly but at the same time the ntpd daemon logged a TIME_ERROR, that the clock is unsynchronized.

What does the overall NTP system status say?

root@ubuntu:~# ntpq -c sysinfo
associd=0 status=c016 leap_alarm, sync_unspec, 1 event, restart,
system peer:        0.0.0.0:0
system peer mode:   unspec
leap indicator:     11
stratum:            16
log2 precision:     -24
root delay:         0.000
root dispersion:    1.455
reference ID:       INIT
reference time:     (no time)
system jitter:      0.000000
clock jitter:       0.000
clock wander:       0.000
broadcast delay:    -50.000
symm. auth. delay:  0.000

Still, no peers found hence no reference time either. That machine won't sync the time, as there are no peers detected. But why?

Verifying ntp.conf

Is there an error in ntp.conf? Maybe a wrong NTP server IP or address? But checking the file turns out to be the standard ntp.conf coming from the Ubuntu package with a local NTP server instead of using the public NTP pools:

root@ubuntu:~# cat /etc/ntp.conf
# /etc/ntp.conf, configuration for ntpd; see ntp.conf(5) for help

driftfile /var/lib/ntp/ntp.drift


# Enable this if you want statistics to be logged.
#statsdir /var/log/ntpstats/

statistics loopstats peerstats clockstats
filegen loopstats file loopstats type day enable
filegen peerstats file peerstats type day enable
filegen clockstats file clockstats type day enable

# Specify one or more NTP servers.

# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board
# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for
# more information.
server ntp1.example.local iburst
server ntp2.example.local iburst
#server 0.ubuntu.pool.ntp.org
#server 1.ubuntu.pool.ntp.org
#server 2.ubuntu.pool.ntp.org
#server 3.ubuntu.pool.ntp.org

# Use Ubuntu's ntp server as a fallback.
server ntp.ubuntu.com

# Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for
# details.  The web page <http://support.ntp.org/bin/view/Support/AccessRestrictions>
# might also be helpful.
#
# Note that "restrict" applies to both servers and clients, so a configuration
# that might be intended to block requests from certain clients could also end
# up blocking replies from your own upstream servers.

# By default, exchange time with everybody, but don't allow configuration.
restrict -4 default kod notrap nomodify nopeer noquery
restrict -6 default kod notrap nomodify nopeer noquery

# Local users may interrogate the ntp server more closely.
restrict 127.0.0.1
restrict ::1

# Clients from this (example!) subnet have unlimited access, but only if
# cryptographically authenticated.
#restrict 192.168.123.0 mask 255.255.255.0 notrust


# If you want to provide time to your local subnet, change the next line.
# (Again, the address is an example only.)
#broadcast 192.168.123.255

In this on-prem infrastructure, the same ntp.conf has been deployed several hundred times and works correctly on other machines. A config error inside ntp.conf can therefore be excluded.

Permission denied

Let's try one more restart and dig through all the logs we can find, not just the ones shown in systemctl status ntp:

root@ubuntu:~# systemctl restart ntp

root@ubuntu:~# journalctl -u ntp
[...]
Sep 27 13:50:42 ubuntu systemd[1]: Stopping Network Time Service...
Sep 27 13:50:42 ubuntu ntpd[3397765]: ntpd exiting on signal 15 (Terminated)
Sep 27 13:50:42 ubuntu systemd[1]: ntp.service: Succeeded.
Sep 27 13:50:42 ubuntu systemd[1]: Stopped Network Time Service.
Sep 27 13:50:42 ubuntu systemd[1]: Starting Network Time Service...
Sep 27 13:50:42 ubuntu ntpd[3402031]: ntpd 4.2.8p12@1.3728-o (1): Starting
Sep 27 13:50:42 ubuntu ntpd[3402031]: Command line: /usr/sbin/ntpd -p /var/run/ntpd.pid -g -u 111:115
Sep 27 13:50:42 ubuntu systemd[1]: Started Network Time Service.
Sep 27 13:50:42 ubuntu ntpd[3402035]: proto: precision = 0.050 usec (-24)
Sep 27 13:50:42 ubuntu ntpd[3402035]: getconfig: Couldn't open </etc/ntp.conf>: Permission denied
[...]

In the middle of NTP log entries, a permission denied error showed up. Conveniently on the most important file: The ntp.conf configuration file!

Let's check the permissions:

root@ubuntu:~# ls -la /etc/ntp.conf
--w----r-T 1 root root 2002 Sep 27 13:51 /etc/ntp.conf

Huh? That looks definitely weird. Let's compare with my (Ubuntu based) Linux Mint workstation on which NTP is correctly working:

ckadm@mintp ~ $ ls -la /etc/ntp.conf
-rw-r--r-- 1 root root 2136 Apr  2  2020 /etc/ntp.conf

Yep, that does look very different.

Let's fix these permissions and remove the weird sticky bit:

root@ubuntu # chmod 0644 /etc/ntp.conf

Another NTP restart:

root@ubuntu # systemctl restart ntp

And finally peers are now showing up:

root@ubuntu # ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 ntp1.example.lo .STEP.          16 u   24   64    0    0.000    0.000   0.000
*ntp2.example.lo 10.161.101.20    3 u   17   64    1    0.389    0.079   0.108
 185.125.190.57  .STEP.          16 u    -   64    0    0.000    0.000   0.000

Another comparison with date proves that time is now synchronized on this Ubuntu machine again:

Add a comment

Show form to leave a comment

Comments (newest first)

Chirstian Le Touzo from wrote on Dec 13th, 2024:

Thank you so much for the help !

---