System Clock Drift Detected on Linux – Quick Fix

Quick answer for advanced users

Run sudo timedatectl set-ntp true and check timedatectl status. If still drifting, force a hardware clock sync with sudo hwclock --systohc and adjust NTP servers in /etc/chrony/chrony.conf.

Why does this happen?

Your system clock drifts because the hardware clock (RTC) isn't perfectly accurate. On physical servers, temperature changes wear down the quartz crystal. On VMs (especially VMware and KVM), the guest doesn't get enough CPU time to keep the clock steady. I've seen this a lot on Ubuntu 20.04 and CentOS 7 boxes running for months without reboots. The kernel reports it as "clock drift detected" in dmesg.

Fix steps

1. Check current status
   timedatectl status
   Look for "NTP service: active" and "System clock synchronized: yes". If NTP is off, you'll see drift.
2. Enable and restart NTP
   sudo timedatectl set-ntp true
sudo systemctl restart systemd-timesyncd
   On older systems use sudo systemctl restart chronyd or ntpd.
3. Force hardware clock sync
   sudo hwclock --systohc
   This writes the system time to the RTC. Without this, the drift comes back after reboot.
4. Tweak kernel parameters
   Edit /etc/sysctl.conf and add:
   

   kernel.printk = 4 4 1 7
   kernel.hung_task_timeout_secs = 0

   
   Run sudo sysctl -p. This stops the kernel from killing tasks when clock jumps.
5. Verify with chrony
   If you use chrony, run chronyc tracking and check the "Last offset" value. If it's above 100ms, you need better NTP servers.

Alternative fixes when the main one fails

Switch to chrony

On Ubuntu, systemd-timesyncd does the job for most setups, but it's not great at handling large drifts. Install chrony:
sudo apt install chrony
sudo systemctl disable systemd-timesyncd
sudo systemctl enable chrony
sudo systemctl start chrony

Force a one-time sync with ntpdate

Old-school but works when the clock is way off. Only run once:
sudo ntpdate pool.ntp.org
Then follow steps 2 and 3 above.

For VMware VMs, disable host time sync

This is a common trap. If the host syncs time and the guest also runs NTP, they fight each other. On the VM, run:
echo 'vmware-guestd' >> /etc/modprobe.d/blacklist.conf
Then reboot. Or disable it in the VM settings: uncheck "Synchronize guest time with host".

Check for hypervisor CPU steal time

On oversubscribed hosts, the VM doesn't get enough CPU cycles. Run cat /proc/stat | grep 'steal'. If steal time is above 5%, your VM is fighting for CPU. Move it to a less loaded host or request more resources.

Prevention tips

• Use chrony instead of systemd-timesyncd – it handles drift way better, especially on VMs.
• Schedule a weekly sync – add this cron job to sync hardware clock every Sunday at 3am:
  

  0 3 * * 0 root /sbin/hwclock --systohc

• Monitor drift – set up a simple check with chronyc tracking and alert if offset is over 500ms. I use a one-liner in a Nagios plugin.
• Keep kernel updated – newer kernels have better timer handling for VMs. On Ubuntu, run sudo apt update && sudo apt upgrade linux-image-$(uname -r).

That's it. If you still see drift after all this, you probably have a dying RTC battery on a physical server. Swap the CMOS battery. On a VM, double-check the hypervisor's time settings. I've had cases where the host's NTP was misconfigured and the VM was just copying bad time.