Latest / Linux Server Admin with Fexingo: Sysadmin, Bash, and Server Engineering / Linux Server Time Synchronization with Chrony
Transcript
- Lucas: Alright, let's talk about something that seems trivial but can break your entire infrastructure if you get it wrong: time synchronization on Linux servers. Luna: Right, because if your servers don't agree on what time it is, logs become useless, certificates fail, and cron jobs fire at the wrong moment. Lucas: Exactly. And the modern standard for keeping time accurate is Chrony. It's the default in RHEL 8 and 9, Fedora, and Ubuntu 20.04 and later. It replaces the older ntpd, and it's designed to handle networks with intermittent connectivity or high latency better. Luna: So if I'm building a fresh server today, I should be using Chrony, not ntpd. What makes it better? Lucas: Chrony can synchronize the clock much faster after a reboot or network outage. It also has better handling of jittery connections—like virtual machines that get suspended and resumed. And it can act as both an NTP client and server in the same daemon. Luna: Let's start with a basic setup. How do I install Chrony on Ubuntu 22.04? Lucas: Simple: sudo apt update && sudo apt install chrony. On rhel based systems, it's dnf install chrony. The package names are the same. After installation, the daemon starts automatically. The main config file is /etc/chrony/chrony.conf. Luna: What does a minimal config look like? I imagine you need at least one NTP server to sync against. Lucas: You need a pool directive. The default config usually has 'pool 2.debian.pool.ntp.org iburst' for Debian-based systems, or 'pool 2.fedora.pool.ntp.org iburst' for Fedora. The iburst option speeds up the initial synchronization by sending a burst of packets. Luna: Right. And you can add multiple pools for redundancy. But what if your servers are in a private network without internet access? Lucas: Then you point them to an internal NTP server. If you're running your own Chrony server that has internet access, you configure it to use public pools, and then other servers in your network point to that internal server. You'd set 'server 192.168.1.10 iburst' in their configs. Luna: How do you verify that Chrony is actually working? I don't want to just trust the logs. Lucas: Use chronyc. The command 'chronyc sources -v' shows the NTP sources and their current status. You'll see a column with ^ for server, * for the currently selected source, and + for other good sources. Also, 'chronyc tracking' shows the current time offset, frequency error, and how often the clock is adjusted. Luna: What should I expect for offset? A few milliseconds is fine, but what's a red flag? Lucas: Anything above 100 milliseconds is concerning. For most applications, you want offset under 10 ms. If you're running financial trading systems or some databases, you might need sub-millisecond accuracy, which requires hardware support like PTP. Luna: Let's talk about the local hardware clock. The RTC—real-time clock—is usually not as accurate as NTP. How does Chrony handle that? Lucas: Chrony can use the RTC as a backup if NTP sources are unavailable. You enable it with the 'rtcsync' directive in chrony.conf. This tells the kernel to sync the RTC to the system clock every 11 minutes. It's a simple but effective safeguard. Luna: So on a laptop that goes to sleep and wakes up, Chrony can correct the drift quickly. That's useful for mobile developers too. Lucas: Exactly. And there's a command 'chronyc -a makestep' that forces an immediate time jump if the offset is too large—say more than 0.1 seconds. But be careful: large jumps can confuse applications like databases. You can configure the threshold in chrony.conf with the 'makestep' directive. Luna: Another common issue: timezones. People mess up by setting the system timezone incorrectly and blaming NTP. How do you set the timezone properly? Lucas: Use timedatectl. 'timedatectl list-timezones' shows all available zones. Then 'sudo timedatectl set-timezone America/New_York'. This doesn't affect the actual time—just the local display. NTP always works with UTC internally. So your logs in /var/log will be in UTC unless you tell syslog otherwise. Luna: That's a good point. Many sysadmins forget to configure log timestamps and end up confused when correlating events across servers in different timezones. Lucas: And that's where Chrony's accuracy directly impacts security. Certificate validation uses timestamps. If your server's clock is off by even a few minutes, you might reject a valid certificate or accept an expired one. Also, Kerberos authentication requires clocks to be within five minutes of each other. Luna: So time sync is not just about logs—it's about trust. What about leap seconds? Chrony handles them gracefully? Lucas: Yes. Chrony can handle leap seconds by either slewing the clock over a longer period or stepping it. By default, it slews, which means it adjusts the clock frequency gradually so that the time never jumps. You can check if a leap second is pending with 'chronyc -a leapstatus'. Luna: Good to know. Let's talk about securing NTP itself. Is there any risk in running an NTP server? Lucas: You should restrict who can query your server. In chrony.conf, you can add 'allow 192.168.0.0/16' to limit clients to your subnet. There's also 'deny all' to block everything, then specific 'allow' statements. And you can use 'bindcmdaddress' to limit command channel access to localhost. Luna: And for the client side, you want to use multiple NTP sources to avoid a single point of failure. But what about NTP authentication? Should I care about that? Lucas: NTP authentication uses symmetric keys to verify that the time information comes from a trusted source. It's important if you're in a hostile network environment. You set up a key file with 'keyfile /etc/chrony/chrony.keys' and then add 'key 1' in the server line. Each server can have a different key. Luna: That's advanced, but good to know. For most internal networks, just restricting access is enough. So, to recap: install chrony, point it to good NTP pools, verify with chronyc, set timezone with timedatectl, and optionally secure it. Lucas: Pretty much. And if you ever need to check if your clock is accurate, you can compare against a known good source like time.google.com using 'chronyc -a -m sources -v'. But remember, it's not just about being accurate—it's about being consistent across your fleet. Luna: Yeah, consistency is key. If every server is off by the same amount, at least logs line up. But ideally, they're all within a few milliseconds of real time. Lucas: Exactly. Now, if today's deep dive into Chrony saved you from a late-night debugging session, that's worth something. For anyone who gets value from these ad-free conversations, the link is buy me a coffee dot com slash fexingo. It's a small gesture that keeps the show going without sponsors. Luna: I'll second that. Even a few coffees help us keep producing the deep dives you all actually use. Lucas: And speaking of deep dives, next time we might look at how to configure Chrony as a high-accuracy time server using hardware timestamping, which gets you into microsecond territory. That's a whole other level. Luna: I'd be into that. For now, double-check your chrony.conf, run chronyc tracking, and make sure your servers are all on the same page—literally.