Latest / Linux Server Admin with Fexingo: Sysadmin, Bash, and Server Engineering / Linux Server SSH Certificate Authentication Setup
Transcript
- Lucas: So let's talk about SSH certificate authentication. It's one of those things that sounds complicated — I mean, 'certificate authority', 'signed keys', it feels enterprise-ish — but once you actually run through it, it's simpler than managing a wall of authorized_keys files. Luna: And probably more secure too, right? I've seen setups where an admin's key file has fifty entries from old contractors. Lucas: Exactly that problem. SSH certificate auth replaces all that with a trust model. You set up a Certificate Authority — a CA — that signs both host keys and user keys. Then your servers only need to trust the CA public key. No more copying individual user public keys around. Luna: So it's like how TLS works for websites, but for SSH. Lucas: Very similar. The CA signs a user's public key and issues a certificate that includes the user's identity and optionally some constraints — like which usernames they're allowed to use, or an expiration time. The server verifies that certificate against the CA's public key. If it's valid, you're in. Luna: And host certificates work the same way the other direction — so clients can verify they're connecting to the real server, not a man-in-the-middle. Lucas: Right. So let's actually build this. I'll walk through the steps on a fresh Ubuntu 24.04 server I've got here. First thing: generate the CA key pair. This is the root of trust, so it needs to live on a secure machine — ideally not one of the servers you're managing, and definitely not on a laptop that could be stolen. Luna: Offline air-gapped machine would be ideal, but even just a dedicated admin jump box with strict access controls is a big step up from nothing. Lucas: Absolutely. So on my CA machine, I run 'ssh-keygen -t ed25519 -f ca_user_key -C user_ca'. That gives me a private key and a public key. The private key is what signs certificates — protect it like root. Luna: Why ed25519? Lucas: It's fast, well-audited, and the keys are short. RSA works too, but ed25519 is the modern default. Then I generate a separate host CA key — same command but with different file name and comment, like 'ca_host_key'. Luna: Separate CAs for users and hosts — that's a good practice. If one gets compromised, the other isn't affected. Lucas: Exactly. Now I need to sign a user key. Let's say I have a user named 'luna' with an existing key pair. On the CA machine, I copy their public key over — via scp or a USB stick — and run: 'ssh-keygen -s ca_user_key -I luna@fexingo -n luna -V +52w luna.pub'. That signs it and produces 'luna-cert.pub'. Luna: The -I flag is an identity string, -n specifies the authorized username, and -V sets the validity period. So that certificate is good for one year? Luna: 52 weeks, yes. You can set shorter windows — 4 weeks, 1 week — depending on your security posture. The key point is, once it expires, the user can't log in until they get a fresh certificate. That's huge for offboarding. Lucas: No more hunting through authorized_keys files to remove someone. You just stop signing their keys, or let their certificate expire. Now I need to configure my server to trust the CA. I copy the CA's public key — ca_user_key.pub — to the server, say to /etc/ssh/ca_user_key.pub. Luna: And then in sshd_config, you add 'TrustedUserCAKeys /etc/ssh/ca_user_key.pub' and 'PubkeyAuthentication yes'. That's basically it. Lucas: Almost. You also want 'AuthorizedPrincipalsFile none' unless you're using principals for fine-grained access. And restart sshd. Now when Luna connects with her signed certificate, the server checks the sig against the trusted CA key — and if the cert is valid, she's authenticated without ever needing an authorized_keys entry. Luna: What about host certificates? That's the flip side — it helps prevent MITM attacks when you SSH into a server for the first time. Lucas: Right. So on the server itself, I generate a host key as usual — but instead of just keeping the public key, I send it to the CA to be signed. On the CA, I run: 'ssh-keygen -s ca_host_key -I server01.fexingo.com -h -n server01.fexingo.com /etc/ssh/ssh_host_ed25519_key.pub'. The -h flag means it's a host certificate. Luna: Then you copy the signed certificate back to the server. And on the clients, you set 'HostCertificate' in sshd_config? No, that's for the server side. Lucas: On the server, you put the signed host cert at /etc/ssh/ssh_host_ed25519_key-cert.pub, and sshd automatically uses it if it finds it. Then on clients, you configure 'GlobalKnownHostsFile' to point to a file containing the host CA public key, or you set '@cert-authority *.fexingo.com' in your known_hosts file. Luna: That way, when you SSH into any server under that domain for the first time, it verifies the host cert against the CA. No more 'The authenticity of host... can't be established' prompts. Lucas: Exactly. And it scales beautifully. I manage about sixty servers — some ephemeral, some long-lived. With certificate auth, I spin up a new VM, install the CA public key in sshd_config, and every user with a valid signed key can log in immediately. No touching each server to add new user keys. Luna: Offboarding is even cleaner. If a contractor leaves, you revoke their certificate by adding a serial number to a Key Revocation List — or you just stop signing new ones and wait for expiration. Lucas: There's also the option of using 'RevokedKeys' in sshd_config to point to a file listing revoked public keys. But honestly, expiration is the cleaner mechanism. Set short validity windows — 30 days — and you force a regular renewal process. That also gives you a heartbeat: if someone's key isn't being renewed, maybe they've left. Luna: The one downside I've seen is that if the CA private key is compromised, an attacker can sign keys for any user. So protecting that CA key is paramount. Lucas: Huge caveat. The CA machine should be locked down — full disk encryption, minimal services, strong SSH access maybe even only via hardware token. Some teams keep the CA key on a hardware security module or a smartcard. For smaller setups, an encrypted USB that you only plug in when signing is a practical compromise. Luna: And you should have a procedure to rotate the CA key if needed. Generate a new one, redistribute the public key to all servers, sign new user and host certs. Lucas: That's a bit of an operation, but it's doable. And let's be honest — rotating a CA key once a year is still less work than managing authorized_keys across a fleet. I've actually scripted the whole signing process. I have a small Bash tool that takes a username, generates a temp key pair on an air-gapped laptop, signs it, and exports the cert. Luna: That's the next step for me — automate the certificate issuance. Right now I'm still doing it manually with ssh-keygen commands, but it's already saving me time. Lucas: You could even wrap it in a simple web interface or a Slack bot. But even manual command-line signing is a huge improvement over the old way. And if you want to go further, you can enforce certificate constraints like source addresses or forced commands. Luna: Speaking of improvements — and this is something our listeners might find useful — if you're into automating your server workflows, you might appreciate the kind of deep-dive content we do here. We keep this show ad-free and listener-supported, and that's only possible because some of you choose to chip in. If today's episode gave you a practical tool or a new idea, you can throw a few bucks our way at buy me a coffee dot com slash fexingo. Lucas: Yeah, it's a small way to keep the server lights on and the Bash examples flowing. No pressure, genuinely — but if you find value, that's where we live. Now, back to constraints: one cool feature is that you can embed a 'force-command' in the certificate. So if you have a user who only needs to run backups, their cert can force them into a specific script. Luna: That's essentially a signed, restricted shell. Great for service accounts or monitoring tools. Lucas: Exactly. And you can also restrict which source IP a certificate is valid from. So even if someone's key is stolen, the attacker can't use it from outside your VPN range. Luna: Let's talk about the actual configs. For a standard setup, you'd have 'TrustedUserCAKeys' in sshd_config, and on the client side, you'd add the signed certificate to the SSH agent with 'ssh-add ~/.ssh/luna-cert.pub' — wait, you add the private key, but the agent picks up the cert if it's next to it. Lucas: Right. If the certificate file is named 'luna-cert.pub' and lives next to the private key 'luna', ssh will automatically try it. Or you can specify 'CertificateFile' in your SSH config. That's useful if you have multiple certs for different roles. Luna: One thing I want to highlight: certificate auth doesn't replace SSH keys entirely — you still need a key pair to be signed. But it does eliminate the need for the server to know about individual user public keys. Lucas: And it eliminates the 'authorized_keys' file management headache. I actually did a quick count on my old setup: I had about 300 lines of authorized_keys across 60 servers. Now I have zero. One line in sshd_config per server: 'TrustedUserCAKeys /etc/ssh/ca.pub'. Luna: And that same line works for every user. So if you hire someone new, you just sign their key. No server changes needed. Lucas: That's the scalability win. The other win is auditing: SSH logs will show the certificate serial number and identity, so you can tie every login to a specific certificate issuance event. That's huge for compliance. Luna: We should also mention that OpenSSH supports certificate revocation lists. You can create a file with revoked serial numbers and point 'RevokedKeys' to it. Though, again, expiration is usually simpler. Lucas: If you want to be really thorough, you can combine both: short expiration plus a CRL for immediate revocation. But for most teams, short-lived certs are enough. I'd recommend starting with a 30-day validity and see how it goes. Luna: So if someone is fired on day 15, their cert expires in 15 more days — but you could also add them to a revocation list immediately. Lucas: Exactly. And you'd push that revocation list out to servers via your config management tool — Ansible, Puppet, whatever. It's a one-liner. So to wrap up: SSH certificate auth is one of those upgrades that gives you immediate security and operational benefits. The setup is a few commands, and the payoff is enormous. Luna: I'd say to anyone using SSH key pairs manually: try setting up a CA on a throwaway server first. Sign a test user key, configure one server, and see how it feels. I think you'll be sold. Lucas: And if you want to take it further, look into SSH CA integration with HashiCorp Vault — that can automate the whole signing lifecycle. But that's probably a topic for another episode.