Latest / Linux Server Admin with Fexingo: Sysadmin, Bash, and Server Engineering / How to Secure Linux SSH with Fail2ban and Key-Only Auth
Transcript
- Lucas: Alright, Luna, let's talk about something that keeps every sysadmin up at night: SSH brute force attacks. If you have a Linux server exposed to the internet with SSH on port 22, you are getting hit—probably every few minutes—by bots trying common usernames and passwords. Luna: Yeah, I've seen the logs. It's almost a rite of passage. You spin up a new VPS, check auth.log the next day, and there are thousands of 'Failed password for root' entries. Lucas: Exactly. And the standard first line of defense is Fail2ban. But a lot of people install it and never tune it. So today I want to walk through a proper Fail2ban setup for SSH, and then go a step further: why you should ultimately move to key-only authentication. Luna: Okay, let's start with Fail2ban. How does it actually work under the hood? Lucas: Fail2ban is essentially a log scanner with an action engine. It watches log files—like /var/log/auth.log—for patterns that indicate failed authentication. When a certain threshold is reached within a defined time window, it executes an action, typically adding an iptables rule to drop all traffic from that IP for a set period. Luna: So it's basically dynamic firewall rules triggered by log events. I like that it's modular—you can define custom filters and actions. Lucas: Right. The default SSH jail in Fail2ban is decent, but I always recommend a few tweaks. First, increase the bantime from the default 10 minutes to something like one hour—or even 24 hours if your users aren't roaming. Second, set a reasonable maxretry—say 3 to 5 attempts—and a findtime of 10 minutes. Luna: And what about ignoring trusted IPs? You don't want to lock yourself out when you're legitimately trying to connect. Lucas: Absolutely. The ignoreip directive in the jail.conf lets you whitelist your office IP or VPN range. I also add a recidive jail that catches repeat offenders—if an IP gets banned multiple times, it gets a much longer ban, like a week. Luna: That makes sense. But even with Fail2ban, you're still allowing password authentication over SSH. Which brings us to the bigger question: why not just turn off password auth entirely? Lucas: Exactly. Fail2ban is a reactive measure. It blocks an IP after a failed attempt. But key-only authentication is proactive—it eliminates the attack surface entirely. If you disable password authentication, even if a bot knows your username, it can't brute force because SSH simply won't accept a password. Luna: So how do you set that up? Walk me through the key generation and configuration. Lucas: First, on your client machine, generate an SSH key pair using ssh-keygen -t ed25519. Ed25519 is faster and more secure than RSA for most use cases. You'll get a private key—which stays on your client—and a public key, which you copy to the server. Luna: And you copy the public key into ~/.ssh/authorized_keys on the server, right? Using something like ssh copy id. Lucas: Exactly. ssh copy id is the easiest way—it appends your public key to the remote user's authorized_keys file. Once that's done, you test that you can log in with the key. Then you edit /etc/ssh/sshd_config and make three critical changes. Luna: What are they? Lucas: First, set PasswordAuthentication to no. Second, set ChallengeResponseAuthentication to no—that covers keyboard-interactive password prompts. Third, set PermitRootLogin to prohibit-password or just no if you never need root SSH access. Then restart the SSH service. Luna: Wait—if you lock yourself out because you forgot to copy the key or you made a syntax error, how do you get back in? Lucas: That's the number one fear. Always, always keep an active SSH session open while testing. And have an out-of-band access method—like a console in your hosting provider's dashboard or a serial console—so you can fix the config if something goes wrong. Alternatively, you can set up a cron job that restores the previous sshd_config if SSH doesn't restart cleanly. Luna: That's clever. I've seen people use a systemd timer that runs a script after a minute to revert changes if a flag file isn't removed. But honestly, the safest approach is to test on a non-production machine first. Lucas: Absolutely. Now, once you have key-only auth, you can even go further: add a passphrase to your private key so that even if someone steals your laptop, they can't use the key without the passphrase. Or use an SSH agent with ssh-agent to cache the passphrase for the session. Luna: And for extra security, some people use hardware tokens like a YubiKey to store the private key. That way the key never leaves the device. Lucas: Right. YubiKey with FIDO2 or PIV support can act as an SSH key. It's a bit more setup, but it's very secure. But even without hardware, just disabling passwords and using Ed25519 keys is a massive improvement over default SSH. Luna: I want to circle back to something you said earlier about Fail2ban being a band-aid. Do you think it's still worth running even after you switch to key-only auth? Lucas: Yes—defense in depth. Fail2ban can still catch other attacks, like web application brute force if you have a web server, or SMTP authentication failures. And even with key-only auth, you might still get connection attempts that fill logs. Fail2ban can keep those logs cleaner by banning repeat scanners. Luna: Good point. So the ideal setup is both: Fail2ban as a perimeter shield, and key-only auth as the fundamental access control. Lucas: Exactly. And the numbers back it up. I had a small web server that was getting about 2000 failed SSH attempts per day. After implementing the Fail2ban tune-ups, that dropped to maybe 200 attempts from IPs that hadn't been banned yet. After switching to key-only auth and disabling passwords, the logs showed zero failed password attempts—because there was nothing to attempt. Luna: That's a huge reduction. And it's not just about security—it's also about log noise. Fewer alerts to sift through means you can actually spot real issues. Lucas: Quick honest thing—productions like this one, deep dives into practical sysadmin work, they take real time to research and produce. A handful of listeners chip in monthly through buy me a coffee dot com slash fexingo, and that's literally what lets us keep making these episodes without ads or sponsors. Luna: Yeah, and it means we can cover exactly what you need to hear—no filler, no fluff. So if this episode saved you a headache, consider tossing a coffee our way. It really does help. Lucas: Alright, back to SSH. One last thing I want to mention is the importance of key management. If you have multiple servers, you can use an SSH config file on your client to define different keys for different hosts. And never, ever share your private key. Luna: And have a backup of your keys—encrypted, of course. I keep mine on an encrypted USB drive in a safe place. Losing your private key is almost as bad as having it stolen. Lucas: Great advice. So to recap: use Fail2ban with tuned settings, switch to Ed25519 keys, disable password and challenge-response authentication, and always have a recovery plan. That combination will harden your SSH dramatically. Luna: And you can sleep a little easier knowing your server isn't an open door. Thanks, Lucas. Lucas: Thanks, Luna. Next time, we'll talk about securing the boot process with UEFI and Secure Boot on Linux. Until then, keep your keys safe.