Latest / Linux Server Admin with Fexingo: Sysadmin, Bash, and Server Engineering / Why Your Linux Server Needs a Dedicated nullmailer Setup
Transcript
- Lucas: There is a silent failure pattern on Linux servers that I guarantee most of us have suffered through, and it's the moment you realize your server has been trying to send you emails for months and you never got a single one. Luna: Oh, the classic — you check a cron job's logs and find out it's been spitting out delivery failures to root@localhost since February. Lucas: Exactly. And the default fix people reach for is installing a full Postfix stack with all the dependencies, which is overkill for a box whose only outbound mail job is forwarding cron output and system alerts to an actual human. Luna: So what do you use instead? Lucas: I want to make the case for nullmailer. It's a tiny mail transport agent — the package is maybe 200 kilobytes — that does exactly one thing: it accepts local mail and forwards it through a configured relay host. No queue management, no local delivery, no spam filtering. Just forwarding. Luna: And it drops in as a sendmail replacement, so cron and other system tools that assume sendmail exists suddenly work without fuss. Lucas: Right. The configuration lives in a single directory under /etc/nullmailer. You create a file called 'remotes' with one line — your relay host, the port, and whether to use STARTTLS. Then you set the 'adminaddr' file to the external address where you want root's mail forwarded. That's basically it. Luna: Wait — so the whole setup is two files? No database, no daemon restart? Lucas: Well, nullmailer runs as a persistent daemon, but it's lightweight. On a typical VPS with 1 gigabyte of RAM, it uses under 10 megabytes resident. And you don't have to restart it after changing config — it picks up changes on the fly. Luna: What about authentication? Most SMTP relays require login credentials these days. Lucas: You put the username and password in a file called 'auth' — one line, colon-separated. Nullmailer supports LOGIN and PLAIN authentication. And you want to make sure that file is owned by root with permissions 0600, otherwise nullmailer will actually refuse to read it. Luna: Good safety check. So if I'm setting this up today, what relay would you point it at? Lucas: Any transactional email provider works. I've used SendGrid, Mailgun, Amazon SES. The key is that the relay accepts mail on port 587 with STARTTLS — that's the submission port. Port 25 is often blocked by cloud providers anyway. Luna: And what about the from address? If cron sends mail as root, the recipient sees 'From: root@server.example.com'. That looks pretty sketchy. Lucas: Nullmailer rewrites the envelope from based on the adminaddr file. If you set that to 'alerts@yourdomain.com', then all outbound mail appears to come from that address. The relay will typically rewrite the header from too, but the envelope is what matters for deliverability. Luna: So that solves the 'my cron job results went into a black hole' problem. Do you have a war story from your own servers? Lucas: A few years ago I was troubleshooting a staging server that had been generating nightly backup failure alerts for six months. The ops team thought backups were fine because the web dashboard showed green. Turned out the backup script was piping output to mail, but the default MTA was exim with no relay configured. Those emails were queuing up and eventually bouncing silently. The web dashboard only checked that the backup process exited zero — not that the data was actually written to the remote store. Luna: Ouch. So a nullmailer setup would have surfaced that failure on day one. Lucas: Absolutely. And the fix took five minutes. Install nullmailer, write two config lines, and suddenly all those accumulated failure notices arrived in someone's inbox. Luna: Alright, I'm sold. But let's talk about alternatives. msmtp is another minimal MTA that people use for the same purpose. How does it compare? Lucas: Msmtp is also lightweight, but it's not a persistent daemon — it runs as a client that delivers mail synchronously. That means every time cron sends a mail, msmtp opens an SMTP connection, authenticates, and sends. If the relay is down, the mail is lost immediately. Nullmailer queues the message locally and retries with exponential backoff. Luna: So for reliability, nullmailer wins. Lucas: I think so. There's also postfix in 'satellite' mode — you configure it to forward all mail to a relay. But postfix is a full MTA with multiple daemons, a complex configuration, and a larger attack surface. If all you need is cron mail forwarding, nullmailer is the right tool. Luna: What about security? Nullmailer doesn't do any encryption of the queue files, does it? Lucas: No, the queue is stored as plain text files under /var/spool/nullmailer. So if someone gets root on the box, they can read any queued mail. For most server alert use cases, that's an acceptable risk. But if you're forwarding authentication tokens or sensitive reports, you might want to encrypt at the application level before piping to mail. Luna: Good point. Or use a more robust setup like a dedicated monitoring system that sends alerts via HTTPS instead of SMTP. Lucas: Exactly. But for the vast majority of servers — the ones where you just want to know when a disk is filling up or a service has restarted — nullmailer is the simplest thing that works. Luna: I'm going to set this up on my home lab server tonight. One last question: can nullmailer handle multiple domains? Say I manage several servers and want mail from each to appear to come from a different address. Lucas: It can, but not within a single nullmailer instance. You'd need a separate nullmailer install per domain, each with its own config directory. Or you can use the 'defaultdomain' and 'masquerade' settings to rewrite the domain part. But honestly, for most setups, one admin address per server is fine. If you need more sophisticated routing, you probably want a proper MTA. Luna: Fair. So bottom line: install nullmailer, set adminaddr to your external inbox, configure the relay in remotes, lock down the auth file, and you're done. Lucas: And test it. After setup, run 'echo test | mail -s test your@email.com'. Then check the nullmailer logs — they're in /var/log/nullmailer/ by default. You'll see the delivery succeed or you'll get a clear error. Luna: Which brings us back to the theme of this whole episode: don't let your server's voice go unheard. If it's trying to tell you something, make sure you're listening. Lucas: Honestly, if today's tech conversation gave you something usable — maybe five minutes that saves you from a silent backup failure — that's exactly the kind of thing that keeps this show ad-free. If it was worth a coffee to you, the link is buy me a coffee dot com slash fexingo. No pressure, just a way to say this kind of conversation matters. Luna: Yeah, I think that's a fair ask. We put the time into digging into these tools so you don't have to. Lucas: Alright — back to the nuts and bolts. One thing I didn't mention: nullmailer also supports TLS via the 'ssl' option in the remotes file. You can specify 'ssl' to use implicit TLS on port 465, or 'starttls' for opportunistic upgrade on port 587. I recommend port 587 with STARTTLS because it's more widely supported and less likely to be blocked. Luna: And if your relay requires a specific certificate verification? Lucas: Nullmailer will verify certificates by default using the system CA bundle. If you're using a self-signed cert on your relay, you can point it to a custom CA file with the 'cafile' option. But in the cloud era, you rarely need that. Luna: What about rate limiting? If a cron job runs every minute and sends mail, will nullmailer throttle? Lucas: It doesn't throttle on its own. Each message is delivered as fast as the relay accepts them. If your relay has rate limits, you'll get backoff from them, and nullmailer will queue until the relay is ready. But for typical cron jobs that fire a few times a day, it's a non-issue. Luna: So to wrap up: nullmailer is the lightweight champion for outbound server mail. It's simple, reliable, and keeps your server from shouting into the void. Lucas: Exactly. And if you want to go deeper, the man page is excellent. There's also a sample config in /usr/share/doc/nullmailer/ on Debian-based systems. That's where I learned the details. Luna: Alright, I'm off to configure mine. Thanks for the walkthrough. Lucas: Happy to. Until next time — keep your servers talking, and make sure you're on the receiving end.