Latest / Linux Server Admin with Fexingo: Sysadmin, Bash, and Server Engineering / How to Use Linux Server Namespaces with Podman for Rootless Containers
Transcript
- Lucas: Alright, so today we're going to talk about running containers without root. Not sudo inside the container—I mean the entire container runtime runs as an unprivileged user on the host. Luna: You mean like Docker's rootless mode? Or something else? Lucas: Close. We're focusing on Podman specifically, because Podman was built from the ground up to be rootless. Docker added rootless mode later as an optional feature, but Podman does it natively. And the key enabler here is Linux user namespaces. Luna: Before we go too deep—Lucas, I know you've talked about this before, but a couple of dollars a month genuinely keeps these shows going. If you've gotten something out of the Linux Server Admin episodes, buy me a coffee dot com slash fexingo. It's ad-free because of listeners like that. Lucas: Yeah, absolutely. It makes a real difference. So back to user namespaces—Podman maps the container's root user to a regular user on the host, say UID 1001. So inside the container, you're root, but outside, you're just a regular user with no special privileges. Luna: That means if someone breaks out of the container, they're just a regular user on the host. They can't modify system binaries or install kernel modules. Lucas: Exactly. And Podman uses a tool called `slirp4netns` for networking in rootless mode, which also doesn't require root. But there are some gotchas—like you can't bind to ports below 1024 without a workaround. Luna: Right, because those are privileged ports. So if you want to run a web server on port 80, you need to either forward from a privileged process or use a higher port and redirect. Lucas: Exactly. One common approach is to run a reverse proxy like nginx in a rootful container that binds to port 80 and 443, and then have your rootless app containers behind it. Or you can use `authbind` or set CAP_NET_BIND_SERVICE on the binary. Luna: Let's walk through a concrete example. How do I actually start a rootless container with Podman? Lucas: Super straightforward. First, you need to have subuid and subgid mappings set up for your user. Most distros handle this automatically when you install Podman. You can verify with `podman info` and look for the user namespace section. Luna: And then you just run `podman run` as a regular user? Lucas: That's it. For example: `podman run -d -p 8080:80 nginx`. That launches an nginx container on port 8080. No sudo, no root. The container's root is mapped to your user's UID on the host. Luna: But what about volume mounts? If I mount a host directory into the container, the files might have different ownership. Lucas: Great question. Since the container root is your user, any files written inside the container will appear on the host as owned by your user. But if you mount a directory that's owned by someone else, the container root can't write to it. You need to either use `podman unshare` to change ownership, or use the `:Z` flag for SELinux relabeling. Luna: So there's a bit of a learning curve for storage. Another thing I've run into is that rootless Podman uses fuse-overlayfs instead of the kernel's overlay filesystem, because overlayfs requires root. Lucas: Right. Fuse-overlayfs is a FUSE implementation of overlayfs that runs in userspace. It's a bit slower, but for most workloads it's fine. You can check if it's in use with `podman info` under 'store.graphDriverName'. Luna: What about performance? Is the overhead significant? Lucas: For the container runtime itself, the overhead is negligible—we're talking microseconds per syscall for the namespace switching. The bigger impact is fuse-overlayfs for write-heavy workloads. But if you're running stateless web servers or batch jobs, you probably won't notice. Luna: And there's also the networking overhead with slirp4netns, right? Lucas: Yeah, slirp4netns does user-mode networking which adds some latency and throughput loss. For high-performance networking, you might want to use the `--network host` option, but that exposes the container's network directly to the host—still safe since it's rootless. Luna: One thing I love about Podman is that it integrates with systemd. You can generate a systemd unit file for a rootless container and have it start on boot. Lucas: Exactly. `podman generate systemd --new --name mycontainer` gives you a unit file. Then you can place it in `~/.config/systemd/user/` and enable it with `systemctl --user enable mycontainer`. The container will start when the user logs in. Luna: But what about starting at boot without login? You need lingering enabled for the user. Lucas: Right. You run `loginctl enable-linger $USER` and then systemd user instances start at boot. That's a critical step that's easy to forget. Luna: So for a production server, you might have a service user like 'app' with linger enabled, and all containers run under that user. Lucas: Exactly. And because it's rootless, even if the container is compromised, the attacker can only mess with files owned by that user. They can't install kernel modules, can't change iptables rules, can't mount filesystems. Luna: That's a huge security win. Docker's rootless mode does something similar, but it still runs a rootful daemon for some operations unless you configure it carefully. Lucas: Yeah, Docker's rootless mode actually runs the Docker daemon as a user namespace root, but it's still a daemon. Podman has no daemon at all—each container is a child process of the Podman command. That means no central point of failure, and no daemon running as root. Luna: So Podman is more 'Unix-philosophy'—do one thing and do it well. Lucas: Exactly. And it uses the same OCI image format as Docker, so you can pull images from Docker Hub or any registry. `podman pull nginx` works the same as `docker pull nginx`. Luna: One thing I've seen in multi-tenant environments is that you can set resource limits per user using cgroups v2. Podman respects those automatically. Lucas: Right, because Podman runs under the user's cgroup slice. If you set a memory limit for user 'app' in systemd, all their containers share that limit. That's great for shared hosting. Luna: Let's talk about a real-world scenario. Say we have a server with multiple developers, each needing to run containers. With rootless Podman, each developer can run their own containers without interfering with each other. Lucas: And without needing sudo. They can install Podman themselves, set up their subuid mappings, and go. The sysadmin just needs to allocate UID ranges in `/etc/subuid`. Luna: Which is typically done automatically by the `podman` package on most distros. But if you need to add a user manually, you edit `/etc/subuid` and add a line like `username:100000:65536`. Lucas: Exactly. That gives the user 65536 subordinate UIDs starting at 100000. Then Podman can map container UIDs into that range. Luna: Are there any security gotchas with rootless containers that people overlook? Lucas: One big one: even though the container is rootless, if you use `--privileged` or mount `/sys` or `/proc` in certain ways, you can still escape the user namespace. So avoid those flags unless you really know what you're doing. Luna: Also, rootless containers can't use some kernel features like AppArmor or SELinux policies that require root to load. But they still benefit from the user namespace isolation. Lucas: Right. And you can still use seccomp profiles to restrict syscalls. Podman applies a default seccomp profile even in rootless mode. Luna: Alright, so what's the takeaway for someone managing Linux servers today? Lucas: If you're running containers, seriously consider going rootless with Podman. It's mature, it's the default on many distros like Fedora and RHEL, and it eliminates a huge class of privilege escalation attacks. The trade-offs in performance and port binding are manageable with proper design. Luna: And you can start small—just run a single test container without sudo and see how it feels. Lucas: Yeah. And once you get used to it, you'll wonder why you ever ran containers as root. That's it for this episode of Linux Server Admin. I'm Lucas. Luna: And I'm Luna. See you next time.