Latest / Linux Server Admin with Fexingo: Sysadmin, Bash, and Server Engineering / How to Use Linux Network Namespaces for Multi-Tenant Isolation
Transcript
- Lucas: If you've ever needed to run two separate network stacks on a single Linux server — maybe for testing, or for isolation between tenants — network namespaces are the tool. They let you create essentially a different view of the network stack per process. Luna: So it's like giving each process its own network? No shared IPs or routes? Lucas: Exactly. Each namespace gets its own interfaces, routing table, firewall rules — the whole TCP/IP stack. And the way you create them is dead simple: 'ip netns add'. Let's say we add two namespaces: ns_red and ns_blue. Luna: But if they're completely isolated, how do they talk to the internet? Or to each other? Lucas: That's where virtual Ethernet pairs come in — veth pairs. Think of them like a two-ended cable. You attach one end to a namespace, the other to the host namespace. Then the namespace can send traffic out through that virtual cable. Luna: Right — I've seen that in container runtimes like Docker. They use veth pairs to connect containers to the host bridge. Lucas: Exactly the same mechanism. But here we're doing it manually to understand what's happening. So after creating the namespace, you create a veth pair: 'ip link add veth_red type veth peer name veth_blue'. Then you move one end into ns_red, assign an IP, and bring it up. Luna: And the other end stays in the default namespace? Or can you put both in separate namespaces? Lucas: You can do either. For one namespace talking to the host, you keep the peer in the default namespace. For two namespaces talking directly, you put each peer into its own namespace. But then you also need to set up routing on the host to forward between them, or use a bridge. Luna: So the host becomes a router. That's fine for a lab, but in production you probably want a bridge so they can talk without the host forwarding? Lucas: Right. You'd create a Linux bridge, attach the host-end of each veth pair to the bridge, and then the namespaces can talk to each other through the bridge. That's essentially how Docker's default bridge network works. Luna: And the bridge also gives them access to the outside world if you add the host's physical interface to the bridge, or set up NAT. Lucas: Exactly. For external connectivity, you'd masquerade the traffic using iptables or nftables — just like a home router. The namespaces get private IPs, and the host does source NAT out the physical interface. Luna: What about DNS? Does each namespace need its own resolv.conf? Lucas: Good question. Each namespace inherits the host's /etc/resolv.conf by default, but you can override it inside the namespace. Just bind mount a custom file or use 'ip netns exec' to edit it. Luna: So a typical workflow: create namespace, create veth pair, move one end in, assign IP, set default route, maybe tweak DNS, and test with ping. Lucas: That's the skeleton. Let me give you a concrete example. I've got a server here — let's walk through the commands. First, 'ip netns add ns1'. Then 'ip link add veth1 type veth peer name veth1_host'. Then 'ip link set veth1 netns ns1'. Luna: And then you assign IPs: inside ns1, 'ip addr add 10.0.0.1/24 dev veth1' and 'ip link set veth1 up'. On the host, you assign 10.0.0.2/24 to veth1_host and bring it up. Lucas: Exactly. Then add a default route in ns1: 'ip netns exec ns1 ip route add default via 10.0.0.2'. And enable IP forwarding on the host: 'sysctl net.ipv4.ip_forward=1'. Finally, set up a masquerade rule. Luna: A lot of steps, but once you script it, it's repeatable. I'd wrap that in a bash function. Lucas: Totally. And that's where network namespaces shine — you can automate the whole thing. In fact, many cloud providers use network namespaces under the hood to create virtual networks for each tenant. It's the same technology, just orchestrated at scale. Luna: So when you spin up a virtual private cloud, it's likely namespaces plus some SDN controller. Lucas: Right. And the beauty is it's all built into the Linux kernel. No extra daemon needed. You can run 'ip netns list' and see what's active. It's incredibly lightweight compared to full virtualization. Luna: One thing I've struggled with: if you have a process that needs to run in a namespace but also access a physical device, like a USB modem or a GPU for networking? Can you move physical interfaces into a namespace? Lucas: Yes — you can move a physical NIC into a namespace using 'ip link set eth1 netns ns1'. But then it's only visible to that namespace. That's useful for dedicated interfaces, but you lose host access to that NIC. So you'd need a separate physical NIC for the host. Luna: Got it. So either you use veth pairs for shared connectivity, or you dedicate a physical interface to a namespace. Lucas: Exactly. And you can also combine both — for example, have a namespace with a veth pair for management traffic and a physical NIC for data traffic. Luna: Okay, let's talk troubleshooting. What's the first thing you check when a namespace can't reach the internet? I've definitely been there. Lucas: First, confirm the namespace exists: 'ip netns exec ns1 ip addr'. Check that the interface is up and has an IP. Then check the default route inside the namespace: 'ip netns exec ns1 ip route'. If there's no default route, add it. Luna: And on the host, you need to make sure forwarding is enabled and the veth peer is up. Lucas: Right. Also check iptables rules — remember, each namespace has its own iptables tables, but the host iptables also applies to traffic coming from veth interfaces if you have FORWARD rules. A common gotcha: the FORWARD policy is DROP, so you need to allow forwarding between the physical interface and the veth peer. Luna: So you'd do something like 'iptables -A FORWARD -i eth0 -o veth1_host -j ACCEPT', and the reverse. Lucas: Exactly. And if you're using nftables, it's a similar flow. Also check that the host has IP forwarding enabled globally — some distros disable it by default. Luna: Another problem I've had: ping works from the namespace to the host, but not to the outside. Usually it's NAT missing. Lucas: Yep. You need a MASQUERADE rule for traffic leaving the physical interface. Something like: 'iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE'. And make sure the default route in the namespace points to the host's veth end. Luna: And if you want to access services inside the namespace from outside? Say a web server running in ns1 on port 80. Lucas: You'd set up a port forward rule on the host to DNAT traffic from a host port to the namespace's IP. For example: 'iptables -t nat -A PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 10.0.0.1:80'. But be aware that by default, traffic from the host itself won't hit PREROUTING — you'd need additional rules for local traffic. Luna: That's a deep rabbit hole. But for most sysadmins, a simple bridge setup with NAT covers the basics. Lucas: Yeah, and that's actually a great segue. If today's conversation gave you something usable — a command, a concept, a troubleshooting tip — consider supporting the show. We keep it ad-free, and listener support on buy me a coffee dot com slash fexingo makes that possible. No pressure, just a sincere nod. Luna: Yeah, it's how we keep doing deep dives like this without commercial interruptions. Appreciate everyone who chips in. Lucas: So back to namespaces: one more practical thing — persistence. Network namespaces are ephemeral; they disappear on reboot unless you script their creation in systemd or a startup script. Luna: Or you could use a tool like systemd-nspawn or even Docker that handles persistence. But sometimes you want a lightweight, no-daemon approach. Lucas: Exactly. For testing or ephemeral workloads, manual namespaces are perfect. And if you need them persistent, just stick the 'ip netns' commands in /etc/rc.local or a systemd oneshot unit. Luna: Alright, I think we gave folks a solid foundation. Any final tip? Lucas: One: always clean up. 'ip netns delete ns1' removes the namespace and its veth pairs automatically. Keeps the system tidy. Luna: Good advice. Thanks, Lucas. Lucas: Thanks, Luna.