Latest / The API Podcast with Fexingo: REST, GraphQL, and Modern Web APIs / How API Gateways Do the Heavy Lifting
Transcript
- Lucas: So I was reading about how a lot of modern APIs are fronted by what's called an API gateway, and it struck me that this is one of those quiet pieces of infrastructure that almost every team ends up using, but hardly anyone talks about what it actually does. Luna: Yeah, I feel like every job posting for a backend engineer mentions 'API gateway experience', but nobody ever explains what it is or why it matters. Lucas: Right. And it's one of those things that you might have used without even knowing it. Like if you've ever called an API that has a key, or a rate limit, or returns JSON even though the backend is in a legacy format, there's a decent chance a gateway was doing that work. Luna: So it's like a bouncer, a translator, and a traffic cop all in one? Lucas: That's actually a pretty good way to put it. And today I want to get into the specifics, because the gateway pattern is powerful, but it's also easy to misuse. And we should be honest about the trade-offs. Luna: Before we go deeper, I've got to say, every time we get into these conversations about API infrastructure, I think about how much work goes into making these things run smoothly. And if you've gotten any value out of this show, a couple of dollars a month genuinely helps keep it going. You can buy me a coffee at buy me a coffee dot com slash fexingo. That's it, no pressure, just if it's made a difference for you. Lucas: Yeah, honestly, it's the listener support that keeps this thing ad-free and independent. So if you're inclined, that link is there. But let's get back to the gateway. Lucas: So let's start with a concrete example. Imagine you're building a food delivery app. You've got a service for restaurants, one for orders, one for payments, maybe another for user profiles. Now your mobile app needs to talk to all of them, but you don't want the app to know about every single service address and manage authentication separately. Luna: Right, so the gateway sits in front and the app just talks to one endpoint, like api dot your app dot com, and the gateway figures out where to route each request. Lucas: Exactly. And this is the core value proposition. It gives you a single entry point for your entire API surface. That means you can centralize things like authentication, rate limiting, request validation, and logging, so your individual services don't each have to implement that boilerplate. Luna: And I imagine that's a huge win for security, right? Because you're not duplicating auth logic across fifty services. Lucas: It's a big win, but it's also a big risk, because now that auth logic lives in one place. If the gateway gets compromised, everything behind it is exposed. So it's not a silver bullet, it's a trade-off. Luna: Okay, so what are some of the specific things a gateway can do that you'd have to build yourself otherwise? Lucas: Let's start with authentication. A gateway can validate a token, like a JWT, and then pass the user identity to the backend via headers. So the service doesn't need to know how to parse tokens, it just trusts that the gateway has done that work. Luna: And that also means you can change your auth provider without touching every service, right? Lucas: Exactly. You swap the logic in one place and all your services get the update. Same thing with rate limiting. Instead of each service implementing its own rate limiter, the gateway can enforce a global limit across your entire API, maybe based on API key or user ID. Luna: So if a client is hammering your endpoints, the gateway can just start returning 429 Too Many Requests without ever hitting your servers. Lucas: Right, and that protects your downstream services from overload. But here's the kicker: the gateway is also a potential bottleneck. If it's not configured properly, or if it doesn't have its own rate limits, a single misbehaving client can saturate the gateway's resources, and then every other client is affected. Luna: So it's like a single point of failure, but also a single point of control. Lucas: That's exactly the trade-off. And that's why you need to think about the gateway's own capacity and clustering. But let's move to another key function: request transformation. A lot of backend services are legacy or built by different teams with different conventions. The gateway can translate between the external contract and the internal service contract. Luna: Like if your public API returns camelCase fields but your internal service uses snake_case, the gateway can do that mapping. Lucas: Exactly. And it can also do things like aggregate multiple service responses into a single response. So maybe the app needs order details plus user info plus restaurant info, the gateway can call those three services in parallel and combine the results into one JSON payload. Luna: That's powerful, but isn't that basically what GraphQL does? I thought we were talking about REST APIs. Lucas: Good question. GraphQL is a query language and a runtime for your clients to request exactly what they need. An API gateway is more about routing and cross-cutting concerns. But a gateway can actually sit in front of a GraphQL service, or you could have a gateway that exposes a GraphQL interface while internally calling REST services. They're not mutually exclusive. Luna: So the gateway is more like the front door, and GraphQL is more about how the rooms are structured once you're inside. Lucas: That's a good analogy. And it highlights that the gateway is often the first line of defense and the first point of control. But let's talk about some of the practical pitfalls. One thing I see all the time is misconfigured timeouts. Luna: What do you mean? Lucas: So if the gateway has a default timeout of, say, ten seconds, but your backend service takes fifteen seconds to generate a report, then the gateway is going to kill the request and return a 504 Gateway Timeout. The client never sees the response, and the backend might still be processing the work, which is wasted compute. Luna: And that can also cause a lot of retries, which makes the problem worse. Lucas: Exactly, and that's why you need to be intentional about timeout settings per route, not just one global value. Another pitfall is when teams use the gateway as a dumping ground for business logic. It's tempting to put request validation or even some orchestration in the gateway, but then you end up with a god object that's hard to maintain. Luna: So the gateway should stay thin and focus on the plumbing, not the business decisions. Lucas: Exactly. Keep the gateway as a policy enforcement point, not a business logic engine. And that's a good segue to talk about the difference between an API gateway and a service mesh, which is something that comes up a lot in microservices discussions. Luna: Yeah, I've heard those terms used interchangeably, but I'm guessing they're not the same. Lucas: They're related but different. An API gateway sits at the edge, facing external clients. A service mesh handles east-west traffic, which is the communication between services inside your network. The mesh typically uses something like a sidecar proxy to handle things like mTLS, traffic splitting, and circuit breaking between services. Luna: So the gateway is the front door, and the service mesh is like the hallway that connects all the rooms. Lucas: That's a great way to put it. And in practice, you might have both. The gateway routes external requests to the services, and the service mesh handles the internal hops. But you don't always need a mesh if your services are simple or if they communicate synchronously over HTTP with good old-fashioned load balancing. Luna: So when should a team actually invest in an API gateway? Like, at what scale does it start paying off? Lucas: I'd say as soon as you have more than one service, or you're exposing APIs to third-party developers, the gateway starts to make sense. Even with a single service, the gateway gives you a central place to enforce security and rate limits without polluting your service code. But if you're just building a prototype, you might be fine with a simple reverse proxy or even just your framework's middleware. Luna: And there are also managed gateway services, right? Like from cloud providers. Lucas: Yes, there are. AWS has API Gateway, and there are other options like Kong, Apigee, and open source ones like Envoy. The managed ones can handle scaling and security updates for you, which is nice, but they also lock you into the provider and can get expensive at high volume. Luna: So for a small team, is it better to just use the managed service or build your own? Lucas: I think for most teams, using a managed service or an open source gateway is the right call. Building your own is a project in itself, and you'll probably get it wrong the first time. But even with a managed service, you still need to think about your gateway's configuration, especially the timeouts and the rate limits, because those are the things that will bite you in production. Luna: And we've definitely seen stories of outages caused by a gateway misconfiguration, right? Lucas: Absolutely. There have been incidents where a single gateway timeout caused a cascade of failures because clients retried at high volume, and the gateway eventually ran out of connections. So the gateway is not just a tool, it's a system that needs its own monitoring and failure scenarios. Luna: So if someone is thinking about adding a gateway to their stack, what's the first thing they should do? Lucas: I'd start by listing the non-negotiable cross-cutting concerns for your APIs. Authentication, rate limiting, request logging, maybe caching. Then ask yourself if each one is better handled by the gateway or by the service itself. And be honest about the complexity you're adding. A gateway is not a silver bullet, but when used well, it's a powerful ally. Luna: That's a good note to end on. Any final thoughts, Lucas? Lucas: Just that the gateway is like a good manager. It delegates, it protects, and it doesn't micromanage. But if you give it too much to do, it becomes the bottleneck. So design it with care, and keep an eye on it in production.