Latest / The API Podcast with Fexingo: REST, GraphQL, and Modern Web APIs / How API Backpressure Prevents System Collapses
Transcript
- Lucas: We talk a lot about how to make APIs faster, but we rarely discuss what happens when they are too fast. Specifically, I want to look at backpressure. It is that specific signal sent from a slow consumer to a fast producer, telling it to slow down before the whole stack collapses. Luna: Right. Most teams just throw hardware at the problem until the bill gets scary. But hardware has limits. When you hit those limits without a signaling mechanism, you get cascading failures instead of graceful degradation. Lucas: Exactly. And the distinction matters because rate limiting is usually enforced at the edge by a gateway. It blocks requests before they enter the system. Backpressure happens inside the system, between services talking to each other. If Service A calls Service B, and Service B is drowning in work, it needs to tell Service A to wait. Luna: I saw this happen last quarter with a logistics client. Their inventory service was getting hammered by a flash sale, but their order processing layer didn't have a backpressure handshake, so everything just queued up until memory ran out. Lucas: That queue overflow is the classic symptom. In traditional REST APIs, there is no native standard for backpressure. You usually see workarounds like polling, where the client keeps asking, 'Are you ready yet?' That wastes bandwidth and CPU cycles on both ends. Luna: Polling feels so primitive now. Isn't gRPC better suited for this kind of internal communication? Lucas: gRPC is built on HTTP/2, which has built-in flow control at the stream level. So yes, technically, it handles backpressure better than raw HTTP/1.1. But most public-facing APIs still use REST over HTTP/1.1 or even HTTP/2 without full stream multiplexing awareness. They need a semantic way to say, 'I am busy.' Luna: So how do they communicate that state without breaking the contract? Lucas: The most common pattern I see is using retry-after headers combined with specific status codes. Some teams use a custom header like X-RateLimit-Remaining set to zero, but that is really just rate limiting masquerading as backpressure. True backpressure requires the client to actively pause its own request generation. Luna: Pause its own generation. That sounds like a client-side responsibility shift. Is that fair to ask developers to implement? Lucas: It is necessary if you want to save your backend. Think about GitHub. They implemented strict backpressure policies on their GraphQL API. If you query too deeply or too frequently, they don't just reject you immediately; they give you feedback on how much capacity you have left and encourage you to batch your requests. Luna: Batching reduces the number of round trips. That helps the network, but does it actually relieve the pressure on the database? Lucas: It does, because it allows the server to process data in larger, more efficient chunks rather than tiny individual transactions. The key is that the client respects the signal. If the client ignores the hint and keeps hammering, the server eventually drops connections. That is the fail-safe. Luna: Dropping connections is harsh. Clients hate that. It looks like downtime to the end user. Lucas: It is better than a five-minute timeout on every single request. A dropped connection can be retried instantly. A hung socket consumes resources until it times out. In high-traffic environments, that difference determines whether your system survives a spike or crashes completely. Luna: I remember reading about Netflix's Chaos Engineering approach. They intentionally inject latency to test exactly this behavior. Did they find that clients usually handled it well? Lucas: Most mature clients handle it well. Mobile apps, in particular, have sophisticated retry logic. But desktop web clients are often built with simple fetch calls that don't understand backpressure semantics. They just spin until the browser kills them. Luna: That seems like a gap in the developer experience. Why isn't this baked into the HTTP specification more heavily? Lucas: HTTP is a protocol for transferring data, not managing load. It assumes a reliable pipe. Backpressure is an operational concern, not a transport concern. That is why we see it emerging in application-layer standards rather than protocol changes. Luna: So we are relying on libraries and frameworks to solve a protocol limitation? Lucas: Yes. Libraries like Feign or Spring Cloud Gateway add these layers on top. But if you are building a microservice from scratch, you have to write the logic yourself. And that is where many small startups fail. They optimize for speed first and forget about congestion. Luna: Optimizing for speed first is the default mindset. Speed feels like progress. Congestion management feels like bureaucracy. Lucas: Bureaucracy that keeps the lights on. Let me share a quick note here. If today's deep dive into API resilience gave you something usable for your next sprint, a handful of listeners chip in monthly through buy me a coffee dot com slash fexingo, and that is literally what funds making this many of these episodes ad-free. Luna: Yeah, it is great to know that direct support keeps the engineering details accurate without needing corporate sponsors dictating the narrative. Lucas: Anyway, back to the mechanics. There is another pattern called circuit breaking. It is related but distinct. A circuit breaker stops requests entirely when a service is down. Backpressure tries to keep the requests flowing but at a reduced rate. Luna: So backpressure is the gentle nudge, and circuit breaking is the hard stop? Lucas: Precisely. You want backpressure first. You only trip the circuit breaker if backpressure fails and the service becomes unresponsive. Using both creates a layered defense strategy. Luna: Layered defense. I like that. It mirrors how we handle physical security or even financial risk. Redundancy saves you when one layer fails. Lucas: Exactly. And in the cloud, redundancy is expensive. Backpressure allows you to run leaner infrastructure because you are actively managing the load rather than just buying more servers to absorb waste. Luna: Cost efficiency through discipline. That is a compelling business case for engineers who usually just care about uptime. Lucas: It is. And as we move toward more distributed systems, that discipline becomes non-negotiable. You cannot scale horizontally if your components are choking each other. Luna: Do you think we will see a new standard emerge for this? Maybe a universal backpressure header? Lucas: Unlikely to be universal. Different systems have different constraints. A database has different bottlenecks than a cache or a compute node. A generic header would be too vague to be useful. Specificity wins here. Luna: Specificity wins. So the lesson is: build contracts that reflect your actual capacity limits, not just your ideal performance metrics. Lucas: Spot on. Design for the worst day, not the best day. That is the core of resilient architecture. Luna: It forces us to admit our systems are fragile. Which is uncomfortable, but honest. Lucas: Fragility is real. Acknowledging it is the first step to strength. Next time you design an endpoint, ask yourself: what happens when ten thousand people hit it at once? Luna: And then listen to what the system tells you it needs. Rather than forcing it to perform beyond its means. Lucas: Let the system speak. That is how you build software that lasts.