Latest / The API Podcast with Fexingo: REST, GraphQL, and Modern Web APIs / How API Backpressure Prevents System Collapses
Transcript
- Lucas: The most dangerous assumption in modern API design is that if a request arrives, it will eventually be processed. We built our entire digital economy on the idea of eventual consistency and infinite scalability, but we forgot to build the brakes. Luna: Right, because without brakes, you just accelerate until you hit a wall. Or in this case, until your database locks up and your customer support tickets explode. Lucas: Exactly. And the specific mechanism that’s missing in most high-traffic architectures is something called backpressure. It’s a term borrowed from electrical engineering and mechanical systems, where a component physically resists force to prevent damage. In software, it’s the ability of a consumer to tell the producer to slow down. Luna: But isn't that what load balancing and auto-scaling do? If the server gets too busy, just add more servers. Lucas: That’s the reactive approach, Luna. Scaling out is expensive and slow. Backpressure is proactive. It happens at the protocol level. Think about HTTP. When you make a standard REST call, you send a request and wait for a response. If the server is drowning, it either times out or throws a five-zero-three error. There is no middle ground where it says, I am still alive, but please reduce your volume by half. Luna: So we’re trading graceful degradation for a hard crash. That sounds inefficient. Lucas: It is catastrophic. Let’s look at a concrete example from last quarter. A major e-commerce platform was integrating with a new payment processor via a synchronous REST API. During a flash sale, traffic spiked four hundred percent. The payment processor didn’t reject the requests immediately; it accepted them and put them in a deep queue. But the e-commerce site kept sending more, assuming the queue would clear. Within twenty minutes, both systems were fully saturated, memory usage hit one hundred percent, and the entire checkout flow went dark for forty-five minutes. Luna: That’s a classic thundering herd problem. They kept knocking on the door while the person inside was already suffocating under the pile of packages. Lucas: Precisely. And the cost of that outage wasn’t just lost sales. It was the engineering hours spent debugging why the queues weren’t draining. The root cause wasn’t bad code; it was a lack of flow control. If that API had supported backpressure, the payment processor could have sent a signal saying, my buffer is at eighty percent, please throttle your input rate. The e-commerce site would have slowed its outbound requests, allowing the queue to drain naturally. Luna: But most REST APIs don’t have a built-in handshake for that. How do developers actually implement this? Lucas: They have to move away from pure synchronous REST for high-volume paths. Two protocols dominate here: gRPC and event-driven streams like Kafka. gRPC uses HTTP/2 multiplexing and has built-in flow control mechanisms. If the receiver’s window size drops, the sender automatically throttles. It’s invisible to the application logic but critical for stability. Luna: And Kafka? Isn’t that just a message broker? Doesn’t it just sit there and hold the data? Lucas: It holds the data, yes, but it also manages the consumer lag. In Kafka Streams, you can configure the consumer group to pause fetching new partitions if the processing latency exceeds a threshold. This is explicit backpressure. The broker stops pushing data to that consumer until it catches up. It turns a potential system collapse into a manageable slowdown. Luna: So the key difference is that synchronous APIs assume the network is reliable and the server is ready, while asynchronous systems acknowledge that the server might be busy and adapt accordingly. Lucas: That’s the shift. You stop treating the network as a pipe and start treating it as a channel with limited bandwidth. But there’s a catch. Implementing backpressure requires changing how you think about success. In REST, success is a two-hundred OK status. In a backpressure-aware system, success might mean the request was delayed by three seconds so it doesn’t get dropped later. Luna: Which makes monitoring much harder. How do you know if the delay is acceptable or if the system is failing? Lucas: You monitor the queue depth and the end to end latency distribution. Specifically, the p ninety-nine latency. If your average latency is low but your tail latency is spiking, you have a backpressure issue hiding in plain sight. You need observability that shows you the pressure points, not just the output. Luna: This feels like a lesson for startups scaling too fast. They optimize for speed, not resilience. Lucas: Always. And speaking of resilience, if today's tech conversation gave you something usable, the way these stay ad-free is listener support. Buy me a coffee dot com slash fexingo helps us keep digging into these structural details. Luna: Yeah, and honestly, the fact that you guys test these edge cases means I don’t have to learn them the hard way during a launch day panic. Lucas: We appreciate that. Now, let’s go deeper into the implementation. Because even with gRPC or Kafka, many teams still fail at the application layer. They write code that ignores the signals. Luna: Like ignoring the headers or the error codes? Lucas: Worse. They write code that assumes the request will always succeed. For example, in a microservices architecture, Service A calls Service B. If Service B returns a five-zero-three service unavailable, Service A often retries immediately. This is retry storms. It amplifies the load instead of relieving it. Effective backpressure requires circuit breakers and exponential backoff with jitter. Luna: So you’re not just slowing down the input; you’re also changing how the output is handled when things go wrong. Lucas: Exactly. You decouple the failure mode. Instead of failing loudly and immediately, you fail gracefully. Maybe you return cached data, maybe you return a partial response, maybe you degrade the user experience slightly but keep the system alive. The goal is survival, not perfection. Luna: I’ve seen companies try this with feature flags. Turning off non-essential parts of the app to save resources for the core transaction. Lucas: That’s a form of application-level backpressure. It’s manual, but it works. However, the ideal is automated. You want the system to detect the pressure and adjust its own behavior without human intervention. This is where sidecars and service meshes come in. Tools like Istio or Linkerd can enforce traffic policies at the infrastructure level, applying backpressure rules regardless of what the application code does. Luna: So the infrastructure becomes the safety net, catching mistakes in the application logic. Lucas: Yes. And this raises an interesting question about cost. Backpressure often means rejecting or delaying requests. From a business perspective, that looks like lost revenue. But compared to a total system outage, it’s a bargain. Luna: Unless your customers interpret the delays as unreliability. Trust is fragile. Lucas: True. So you have to communicate. If you’re throttling users, you should tell them. Not with a generic error page, but with a clear status indicator. Slow loading spinner, or a message saying we’re experiencing high demand. Transparency mitigates the frustration. Luna: It’s the difference between a broken machine and a busy shop. One implies failure, the other implies popularity. Lucas: Right. And popularity is a brand asset. Failure is a liability. So designing for backpressure isn’t just an engineering decision; it’s a product decision. You are designing the user experience for stress. Luna: That’s a reframing I hadn’t considered. Usually, UX designers hate talking about failure states. Lucas: They should. Because the failure state is where the real relationship with the user is tested. Can they recover? Do they trust you to fix it? Backpressure gives you the time to answer those questions. Luna: So, if we’re building a new API suite today, what’s the first step? Do we pick gRPC over REST? Lucas: Not necessarily. REST is fine for low-volume interactions. But for any internal service to service communication or high-throughput public endpoint, you need a protocol that supports flow control. If you stick with REST, you must implement custom headers for congestion control, which is messy. Better to choose the right tool for the job. Luna: Or use an API gateway to enforce the limits before they hit your services. Lucas: Yes. Gateways are great for coarse-grained backpressure. They can limit the total number of connections per IP address. But they can’t see into the payload or understand the context of the request. That’s why the application layer still needs to be aware. Luna: So it’s a layered defense. Gateway at the edge, protocol in the middle, application logic at the core. Lucas: Exactly. Each layer adds resilience. And if one layer fails, the next one picks up the slack. That’s how you build systems that don’t just survive, but thrive under pressure. Luna: I guess the ultimate goal isn’t to avoid pressure, but to learn how to carry it. Lucas: Now that is a thought worth carrying forward. How else might we design systems that absorb shock rather than shatter under it?