Latest / The API Podcast with Fexingo: REST, GraphQL, and Modern Web APIs / How API Latency Shapes Customer Experience
Transcript
- Lucas: Think about the last time you clicked 'buy' on an online store and felt that tiny, almost imperceptible hesitation before the confirmation page loaded. That pause is where millions of dollars vanish, and it's not because of bad product or high prices. It's because of latency. Today we are looking at how sub-second delays in API responses directly kill conversion rates, using a specific case study from a major retailer who lost one percent of sales for every hundred milliseconds of delay. Luna: That sounds extreme until you remember how many micro-interactions happen between click and confirmation now. Is this about server speed or something deeper in the code? Lucas: It's both, but mostly it's about architecture choices. Let me take a moment here. If today's conversation gave you something usable for your team's roadmap, the way these stay ad-free is listener support — buy me a coffee dot com slash fexingo. Luna: Appreciate that, Lucas. It helps keep the signal to noise ratio high. So back to that retailer, did they just upgrade their hardware? Lucas: No, they actually didn't change their underlying infrastructure at all. They changed how they composed their API calls. For years, the frontend had been waiting for three separate backend services to respond sequentially: inventory check, pricing engine, and shipping calculator. The total time was adding up to nearly four hundred milliseconds. Luna: So they switched to parallel processing? That feels like a standard move for senior engineers. Lucas: Exactly, but the twist is in the data structure. Instead of just making three calls at once, they introduced a lightweight composition layer right at the edge. This layer aggregated the responses into a single JSON object before sending it to the client. It reduced the round trips from three to one, cutting the perceived latency by sixty percent. Luna: I see. But doesn't aggregating data increase the payload size? Wouldn't that slow down the initial download? Lucas: A valid concern. In fact, the payload grew by roughly twenty percent, but because the network overhead of establishing connections is so much higher than transmitting a few extra kilobytes of JSON, the net effect was still a massive win. The browser didn't have to wait for multiple TCP handshakes. It just got one big package of truth. Luna: That makes sense. It's trading bandwidth for time. Are there any downsides to this approach that teams often overlook? Lucas: The biggest risk is coupling. When you bundle inventory, price, and shipping together, you're creating a tight dependency. If the shipping service goes down, the whole checkout flow breaks, even if the user only wanted to check the price. You lose granularity in failure modes. Luna: Right. So you gain speed but lose resilience. How do successful companies balance that trade-off? Lucas: They use circuit breakers and fallback strategies. In our case study, the composition layer was smart enough to return cached shipping estimates if the live service timed out. It wasn't perfect data, but it kept the page loadable. The alternative was a spinning wheel of death, which converts to zero. Luna: Cached data for shipping? That feels risky for customer trust if the estimate is wrong later. Lucas: It is risky, but it's managed risk. The key is transparency. The UI clearly labeled the shipping cost as 'estimated' when using cached data. Most users will accept a slight uncertainty in exchange for instant feedback. The psychology of immediacy outweighs the precision of accuracy in most consumer contexts. Luna: So it's about managing expectations, not just engineering speed. Does this apply to B2B software too, or is it mostly consumer-facing? Lucas: Absolutely applies to B2B. Look at Salesforce or HubSpot. Their dashboards are essentially complex API compositions. When a sales rep loads an account view, they want to see contact history, open deals, and recent activity simultaneously. If those APIs talk sequentially, the dashboard takes forever to render, and the rep stops using the tool. Luna: I can imagine. Friction in enterprise tools leads to shadow IT pretty quickly. People go back to Excel spreadsheets. Lucas: Precisely. And Excel doesn't scale. So the pressure to optimize API latency isn't just about UX polish; it's about retaining adoption. If your internal tool is slower than the manual process it replaced, you've failed the value proposition test. Luna: That's a sharp way to put it. Value proposition over vanity metrics. What about mobile apps? Isn't network variability worse there? Lucas: Much worse. On cellular networks, latency jitter is significant. A sequential request might fail entirely if the connection drops during the second hop. Composition helps here too, but it also requires aggressive caching on the device side. The app needs to serve the user from local storage while refreshing in the background. Luna: So offline-first becomes a requirement for fast experiences. Is that hard to implement? Lucas: It's complex, but frameworks like React Query or Apollo Client make it manageable. The lesson is that latency optimization isn't a one-time fix. It's a continuous practice of balancing consistency, availability, and partition tolerance in ways that prioritize the user's immediate need for information. Luna: Balancing the CAP theorem for human patience. I like that framing. Do you think smaller startups should worry about this early on? Lucas: Yes, because technical debt compounds faster than revenue growth does. If you build a monolithic backend that serializes requests, fixing it later requires rewriting the core logic. It's cheaper to design for parallelism from day one, even if you don't have the traffic to prove it yet. Luna: Cheaper upfront than painful later. That's the golden rule of engineering economics. Any final thoughts on how listeners can audit their own APIs for latency leaks? Lucas: Start by mapping your critical paths. Identify the three most important screens in your application and measure the time from user action to visual update. If it's over two hundred milliseconds, investigate the API chain. You'll likely find a sequential call hiding in plain sight. Luna: Practical advice. Measure first, then compose. Thanks for breaking this down, Lucas. Lucas: Happy to. It's a small change with outsized impact. Next week, we might dig into how GraphQL handles some of these composition challenges differently. Luna: Looking forward to that comparison. Until then, keep those requests parallel. Lucas: Will do. See you next time.