Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / How Cloud Invoice Audits Reveal Hidden AI Costs
Transcript
- Lucas: So, Luna, I was digging through a mid-sized SaaS company's cloud invoice last week — not naming names — and I found something that made me stop. Their AI inference charges — just the inference charges — came to twenty-three percent of their entire monthly cloud bill. Luna: Twenty-three percent just for running models? That seems like a lot. Were they expecting that? Lucas: Not at all. They knew AI was growing, but the line items were spread across so many service categories that nobody could see the real total until I pulled everything together and added it up. And this is becoming a pattern. The era of the cloud bill you can actually understand is over, and AI is the reason. Luna: Right, because traditional cloud costs — compute, storage, networking — they follow pretty predictable formulas. But AI billing is a whole different animal. Lucas: Exactly. And it's not just the big three — AWS, Azure, GCP. Every vendor that offers AI services is layering on new pricing dimensions. Token-based pricing for model calls, GPU instance hours, data processing per request, model endpoint uptime. Some of these line items didn't exist two years ago. Luna: So how is a typical FinOps person supposed to keep up? I mean, most teams are still struggling with the basics like tagging and reserved instances. Lucas: That's the thing — the old playbooks don't translate. With traditional cloud, you could forecast pretty well because usage patterns were steady. But AI workloads are spiky and unpredictable. A single model rollout can double your spend overnight, and the invoice doesn't always tell you why. Luna: And that's if you can even find the AI charges. I've seen invoices where AI inference is mixed in with storage egress or database connections. It's almost like the vendors are making it intentionally opaque. Lucas: I don't think it's malicious as much as it is just how fast the services have been built. But the effect is the same — teams can't get a clear picture, and that's dangerous. Because when you can't see the cost, you can't control it. Luna: So what do you do? How do you actually start to unwind this? Because I imagine most teams don't have the luxury of a dedicated AI cost analyst. Lucas: Well, the good news is that the cloud providers themselves are starting to add ai specific cost tools. AWS has something that breaks down spend by model and even by endpoint. Azure has similar capabilities in their cost management portal. But the data is only as good as your tagging strategy, and that's where most teams fall down. Luna: Tagging. Of course. It's always tagging. Lucas: It's the unsexy work that makes everything else possible. If you're not tagging AI workloads from day one — which model, which version, which team, which use case — you're basically flying blind. And retroactively tagging is a nightmare. Luna: I've heard of teams that have to do post-hoc analysis on their bills just to figure out which department should be charged for what. That's not a recipe for harmony between engineering and finance. Lucas: Exactly. And that friction is real. Engineering says, 'We need this model to ship a feature,' and finance says, 'Why is our bill up fifty percent?' Nobody's wrong, but everybody's frustrated. Luna: And if these conversations are useful for what you're building or running, you might have noticed we don't do ads on these shows. That's a deliberate choice. If you want to support that choice, the link is buy me a coffee dot com slash fexingo. Lucas: Yeah, it's a small way to keep this independent and ad-free. Appreciate anyone who does. But back to the AI bill problem — there's another layer that makes this harder, and it's about how the vendors price things. Luna: What do you mean? Are they changing the pricing models themselves? Lucas: So, for a while, the big three were competing on lowering inference prices — making it cheaper to run models. But now that the initial price war is cooling off, they're getting more creative with how they charge. You're seeing things like per-token pricing that varies by model complexity, or charges for context caching, or even fees for the infrastructure that keeps your model endpoint warm. Luna: Context caching — that's when the model remembers previous interactions, right? So you're paying for that memory. Lucas: Right. And it's genuinely useful — it makes responses faster and cheaper for the user. But it also means your bill can fluctuate based on how many conversations are being cached at any given moment. And that's not something you can predict with a spreadsheet. Luna: So what's the actual solution? Do you just have to accept that AI costs are inherently messy and build in a lot of buffer? Lucas: I think there are two things that help. First, adopt a FinOps framework specifically for AI. That means setting up chargeback mechanisms, creating cost centers for each AI use case, and reviewing the AI line items every single week, not just at month-end. Luna: Weekly reviews. That's a big commitment, but I guess with AI costs moving this fast, it makes sense. Lucas: The second thing is to negotiate with your cloud provider. Most teams don't realize that these AI services are often negotiable, especially if you have steady-state inferencing. Some vendors will give you custom pricing if you commit to a certain volume or if you agree to run your workloads in specific regions. Luna: Wait, negotiate with AWS or Azure? I always thought their pricing was take it or leave it. Lucas: That's the myth. The list prices are stickers. Enterprise teams — especially ones with significant spend — have leverage. And with AI spend growing as fast as it is, the vendors are actually pretty eager to sign you into a custom deal. Luna: So the advice is: tag your stuff, meet weekly, and start the negotiation. Anything else? Lucas: One more thing. Look at the financial statement disclosures from the big public companies that are heavy AI users. Some of them, like the big tech names, are already showing how AI spend is hitting their margins. That's a signal that this isn't just a mid-market problem — it's affecting the giants. Luna: So there's a real incentive for vendors to keep the costs under control, or at least make them more understandable, because their customers are screaming. Lucas: Exactly. And as a listener, you can get ahead of this curve. The companies that build the FinOps muscle for AI now are the ones that will be able to scale without blowing up their P&L. Luna: It's like the early days of cloud adoption all over again, just with a new set of cost dimensions. Lucas: Right. First it was storage and compute. Then it was data transfer. Now it's tokens and GPU hours. The lesson is the same: you've got to understand what you're paying for, and you've got to have the right tools and processes in place to manage it. Luna: And it sounds like the tools are still catching up. So there's an opportunity for teams that are willing to be early adopters of the new cost-management features. Lucas: Absolutely. I'd say the teams that treat AI cost management as a competitive advantage, rather than an afterthought, are going to be the ones that win. Because AI isn't going anywhere, and the bills are only going to get bigger. Luna: So the takeaway for our listeners: start auditing your AI costs now, even if it's just a manual review of the invoice. Because the sooner you get visibility, the better. Lucas: And maybe start that conversation with your cloud account team before they come to you about renewals. You might be surprised what you can negotiate. Luna: Great advice. Thanks for breaking it down. Lucas: Anytime. And if you've got your own story about an AI invoice that made you do a double take, I'd love to hear it.