Latest / MBA Training Data / Cutting cloud data costs without breaking analytics
Transcript
- 0:00Welcome to the Leaders Insights podcast.
- 0:08Today's episode, cutting cloud data costs without breaking analytics.
- 0:12Enterprises are torching money on cloud data bills right now.
- 0:15Give me a number.
- 0:16Fine.
- 0:17I've seen a midsize retailer spending $2.4 million a year on their cloud data warehouse,
- 0:23and roughly 40% of that was pure waste.
- 0:26Not, we could trim it, waste.
- 0:2940%?
- 0:30That sounds like the kind of figure a consultant invents to justify their invoice.
- 0:34It does, and I'd be suspicious too.
- 0:37But the waste is boring and mechanical, which is why it's believable.
- 0:41Nobody's making dramatic mistakes.
- 0:43They're just leaving the lights on in every room of a very expensive house.
- 0:47Start me at the beginning.
- 0:49What am I actually paying for in a cloud data bill?
- 0:51Two things, mostly.
- 0:53Storage, die, keeping the data sitting there, grub, and compute.
- 0:58The machines that actually run your queries and crunch numbers.
- 1:02Storage is cheap.
- 1:03Compute is where they get you.
- 1:04Why compute?
- 1:05Because modern platforms, Snowflake, Databricks, Google's BigQuery, bill you for compute by
- 1:12the second the engine is running.
- 1:14Every dashboard refresh.
- 1:16Every analyst's clumsy query.
- 1:18Every scheduled job at 3am that nobody remembers setting up.
- 1:22It all runs the meter.
- 1:23So where's the 40% hiding?
- 1:26Great place, idle warehouses.
- 1:28A compute cluster, the cluster being the pool of machines doing the work, left running
- 1:33when no one's querying.
- 1:34It's like paying a taxi to idle outside your office all night in case you fancy a ride.
- 1:40The fix is auto suspend, turning it off after a few idle minutes.
- 1:44Costs nothing.
- 1:45That alone was 300 grand for the retailer.
- 1:48That's almost insultingly simple.
- 1:51Why hadn't they done it?
- 1:52Because someone years ago turned it off so dashboards would load instantly.
- 1:56And then that person left.
- 1:58And fear did the rest.
- 2:00Most cloud waste is archaeology, not stupidity.
- 2:03Alright, idle machines are the easy win.
- 2:06What's next?
- 2:07Oversized warehouses.
- 2:09Teams pick an extra-large cluster because bigger feels safer.
- 2:12Then run queries that a small would handle.
- 2:15On Snowflake, each size up roughly doubles the cost per second.
- 2:19So people are paying quadruple for a job that finishes two seconds faster.
- 2:23How do I even know I'm oversized?
- 2:26Every platform gives you query history, a log of what ran, how long, on what size.
- 2:32Sort by cost, look at the top 20 queries.
- 2:35That's your afternoon.
- 2:3690% of your spend usually hides in maybe 5% of queries.
- 2:41Give me the next lever.
- 2:42Storing everything at full readiness forever.
- 2:45Data you queried once in 2023, sitting in premium hot storage.
- 2:50Move cold data, stuff nobody touches, to cheaper archive tiers.
- 2:54It's the difference between a walk-in fridge and a box in the garage.
- 2:59You don't need the ice cream in both.
- 3:01This is where I get nervous.
- 3:02Cheap storage sounds like the report breaks when the CFO wants it.
- 3:06That's the right instinct.
- 3:08And it's why sequence matters.
- 3:09You archive data with no query hits in, say, 180 days.
- 3:14The platform tells you exactly what that is.
- 3:17You're not guessing.
- 3:18If someone needs it, it's slower to pull, not gone.
- 3:21Fine.
- 3:22But at some point, aggressive cost-cutting does break analytics.
- 3:25Where's that line?
- 3:27The line is when you start throttling the queries people actually use to make decisions.
- 3:32Cutting a warehouse.
- 3:33The finance team runs every morning to save $40 a day.
- 3:37That's where you've lost the plot.
- 3:39The rule?
- 3:40Cut idle and oversized ruthlessly.
- 3:42Such live analytical workloads never without asking the humans who run them.
- 3:47What about the vendors?
- 3:48They're happy to sell me a cost optimization tool for this.
- 3:51They are.
- 3:53Companies like Kibo or Blusky will automate warehouse sizing.
- 3:56And Snowflake's own numbers say customers can trim 20 to 30 percent, which, being their
- 4:01pitch, I'd have in my head.
- 4:04Honestly, a spreadsheet and a disciplined engineer get you most of the way for free
- 4:09first.
- 4:10So if a CDO does one thing tomorrow morning, what is it?
- 4:13Pull your query history, sort by cost, and stare at the top 20 queries.
- 4:18You'll find one scheduled job nobody owns burning real money.
- 4:22Kill it before lunch.
- 4:23That's your first $1,000.
- 4:25And it took an hour.
- 4:26That's a wrap.
- 4:27Fresh CDO briefings drop daily at MBA-training.com.