Latest / The Cloud Business Podcast with Fexingo: AWS, Azure, GCP, and Enterprise Infrastructure / Cloud Contracts Now Charge for Terraform State File Locking
Transcript
- Lucas: Luna, you ever run a terraform apply that took fifteen minutes because it was spinning up a whole new environment? Luna: More times than I'd like to admit. You sit there watching the progress bar, hoping nothing fails at minute fourteen. Lucas: Right. Well, those fifteen minutes are about to get more expensive. Cloud providers have started charging for the time your Terraform state file is locked during an apply operation. Luna: Wait — they're metering the lock duration? That's a new one. Lucas: It is. And it's rolling out across AWS, Azure, and GCP. The standard Terraform backend uses a locking mechanism — on AWS it's DynamoDB, on Azure it's Blob Storage leases, on GCP it's Cloud Storage object locks — to prevent two engineers from running apply at the same time and corrupting state. Luna: So they're charging for the time the lock is held. How much are we talking? Lucas: AWS is charging $0.00002 per second of lock time per state file. Azure is a bit different — they charge per lease acquisition, about $0.001 per acquire. GCP charges per minute of lock held at $0.0001 per minute. Seems tiny, but it adds up fast. Luna: Let's run the numbers. Say you have fifty state files, each locked for ten minutes per deployment, and you deploy ten times a day. Lucas: On AWS: fifty files times ten minutes is five hundred minutes of lock time, times ten deployments is five thousand minutes, which is three hundred thousand seconds. At twenty cents per ten thousand seconds, that's six dollars per day. Not huge. Luna: But scale that to a large enterprise with two hundred engineers, hundreds of state files, and deployments around the clock. Suddenly you're looking at thousands a month. Lucas: Exactly. And there's a real-world case I looked at: a mid-size e-commerce company I'll call ShopGrid. They have about eighty state files, each locked an average of fifteen minutes per apply, with about forty applies per day across their CI/CD pipeline. On AWS, their monthly lock fee came to roughly two hundred and thirty dollars. Luna: Two hundred thirty bucks just for locks. That's a line item nobody budgeted for. Lucas: And it's not just the money — it's the behavior change. Teams are now incentivized to minimize lock duration. That means faster applies, smaller state files, and perhaps splitting monolithic state into smaller pieces. Luna: But doesn't splitting state into separate files increase complexity? You end up with cross-state references and dependency management. Lucas: It does. But the cloud providers would argue that the lock fee reflects actual resource consumption — they're maintaining a distributed lock table, handling concurrency, and ensuring consistency. If you hold the lock, you're consuming that infrastructure. Luna: I get that argument, but it feels like death by a thousand cuts. First it was data egress, then API calls, now lock time. What's next, charging for the CPU cycles used by the Terraform binary itself? Lucas: Don't give them ideas. But seriously, the providers frame it as 'aligning cost with usage.' And to be fair, locking does use resources. DynamoDB's lock table has read and write capacity, and if you hold a lock for an hour, you're blocking other operations. Luna: But the alternative is no locking, which would lead to state corruption and even worse costs from recovery. Lucas: Precisely. So it's a necessary evil. The question is whether the pricing is fair. AWS's rate comes out to about seventy-two cents per hour of lock time per state file. If you have a single file locked for an hour during a production rollout, that's less than a dollar. Hard to complain about. Luna: Except when you aggregate across hundreds of files, it becomes a real line item. What are teams doing to cope? Lucas: Some are adopting Terraform workspaces to reduce the number of state files. Others are optimizing their deployments to be faster — using parallelism, avoiding unnecessary resources, and refactoring modules to reduce apply time. Luna: And I've heard of teams moving to a 'partial state' approach where they only lock the specific resource being modified, not the entire file. Is that possible? Lucas: Partially. There's a feature called 'state locking at resource level' that some third-party backends offer, but native Terraform doesn't support it yet. HashiCorp has discussed it, but no timeline. Luna: So for now, it's about discipline: smaller applies, shorter locks, maybe even scheduling deployments during off-peak hours when fewer people are applying. Lucas: Actually, that's a smart take — if you can shift your CI/CD pipeline to run applies in a batch window, you reduce the concurrency and thus the total lock time across your team. Luna: Speaking of smart takes, if today's episode helped you think about your own cloud costs in a new way, you know what keeps these conversations ad-free and independent. It's listener support at buy me a coffee dot com slash fexingo. Lucas: Yeah, we don't run ads, so every little bit helps us keep digging into these contract details that actually affect your bottom line. Luna: Alright, back to locks. So the providers are clearly watching how teams use Terraform and adjusting their pricing accordingly. Lucas: They are. And this is part of a broader trend of 'granular metering' in cloud contracts. Every API call, every byte stored, every second of compute is now tracked and billed. Luna: But is there a risk that these granular fees discourage teams from using infrastructure as code altogether? Lucas: I think the opposite is happening — it's encouraging teams to be more thoughtful about how they use IaC. The days of 'just run terraform apply and walk away' are over. You need to script it, time it, and optimize it. Luna: So you see this as a net positive for engineering culture? Lucas: In a weird way, yes. It forces you to understand the cost of every operation. That's good for the business. But I worry about smaller teams who don't have the bandwidth to fine-tune their Terraform code. They'll just see the bill go up without knowing why. Luna: So the onus is on the cloud providers to be transparent about these charges. Are they? Lucas: AWS added a line item in the billing console for 'DynamoDB Lock Duration' — but it's not obvious. You have to dig into the cost and usage report. Azure bundles it into blob storage transactions. GCP shows it under 'Cloud Storage Lock Minutes.' It's not exactly front and center. Luna: Classic. So a finance team reviewing the cloud bill might see a ten percent increase and not connect it to Terraform locks. Lucas: That's where the engineering team needs to step in and educate. Or use third-party cost monitoring tools that tag these charges. Luna: Have you seen any tooling that specifically tracks lock costs? Lucas: CloudHealth and Vantage have started adding tags for lock-related resources. And Infracost is working on a feature that estimates lock costs during planning, so you see the projected fee before you apply. Luna: That would be huge — imagine getting a warning: 'This apply will cost $0.42 in lock fees.' Lucas: Exactly. It changes behavior. If you know a fifteen-minute apply costs two cents, you might not care. But if it's two dollars, you'll think twice. Luna: Let's circle back to the technical side. How does the lock mechanism actually work in Terraform? Lucas: When you run terraform apply, the backend — say S3 with DynamoDB for AWS — first acquires a lock by writing a record to DynamoDB with a time to live. If another process tries to acquire the same lock, it gets an error. The lock is released when the apply completes or fails. Luna: And the cloud provider charges for that time based on the lock's duration. But if the apply fails, you might have a stale lock? How does that affect billing? Lucas: Good question. Terraform has a force-unlock command, but if you don't use it, the lock can expire based on the TTL. During that time, you're still being charged. So a failed apply can lead to a lock that persists for minutes, accruing cost. Luna: That's a hidden cost of failure. Another reason to build robust error handling into your CI/CD. Lucas: Absolutely. And we're seeing teams implement lock monitors that alert if a lock is held longer than expected. Luna: So what's your prediction — will other providers match these fees, or will they differentiate? Lucas: I think they'll all converge. It's too easy a revenue stream to ignore. But the differentiation will come in how they implement it — maybe a free tier of a certain number of lock seconds per month, or bundling it with other IaC services. Luna: Interesting. So for now, the takeaway for listeners is: check your cloud bill for lock-related charges, and start optimizing your Terraform code for speed. Lucas: And consider using workspaces or partial state to reduce the number of files. Every second counts. Luna: Thanks, Lucas. Good episode. Lucas: Thanks, Luna. See you next time.