Serving LLMs in Production: Performance, Cost & Scale // CAST AI Roundtable

Agentic Conversations (formally mlops.community) · Feb 19, 2026 · 1h 5m · transcript

Roundtable CAST AI episode: Serving LLMs in Production: Performance, Cost & Scale.

Join the Community:

https://go.mlops.community/YTJoinIn

Get the newsletter: https://go.mlops.community/YTNewsletter

MLOps GPU Guide:

https://go.mlops.community/gpuguide

// Abstract

Experimenting with LLMs is easy. Running them reliably and cost-effectively in production is where things break.

Most AI teams never make it past demos and proofs of concept. A smaller group is pushing real workloads to production—and running into very real challenges around infrastructure efficiency, runaway cloud costs, and…