Latest / AI Fire Daily / #97 Robin: Google's $0.75 Coding Miracle? Why Gemini 3.8 Flash Just Triggered an API Price War

Transcript

#97 Robin: Google's $0.75 Coding Miracle? Why Gemini 3.8 Flash Just Triggered an API Price War · AI Fire Daily

• 0:00
  The AI race used to be a very simple contest.
• 0:03
  Everyone just wanted to build the absolute smartest
• 0:05
  model possible. You know, just build the biggest
• 0:07
  brain. Right. The only goal was raw power. Exactly.
• 0:10
  Yeah. But recently, that entire game has shifted.
• 0:14
  Beat. It is no longer just about raw intelligence.
• 0:18
  No, it's about building a model that is good
• 0:20
  enough and shockingly cheap. Google is Gemini
• 0:24
  3 .8 Flash. Just flip the board. Welcome to the
• 0:28
  Deep Dive. We are really glad you were here with
• 0:29
  us today. We've got a great one for you. We are
• 0:31
  looking at a comprehensive breakdown of Google's
• 0:34
  newest release. Down to 3 .8 flash. And we have
• 0:37
  a fascinating roadmap for you today. We are going
• 0:40
  to explore its weirdly spiky benchmark scores.
• 0:44
  Very spiky. Yeah. And we'll unpack the aggressive
• 0:47
  new pricing strategy that kind of changes everything.
• 0:50
  Plus, we are looking at a secret cybersecurity
• 0:51
  version most people can't even touch. I'm excited
• 0:54
  for that part. Me too. And finally, we will see
• 0:56
  what happened when someone asked it to code a
• 0:58
  Tesla landing page. and a playable Doom clone.
• 1:02
  Let's start with the raw numbers. We need to
• 1:04
  establish what Gemini 3 .8 Flash actually is
• 1:07
  mechanically. We have to understand its brain
• 1:09
  before we look at its price tag. That makes total
• 1:11
  sense, and looking at the data... This model's
• 1:15
  performance profile is incredibly uneven. It
• 1:18
  is definitely not a clean sweep across the board
• 1:21
  like we're used to seeing from major frontier
• 1:22
  models. On some tests, it looks like the absolute
• 1:26
  best AI available on the planet. But on others?
• 1:29
  On others, it falls all the way back into the
• 1:31
  middle of the pack. Let's look at where it actually
• 1:33
  succeeded first. It scored incredibly well on
• 1:37
  the DeepSWE version 1 .1 benchmark. Oh, this
• 1:40
  is arguably the most important result for software
• 1:43
  developers. It scored 73 .7 % here. Wow. Yeah,
• 1:47
  that ties it directly with Cloud Opus 5. And
• 1:50
  it actually beats GPT 5 .6 Sol, which scored
• 1:54
  72 .7%. Beat? Oh, I want to pause on that. DeepSWE
• 1:58
  measures performance on long -horizon tasks.
• 2:01
  Yes. Let's make sure we're on the same page there.
• 2:03
  Long -horizon tasks are tasks where AI reads
• 2:05
  code, plans steps, and works without breaking.
• 2:08
  That's a perfect definition. It tests software
• 2:10
  engineering work that requires real sustained
• 2:13
  focus. Right. It's not just a quick script. Exactly.
• 2:16
  We aren't talking about writing a quick Python
• 2:18
  script to sort an array. We're talking about
• 2:20
  giving the AI a sprawling, messy GitHub issue.
• 2:25
  Like asking it to fix a bug deep in a legacy
• 2:27
  code base. Yeah, the model has to understand
• 2:30
  an entire repository of files. It has to plan
• 2:33
  a solution across many different steps. It has
• 2:35
  to write the code, run a test, read the error
• 2:38
  message, and fix its own mistakes. Deep SWE feels
• 2:41
  much closer to actual real world coding work.
• 2:44
  It isn't just a static multiple choice test.
• 2:47
  Two sec silence? Yeah. But then we look deeper
• 2:49
  and we see these weird extremes in the data.
• 2:52
  The spikes. Right. It ranks number one on the
• 2:54
  Harvey Legal Agent benchmark. It scored 61 .4
• 2:58
  % there. It also took the number one spot on
• 3:01
  Humanity's last exam. Which is wide. Scoring
• 3:03
  55 .9. Right, and those are incredibly difficult
• 3:07
  specialized tests. Humanity's last exam involves
• 3:10
  questions so niche that regular subject matter
• 3:12
  experts often failed them. Winning those benchmarks
• 3:15
  is a massive technical achievement. But then
• 3:17
  we look at system control, it completely bombs
• 3:19
  Terminal Bench 4 .0. Yeah, it really does. It
• 3:21
  only scored 19 .1%. For context, Claude Opus
• 3:25
  5 scored 51 % on that exact same test. It's a
• 3:29
  huge gap. I find that baffling. How does it fail
• 3:31
  so badly? Well, it comes down to the mechanics
• 3:33
  of interacting with an environment. Terminal
• 3:36
  Bench 4 .0 gives the AI a Linux terminal. The
• 3:40
  model has to navigate folders, install packages,
• 3:43
  and configure servers. So it's about sequential
• 3:45
  state tracking. Exactly. If you delete the wrong
• 3:48
  directory on step 2, step 15 is impossible. The
• 3:51
  AI needs a persistent spatial memory of the operating
• 3:54
  system. And it just loses track. Yeah. For whatever
• 3:57
  reason, the architecture of Gemini 3 .8 Flash
• 4:00
  struggles to maintain that specific kind of state
• 4:02
  memory. It also trails significantly on standard
• 4:05
  knowledge work. Took a big hit on GDPVal. It
• 4:07
  scored 1545 there. Which is pretty low. Yeah.
• 4:10
  Opus 5 hit 1824 and Sol hit 1710. GDPVAL measures
• 4:14
  things like complex PDF extraction and data analysis.
• 4:18
  It tests presentation creation. It's kind of
• 4:20
  the daily grunt work a business analyst needs.
• 4:22
  It's like dealing with a highly specialized savant.
• 4:24
  That's a good way to put it. It reminds me of
• 4:26
  a master chef. You know, this chef can perfectly
• 4:29
  execute a complex Michelin star souffle. That's
• 4:33
  the deep SWE score. Right. But somehow, this
• 4:36
  same chef burns a simple piece of toast. That's
• 4:40
  the GDPVAL score. That's a perfect analogy. The
• 4:44
  underlying architecture is likely heavily weighted
• 4:46
  toward code and logic data, but severely pruned
• 4:50
  in areas like document formatting to save compute
• 4:52
  costs. Does cheap token pricing hide the fact
• 4:56
  that it might waste tokens failing at hard tasks?
• 4:59
  That is the smartest metric to watch. Cheap pricing
• 5:02
  alone can definitely lie to you. A model might
• 5:05
  cost half as much, but it might burn twice as
• 5:07
  much compute trying to finish one single workflow.
• 5:10
  But the data shows Gemini 3 .8 Flash actually
• 5:13
  succeeds in its specialized domains without looping
• 5:16
  infinitely. So it lands in the sweet spot. Strong
• 5:19
  scores and low average task cost. You nailed
• 5:21
  it. That's why the developer community is suddenly
• 5:24
  paying such close attention to this release.
• 5:26
  So if it's burning toast on basic research tasks,
• 5:28
  why are developers obsessing over it? It comes
• 5:30
  down to one fundamental thing. The economics
• 5:33
  of failure. Yes. The pricing numbers are the
• 5:36
  real headline here. They're absolute game changers
• 5:39
  for anyone building AI software. Let's get into
• 5:41
  those numbers. Gemini 3 .8 Flash is charging
• 5:44
  $0 .75 per 1 million input tokens. Let's define
• 5:48
  that quickly for anyone new. Tokens are small
• 5:50
  chunks of data the AI reads. Perfect. Yes, $0
• 5:52
  .75 per million chunks of input data. And it
• 5:56
  costs $0 .75 per $1 million output tokens. Those
• 6:00
  are the words it generates back to you. Exactly.
• 6:02
  Let's compare that directly to the leading competition.
• 6:04
  Take GTC 5 .6 Terra. Terra costs $2 for input
• 6:09
  and $12 for output. So Gemini 3 .8 Flash is running
• 6:12
  at roughly 20 % to 30 % of the cost of its direct
• 6:14
  arrival. That's a staggering price drop. But
• 6:17
  let's dig into the why. Why does that specific
• 6:19
  price difference matter so much in practice?
• 6:22
  It matters massively for autonomous agents. Think
• 6:24
  about how a coding agent actually functions in
• 6:26
  the wild. It doesn't just write code once and
• 6:28
  magically succeed. Never. It has to iterate.
• 6:30
  A lot. It reads multiple files to understand
• 6:33
  the context. That takes thousands of input tokens.
• 6:36
  It writes a draft of the code. That's output
• 6:38
  tokens. Then it runs tests on that code. It inevitably
• 6:42
  fails. It has to read the new error log. It's
• 6:44
  calling the model again and again. Every single
• 6:47
  time, it sends the entire history of the conversation
• 6:50
  back to the server. And those loops are where
• 6:53
  the financial danger lies. It loops dozens of
• 6:56
  times. Yeah. Every single loop consumes thousands
• 7:00
  of tokens. Those costs compound incredibly fast
• 7:04
  during a long workflow. I have to admit something
• 7:06
  here. I still wrestle with budget anxiety myself
• 7:10
  when letting autonomous agents run. Oh, absolutely.
• 7:12
  You just watch the terminal spinning. It feels
• 7:15
  like watching a taxi meter ticking up in rush
• 7:18
  hour traffic. We have all felt that panic. Agent
• 7:21
  loops can drain a small development budget overnight
• 7:23
  if they get stuck in a failure loop. Which happens
• 7:25
  a lot. Right. That's why a model that's 80 %
• 7:29
  cheaper is so fundamentally disruptive. It changes
• 7:32
  what you can afford to experiment with. But there's
• 7:34
  a catch hidden in the fine print of this pricing.
• 7:36
  Is it there? Yeah. Unfortunately, there is. This
• 7:39
  is just introductory pricing. It actually expires
• 7:41
  at the end of the year. Wait. So this is basically
• 7:43
  a bait and switch. They get you hooked on cheap
• 7:44
  agent loops and then double the price in January.
• 7:47
  It definitely feels a bit opaque. The real listed
• 7:50
  price kicks in next year. It goes up to $1 .50
• 7:52
  for input and $7 .50 for output. The source material
• 7:56
  notes some real frustration from developers about
• 7:58
  how that was presented. It is a fair criticism.
• 8:01
  If the future price is already locked in, it
• 8:03
  should be the main number printed on the tin.
• 8:05
  Right. But honestly, we have to look at the math.
• 8:08
  Even at $1 .50 and $7 .50, it remains substantially
• 8:12
  cheaper than a comparable Terra class model.
• 8:15
  With pricing models shifting this fast, is loyalty
• 8:19
  to one AI provider dead. I think it has to be.
• 8:23
  These new models are getting way too specialized.
• 8:25
  One model crushes legal work. Another dominates
• 8:28
  browser control. Right. Another is best for long
• 8:30
  coding tasks. You just can't blindly default
• 8:33
  to the newest open AI or anthropic release anymore.
• 8:36
  Right. Companies must build internal tests using
• 8:39
  real tasks instead of blindly trusting benchmarks.
• 8:41
  Yeah, you have to run your real files through
• 8:43
  it. You have to test your real failure cases.
• 8:46
  That is the only way to know what actually works
• 8:48
  for your specific budget. Since Google is clearly
• 8:51
  prioritizing cost efficiency and deep specialization,
• 8:55
  their next move in the source material makes
• 8:57
  perfect sense. It really does. They quietly released
• 9:01
  a highly specialized enterprise -grade variant
• 9:03
  of this model. We need to talk about Gemini 3
• 9:06
  .8 Flash Cyber. This is the restricted version
• 9:09
  most people can't touch. Why do we need a separate
• 9:13
  model just for cybersecurity? It comes down to
• 9:16
  safety guardrails. Normal frontier models are
• 9:19
  heavily restricted from writing or analyzing
• 9:21
  malicious kid. Obviously. But security professionals
• 9:23
  actually need AI to find exploits so they can
• 9:26
  patch them. So it's built specifically for vulnerability
• 9:29
  discovery. Yeah. Think of it as the cyber focus
• 9:32
  version of 3 .8 Flash. But the safety guardrails
• 9:35
  have been explicitly adjusted for security professionals.
• 9:38
  But I can't just sign up and use it on a weekend
• 9:40
  project. No. It's heavily restricted. It is only
• 9:42
  available to what Google officially calls trusted
• 9:45
  defenders. You have to access it through a specific
• 9:48
  initiative called Google's Fair Wind program.
• 9:51
  You can apply, but the vetting process is strict.
• 9:55
  The early performance numbers from the source
• 9:56
  are fascinating, though. They really are. On
• 9:59
  the standard Cyber Gem Pest, it scored 86 .2
• 10:02
  percent. How does that compare to the heavyweights
• 10:04
  in the industry? Well, it beat GPT 5 .5 Cyber,
• 10:08
  which scored 85 .6 percent. And remember, that
• 10:11
  GPT model was purpose -built for cyber work.
• 10:14
  It also beat GPT 5 .6 Sol, which scored 83%.
• 10:19
  That is a massive deal for enterprise security
• 10:21
  teams. It's huge. Security engineers spend hundreds
• 10:24
  of hours reviewing massive complex code bases
• 10:27
  looking for risky patterns. Having a cheap specialized
• 10:30
  model do the first pass is a game changer. Definitely.
• 10:32
  But there is another data point in the research
• 10:34
  that's even more interesting. The internal benchmark
• 10:37
  across 20 programming languages. Yes. unpack
• 10:40
  why that matters. The standard SummerGem test
• 10:42
  only focuses on C and C++N, but real enterprise
• 10:46
  security environments are a mess. Exactly. The
• 10:51
  landscape is deeply fragmented. The AI needs
• 10:54
  to be able to trace a vulnerability as it moves
• 10:58
  across all of those different languages. Like
• 11:00
  they have a 15 -year -old Java backend, a Python
• 11:03
  data pipeline, and a modern React frontend. Spot
• 11:06
  on. And Google tested the cyber model across
• 11:09
  20 different languages. How did it do? The internal
• 11:11
  results showed a massive jump in capability over
• 11:14
  their older Gemini 3 .7 and 3 .5 flash models.
• 11:18
  It is a huge leap forward in multi -language
• 11:21
  vulnerability detection. Whoa! Imagine scaling
• 11:26
  that. Yeah. Imagine being able to review a billion
• 11:29
  lines of multi -language enterprise code instantly.
• 11:31
  It changes the entire posture of network defense.
• 11:34
  You go from reactive patching to proactive systemic
• 11:37
  auditing at an unimaginable scale. Given Google
• 11:40
  didn't test competitors on the 20 -language benchmark,
• 11:43
  how much weight should we give it? You always
• 11:45
  have to be careful with internal, unverified
• 11:47
  benchmarks. Since they didn't test competitor
• 11:49
  models on this specific test, we can't declare
• 11:52
  it the undisputed winner. Right. This is just
• 11:54
  their numbers. Yeah, it's a really promising
• 11:55
  data point, but it's not independent verification.
• 11:58
  We treat it as a highly promising direction,
• 12:00
  but definitely not final proof. Well said. But
• 12:03
  it absolutely proves Google is building highly
• 12:06
  specialized versions for serious enterprise use
• 12:09
  cases. We've talked a lot about stats. We've
• 12:11
  covered enterprise cyber defense. But let's move
• 12:14
  from the enterprise server room to an indie developer's
• 12:16
  laptop. This is my favorite part. What actually
• 12:19
  happens when a normal developer prompts this
• 12:21
  thing? to build something visual. This is honestly
• 12:25
  where the deep dive gets really fun. Benchmarks
• 12:27
  are useful for analysts, but hands -on demos
• 12:30
  reveal the model's actual personality. The source
• 12:32
  material runs through some very uneven hands
• 12:35
  -on testing results. Uneven is the perfect word.
• 12:38
  The clean pattern we see is this. Its interactive
• 12:41
  coding logic is absolutely fantastic, but its
• 12:44
  visual design polish is severely lacking. Let's
• 12:47
  walk through the actual tests. Test one involved
• 12:49
  generating 3D low -poly biomes. The prompt was
• 12:52
  simple. Create seven 3D low -poly biomes. It
• 12:56
  did a pretty good job with a basic beach scene
• 12:58
  and a farm scene. They looked clean and readable
• 13:01
  as code -generated graphics. But they were artifacts.
• 13:04
  Yeah, weird glitchy visual artifacts. For example,
• 13:07
  in an ice lake scene, the water was pushing out
• 13:10
  from the biome in a very strange, unnatural way.
• 13:14
  What does that happen? AI models often struggle
• 13:16
  with 3D spatial consistency. They understand
• 13:18
  the text concept of water and ice, but mapping
• 13:21
  that math perfectly in a three -dimensional coordinate
• 13:24
  space is difficult. So it landed around the GLM
• 13:28
  5 .3 level? Pretty much. It was below Fable 5
• 13:31
  .1 and definitely below GPT 5 .6 Sol. Right.
• 13:35
  It's good enough to prototype a basic scene,
• 13:37
  but you'd never trust it for final design polish
• 13:39
  in a real game. Test 2 was generating product
• 13:42
  landing pages. I was looking at the Tesla test,
• 13:45
  and the visuals were honestly jarring. Oh man,
• 13:47
  yeah. It looked like MS Paint. How is this considered
• 13:49
  a success? I know, the visual render of the car
• 13:52
  looked terrible. It didn't look like a premium
• 13:54
  electric vehicle at all. It looked like a quick
• 13:56
  bad sketch. Right, and yet. Yet the underlying
• 13:59
  configurator mechanics worked flawlessly. Unpack
• 14:02
  that. What do you mean by the mechanics? the
• 14:04
  DOM logic, the document object model. You could
• 14:07
  interact with the web page perfectly. You could
• 14:09
  click to change the car's trims. You could switch
• 14:12
  the paint colors. You could estimate gas savings
• 14:15
  using a built -in calculator. Wow. It even had
• 14:18
  a clean animation for the vision -only autopilot.
• 14:21
  The underlying logic was bulletproof. The rasterized
• 14:25
  visuals were just rough. I love that. It's like
• 14:28
  a brilliant backend engineer. Yeah. They build
• 14:30
  you a flawless high -speed database. But then
• 14:34
  they draw stick figures when asked to design
• 14:37
  the user interface. That is exactly what the
• 14:39
  model feels like. It understands the mechanics
• 14:41
  of the web code perfectly. It just doesn't have
• 14:43
  a trained eye for aesthetic. They tested other
• 14:45
  product pages too, right? Yeah. An Apple product
• 14:47
  page was generated, but it was way too simple.
• 14:50
  It lacked a strong hero image and had a weak
• 14:52
  checkout flow. A Galaxy Z Fold page had a really
• 14:56
  cool draggable fold feature. But again, the rendering
• 14:59
  of the phone itself just looked wrong. Let's
• 15:01
  look at test three. The PowerPoint deck. This
• 15:03
  was a presentation deck about data centers. Gemini
• 15:06
  3 .8 Flash actually nailed the brand colors and
• 15:09
  typography. The slide structure made total narrative
• 15:12
  sense. But no polish. Right. Again, it lacked
• 15:15
  that premium polish. If you needed a deck for
• 15:18
  a high stakes client meeting tomorrow, you would
• 15:21
  still lean toward a clod model to generate it.
• 15:24
  But test four was the absolute standout in the
• 15:27
  source material, the Mount Everest test. This
• 15:30
  was incredibly impressive from a technical standpoint.
• 15:32
  The prompt asked for a 3D topographic map of
• 15:35
  Mount Everest, and Gemini 3 .8 Flash built a
• 15:39
  fully interactive, draggable, zoomable 3D map.
• 15:42
  And it had incredibly complex math controls built
• 15:45
  into it. Yes. It included a cross -section cut
• 15:48
  angle so you could slice through the terrain.
• 15:50
  It had a crustal depth offset for studying geology.
• 15:53
  It had a solar azimuth control. That requires
• 15:56
  serious technical spatial reasoning. It's calculating
• 15:58
  the angle of the sun and how the light hits specific
• 16:01
  mountain ridges. Exactly. It isn't just drawing
• 16:03
  a pretty picture. It is generating the mathematical
• 16:05
  relationships of the topography. That's wild.
• 16:08
  It even included vertical exaggeration controls
• 16:10
  and a 2D projection view alongside the 3D map.
• 16:13
  And it accurately labeled waypoints like Camp
• 16:15
  2. For interactive technical visuals, it's genuinely
• 16:19
  powerful. Who actually uses that kind of output?
• 16:21
  Geologists, game designers, data scientists.
• 16:25
  Being able to generate that from a single text
• 16:27
  prompt replaces days of manual 3D modeling work.
• 16:31
  The final test was recreating Doom. From a single
• 16:34
  prompt. It created a playable Doom -style prototype
• 16:37
  in the browser. You could move around. You could
• 16:40
  find enemies. You could shoot them. The game
• 16:42
  loop worked. It's basic, but as a one -prompt
• 16:45
  prototype? It's phenomenal. If the visual polish
• 16:47
  is so rough, does that defeat the purpose of
• 16:50
  using it for front -end work? Not at all. The
• 16:53
  flawless interactive logic provides an incredible
• 16:56
  starting point for prototyping. The raw visual
• 16:58
  output is rough, but that underlying interactive
• 17:00
  logic is incredibly valuable. You're getting
• 17:02
  the engine of the car built for pennies. You
• 17:04
  just need to paint it yourself. Let's take a
• 17:06
  quick break here, sponsor. Let's calmly synthesize
• 17:10
  the journey we've been on today. We've looked
• 17:12
  at the spiky benchmarks, the aggressive economics,
• 17:15
  the specialized cyber model, and these wild hands
• 17:18
  -on tests. It is a massive amount of data to
• 17:20
  process. The big takeaway here is clarity of
• 17:23
  purpose. Gemini 3 .8 Flash is not the absolute
• 17:27
  smartest model in the room. Google isn't claiming
• 17:30
  it is. Right. If you need deep general knowledge
• 17:32
  or we need perfect design polish, Claude Opus
• 17:35
  5 and GPT 5 .6 sole still win. Yeah, absolutely
• 17:40
  do. You wouldn't use this model to write a beautiful
• 17:42
  novel or design a final marketing campaign. But
• 17:45
  at its price point, the developer math completely
• 17:48
  changes. It has absolute dominance in niche areas
• 17:52
  like legal work. Yeah. It ties for the lead in
• 17:54
  long horizon coding tasks like deep SWE. And
• 17:58
  as we saw with the Mount Everest map and the
• 18:00
  Tesla Pagelogic, it is incredible for interactive
• 18:03
  prototyping. It's undeniably worth testing on
• 18:06
  your own real -world tasks. The cost savings
• 18:08
  on agentic loops are just too massive to ignore.
• 18:10
  We encourage you to add Gemini 3 .8 Flash to
• 18:12
  your workflow testing. Run your own agents. See
• 18:15
  if it saves your budget without lowering your
• 18:17
  output quality. Don't rip out your whole tech
• 18:18
  stack just yet. Just add it to your internal
• 18:20
  benchmark list. You might be really surprised
• 18:22
  by where it succeeds. Sweet. We're going to leave
• 18:25
  you with the final thought today. a shift in
• 18:27
  perspective. If the future of AI isn't one godlike
• 18:32
  model that does everything perfectly, but rather
• 18:35
  a highly fragmented ecosystem of cheap hyper
• 18:39
  -specialized models, what does that mean for
• 18:41
  the dream of a single artificial general intelligence?
• 18:45
  Are we actually building a giant unified brain?
• 18:47
  Beat. Or are we just building a really efficient
• 18:50
  assembly line of savants? Think about that next
• 18:52
  time you run a prompt. Thank you for listening.
• 18:54
  See you next time.