Latest / AI Fire Daily / #582 Neil: Gemini 3.7 Hits 340 Tokens a Second in Real Coding Tests

Transcript

#582 Neil: Gemini 3.7 Hits 340 Tokens a Second in Real Coding Tests · AI Fire Daily

• 0:00
  For a year, Gemini has been the guy in the track
• 0:02
  jacket. He stood right outside the heavy boxing
• 0:05
  ring. He just watched Chad GPT and Claude trade
• 0:08
  heavy, exhausting blows. Right. They kept dropping
• 0:11
  new models every single month. It felt like a
• 0:14
  relentless heavyweight title match. Exactly.
• 0:16
  And then other contenders suddenly jumped in
• 0:18
  too. Yeah. Models like Pimi and Quinn threw wild
• 0:21
  punches. They completely messed up the global
• 0:24
  leader boards constantly. Meanwhile, Gemini just
• 0:27
  looked like that one hesitant friend. The friend
• 0:30
  who claims they just haven't fought yet. Right.
• 0:32
  Of course, nobody in the crowd really believed
• 0:34
  him. But now, Gemini 3 .7 Flash just stepped
• 0:37
  over the ropes. Beat. Welcome to the deep dive.
• 0:41
  We have a really exciting roadmap for you today.
• 0:44
  We are exploring Google's brand new Gemini 3
• 0:47
  .7 Flash model. Finally. Yeah, finally. We are
• 0:51
  going to unpack its incredible raw generation
• 0:54
  speed. We will also examine its surprisingly
• 0:57
  halved price tag. Which is a huge deal. Absolutely.
• 1:01
  And finally, we will test it inside the newly
• 1:02
  released Deepsea Karnas. We really want to see
• 1:04
  if it actually changes how you cured. Because
• 1:06
  this is not just another basic model update release.
• 1:09
  No, it is not. It is about fundamentally changing
• 1:12
  your entire developer workflow. But to understand
• 1:16
  why this specific model actually matters, we
• 1:19
  have to look at the raw numbers first. Yeah.
• 1:21
  We must check the specs before hitting the track.
• 1:23
  So Google kept the same fast direction for this
• 1:26
  flash lineup? They did. But this version focuses
• 1:29
  heavily on practical coding workflows. Right.
• 1:32
  The clearest upgrade here is definitely in software
• 1:34
  engineering. Let's examine the frontier code
• 1:37
  1 .1. main benchmark results first. Gemini 3
• 1:41
  .7 hit 43 .6 % on that specific test. And what
• 1:45
  was the previous score for context? That is up
• 1:48
  from 34 .4 % for a 3 .6 flash. Wow, that is almost
• 1:53
  a 10 % absolute performance camp. Yeah, it is
• 1:56
  huge. In the AI space, that is a massive leap.
• 1:59
  It means the model solves one in 10 more problems
• 2:02
  perfectly. It ultimately means fewer frustrating
• 2:04
  software bugs for you. It even scored slightly
• 2:06
  higher than Cloud Sonnet 5 there, and it easily
• 2:09
  beat GPT 5 .6 tera in Google's own comparisons.
• 2:12
  But short bursts of coding are actually quite
• 2:14
  easy. The real nightmare is when an AI loses
• 2:18
  the plot entirely. How does it hold up? when
• 2:21
  the task drags on. Well, that brings us to the
• 2:24
  DeepSWE v1 .1 benchmark test. Right. This measures
• 2:29
  an AI's ability to handle long workflows. Gemini
• 2:33
  3 .7 reached an incredibly impressive 65 .3 %
• 2:37
  score here. And the previous version. It only
• 2:39
  hit 48 .6 % on this exact same test. That shows
• 2:43
  a massive improvement in holding a coherent thought.
• 2:45
  Yeah, it really does. It doesn't lose the thread
• 2:48
  halfway through a complex task because refactoring
• 2:50
  an entire code base requires maintaining intense
• 2:53
  focus. Absolutely. This model clearly proves
• 2:56
  it can handle that heavy burden. Web development
• 2:59
  follows that exact same positive growth pattern,
• 3:01
  too. Gemini 3 .7 reached a highly competitive
• 3:04
  ranking of 1588 ELO. Okay. This was officially
• 3:07
  recorded on the CodeArena platform. That is actually
• 3:09
  the highest score among all the charted models.
• 3:12
  But a smart model is totally useless if it bankrupts
• 3:14
  you. Oh, for sure. Good benchmark scores are
• 3:16
  really just pure potential energy. Pricing matters
• 3:19
  immensely when you run these models constantly.
• 3:21
  Yeah, and the cost drop here is actually very
• 3:23
  aggressive today. It starts at 75 cents per million
• 3:26
  input tokens, and it is $3 .75 for output tokens.
• 3:31
  Wow. Google says this is half the original launch
• 3:34
  price. It is like finding a high -performance
• 3:36
  sports car engine. But you get it for the price
• 3:40
  of an economy compact. Beat. It completely changes
• 3:44
  the math on running it daily. Does this cheaper
• 3:47
  cost matter more than raw intellect here? Well,
• 3:51
  for daily high volume tasks, cost efficiency
• 3:53
  absolutely beats marginal genius. Right. You
• 3:56
  want something that is smart enough. but mostly
• 3:59
  cheap. I mean, imagine running an autonomous
• 4:01
  agent hundreds of times a day. Yeah, the EPI
• 4:03
  bills stack up fast. Exactly. You simply can't
• 4:06
  afford top tier models for every single step.
• 4:09
  The cheaper speed wins if it's smart enough.
• 4:11
  Exactly. That is when a model becomes truly useful.
• 4:14
  Good benchmark scores and low costs are just
• 4:17
  potential energy. We need to see how this translates
• 4:20
  to kinetic energy. We really need to look at
• 4:22
  the actual waiting time for you. Two -second
• 4:25
  silence. Because raw processing speed sounds
• 4:28
  very impressive on paper. Right. But one benchmark
• 4:30
  number doesn't show the real human feeling. Let's
• 4:33
  start with the pure text generation speed first.
• 4:36
  Artificial analysis tracked this specific metric
• 4:39
  very closely indeed. Gemini 3 .7 running at high
• 4:43
  reasoning generates incredibly fast. It hits
• 4:46
  around 340 output tokens per second constantly.
• 4:50
  Let's quickly define tokens per second for a
• 4:52
  moment. It is the speed of spitting out word
• 4:54
  chunks. Yeah, and 340 is a massive number in
• 4:56
  daily practice. That is nearly three times faster
• 4:59
  than its direct rivals. Really? Yeah, it beats
• 5:01
  GBT 5 .6 Tera and GLM 5 .2 in the exact same
• 5:06
  test. That number makes Gemini 3 .7 stand out
• 5:09
  right away. Absolutely. But raw generation speed
• 5:11
  is still only one small part. Right, you must
• 5:13
  look at the time to finish the task. Artificial
• 5:16
  analysis measured an average time per task too.
• 5:18
  What was it? The average task time is now just
• 5:20
  1 .7 minutes. Wow. And that is with the model
• 5:23
  running at high reasoning mode. The high output
• 5:26
  speed brings that total time way down. Yeah.
• 5:28
  That 1 .7 minute number is very easy to comprehend.
• 5:31
  Whoa. I mean, imagine a standard 14 -second wait
• 5:35
  for a chatbot. Imagine that wait dropping down
• 5:37
  to just two seconds. That is totally game -changing.
• 5:40
  It is. Users don't really need to know the exact
• 5:42
  token speed. They just see the complex answer
• 5:45
  appear almost instantly. You stop watching those
• 5:47
  three dots spin endlessly on screen. Exactly.
• 5:50
  Your psychological flow state remains entirely
• 5:53
  unbroken while working. Because when you wait
• 5:55
  14 seconds... you often check your phone. Oh,
• 5:58
  every single time. You entirely lose your delicate
• 6:01
  train of thought. Two seconds keeps your brain
• 6:03
  completely locked in. It really does. But benchmarks
• 6:06
  usually happen in a very clean, sterile setup.
• 6:10
  Yeah. Real coding agents have a lot more messy
• 6:12
  work. Right. Maho Mohan ran a very complex security
• 6:16
  review test. It was a real software project with
• 6:19
  many files involved. Gemini 3 .7 needed time
• 6:22
  at the start just to read. It had to use several
• 6:25
  tools to analyze the project. So it couldn't
• 6:28
  just start typing instantly? No, it had to think.
• 6:31
  But even with that extra work, the total run
• 6:33
  was fast. It's finished in around 2 minutes and
• 6:36
  38 seconds. That is still incredibly quick. Yeah,
• 6:39
  and the counter went through roughly 90 ,000
• 6:41
  tokens total. Does this blazing generation speed
• 6:45
  mean it skips thinking time? No. The model still
• 6:48
  has to pause for tool calls. It must physically
• 6:50
  stop and read the context carefully. Right. But
• 6:53
  the actual text output generation is basically
• 6:55
  instant. Right. Tools take time, but generation
• 6:57
  is lightning. There is one more crucial efficiency
• 7:00
  metric to mention here. Steve Ibe ran a direct
• 7:03
  comparison test with 3 .6 Flash. Gemini 3 .7
• 7:07
  actually used about 20 % fewer tokens. So it
• 7:10
  is fundamentally faster, cheaper, and far more
• 7:12
  concise. Exactly. It gets straight to the point
• 7:14
  without wasting any words. That is why this specific
• 7:17
  speed matters outside of simple benchmarks. When
• 7:20
  an AI talks to another AI, pleasantries waste
• 7:24
  compute. By being concise, it saves money on
• 7:27
  every single tool call loop. It flawlessly combines
• 7:30
  shorter task times and faster tool workflows.
• 7:33
  Flash makes much more sense for complex agent
• 7:35
  workflows now. Sponsor. Speed is completely useless
• 7:39
  without proper reliable control. Oh, 100%. A
• 7:43
  fast car still needs a steering wheel and dashboard.
• 7:45
  To actually harness 340 tokens a second, you
• 7:49
  need deep visibility. You need a clear dashboard
• 7:53
  to see what the AI does. Enter the newly released
• 7:56
  DeepSeq harness framework. DeepSeq just dropped
• 7:59
  DeepSeq Harness version 0 .1 for developers.
• 8:02
  Nice. It is a developer preview, but it is an
• 8:05
  official release. It is a plug and play harness
• 8:08
  for building highly functional agents. Let's
• 8:10
  clarify what agent workflows actually mean today.
• 8:13
  It means AI doing multi -step tasks on its own.
• 8:16
  Exactly. It is built around a very simple, elegant
• 8:19
  idea. Almost every part of it works perfectly
• 8:21
  like a plugin. You don't have to do everything
• 8:23
  inside a dark terminal. You just type a simple
• 8:25
  terminal command to start it. PX command right?
• 8:28
  Yeah. it immediately opens a clean browser interface
• 8:31
  for you. You aren't just staring at a scrolling
• 8:33
  command line anymore. Yeah. No, not at all. You
• 8:35
  get a full visual workspace to manage things
• 8:38
  effectively. Once it is running, you choose your
• 8:41
  active coding workspace. The harness reads your
• 8:43
  project files for immediate local context. Okay,
• 8:46
  that is super helpful. You can also choose different
• 8:49
  operational modes for the working agent. There
• 8:51
  is standard, code, minimal, and creator mode
• 8:55
  available. You match the specific setup to your
• 8:57
  exact tasks. You don't have to rebuild the entire
• 9:00
  setup every time. No, you can easily change plugins
• 9:03
  and agent presets too. What kind of metrics does
• 9:06
  this visual dashboard actually show you? You
• 9:09
  get deep real -time metrics while the agent is
• 9:11
  running. You watch the tokens per second live
• 9:14
  on the screen. You see the exact time to the
• 9:18
  first generated token. You also track the cache
• 9:21
  hit rate continuously during execution. Let's
• 9:24
  define a cache hit for clarity here. It means
• 9:27
  reusing saved memory to go faster. Yeah, but
• 9:30
  the absolute best feature is the trajectory view.
• 9:33
  Oh, tell me about that. It shows exactly how
• 9:35
  the agent moves through the task. It details
• 9:38
  every single tool call the autonomous agent makes.
• 9:41
  It displays the exact context used during the
• 9:44
  entire workflow. I still wrestle with prompt
• 9:47
  draft myself, honestly. Beat. It is deeply frustrating
• 9:51
  when a model forgets its original goal. Oh, I
• 9:54
  know the feeling. It starts building a calculator
• 9:57
  and ends up writing poetry. Exactly. Seeing the
• 10:00
  actual trajectory on screen is a total lifesaver.
• 10:04
  Why do we need to watch its trajectory, though?
• 10:06
  Especially if it eventually gets the right answer
• 10:08
  anyway. Well watching the trajectory helps you
• 10:11
  debug the underlying agent mechanics. Okay. You
• 10:14
  see exactly why an agent went completely off
• 10:16
  track. You understand why it suddenly slowed
• 10:18
  down midway through. Right. You simply can't
• 10:21
  fix a problem if you can't see the steps. Got
• 10:23
  it. Visibility helps us fix bad behavior. Now
• 10:27
  we have the ultimate high -speed engine totally
• 10:29
  ready. We do. We also have the ultimate dashboard
• 10:32
  to monitor it closely. What happens when we put
• 10:35
  them together for a real test? Two -sec silence.
• 10:38
  First, you have to connect Gemini 3 .7 inside
• 10:41
  the harness. You do this easily using an OpenRouter
• 10:43
  API key. Okay, so a standard integration? Yeah.
• 10:47
  Inside the harness, you navigate to the Settings
• 10:49
  menu tab, you click on Models, then you select
• 10:52
  Add Provider. Simple enough. Select OpenRouter
• 10:55
  and simply paste your secret key to save it.
• 10:58
  Then you just select Gemini 3 .7 from the drop
• 11:00
  -down menu. So you were running Google's models
• 11:03
  securely in DocSix's environment. Right, and
• 11:05
  watching all those critical performance metrics
• 11:07
  live. Steve -Eye ran a truly fascinating test
• 11:11
  with this exact setup. What did he build? He
• 11:13
  asked Gemini 3 .7 to build a 3D Rubik's Cube.
• 11:17
  Oh, wow. It was a heavy coding task requiring
• 11:19
  many distinct steps. The complex task finished
• 11:23
  in exactly 27 total minutes. It went through
• 11:26
  52 separate discrete steps to build it. It ran
• 11:29
  at about 120 tokens per second overall. Does
• 11:32
  that drop to 120 tokens per second mean the initial
• 11:35
  benchmark was misleading? Not exactly. Benchmarks
• 11:38
  strictly measure pure text streaming speed. OK,
• 11:40
  just raw output. Yeah. That is just generating
• 11:43
  one long continuous text answer quickly. Complex
• 11:46
  agent tasks require stopping to execute external
• 11:49
  coding tools. Right. The model has to think carefully
• 11:52
  between each discrete step. Makes sense. Harder
• 11:55
  tasks require more computing breaks. Exactly.
• 11:57
  And it handles those hard, multi -step tasks
• 12:01
  incredibly well. Did anyone else test it? Yeah.
• 12:04
  Another user named Kiseki Codes tested it with
• 12:08
  a video game. Oh, what game? They asked the model
• 12:10
  to build an interactive Mario game. This proved
• 12:14
  the model handles complete software engineering
• 12:16
  architectural work. So it is not just short files.
• 12:19
  No, it isn't just writing short snippets of simple
• 12:22
  code. It manages the entire project architecture
• 12:25
  from the ground up. It holds the whole interactive
• 12:28
  game state in its context. Yeah, exactly. Deep
• 12:31
  Sea Karnas makes all this underlying complexity
• 12:33
  much clearer visually. You see the speed changing
• 12:35
  dynamically with the task difficulty. You really
• 12:38
  do. Let's pull all these distinct threads together
• 12:40
  right now. We have the remarkably low cost and
• 12:43
  the raw speed. We have practical application
• 12:47
  inside a clean visual agent harness. Where does
• 12:50
  this actually fit into your daily life? Two secs
• 12:53
  silence. It really comes down to four main practical
• 12:56
  use cases. OK, let's hear them. First, use it
• 12:59
  for incredibly fast iteration when you code.
• 13:01
  You can run a prompt, check it, and change it
• 13:03
  quickly. Right, the speed helps there. Second,
• 13:05
  use it as a dedicated software coding assistant
• 13:08
  constantly. It shows clear, undeniable improvements
• 13:12
  in long coding workflows today. OK, what is the
• 13:15
  third? Third, use it for high -volume workloads
• 13:18
  with many API requests. The raw speed helps large
• 13:21
  token usage tasks finish much faster. And the
• 13:24
  fourth. Fourth, use it for low -latency chatbots
• 13:27
  that desperately need speed. But we need to provide
• 13:30
  a realistic counterweight here today. Blazing
• 13:32
  speed is not the only metric that truly matters.
• 13:35
  Right. For deep reasoning, you need something
• 13:37
  much heavier still. Yeah. For difficult strategic
• 13:40
  decisions or maximum accuracy, proceed with some
• 13:43
  caution. You should still benchmark it against
• 13:45
  a stronger, slower frontier model. Will this
• 13:48
  kind of speed ultimately replace the need for
• 13:50
  the slower frontier models? Well, there will
• 13:52
  always be a profound need for deep strategic
• 13:55
  thinkers. Some difficult problems require massive
• 13:58
  compute and very slow deliberation. But extremely
• 14:01
  fast models will gladly handle 90 % of daily
• 14:04
  grunt work. No, but it redefines our everyday
• 14:07
  coding baseline. The new AI competition isn't
• 14:11
  about pure benchmark supremacy anymore. It is
• 14:14
  about who makes the end user wait the absolute
• 14:17
  least. You want to strictly minimize waiting
• 14:19
  while still doing a great job. The entire paradigm
• 14:22
  is shifting from pure power to practical velocity.
• 14:25
  It really is. You should really try spinning
• 14:27
  up DeepSea Karnas yourself today. Yeah, absolutely.
• 14:29
  Grab an open router key and plug it right into
• 14:32
  the dashboard. Watch the complex trajectory of
• 14:35
  a remarkably fast model directly. It changes
• 14:37
  everything. It completely changes how you fundamentally
• 14:40
  understand these AI agents. We will leave you
• 14:42
  with this calm, lingering thought today. Beat.
• 14:46
  These newer models are getting unimaginably fast
• 14:50
  at building complex apps. They can literally
• 14:53
  build entire functional applications in just
• 14:55
  two minutes. The software generation process
• 14:58
  is becoming virtually instantaneous for developers
• 15:00
  now. How long until the human bottleneck becomes
• 15:03
  the ultimate glaring problem? Simply reviewing
• 15:06
  that code will soon become the slowest part of
• 15:08
  the entire workflow. Thank you for joining us
• 15:11
  on this deep dive. Catch you next time. OTRO
• 15:13
  music.