Latest / AI Fire Daily / #586 Neil: DeepSeek V4 Pro and Grok 4.6 Push AI Prices Much Lower

Transcript

#586 Neil: DeepSeek V4 Pro and Grok 4.6 Push AI Prices Much Lower · AI Fire Daily

• 0:00
  For a very long time, the AI race was, it was
• 0:03
  simple. Yeah, it really was. It was always about
• 0:06
  building the biggest brain possible. Everyone
• 0:08
  just wanted the smartest, most powerful system
• 0:10
  on the market. Beat. But today, a quiet revolution
• 0:15
  is actually happening. The real battle is now
• 0:18
  about price and efficiency. It is about getting
• 0:20
  real work done without going broke. The entire
• 0:24
  narrative of the industry is shifting rapidly.
• 0:26
  I mean, developers care deeply about unit economics
• 0:29
  right now. Raw intellectual dominance is, well,
• 0:32
  it's simply no longer enough. Welcome to the
• 0:35
  deep dive. You know, we're unpacking a sudden
• 0:37
  and fascinating shift today. Two highly competitive
• 0:40
  models just arrived on the scene. And they are
• 0:42
  making serious waves. Right. We're looking closely
• 0:45
  at Grok 4 .6 and DeepSeek V4 Pro. Both are getting
• 0:49
  shockingly close to the most expensive models.
• 0:51
  But they're doing this for pennies on the dollar.
• 0:53
  Which is just crazy to see. It is. So we will
• 0:56
  look at their raw performance numbers first.
• 0:58
  But as we dug into the real world testing, we
• 1:00
  found something. We found a massive plot twist
• 1:03
  regarding token usage. Oh, yeah. The real world
• 1:06
  data completely changes how we view these models.
• 1:09
  Ultimately, we will figure out which model you
• 1:11
  should actually choose for your workflow. So
• 1:13
  let's start by looking closely at Grok 4 .6.
• 1:16
  Let's do it. This model represents a truly major
• 1:19
  shift in the market. It directly challenges the
• 1:22
  established heavyweights in the industry. It
• 1:25
  blurs the line around top tier performance and
• 1:28
  everyday accessibility. It really does. And this
• 1:30
  brings us to the core concept of frontier models.
• 1:33
  Right, which are the absolute smartest, most
• 1:36
  capable AI systems available right now. Exactly.
• 1:38
  And Grok 4 .6 is basically knocking heavily on
• 1:41
  their door. Cursor notes, this version was built
• 1:44
  for complex multi -step tasks. It targets large
• 1:48
  coding projects and serious knowledge work. You
• 1:50
  can see Grok 4 .6 as a dedicated worker. It's
• 1:54
  definitely not just for casual chatting anymore.
• 1:56
  No, not at all. The performance numbers tell
• 1:58
  an incredibly fascinating story here. Let's examine
• 2:01
  the artificial analysis intelligence index. That
• 2:04
  is the big one. Yeah, that index combines nine
• 2:06
  different major benchmarks together. It gives
• 2:09
  us one comprehensive capability score for models.
• 2:13
  Grok 4 .6 scores a solid 61 on that specific
• 2:17
  index. Wow. I know, right? That perfectly matches
• 2:20
  the powerful GPT 5 .6 Solmax model. Wait, really?
• 2:24
  How does a cheaper model match that expensive
• 2:27
  standard? What exactly is happening under the
• 2:29
  hood there? Well, it comes down to highly specialized
• 2:32
  training techniques. They stopped trying to make
• 2:34
  the model deeply conversational. Oh, I see. Yeah,
• 2:36
  they focused entirely on intense logic and deep
• 2:39
  reasoning. It actually trails Fable 5 Max by
• 2:42
  just one single point. The performance gap is
• 2:45
  incredibly small across the board. That is wild.
• 2:47
  Look at GDP Valet version 2. Grok 4 .6 scores
• 2:50
  1 ,753 on that specific test. That actually beats
• 2:54
  GPT 5 .6 Solmax entirely. Really? Yeah. It also
• 2:58
  beats the highly capable Fable 5 Max. The coding
• 3:01
  scores are particularly interesting to me, though.
• 3:03
  I mean, coding requires rigorous logic and strict
• 3:06
  structural discipline. Exactly, and the coding
• 3:08
  benchmarks highlight its greatest hidden strengths.
• 3:10
  Let's look closely at Cruiserbench 3 .2. Grok
• 3:13
  4 .6 high scores, 69 .9 % on that rigorous test.
• 3:17
  That is very close to Fable 5 Max at 70 .5%.
• 3:21
  And it completely beats GPT 5 .6 Solmax at 67
• 3:25
  .2%. There was also an extra high version available,
• 3:27
  right? Yep. Grok 4 .6 extra high reaches 70 .8
• 3:31
  % on the leaderboard. Then we have Frontier Code
• 3:33
  1 .1 extended to consider. How does it do there?
• 3:36
  Grok 4 .6 scores an impressive 61 .3 % there.
• 3:40
  That is notably higher than GPT 5 .6 Solmax.
• 3:43
  But it still sits just below Fable 5 Max. So
• 3:46
  Grok does not win every single benchmark. Right.
• 3:49
  But is now incredibly close to the leading models.
• 3:52
  You know, benchmarks are basically like taking
• 3:53
  a standardized test. They look great on paper
• 3:55
  for comparing raw potential. Absolutely. But
• 3:58
  real world demos represent the actual job interview.
• 4:01
  How is Grok performing in the real world? Oh
• 4:03
  man, developers are sharing some genuinely crazy
• 4:05
  early results online. TinJS ran a very simple
• 4:08
  direct pump test. He asked the model to create
• 4:11
  a racing game. Okay, a racing game. Yeah, and
• 4:13
  he wanted the entire game written purely in HTML.
• 4:18
  Grok 4 .6 built a fully playable game immediately,
• 4:21
  and it did that in about one single minute. One
• 4:24
  minute to build a playable game is staggering.
• 4:26
  That requires understanding game loops, rendering,
• 4:30
  and user input. It is insane. Another complex
• 4:33
  test showed something even more visually impressive.
• 4:36
  It showed Grok 4 .6 creating a Falcon 9 booster
• 4:39
  return simulation. Oh, wow. Yeah, it handled
• 4:42
  that complex physics simulation perfectly on
• 4:44
  the first try. These examples prove its coding
• 4:47
  ability turns into highly usable outputs. We
• 4:49
  really need to talk about the pricing structure
• 4:51
  now. Yeah. Because this is where the math gets
• 4:53
  genuinely fascinating. It really does. XAI lists
• 4:56
  Grok 4 .6 at $2 per million input tokens. It
• 5:00
  charges $6 per 1 million output tokens. Which
• 5:02
  is crazy. You have to compare that carefully
• 5:04
  to the big players. GPT 5 .6 all is $5 for input
• 5:08
  tokens. It jumps to $30 for output tokens. Maple
• 5:11
  5 is even more expensive than that baseline.
• 5:13
  What? They charge $10 for 1 million input tokens.
• 5:16
  They charge $50 for 1 million output tokens.
• 5:19
  So the difference is huge. Right. Grok 4 .6 is
• 5:22
  significantly cheaper for heavy API users. That
• 5:26
  massive price gap changes a company's monthly
• 5:28
  cost. Beat. Though prices do rise if prompts
• 5:32
  exceed 200 ,000 tokens. Yeah, you have to monitor
• 5:35
  that upper limit very carefully. Is Grok sacrificing
• 5:38
  deep reasoning to achieve this lower price point,
• 5:40
  or is it genuinely this efficient? Well, it is
• 5:43
  genuinely efficient for highly targeted developer
• 5:45
  tasks. The creators stripped away the unnecessary
• 5:48
  general chatter capabilities. They focused the
• 5:51
  computing power strictly on multi -step coding.
• 5:54
  It relies heavily on specialized training for
• 5:56
  complex development. It directs energy precisely
• 5:59
  where developers actually need it. So it optimizes
• 6:01
  computing powers strictly for complex coding
• 6:03
  tasks. Precisely. So Grok is dominating in coding
• 6:06
  efficiency. But coding is only half the battle.
• 6:09
  What happens when developers process massive
• 6:11
  of document libraries. That is a whole different
• 6:13
  game. It is. And that brings us directly to DeepSeek
• 6:16
  V4 Pro. Grok 4 .6 seemed like the best deal in
• 6:19
  town initially. But then DeepSeek actively shattered
• 6:22
  the absolute price floor. They really did. Grok
• 6:25
  4 .6 is already much cheaper than many frontier
• 6:28
  models. But DeepSeek pushes the final cost even
• 6:31
  lower. The pricing on this specific model is
• 6:34
  incredibly aggressive. DeepSeek charges just
• 6:37
  $1 .32 per million input tokens. Output tokens
• 6:41
  are priced at $3 .96 per million. And that is
• 6:45
  the standard rate during peak hours. Right. If
• 6:47
  you run your tasks off peak, pricing drops further.
• 6:50
  The pricing is literally 50 % lower during off
• 6:53
  peak hours. That is staggeringly cheap for this
• 6:56
  high level of capability. Yeah. It focuses heavily
• 6:58
  on large agent workflows and longer tasks. It
• 7:01
  also supports a massive 1 million token context
• 7:04
  window. We should quickly define what a context
• 7:07
  window actually is. Sure, it is the total amount
• 7:09
  of text an AI can remember at once. This is basically
• 7:11
  the model's short -term working memory. A million
• 7:14
  tokens is roughly a dozen standard novels. Beat.
• 7:18
  I still wrestle with prompt drift myself. Oh,
• 7:20
  we all do. Managing a one million token context
• 7:22
  window is incredibly difficult. It's like stacking
• 7:25
  Lego blocks of data. If the foundation gets wobbly,
• 7:28
  the AI loses track of the instructions. That
• 7:30
  prompt drift is a real challenge for any developer.
• 7:33
  When an AI reads too much, it often forgets the
• 7:36
  beginning. But DeepSeek handles specific, highly
• 7:38
  -focused tasks very well. Let's look at the official
• 7:41
  DeepSeek V4 Pro benchmarks. On Terminal Bench
• 7:44
  2 .1, it scores an impressive 87 .9. OK. That
• 7:49
  sits just slightly below Fable 5 at 88 .0. I
• 7:53
  see in our notes it actually wins on Cyber Gym.
• 7:54
  It does. It scores 83 .3 on the Complex Cyber
• 7:57
  Gym benchmark. That is slightly higher than Fable
• 7:59
  5 at 83 .1. It also wins on Automation Bench
• 8:02
  with a 31 .8 score. Nice. That puts it clearly
• 8:05
  above Fable 5 at 29 .1. But it still struggles
• 8:08
  on a few specific programming benchmarks, doesn't
• 8:10
  it? Yeah, it does. On DeepSWE, it scores a modest
• 8:14
  62 .7 overall. It stays behind Kimi K3 at 67
• 8:18
  .5. And it remains well behind Fable 5 at 70
• 8:21
  .0. OK, so these results show DeepSeat doesn't
• 8:24
  lead every single benchmark. Right. However,
• 8:27
  you still get performance remarkably close to
• 8:29
  frontier models. It performs incredibly well
• 8:31
  on terminal cybersecurity and automation tasks.
• 8:34
  And you are paying drastically less for the API
• 8:36
  access. Exactly. Eepsic V4 Pro makes perfect
• 8:40
  sense for specific workloads. It is perfect where
• 8:42
  cost, scale, and long contexts matter most. High
• 8:45
  volume agent workflows benefit heavily from its
• 8:48
  lower API pricing. You can process mountains
• 8:50
  of data very cheaply. You really care. The one
• 8:53
  million token context window is a massive feature.
• 8:56
  It really helps with large code bases and huge
• 8:59
  legal documents. Pete, with a 1 million token
• 9:03
  context at half the price off peak, does this
• 9:05
  change how developers design agent workflows?
• 9:08
  Oh, it absolutely changes the entire development
• 9:11
  game. Developers no longer have to carefully
• 9:13
  filter data beforehand. They can afford to feed
• 9:16
  massive libraries directly inside. They can dump
• 9:19
  entire massive code bases into autonomous agents.
• 9:21
  They run complex terminal tasks overnight while
• 9:24
  everyone sloops. They achieve massive operational
• 9:27
  scale without breaking the bank. Developers can
• 9:29
  now run massive t - overnight without any worry.
• 9:31
  Exactly. We'll take a very short break right
• 9:33
  here. Sounds good. Midroll sponsor, read placeholder.
• 9:36
  Midroll sponsor, read placeholder. Welcome back
• 9:38
  to The Deep Dive. We just talked a lot about
• 9:39
  basic price per token. You did. But a lower price
• 9:42
  tag per token is actually deceptive. It doesn't
• 9:45
  matter if the AI struggles with the logic. What
• 9:47
  if the AI uses 10 times as many tokens? Right.
• 9:50
  It needs to actually solve the problem efficiently.
• 9:52
  This is where the story gets really wild today.
• 9:55
  Benchmarks only give you a useful theoretical
• 9:57
  reference point. True. Real -world tests show
• 10:00
  a much clearer, often shocking difference. A
• 10:03
  few dedicated users compared these models on
• 10:06
  real workflows. The results uncover a massive,
• 10:10
  hidden operational cost. I was looking at the
• 10:13
  Day 7 tech test earlier today. I honestly thought
• 10:16
  there was a typo in our notes. I felt the same
• 10:18
  thing. They used the exact same prompt for both
• 10:20
  AI models. They used the exact same harness set
• 10:23
  up for the test. Walk me through. through what
• 10:24
  actually happened during that comparison. So
• 10:26
  they gave both models a complex multi -step coding
• 10:29
  task. Grok 4 .6 finished the entire task in one
• 10:33
  hour. Well, one hour and 12 minutes to be specific.
• 10:36
  OK. It used 603 ,000 tokens to get the job done.
• 10:40
  DeepSeq V4 Pro took slightly longer to finish
• 10:43
  the work. It finished in one hour and 17 minutes.
• 10:46
  That's not too bad. But here is the massive plot
• 10:48
  twist. DeepSeek used a staggering 8 .37 million
• 10:53
  tokens. I mean, imagine scaling to a billion
• 10:58
  queries and realizing your cheaper model just
• 11:01
  burned through 14 times the tokens. It completely
• 11:03
  destroys the initial attractive cost advantage.
• 11:06
  When DeepSeek hits a complex agent workflow,
• 11:08
  it tends to overthink. Ah, I see. It tries a
• 11:11
  solution, fails, and tries again repeatedly.
• 11:13
  It burns thousands of tokens in a confusing loop.
• 11:16
  Grok takes a much more direct, linear path forward.
• 11:19
  You mentioned they used the exact same harness
• 11:21
  set up earlier. What exactly is a harness set
• 11:24
  up anyway? It's the surrounding code environment
• 11:26
  letting an AI interact with your system. So the
• 11:29
  environment itself dictates the AI's overall
• 11:31
  efficiency. Bohuteng tested both models with
• 11:34
  completely different setups. He did. He used
• 11:36
  the DeepSeq harness and the Pi framework. He
• 11:38
  wanted to see how the agent setup changes token
• 11:40
  use. And what happened? For DeepSeq v4 Pro, the
• 11:43
  DeepSeq harness was highly inefficient. It used
• 11:45
  about 5 .3 times more tokens than Pi. It ultimately
• 11:48
  cost 3 .8 times more money to run. How did Grok
• 11:51
  handle that exact same harness test? For Grok
• 11:54
  4 .6, the efficiency gap was much smaller. It
• 11:57
  used around 2 .7 times more tokens overall. It
• 12:01
  cost 2 .4 times more money to complete. This
• 12:05
  test proves the model is only one part of your
• 12:07
  cost. The setup itself literally changes your
• 12:10
  final monthly bill. If the harness is poorly
• 12:13
  designed, the AI spins its wheels. Exactly. Jonesong
• 12:16
  also ran some very important tests recently.
• 12:19
  He tested both models on his own real agent framework.
• 12:22
  Yeah, he used complex real -world agent tasks
• 12:25
  for his testing. He found that both models look
• 12:27
  far less impressive. They didn't match their
• 12:29
  shiny public benchmark scores at all. Complex
• 12:32
  multi -step tasks clearly change the reality
• 12:35
  of performance. The way a model uses tools and
• 12:37
  keeps context matters greatly. Handling several
• 12:40
  sequential steps introduces many points of failure.
• 12:43
  If token efficiency drastically alters the final
• 12:46
  bill, are we measuring the wrong things on these
• 12:48
  public leaderboards? We are definitely measuring
• 12:50
  the wrong things right now. Leaderboards measure
• 12:52
  raw capability in perfectly isolated sterile
• 12:55
  conditions. They assume the model always walks
• 12:58
  a perfectly straight line. They completely ignore
• 13:00
  the messy reality of actual development work.
• 13:03
  They ignore how many costly dead ends a model
• 13:06
  explores. Leaderboards measure perfect conditions,
• 13:09
  completely ignoring the costly dead ends. Yes,
• 13:12
  beautifully said. Benchmarks and raw token prices
• 13:15
  can clearly be deceiving. Let's transition to
• 13:18
  a practical down -to -earth guide for our listeners.
• 13:21
  After looking at tests, how do you choose based
• 13:23
  on your daily work? You have to choose strictly
• 13:26
  based on your specific workload. You cannot just
• 13:28
  look at which model ranks nominally higher. If
• 13:31
  you mainly write code, the choice is pretty easy.
• 13:33
  If you already work inside Cursor, you should
• 13:36
  use Grok 4 .6. It has incredibly strong proven
• 13:39
  coding capabilities. Plus, it has deep native
• 13:42
  Cursor support integrated fully. Yeah, you get
• 13:44
  seamless support across desktop, web, and CLI
• 13:47
  interfaces. You can use Grok 4 .6 effortlessly
• 13:50
  in complex agent workflows. Grok Build is also
• 13:53
  fantastic for longer, intricate coding tasks.
• 13:56
  It integrates perfectly into a developer's existing
• 13:58
  daily habits. Now, what if you have much larger
• 14:02
  data -heavy workloads? What if you process massive,
• 14:05
  sprawling document libraries? That is where DeepSeq
• 14:07
  V4 Pro really shines brightest. It is the absolute
• 14:11
  best choice for high -volume agents. Totally.
• 14:13
  If you run heavy, off -peak terminal tasks, choose
• 14:16
  DeepSeq. If you run massive cybersecurity log
• 14:19
  analyses, choose DeepSeq. The massive 1 million
• 14:23
  token context window is simply unbeatable. The
• 14:26
  rock bottom API cost makes it the clear winner
• 14:28
  for scale. You can process data like running
• 14:31
  a dishwasher at night. That's a great way to
• 14:33
  put it, but there are still crucial reasons to
• 14:35
  stick with the giants. You should keep GPT 5
• 14:38
  .6 Sol or Fable 5 available. Why is that? Well,
• 14:42
  tasks demanding absolute peak reasoning require
• 14:44
  the absolute best tools. They still give you
• 14:46
  clearly many results on complex edge cases. They
• 14:49
  handle deep nuance better than the cheaper models.
• 14:52
  A slightly higher benchmark score matters so
• 14:54
  much less practically. The actual quality you
• 14:57
  get in your workflow is what matters. Beat. Should
• 15:01
  developers completely stop chasing leaderboard
• 15:03
  ranks and just AD test everything locally? I
• 15:06
  strongly believe that is the only logical way
• 15:08
  forward. A cheaper model may perfectly fit your
• 15:11
  specific daily workflow. It might actually use
• 15:14
  fewer tokens overall to solve problems. It might
• 15:16
  stay more reliable on your specific company data.
• 15:20
  Local testing reveals the true hitting costs
• 15:22
  immediately. Leaderboards simply cannot predict
• 15:25
  your actual production bill. Local testing alone.
• 15:28
  reviews the actual hidden costs and performance.
• 15:31
  We have covered a massive amount of ground today.
• 15:33
  The AI race has officially entered a totally
• 15:36
  new phase. It really has. It is no longer just
• 15:38
  a quest for raw intellectual dominance. It is
• 15:41
  now fundamentally about basic, grounded unit
• 15:43
  economics. The raw capability gap between top
• 15:46
  models is getting much smaller, but the operational
• 15:49
  price gap can still be very large. API price
• 15:51
  matters just as much as benchmark performance
• 15:53
  now. Token efficiency is absolutely crucial for
• 15:56
  scaling any real business. Oh, without a doubt.
• 15:58
  A lower token price does not guarantee a cheaper
• 16:01
  final bill. The specific harness and setup you
• 16:04
  use change everything completely. You must optimize
• 16:07
  the environment around the AI model. If token
• 16:10
  efficiency is the true hidden cost of AI, consider
• 16:14
  this. How quickly a model gets to the right answer
• 16:17
  is crucial. Will the next massive breakthrough
• 16:20
  be less about building smarter models? Will it
• 16:23
  be more about building models that know how to
• 16:25
  think in a straight line without getting distracted?
• 16:27
  That is a profoundly fascinating way to look
• 16:29
  at the future. Efficiency might actually become
• 16:32
  the new definition of intelligence. Just try
• 16:39
  it out.