Latest / AI Fire Daily / #125 Robin: Jev Makes Claude Code Faster and Cheaper by Handling the Decisions Claude Shouldn’t Waste Tokens On

Transcript

#125 Robin: Jev Makes Claude Code Faster and Cheaper by Handling the Decisions Claude Shouldn’t Waste Tokens On · AI Fire Daily

• 0:00
  I want you to think about speed, just for a moment.
• 0:04
  Think about how we interact with AI. We just
• 0:08
  accept the delay. We type a prompt, and we watch
• 0:11
  the cursor blink. Yeah, you set there waiting.
• 0:13
  Right. We watch the word slowly type out on the
• 0:16
  screen. And it feels like waiting for an AI to
• 0:19
  think is, well, it's just a lot of physics. Like
• 0:22
  it's just built into the system. Exactly. We
• 0:24
  expect it to be slow. But this is completely
• 0:28
  different. It really operates on a totally different
• 0:31
  wavelength. I mean, it's instantaneous. It is
• 0:34
  practically instantaneous. TypeSafe is actually
• 0:36
  reporting response times under a single second.
• 0:40
  Which is wild. 70 to 500 milliseconds, to be
• 0:44
  exact. That changes the entire calculation for
• 0:47
  how we build things. Absolutely. So welcome to
• 0:51
  this deep dive. Today we are exploring a brand
• 0:54
  new approach. We are looking at an ultra low
• 0:57
  -cost decision model. It's called GEV, and it
• 1:00
  is built by TypeSafe AI. And we're going to unpack
• 1:03
  all the mechanics of it today. We will look at
• 1:06
  how you actually wire it up to cloud code, practically
• 1:09
  speaking. Right. Then we'll walk through three
• 1:12
  distinct levels of workflow automation because
• 1:15
  you can scale this, you know, from just your
• 1:17
  local machine all the way to a massive enterprise.
• 1:20
  The narrative through line for you today is pretty
• 1:21
  profound. We are exploring how separating thinking
• 1:25
  from deciding can completely revolutionize your
• 1:28
  workflow. Yeah, it changes everything about how
• 1:31
  you architect an app. But before you start building
• 1:34
  pipelines, we really need to understand the foundation.
• 1:38
  We do. We need to explore why GEM exists in the
• 1:41
  first place. We have to see how its economics
• 1:43
  completely changed the game. Right, and I think
• 1:46
  the best place to start there is the creator.
• 1:48
  Gem was built by TypeSafe AI, which is a startup
• 1:51
  led by Diogo Almeida. He was previously a researcher
• 1:55
  over at OpenAI, but more importantly, he actually
• 1:58
  co -invented RLHF and InstructGPT. Meaning he
• 2:02
  helped build the actual methods behind ChatGPT
• 2:05
  itself. Exactly. And just to clarify for everyone,
• 2:08
  RLHF is training AI with human feedback to behave
• 2:12
  safely. Spot on. So Almeida deeply understands
• 2:16
  how frontier models operate. He knows all their
• 2:19
  strengths, but he also knows they are incredibly
• 2:22
  heavy. Right. Heavy, slow, and expensive to run.
• 2:25
  Yeah. And that deep knowledge brings us directly
• 2:28
  to the economics of Jev. Because Jev costs just
• 2:32
  $0 .042 per 1 million input tokens. That's practically
• 2:36
  nothing. It is. But here is the massive shift.
• 2:39
  The output is entirely free. Wow. I really want
• 2:43
  to pause on that. Free output. is basically unheard
• 2:47
  of in this space. It really doesn't happen. Let's
• 2:50
  compare that to the Claude models you might be
• 2:52
  using right now. JEV is roughly 2 .4 times cheaper
• 2:56
  on input than Haiku 5 .5, and it is a staggering
• 3:00
  238 times cheaper than Fable 5 .1. And that price
• 3:06
  difference, you know, it isn't some marketing
• 3:07
  trick to get users. It exists because of how
• 3:10
  JEV physically operates under the hood. Right.
• 3:12
  It's architectural. Exactly. Standard models
• 3:14
  predict the next thousand words of conversational
• 3:17
  filler, right? That requires massive compute.
• 3:20
  Sure. But Jev does not write essays. It does
• 3:22
  not write your code. It gives you four very specific,
• 3:26
  highly constrained typed answers. So it's intentionally
• 3:28
  narrow. Very narrow. What are the four types
• 3:31
  it actually outputs? So the first is just a simple
• 3:33
  yes or no. It's a binary choice, and it includes
• 3:35
  a probability score with it. OK, that's super
• 3:38
  useful. The second is a choice type. You feed
• 3:40
  it a list of options, and it just picks one single
• 3:42
  option from that list. Nice and clean. The third
• 3:45
  is a score, so it provides a numeric rating on
• 3:48
  whatever scale you define, like 1 to 10. Right.
• 3:51
  And the fourth is highly experimental right now.
• 3:52
  It's called box. It actually returns spatial
• 3:55
  bounding boxes for objects inside of images.
• 3:57
  Oh, wow. So by eliminating all that conversational
• 4:01
  generation, all the pleasantries, the compute
• 4:04
  cost just plummets. It drops to almost zero.
• 4:07
  Think of it kind of like stacking Lego blocks
• 4:09
  of data. Jeff sorts the bricks by color instantly,
• 4:12
  so Claude can focus. on actually building the
• 4:15
  castle. That is a brilliant analogy. Yeah. Or
• 4:18
  think of it like a hospital triage system. OK.
• 4:21
  Jev is the nurse at the front desk. The nurse
• 4:23
  takes your vitals in 30 seconds and just decides
• 4:25
  which ward you need to go to. Right. Making that
• 4:27
  immediate routing call. Yeah. And then Claude
• 4:30
  is the specialized surgeon in the operating room.
• 4:32
  You wouldn't pay a world class neurosurgeon to
• 4:34
  sit in the lobby, right? Of course not. You wouldn't
• 4:36
  have them checking patient temperatures. That
• 4:38
  is a massive waste of brilliant compute. And
• 4:41
  this actually mirrors human psychology perfectly.
• 4:44
  You mean Daniel Kahneman's work. Exactly. Kahneman
• 4:46
  wrote Thinking Fast and Slow. He describes these
• 4:50
  two core systems of human cognition. System one
• 4:52
  and system two. Right. So system one is fast.
• 4:55
  It's highly intuitive. It relies on like quick
• 4:58
  pattern matching judgments just to survive. Right.
• 5:01
  It's instinctual. Yeah. And system two is much
• 5:04
  slower. It's methodical complex reasoning. TypeSafe
• 5:07
  actually explicitly calls Jev a system one model
• 5:10
  for software. So Claude operates as our system
• 5:13
  too. Right. Humans do not use complex reasoning
• 5:15
  to dodge a moving car. We just use our fast instincts.
• 5:19
  So software shouldn't use a massive frontier
• 5:21
  model just to classify a simple email. That makes
• 5:24
  total sense. But we do need to discuss the quality
• 5:26
  threshold here because Jev is absolutely not
• 5:29
  a replacement for those frontier models. No,
• 5:31
  not at all. The sources highlight a really great
• 5:34
  test on this. Dan Shipper ran a strict side -by
• 5:37
  -side comparison between the models. Yeah, the
• 5:39
  defect test. Right. He planted seven specific
• 5:42
  writing defects into a single document. Then
• 5:45
  he tested both models to see what they'd catch.
• 5:47
  And what were the results? Fable 5 .1 caught
• 5:50
  all seven out of seven defects. It was perfect.
• 5:53
  Jev only caught six out of the seven defects.
• 5:56
  See, and that sounds like a failure until you
• 5:58
  actually look at the resources it took. Exactly.
• 6:00
  Jev completed that entire scan 25 times faster
• 6:04
  than Fable did, and it was 580 times cheaper
• 6:07
  to execute. But if I am building a reliable pipeline,
• 6:10
  I have to ask, if Jev misses defects that Fable
• 6:13
  easily catches, why should we just pay the premium
• 6:16
  for perfect accuracy every time? Well, because
• 6:19
  demanding perfection at the very first step,
• 6:21
  it just scales terribly. Right. You want to use
• 6:24
  Jev to filter out massive amounts of basic noise
• 6:27
  for literal pennies. Then you deliberately escalate
• 6:31
  only the really critical high stakes tasks up
• 6:34
  to the expensive model. So Jev categorizes things
• 6:37
  instantly, but Claude does the heavy intellectual
• 6:39
  lifting. Precisely. You engineer a workflow that
• 6:42
  leverages both of their strengths. So the economics
• 6:44
  make total sense, but economics don't matter
• 6:46
  if the actual integration is a nightmare for
• 6:48
  you. Oh, totally. If you want Claude to hand
• 6:51
  off the sorting to Jev right now, We need to
• 6:54
  know how they actually talk to each other without
• 6:55
  breaking. Yeah. The plumbing has to work. There
• 6:58
  are two main access routes you can take for this.
• 7:00
  The first option is completely direct. You can
• 7:03
  access GEV via the TypeSafe console. You just
• 7:07
  make a standard post request to their slash V1
• 7:10
  slash system one endpoint. Got it. Very straightforward
• 7:13
  if you're a developer. And the second route is
• 7:15
  through OpenRouter. But the documentation flags
• 7:17
  a pretty major catch here. It is a critical catch.
• 7:20
  Yeah. OpenRouter serves Jev on a completely separate
• 7:23
  decisions endpoint. It does not sit on their
• 7:26
  usual chat completions endpoint like everything
• 7:29
  else. Why the separation? Well, because standard
• 7:31
  endpoints expect conversational chat architecture.
• 7:34
  And as we said, Jev does not chat. Right. It
• 7:37
  just outputs those strict data types. Exactly.
• 7:39
  So you have to explicitly tell Claude code about
• 7:42
  this endpoint difference in your settings. If
• 7:44
  you forget this step, Claude will try to prompt
• 7:46
  Jev like a normal chat model, and the request
• 7:49
  will instantly fail. Okay, that's a crucial tip.
• 7:51
  We also need to talk about pinning your model
• 7:53
  versions. Yes. The current release is JEV 1 .14.
• 7:57
  It officially dropped on October 14th, 2026,
• 8:00
  and it features a massive context window. It
• 8:03
  really does. Just as a quick reminder, our context
• 8:05
  window is the maximum amount of text an AI remembers
• 8:08
  at once. Right, and JEV 1 .14 has a 262 ,000
• 8:13
  token context window. That is huge. It's astronomically
• 8:16
  large for just a simple routing model. It basically
• 8:19
  means you can feed it entire massive code bases
• 8:22
  just to make a simple yes or no routing decision.
• 8:25
  But you must strictly pin your version in the
• 8:27
  configuration file. You have to literally write
• 8:29
  out jev -1 .14. You absolutely do. Because TypeSafe
• 8:34
  is constantly iterating on these models, they
• 8:36
  only give about 30 to 60 days of notice before
• 8:39
  retiring older models. So if you just use a generic
• 8:42
  tag, your carefully tuned routing thresholds
• 8:44
  could just suddenly break overnight. I still
• 8:47
  wrestle with prompt drift myself, so testing
• 8:50
  those basic API endpoints first is a relief.
• 8:53
  Oh yeah. Building complex routing logic before
• 8:56
  testing the actual connection is just terrifying
• 8:58
  to me. It's a nightmare. So we need a solid testing
• 9:01
  protocol. Testing is absolutely crucial here.
• 9:04
  Do not build complex automation on day one. That
• 9:08
  is just a recipe for endless debugging headaches.
• 9:11
  Instead, you want to have Claude Code send a
• 9:13
  remarkably simple task to Jev first. Walk us
• 9:16
  through what a basic test prompt looks like.
• 9:18
  You keep it incredibly basic. You literally tell
• 9:21
  Claude Code to ask Jev, classify this task as
• 9:24
  simple, medium, or complex. Then you just review
• 9:28
  the raw JSON response that comes back. If you
• 9:30
  get a clean typed answer back with a probability
• 9:32
  score, you know the plumbing is solid. So what
• 9:35
  is the absolute biggest mistake people make when
• 9:38
  first bridging Claude and Jev? Oh, overengineering
• 9:41
  right out of the gate. People write these massive,
• 9:45
  intricate routing scripts before they even check
• 9:47
  if their API key connects to the right endpoint.
• 9:49
  Start small, verify the basic connection, then
• 9:52
  build your complex routing logic. That is the
• 9:55
  golden rule of any new API integration, really.
• 9:58
  Now that the wiring is secure and tested, we
• 10:00
  can actually unleash this system. This is the
• 10:03
  fun part. This is where we scale. We move from
• 10:06
  just saving a few pennies locally to fundamentally
• 10:09
  restructuring how applications operate. Let's
• 10:12
  explore the three distinct levels of using Jev
• 10:15
  with Cloud Code. So level one is upgrading your
• 10:18
  own personal Cloud Code workflow on your machine.
• 10:22
  This level is entirely about automatic model
• 10:25
  routing. Right. So in this setup, Jev essentially
• 10:28
  becomes the traffic cop for your tasks. Exactly.
• 10:31
  It analyzes an incoming prompt or task and then
• 10:34
  it decides where that task should go. Right.
• 10:37
  Does it go to the faster Haiku 5 .5? Does it
• 10:40
  need the balance of Sonnet 5 .5? Or does it actually
• 10:44
  require the deep reasoning of Opus 5 .5? Yeah,
• 10:48
  because why on earth would you send a simple
• 10:50
  file path search to Opus? Right. It just burns
• 10:53
  through your compute budget for absolutely no
• 10:55
  reason. I actually ran a test on this locally
• 10:57
  myself. My routing setup cut my token costs by
• 11:01
  roughly 70%. Wow, 70%. Yeah, because most of
• 11:06
  my daily tasks just do not need Opus. If I ask
• 11:10
  Claude to find a specific component file, Jev
• 11:12
  intercepts it. It routes it instantly to Haiku.
• 11:15
  It saves me money and time. And level one also
• 11:18
  revolutionizes skill selection, which is huge.
• 11:21
  Let's say you're running a massive library of
• 11:22
  like 145 different developer skills in your workspace.
• 11:26
  OK. Normally, Claude has to read through that
• 11:28
  entire list every single time just to find the
• 11:31
  right tool, and that burns heavy compute time.
• 11:33
  The sources show a really stark comparison here
• 11:35
  on that exact point. With 145 skill library,
• 11:39
  Jeff found the correct skill in just five seconds.
• 11:42
  Opus 5 took 30 seconds to do the same exact thing.
• 11:46
  Over an eight -hour workday, that speed difference
• 11:48
  is completely transformational. It really is.
• 11:50
  It adds up so fast. And the best part is you
• 11:52
  don't even have to build the router from scratch.
• 11:55
  Really? Yeah. TypeSafe actually offers a completely
• 11:57
  free dev router over on OpenRouter. It's a ready
• 12:00
  -made option to handle all this traffic cop logic
• 12:02
  for you. That's incredibly useful. Which brings
• 12:05
  us to level two. Yeah. This is where the economics
• 12:09
  get truly wild. We are talking about high -volume
• 12:12
  business automation. There is a very specific
• 12:15
  formula for success at level 2. You need high
• 12:18
  volume combined with a simple repeated question.
• 12:21
  High volume plus a simple question. Exactly.
• 12:24
  Give us a concrete example of that formula in
• 12:26
  action. Sure. So imagine your company receives
• 12:29
  thousands of incoming sales emails every single
• 12:32
  day. Okay. A huge inbox. Right. And you need
• 12:34
  to classify every single one of them. Is this
• 12:36
  a warm lead or a cold lead? Jev can categorize
• 12:39
  each one in milliseconds. Wow. Or think about
• 12:43
  fintech. You know, you could run anomaly detection
• 12:45
  on millions of invoices to spot fraud. Or you
• 12:49
  could route thousands of angry customer support
• 12:51
  tickets to the precise department they need instantly.
• 12:54
  Whoa. Imagine scaling to a billion queries. The
• 13:00
  cost savings there would be absolutely astronomical.
• 13:03
  literally changes the entire business model for
• 13:05
  certain software companies. I mean, when complex
• 13:08
  decisions cost fractions of a single penny, you
• 13:11
  can automate things that previously required
• 13:13
  a whole team of humans. It used to have to pay
• 13:15
  people just to sit there and click yes or no
• 13:17
  all day long. Which naturally pushes us toward
• 13:20
  level three, building entirely new apps and novel
• 13:24
  product features. This is the true frontier of
• 13:27
  the technology, I think. We are talking about
• 13:29
  deploying features that were simply just too
• 13:31
  expensive to run at scale before Jev existed.
• 13:34
  Semantic Media Search is a brilliant example
• 13:36
  of this. Oh, totally. Because usually you search
• 13:39
  your computer by file name. You type the word
• 13:41
  Claude, and you get files literally named Claude.
• 13:44
  Right. But semantic search finds things by their
• 13:47
  underlying meaning. Yeah. So instead of relying
• 13:49
  on manual tags, Jev can instantly classify thousands
• 13:52
  of images by what is actually inside them. Right.
• 13:55
  It understands the visual context without you
• 13:57
  ever typing a description. Another amazing level
• 14:00
  three example is always on background tools.
• 14:03
  There is a new Chrome extension called Unclutter.
• 14:06
  I love that one. It's so cool. It uses Jev to
• 14:08
  instantly classify page elements as you browse
• 14:11
  the web. It looks at the web page's document
• 14:13
  object model. And it instantly decides, is this
• 14:17
  a pop -up ad? Is this a GDPR cookie banner? And
• 14:20
  it removes them in milliseconds before you even
• 14:22
  scroll the page. And that's key, because a browser
• 14:25
  extension has to be blindingly fast. Right. If
• 14:28
  it slows down the page load even slightly, users
• 14:32
  will instantly uninstall it. Speed is the only
• 14:34
  feature that really matters there. Exactly. And
• 14:37
  Jev makes that real -time processing possible.
• 14:39
  But there is a very important caveat we need
• 14:41
  to address, especially for these level three
• 14:44
  features. Yeah, you're talking about the multimodal
• 14:46
  capabilities. Right. The box answer type we mentioned
• 14:49
  earlier. Right. Right. TypeSafe has officially
• 14:52
  placed those visual features under an opt -in
• 14:54
  danger zone flag. Yeah, they have. They explicitly
• 14:57
  warned developers that multimodal input performs
• 14:59
  significantly worse than text right now. They
• 15:02
  strongly advise against using it for production
• 15:04
  environments. It's just not ready yet. So if
• 15:07
  the image search is so powerful, why is TypeSafe
• 15:09
  actively warning people to keep it in the danger
• 15:11
  zone? Because pure text classification is highly
• 15:14
  reliable and deterministic right now. But image
• 15:18
  classification is still pretty unpredictable.
• 15:20
  The model might hallucinate spatial coordinates
• 15:22
  for a bounding box, you know, or it might completely
• 15:25
  misclassify a visual element. And in a production
• 15:28
  app, that hallucination could just break the
• 15:30
  entire user experience. image testing and prototyping
• 15:33
  and don't rush it to production yet. Exactly
• 15:35
  right. Stick strictly to text analysis for anything
• 15:38
  mission critical for now. We need to take a very
• 15:40
  quick break. Mid -roll sponsor read placeholder.
• 15:43
  And we are back. Let's get into it. Let's take
• 15:46
  a step back and synthesize the overarching theme
• 15:49
  of this deep dive. We are witnessing a fundamental
• 15:52
  architectural shift in how we build. We really
• 15:55
  are. Jeff is not trying to replace the brilliant
• 15:57
  frontier models. Not at all. It's about radically
• 15:59
  optimizing how we use them. It's a true division
• 16:02
  of labor in software engineering. You send the
• 16:04
  repetitive, mundane classification work to a
• 16:07
  model that costs pennies. A model that responds
• 16:10
  in milliseconds instead of seconds. Right. And
• 16:13
  by offloading that basic noise, you reserve the
• 16:16
  really expensive deep reasoning models for the
• 16:18
  tasks that actually require human -like nuance.
• 16:21
  Exactly. You let Claude write the brilliant algorithms
• 16:24
  and you let Jev instantly sort the chaotic inbox.
• 16:27
  It is a division of labor that makes AI workflows
• 16:31
  practical at an enterprise scale. It finally
• 16:34
  makes the economics of AI actually work for normal
• 16:38
  businesses. Yeah, it's Kahneman's System 1 and
• 16:40
  System 2 concept fully realized in software architecture.
• 16:44
  We finally have the fast instincts to pair with
• 16:46
  the slow logic. Which leaves us with a pretty
• 16:49
  provocative thought to consider today. Let's
• 16:52
  hear it. If these System 1 decision models become
• 16:54
  virtually free and instantaneous, We are rapidly
• 16:58
  approaching a world where literally every digital
• 17:01
  interaction can be pre -screened Every single
• 17:04
  email every breaking news notification every
• 17:07
  swipe on your phone all scored filtered and routed
• 17:10
  by AI Before it ever reaches your actual screen
• 17:15
  What happens to our own human system one instincts?
• 17:19
  When an AI is doing all of our fast -thinking
• 17:22
  for us. That is the big societal question. We
• 17:25
  will inevitably have to face next I mean, when
• 17:28
  the software handles all the triage for us, human
• 17:32
  intuition might just atrophy entirely. It's something
• 17:35
  you really have to think about. Thank you for
• 17:37
  joining this deep dive. I highly recommend you
• 17:40
  test Jev on your own Claude code setup. Definitely
• 17:42
  give it a try. Run the basic tests. Compare the
• 17:45
  remarkable speed. Look at the immediate cost
• 17:48
  savings in your dashboard. See the difference
• 17:50
  for yourself. Outro music.