Latest / AI Fire Daily / #133 Robin: Jev Classified 1,000 Emails in 6 Seconds for $0.09 - Here’s Where It Actually Makes Sense

Transcript

#133 Robin: Jev Classified 1,000 Emails in 6 Seconds for $0.09 - Here’s Where It Actually Makes Sense · AI Fire Daily

• 0:00
  1 ,000 emails, seven classification rules, just
• 0:03
  six seconds, and it costs exactly nine cents.
• 0:07
  The old way took five minutes and cost 62 cents.
• 0:11
  Yeah, it's just, it's a staggering difference
• 0:13
  when you look at the raw math. Welcome to today's
• 0:15
  Deep Dive. We are dissecting type safe GEV today.
• 0:19
  It's a radical new AI model. Our mission is to
• 0:22
  figure out exactly how it achieves those numbers.
• 0:24
  Right. We'll explore how it completely abandons
• 0:27
  conversational chat. It focuses entirely on high
• 0:30
  speed decision making. Right. And we're going
• 0:32
  to examine how that plays out in the real world.
• 0:34
  We'll look at massive batch processing and we'll
• 0:37
  look at real time use cases at the edge, you
• 0:39
  know, from filtering your ex feeds to running
• 0:41
  Bitcoin bots. And most importantly, we're going
• 0:44
  to reveal how to rigorously test its accuracy.
• 0:47
  Definitely. You need to know how to validate
• 0:49
  this for your own workflows before you just let
• 0:51
  it loose. Because speed doesn't matter if it's
• 0:54
  making the wrong decisions, especially in milliseconds.
• 0:57
  Exactly. Right. To understand how processing
• 0:59
  1 ,000 emails in six seconds is even possible,
• 1:03
  we first have to strip away what we assume AI
• 1:05
  actually does. Right. We need to look at what
• 1:08
  JEV fundamentally is under the hood. It's a complete
• 1:11
  departure from the generative models we're used
• 1:13
  to. JEV refuses to write long text. It will not
• 1:18
  converse with you. It's purely a decision -making
• 1:20
  model from TypeSafe AI. It's built for one highly
• 1:23
  specific job. Exactly. Giving you structured
• 1:26
  outputs based on strict criteria. And it achieves
• 1:29
  that because of how it was trained. The documentation
• 1:32
  mentions a specific architecture called RLCD.
• 1:35
  Yeah, RLCD. Which stands for Reinforcement Learning
• 1:38
  for Calibrated Decisions. But let's demystify
• 1:41
  that for a second. Sure. How would you define
• 1:43
  RLCD in plain English? It's AI that grades its
• 1:46
  own homework to give measurable confidence scores.
• 1:49
  I like that. That measurable confidence isn't
• 1:51
  the entire point. Because it isn't generating
• 1:53
  paragraphs. It's handling three very specific
• 1:56
  types of decisions. Right. The source gives us
• 1:59
  some really clear examples here. First, you have
• 2:01
  the simple yes -no decision. Okay. So imagine
• 2:04
  you get a massive influx of customer support
• 2:06
  tickets. Sounds like a nightmare. It is. Instead
• 2:09
  of a human reading them, Jev just asks a binary
• 2:11
  question like, is this ticket urgent? And it
• 2:13
  returns a strict yes or no. Right. But critically,
• 2:17
  because of that calibrated training, it outputs
• 2:19
  a confidence score too, like 99%. Then you have
• 2:23
  the pick one decision. You have a vague ticket.
• 2:26
  Which department should actually handle it? Exactly.
• 2:29
  Billing, technical, or general. Jev analyzes
• 2:31
  the context. It picks one option from your predefined
• 2:34
  list and routes it directly to the technical
• 2:36
  team. It doesn't explain its reasoning unless
• 2:39
  you force it to. It just gives you the route.
• 2:41
  Yeah. Finally, there's the score decision. You
• 2:44
  feed it a transcript and you ask, how frustrated
• 2:47
  is this customer? Jeff reads the text and instantly
• 2:50
  assigns a frustration level, say 8 out of 10.
• 2:53
  Right. And because it outputs this data cleanly,
• 2:56
  like in a JSON file, your software can instantly
• 2:59
  trigger an alert. To a manager or someone. That
• 3:02
  predictable structure is exactly why it's so
• 3:04
  fast. Oh, yeah. The company claims GEV is 20
• 3:07
  to 200 times faster than traditional models.
• 3:10
  It's wild. And it's 40 to 400 times cheaper.
• 3:14
  They even offer free output tokens, which is
• 3:18
  kind of hard to believe. It completely changes
• 3:20
  the unit economics of AI, but, you know, we have
• 3:22
  to be realistic about the trade -offs. Right.
• 3:25
  It's incredibly fast, but it is not a magic bullet.
• 3:28
  Right. It has a surprisingly small context window.
• 3:31
  We're talking only 64k tokens. Yeah, tiny. When
• 3:35
  you consider that some leading chat models support
• 3:38
  a million tokens, Jeff looks pretty limited.
• 3:41
  Very limited. It can't summarize a 500 page legal
• 3:45
  document. No. It can't discover overarching themes
• 3:47
  across a giant data set on its own. It's completely
• 3:50
  blind to the macro picture. Exactly. It evaluates
• 3:53
  the exact piece of data you hand it and nothing
• 3:56
  else. I like to compare Jev to a hospital triage
• 4:00
  nurse. OK, I like that. A triage nurse isn't
• 4:03
  there to perform complex surgery. They aren't
• 4:05
  going to write out a detailed multi -month treatment
• 4:08
  plan for you. No, of course not. They just evaluate
• 4:10
  your vital signs instantly. They figure out how
• 4:13
  critical you are and route you to the correct
• 4:15
  room so the doctors can take over. That's a great
• 4:17
  analogy. But remember, even a triage nurse has
• 4:20
  years of broader medical context and memory.
• 4:24
  True. Jev has that tiny 64K window. It has a
• 4:28
  very short memory. It's a pure, stateless gatekeeper.
• 4:32
  Which is actually a relief. Because honestly,
• 4:34
  I still wrestle with prompt drift myself. Oh,
• 4:36
  don't we all? You try to make regular, intelligent
• 4:39
  chat models do this strict, repetitive sorting,
• 4:42
  and they constantly get distracted. Yeah, they
• 4:45
  start over -explaining their reasoning. Or hallucinating
• 4:48
  extra categories you didn't even ask for. Because
• 4:50
  they're desperate to chat. Exactly. That's what
• 4:52
  they were optimized to do. Exactly. Jeff doesn't
• 4:54
  have that impulse. So it's purely a router, not
• 4:57
  a creator. That's exactly the framework. The
• 4:59
  modern workflow is all about combining models.
• 5:01
  Okay. How so? You let Jeff classify, say, 5 ,000
• 5:05
  YouTube comments in seconds. It identifies the
• 5:08
  50 comments that are actually angry and need
• 5:10
  a customer service reply. Right. Then your system
• 5:12
  sends only those 50 flagged comments to an expensive
• 5:15
  chat model, and that model drafts the thoughtful,
• 5:18
  empathetic responses. Exactly. It categorizes
• 5:20
  the data so the chat models can write the responses.
• 5:24
  You stop paying top dollar for an intelligent
• 5:26
  model to read thousands of comments that just
• 5:28
  say, great video. Because Jev is a pure router,
• 5:32
  its true power unlocks when you feed it massive
• 5:34
  walls of messy data. Oh, absolutely. That batch
• 5:37
  classification is where the unit economics get
• 5:39
  genuinely absurd. Let's dig into that benchmark
• 5:42
  test from the source. They took a data set of
• 5:45
  1 ,000 emails. Right. They needed to sort them
• 5:48
  across seven different classification rules simultaneously.
• 5:51
  They were looking for invoices. They were hunting
• 5:53
  for brand deals, filtering out scams, checking
• 5:57
  for urgency, and evaluating sponsor fit. It's
• 6:01
  a pretty complex sorting task. It is. So they
• 6:04
  ran this through a traditional model called Luna
• 6:06
  first. Luna took about five minutes to process
• 6:09
  all 1 ,000 emails. And the total cost was 62
• 6:12
  cents. Honestly, compared to a human doing it,
• 6:15
  5 minutes and 62 cents is already incredible.
• 6:17
  True. But then they swapped in Jeff. The initial
• 6:21
  setup took about 70 seconds and cost 9 cents.
• 6:24
  But here's where the mechanics really matter.
• 6:27
  They optimized the backend for parallel processing.
• 6:30
  Right, and this is crucial. Because Jeff is just
• 6:33
  giving a single structured output, it isn't generating
• 6:35
  sequential text. Exactly. Every word doesn't
• 6:37
  depend on the last one. Right. So you can fire
• 6:40
  hundreds of queries at the API at the exact same
• 6:43
  time. And when they did that, they processed
• 6:46
  all 1 ,000 emails in six seconds. While keeping
• 6:49
  the cost at exactly nine cents. That is roughly
• 6:52
  50 times faster. And seven times cheaper. Just
• 6:56
  by switching to a model built strictly for routing.
• 6:59
  They ran a simpler receipt hunting test, too.
• 7:01
  They had an inbox of 1 ,000 emails. And they
• 7:04
  just needed to find the invoices and receipts
• 7:06
  hidden inside. That task took Jev... four seconds.
• 7:10
  It cost five cents. It successfully identified
• 7:14
  237 matches and instantly rejected the other
• 7:18
  763 emails. When you abstract that capability,
• 7:22
  the broader applications are just endless. Oh
• 7:25
  yeah. Think about content creators managing chaotic
• 7:27
  communities. You can classify a thousand YouTube
• 7:30
  comments in five seconds for a nickel. You can
• 7:33
  group them by sentiment. You can extract video
• 7:36
  ideas that your audience is asking for. You can
• 7:39
  gauge the difficulty of questions to see if they
• 7:41
  need a technical response or just a basic one.
• 7:44
  It's the same for internal business workflows.
• 7:46
  You could dump all your scattered company data
• 7:49
  into it. It's less like building a Lego castle
• 7:52
  and more like stacking Lego blocks of data, dumping
• 7:55
  a massive bucket of mixed Legos through a sieve.
• 7:59
  That instantly sorts them by color into separate
• 8:01
  bins. Exactly. So you can actually start building.
• 8:04
  Yeah, that's exactly what it feels like. It just
• 8:06
  structures the chaos instantly. But here's the
• 8:10
  insight that really caught my eye. Okay. The
• 8:13
  real magic isn't just labeling the data. Okay.
• 8:15
  It's spotting systemic patterns without needing
• 8:17
  deep analysis. How do you mean? What kind of
• 8:19
  patterns? So imagine you have 500 meeting transcripts
• 8:23
  from the last quarter. You don't want a summary
• 8:25
  of every meeting. No, that's too much. You just
• 8:27
  asked you have one binary question. Does the
• 8:30
  end of this transcript contain clearly assigned
• 8:32
  next steps? Okay, a very simple classification
• 8:35
  task. Right. And if JEV flags that 70 % of your
• 8:38
  company's meetings lack clear -neck steps, you
• 8:40
  didn't just label data. Oh, wow. You just found
• 8:43
  a broken company process. And you did it for
• 8:46
  pennies in seconds. Exactly. You bypassed the
• 8:49
  need to have a generative model deeply analyze
• 8:52
  the cultural subtext of the meetings. You just
• 8:55
  found the structural flaw. The cost savings at
• 8:58
  scale are hard to wrap your head around. During
• 9:00
  this broader testing phase, the developers processed
• 9:03
  nearly 20 ,000 JEV requests. And the total cost
• 9:06
  for 20 ,000 decisions was just 85 cents. Yeah.
• 9:10
  Whoa. Imagine scaling to a billion queries. But
• 9:13
  I have to push back a little here. Can it really
• 9:15
  evaluate creative value? Or is it just hunting
• 9:18
  keywords? Good question. Like, if I ask it to
• 9:21
  evaluate the hook strength of a video, isn't
• 9:25
  that incredibly subjective? How does a model
• 9:27
  that refuses to think deeply grasp a qualitative
• 9:31
  human concept like a good hook? It's a very fair
• 9:35
  question, but you have to separate text generation
• 9:38
  from semantic comprehension. Okay. It isn't blindly
• 9:41
  hunting for the phrase watch until the end. Jev
• 9:44
  is built on a modern transformer architecture.
• 9:46
  It still possesses a deep, nuanced understanding
• 9:49
  of language and human context. So it understands
• 9:53
  the tension in the script. It just won't write
• 9:55
  an essay about it. Exactly. It recognizes the
• 9:58
  pattern of high tension or curiosity inducing
• 10:01
  phrasing. It just maps that semantic understanding
• 10:04
  to a numeric score. Right. It can identify quotable
• 10:07
  moments or flag clips that are worth posting
• 10:10
  on their own. It makes those qualitative judgments
• 10:13
  accurately. It actually grasps context, making
• 10:16
  qualitative judgments like hook strength without
• 10:19
  deep analysis. Yes. It's exceptionally fast,
• 10:22
  but it is not blind. So batch processing old
• 10:25
  data obviously saves money, but what happens
• 10:27
  when decisions must be made in the exact millisecond
• 10:30
  the data is created? Right, speed at the edge.
• 10:33
  Exactly. We're talking about speed at the edge.
• 10:35
  This is where the latency of a smaller model
• 10:37
  really shines. Let's look at the first real -time
• 10:40
  use case from the sources, an XFeed filter. Scrolling
• 10:43
  through Twitter or X is usually a chaotic flood
• 10:47
  of information. That's exhausting. So someone
• 10:50
  built a Chrome extension powered directly by
• 10:52
  the Jev API. It intercepts the posts as they
• 10:55
  load and labels every single one automatically.
• 10:57
  And it does this before you even finish reading
• 11:00
  the tweet? Yeah. That requires some serious millisecond
• 11:03
  latency to check the label into the browser that
• 11:05
  fast. It does. The labels appear seamlessly on
• 11:08
  your screen, classifying the feed into three
• 11:10
  buckets. First, you have breaking news. These
• 11:13
  are the important time -sensitive updates worth
• 11:15
  checking immediately. Second is golden nuggets.
• 11:18
  Right. High -value insights, deep threads, things
• 11:20
  you actually want to save or learn from. And
• 11:22
  third is AIslop, the engagement bait and low
• 11:25
  -value content that you can just completely skip.
• 11:28
  It fundamentally changes how you consume information.
• 11:30
  You're pre -filtering the chaos seamlessly. Totally.
• 11:34
  But the second real -time use case is even more
• 11:36
  aggressive. It's a Bitcoin paper trading bot.
• 11:39
  This one operates on a totally different frequency.
• 11:43
  It pulls in live market data and analyzes the
• 11:45
  state of the order book every single second.
• 11:49
  I have to admit, I'm pretty skeptical here. Yeah,
• 11:52
  understandable. Predicting Bitcoin. We're talking
• 11:55
  about a nine -cent model doing financial analysis.
• 11:57
  How is it predicting outcomes like up, down,
• 12:01
  or unclear without just hallucinating? Well,
• 12:03
  it's not writing a thesis on macroeconomics.
• 12:06
  It's taking a highly structured array of numerical
• 12:08
  market data, price action, volume, and running
• 12:11
  it through a strict classification prompt. It
• 12:14
  outputs a directional probability. And because
• 12:17
  of that RLCD training, it gives a confidence
• 12:20
  score. So the bot uses that specific confidence
• 12:24
  score to decide if it should execute a simulated
• 12:26
  buy or sell order. Exactly. And because the model
• 12:30
  is so cheap, it runs continuously, evaluating
• 12:33
  every second for about $2 a day. It's a fascinating
• 12:37
  proof of concept for edge computing, but we need
• 12:41
  to heavily caveat this crypto bot. Absolutely.
• 12:44
  It is strictly an unvetted proof of concept.
• 12:48
  A cheap AI predicting price movement absolutely
• 12:50
  does not guarantee you a profit. No way. Especially
• 12:54
  in high -frequency trading, your transaction
• 12:56
  fees are going to completely dwarf whatever you
• 12:58
  saved on API costs. The fees will eat you alive.
• 13:02
  It's a demonstration of processing speed, not
• 13:04
  financial advice. Right. We're seeing similar
• 13:06
  experiments with browser and gaming automation,
• 13:08
  too. Jev is heavily limited in those environments,
• 13:12
  though. It can observe the screen state and make
• 13:14
  the decision. But it cannot physically act. Right.
• 13:17
  It can't type into a text box. It can't move
• 13:20
  a mouse or control a game character. So in these
• 13:23
  real -time setups, Jev is basically the brain,
• 13:26
  not the hands. Exactly. Jev looks at the DOM.
• 13:30
  or the screen, and decides what the very next
• 13:32
  action should be. But you still have to wire
• 13:34
  it up to an external automation tool, like Puppeteer,
• 13:38
  to actually execute the click or make the trade.
• 13:41
  Right. JAV is just a fast decision layer. Another
• 13:44
  tool must execute the action. Exactly. We'll
• 13:47
  be right back with the rest of the deep dive.
• 13:49
  Just a quick break. Sponsor. Hey, listeners.
• 13:52
  Let's take a quick break to thank today's sponsors.
• 13:55
  We are incredibly grateful for their support
• 13:57
  in keeping this deep dive running smoothly. Be
• 13:59
  sure to check the description for more details
• 14:01
  on how their tools can optimize your daily operations.
• 14:04
  We've talked a lot about speed and cost, but,
• 14:07
  frankly, speed and cost do not matter at all
• 14:09
  if the AI is just confidently wrong. Right. Since
• 14:12
  JEV is blind to a larger context, you can't just
• 14:15
  trust it blindly with your business. If it confidently
• 14:18
  routes a furious VIP client to the spam folder
• 14:21
  in six milliseconds, you've just automated a
• 14:24
  disaster. Which brings us to the implementation
• 14:26
  phase. Before you deploy this, we have to talk
• 14:28
  about testing accuracy. Definitely. You can't
• 14:31
  just flip a switch and hope it understands your
• 14:33
  specific business roles. Step one is testing
• 14:36
  against a golden dataset. You need to gather
• 14:38
  exactly 100 real test cases from your own data.
• 14:42
  Like what? These could be past customer support
• 14:44
  tickets, old sales leads, or meeting transcripts.
• 14:48
  And you, the human, manually label the correct
• 14:51
  answers for each one of those 100 cases. Right.
• 14:55
  That builds your ground truth. Then you run those
• 14:58
  exact same 100 cases through JEV, but you don't
• 15:02
  stop there. You also run them through a massive
• 15:04
  expensive model like GPT -4 or Claude. Then you
• 15:08
  line the results up side by side. You evaluate
• 15:10
  the total accuracy, the total latency, and the
• 15:14
  final cost. Yeah. And this is where you have
• 15:16
  to make a hard business decision about trade
• 15:18
  -offs. Let's say Jev gets 95 out of 100 correct.
• 15:21
  OK. And the massive expensive model gets 97 out
• 15:25
  of 100 correct. As the user, you have to decide,
• 15:28
  does the massive cost and time savings of Jev
• 15:32
  justify taking a 2 % drop in accuracy? Exactly.
• 15:35
  For sorting YouTube comments? Absolutely. For
• 15:38
  routing critical legal documents? Maybe not.
• 15:41
  That's why step two is to start small. Don't
• 15:44
  rebuild your entire infrastructure on day one
• 15:46
  based on a benchmark. Build a simple internal
• 15:49
  dashboard first. Use it to automatically classify
• 15:52
  incoming emails, just for your team to observe.
• 15:55
  Yeah. Let Jeff handle the repetitive decisions
• 15:57
  in the background quietly. You measure those
• 16:00
  real -world results first. You only wire up your
• 16:02
  complex automated replies once you actually trust
• 16:05
  the routing. I do have a mechanical question,
• 16:07
  though. Why exactly 100 test cases for the golden
• 16:11
  data set? Is that just an arbitrary number? It's
• 16:13
  the statistical sweet spot for this kind of evaluation.
• 16:17
  Really? Yeah. It provides a large enough sample
• 16:19
  size to confidently calculate a baseline percentage.
• 16:22
  You avoid the noise of a tiny sample. But it
• 16:26
  isn't overwhelmingly tedious for a human to build
• 16:28
  manually in an afternoon. It offers a statistically
• 16:32
  practical baseline to measure true accuracy before
• 16:35
  scaling up. Exactly. It saves you from building
• 16:37
  a complicated, brittle system before you even
• 16:39
  know if the prompt works. Pulling back from the
• 16:44
  technical implementation for a second, let's
• 16:47
  look at the broader shift happening here. It's
• 16:48
  huge. This fundamentally changes how we interact
• 16:51
  with data. We are rapidly moving away from relying
• 16:54
  entirely on massive do -everything models. Right.
• 16:58
  Trying to use a frontier chat model to sort a
• 17:00
  CSV file is like using a supercomputer to calculate
• 17:03
  a restaurant tip. That's a great way to put it.
• 17:06
  It's expensive, it's slow, and it's massive overkill.
• 17:08
  Jeb proves that a new blueprint is taking over.
• 17:11
  You pair a highly optimized dirt cheap classification
• 17:14
  model, that's your router, with a deeply intelligent
• 17:17
  conversational model, that's your thinker. It
• 17:20
  isolates the heavy lifting. It's the ultimate
• 17:22
  blueprint for processing massive volumes of information
• 17:24
  efficiently without burning through your budget.
• 17:26
  It lets AI actually run in the background of
• 17:29
  our lives rather than just being a chat box we
• 17:31
  visit. So I want to ask you to examine your own
• 17:35
  workflows today. What messy data are you currently
• 17:38
  drowning in? What repetitive, mind -numbing routing
• 17:42
  task are you currently overpaying for? Whether
• 17:44
  you're paying in wasted hours of your own time
• 17:46
  or in wildly expensive API costs for a model
• 17:49
  that's doing too much. It's worth sitting down
• 17:51
  and looking for those bottlenecks. But before
• 17:54
  we wrap up, I want to leave you with a more provocative
• 17:56
  thought. Okay. Let's build on that XFeed filter
• 17:59
  we talked about earlier. The one instantly sorting
• 18:01
  out the breaking news from the AI slop. Right.
• 18:04
  If micro decisions become functionally free and
• 18:06
  latency drops to zero, We're approaching a world
• 18:09
  where every single piece of digital information
• 18:11
  we encounter is pre -filtered. It's scored, categorized,
• 18:15
  and sanitized for us before it ever even reaches
• 18:17
  our screens. Which sounds incredibly convenient.
• 18:20
  But what happens to us? If the machine always
• 18:23
  cleans the room before we walk in, do we lose
• 18:26
  our human ability to critically evaluate raw,
• 18:29
  messy information for ourselves? That's a profound
• 18:32
  question. If we never see the slop, we might
• 18:34
  just outsource our critical thinking entirely
• 18:37
  to the router. It's something to seriously consider
• 18:40
  as this technology scales into every app we use.
• 18:44
  Thank you for joining us on this deep dive. It's
• 18:46
  been great. Take a hard look at your workflows,
• 18:48
  keep asking the right questions, and we will
• 18:50
  see you next time.