Latest / TeqTalk with Jas / Jev Explained in 14 Minutes | What is Jev and How It Works
Transcript
- Teqfocus: For the last few years, anytime software needed to be smart, we did the same thing. We reached for an LLM. And look, that makes sense for the big stuff writing, summarizing, reasoning, coding. But we have also started using LLMs for tiny little jobs. Which support queue should this ticket go to? Is this payment specious? Which tool should this agent call next? Should we escalate this? Well, on September 15th, a San Francisco lab called TypeSafe AI came out with a model that reads the situation. It looks at the options you give it and it hands back a decision with how sure it is. And that's called JEV. I am Jess Carr and welcome to Tech Talk. By the end of this video, you will really have a simple test for which decisions in your workflows actually need an LLM and which ones probably never did. Let's get dive into it. So what is JEV really? Here is the easiest way to think about it. You give it some information, you ask it a question, and you tell it which answers you will accept. Jeff picks one and tells you how confident it is. That's it. No paragraph, no explanation, just an answer your software can use right away. TypeSafe has a fancy way of saying this. Unstructured state in typed probabilistic decisions out. But they also have a simpler name for it. And honestly, I like this one way more. They call them SMARDF statements. Now, TypeSafe calls Jeff a System 1 model. And once you get what that means, the whole thing clicks. The name comes from Daniel Kahnman. He described two ways we think. System 1, which is fast and gut level. System 2, which is slow and careful. Let me give you two questions and you will feel the difference. Plan a fun weekend trip for my family. You have to sit with that one. Where do we go? What's the budget? What the kids enjoy, this is system two. Now, should we carry an umbrella now? Yes or no. That's a decision. You do not need to think it through. You just look at the sky and decide. That's system one. And that second kind of question is exactly what Jev is built for. Quick decisions made over and over where you already know what the possible answers are. So no, Jeff isn't trying to copy the human brain. System one is just type safe's way of saying. Fast judgments for machines. But hang on, you might be thinking, can't an LLM already do this? Great question. And let me answer it with one example. And we are going to stick with this example for the rest of our video. A customer writes into support. I was charged twice for my order and I need this fixed today. Now say I ask an LLM. Read this customer's history and write a reply that calms them down. I do not know what the perfect reply looks like. It depends on the tone, the history, what we can offer. I need the model to understand the situation and write something. And that is an LLM job. But before anyone writes anything, the support system has a much simpler question. Which team should handle this? Billing, technical, shipping, or account? So here I already know the possible answers. I just do not know which one is the right. That is a Jeff job. So here is the simplest way to put it. An LLM generates an answer, Jeb selects an answer. Now, to be fair, LLMs can absolutely sort and route things. We do it every day. But software usually doesn't need a paragraph. It just needs to know what to do next: approve or reject. Send it here or there. Keep going or stop. So when does JEV really shine? When a decision has to be made very fast and made thousands of times a day. Picture a company getting 100,000 support requests a day. Every single one needs a few quick calls. Is this billing, technical, account access, sales, or fraud? How urgent it is? Does a human need to see it? You do not need a big language model writing a paragraph for each one of those. You need a fast answer picked from those shortlists with a confidence score. And that's exactly the kind of work Jeff is built for. So what does it actually look like when your software asks Jeff something? Let's open one up. You send Jeff two things. First, the thing you wanted to look at. For us, that's the support ticket. Second, your question about it. And Jem only understands three kinds of questions. TypeSafe calls them NOL, Choice and Score. Now is a yes or no. Is this customer asking for a refund? Second is choice, which means pick one. Which team should it take to? Billing, technical shipping or account? Third is score, means it gives a number. How urgent is this? From 1 to 5. And here is the clever part. You do not ask one and then wait and then ask the next one. You send all three together and Jeb answers all three at the same time. Plus, every answer tells you how sure Jeb is. It doesn't just say billing, it says billing and I am 95% sure. Remember that 95%. It's going to matter a lot later. Because a confidence score is only useful if it is honest. And whether it's honest comes down to how Jeff really was trained. So most chat models were trained with something called RLHM, reinforced learning from human feedback. Basically, the model learns to write answers that people like. And TypeSafe's argument is that this is exactly why chat models tend to sound overconfident and can be unreliable sometimes. And it is why they still need a human watching over them. So TimeSafe builds a different training method and they call it RLCD, which is reinforcement learning for calibrated decisions. And the goal is really very simple. When the model says 90%, it should actually be right about 90% of the time. There is a second difference, and it is about speed. The chat models write one piece at a time. Because it could be writing anything, a sentence, or a thousand lines of code. Jeff doesn't do that. The possible answers are set before you even ask, so it checks them all in one go. That's a trade-off. Jeff cannot do what an LLM does, but for the jobs it's built for, it doesn't need to. Let's go back to our support desk and watch this actually work. Remember those hundred thousand support requests? Here is one of them. I was charged twice for my order and I need the fix today. Before anyone replies, the system has to answer six quick questions. Which team? How urgent? Is the customer upset? Is it a refund request? Can we refund automatically? Does a person need to step in? Those are in six essays. They are six decisions. And each one is a noun, a choice, or a score. Now multiply that. Six decisions on every ticket, 100,000 tickets a day. That's 600,000 decisions. At that volume, every second and every cent really adds up. And that's Jeff's sweet spot. Fast, repetitive, high volume decision. So if Jeff is making all these decisions, does the LLM just disappear? Well, not at all. It moves to where it is actually strongest. Because right now, in a lot of companies, one LLM does everything. It reads the request, sorts it, decides what to do, picks a tool, and writes the reply. And it works. But that's a lot of very different kinds of jobs on one model. So split it up. The LLM reads and understands, Jeff makes the quick decisions, regular software follows the rules and does the work, and then the LLM comes back at the end to write that calm reply to our customer. And if Jeff isn't sure about something, the ticket goes to a real support agent. Now of course, splitting things up only makes sense if the numbers work. So let's talk about money. And speed now. Let's start with the price. JEV costs about 0.042 per million input tokens, and there is no charge for the output at all because it's not writing any text. On speed, TypeSafe says a full response comes back in roughly 70 to 500 milliseconds, and that is well under a second. On their own tests, TypeSafe says JEV is about 194 times faster and 440 times faster. Cheaper than any LLM. But here is what I appreciate. Type safe themselves say those numbers are at the high end of what you would see in the real world. And their own team build those tests. So treat them as vendors' number, not a guarantee. Well, they are also upfront that they cannot yet prove this early pricing isn't subsidized. Though they expect prices to go down over time, not up. So if you're a business looking at this, you can ask. For this one specific decision, does it hit our accuracy bar cheaper and faster? But honestly, price and speed aren't even the most important part. Remember that 95% I asked you to hold on to? That's where it matters. Imagine two models, both are right, 92% of the time. Model A sounds super confident every single time, even when it is very strong. Model B makes mistakes too, but when a case is hard, its confidence drops. Which one would you trust to run on its own? Well, Model B, because you can see when it is unsure. Type safe puts it really important. A model that is right 95% of the time is still useless for automation if it cannot tell you which 5% it is unsure about. So you build rules around that confidence. Let's go back to our ticket. Can we refund automatically? If JEV is 99% sure, the system just issues a refund. If it is around 70%, a smarter model double checks. And if it is 52%, basically a coin flips, it goes straight into a support agent. Now those numbers are just examples. Every company sets their own limits based on how much risk it can live with. But the idea is the model makes a call and you decide how sure it needs to be before anything happens. And this gets really interesting when the decision is about AI itself. Like which model should handle this request? Because most companies now use more than one AI model. A big, powerful one for hard problems, a small, cheap one for high-volume work, maybe an internal work for sensitive data. So now you have got one new problem. Who decides which model gets which request? You could hard code the rules, you could ask another LLM, or you could use a decision model. And routing is actually one of the jobs. Type safe itself lists for Jeb. A simple question goes into a cheap model, a hard reasoning problem goes to the powerful one, anything sensitive stays in house. So the decision layer figures out how much intelligence a problem needs before you actually pay for it. And that brings me to my favorite part of this whole story: the name. Jeb is named after William Stanley Jevons, a British economist from 1800s. And he noticed something that sounds Backwards. When steam engines got more efficient, Britain didn't burn less coal. It burned way more. Because once coal got cheaper to use, people found a hundred new things to use it for. That is what we now call the Jevon's paradox. And time safe is betting the same thing happens with AI. Think about our support desk. Today you would never pay an LLM to check every single message for fraud or tone or any refund request. Too slow, too expensive, so we ration it. But if each decision cost almost nothing and takes a fraction of the second, you would check everything, every workflow setup, every tool call, every transaction, every document, every agent action. So the bigger story might not be cheaper AI decisions, it might be a lot more AI decisions. But before we get too excited, let's be honest about what Jeff cannot do. First, it's new, it's still under early access. And there is just isn't much independent testing yet compared to the big LLMs. Second, Jeff never explains itself. You get a number, not a reason. And TypeSafe hasn't shared a detail how the model is built. So you can check later whether its confidence was right, but not why it chose what it chose. And third, TypeSafe says Jeff cannot hallucinate. But let's be clear on what that means because you said the possible answers Jeff cannot make up an option you never allowed. And if you say A, B or C, you'll get A, B or C. But it still can pick B when the right answer was A. So remember this, a neat answer isn't always the right answer. So if you're testing model like this, check four things. Accuracy, confidence, cost and speed. And the test where it fails using your own real data. Let me bring this all together. What makes JEV interesting to me isn't that it is another AI model, there are plenty of those. It is that it makes us ask a question we probably should have asked a while ago. Does every smart step in a workflow really need an LLM? I think we are heading toward a more specialized AI setup, LLMs where we need language and deep thinking, decision models where we need fast sorting, scoring, routing, regular software where we need rules and control. And people where the risk is high and someone has to be accountable. So next time you look at an AI workflow, start with a simpler question. Does this step need to generate something or does it need to make a decision? That's a test I promised you at the start and honestly it's the simplest way to understand why JEP matters.