Latest / Agentic Conversations (formally mlops.community) / Getting AI Apps Past the Demo // Vaibhav Gupta // #319

Transcript

Getting AI Apps Past the Demo // Vaibhav Gupta // #319 · Agentic Conversations (formally mlops.community)

• 0:00
  Like full expressive systems like if statements, for loops,
• 0:03
  while loops where you can conditionally call an agent,
• 0:06
  abort to a human worker, Human annotation system, come back to
• 0:10
  it, that kind of system. Wow.
• 0:14
  Yeah, hey everyone, I'm Vaibhav. I'm one of the Co creators of
• 0:22
  Battle. I work for a company called
• 0:24
  Boundary as the founder and we do not really drink coffee.
• 0:29
  Actually no one on the team drinks coffee out here.
• 0:32
  I mostly drink water and every now and then I will have some
• 0:35
  oranges in the morning. All.
• 0:38
  Right dude, what the fuck is Bammel and why do I keep hearing
• 0:42
  all about it? What is bammel?
• 0:47
  I mean Bammel's. I mean what it tangibly is this
• 0:50
  programming language, what it really is.
• 0:52
  This a crazy idea me and my Co founder started out with about
• 0:55
  two years ago, about a year and a half ago now that just started
• 1:00
  with the idea that we can't possibly write prompts using
• 1:04
  Lang chain like that. Just feels wrong ethically.
• 1:10
  'Cause you were doing it a lot. No, I just cause every single
• 1:13
  piece of code that I saw was ugly and like I think so many
• 1:18
  people I talked to are just like they talk about.
• 1:22
  We talked to everyone and like at least everyone I've talked to
• 1:25
  has always said that AI pipeline code feels like a demo code.
• 1:29
  It's like you throw it away and like with this vibe coding thing
• 1:31
  that's going on ever just like vibe code and then ship it And
• 1:34
  like that's, that works if you're starting from zero and
• 1:37
  for front engineering where everything is strongly
• 1:39
  componentized, that works really well.
• 1:42
  But if you're building a back end that has like 15 intricacies
• 1:44
  with software you wrote five years ago, it's not that it's
• 1:49
  bad. It's just like I wouldn't let a
• 1:53
  junior engineer or new hire submit code without reading it
• 1:56
  carefully or like talking about it with them.
• 1:59
  And people just don't have that natural tendency when they vibe
• 2:02
  code. It's not like, so I think if you
• 2:05
  add pull request that process, you get to that.
• 2:07
  But anyway, the point is every single piece of prompted during
• 2:10
  code I've seen even before vibe coding felt like it was vibe
• 2:12
  coded. It looked like it was vibe
• 2:14
  coded. It was a phenomenon before it
• 2:17
  was a name. Yeah, And like, I see, I think
• 2:22
  it's ugly. Code is one of those things that
• 2:26
  I think not a lot of people put a lot of thought into.
• 2:29
  But there's a reason that like every single company in the
• 2:33
  world has a CICD system, which includes A lender, right?
• 2:38
  Because like, it doesn't even matter if your lender is good,
• 2:40
  right or bad. All that matters is you as a
• 2:42
  company have a process and an opinionated system of how you
• 2:45
  write code because that helps you build pipelines that anyone
• 2:49
  can hop into from a different company that just joined you and
• 2:51
  iterate on. And whether it's an AI agent
• 2:53
  writing that code or whether it's a human, you need that.
• 2:56
  And AI pipelines just don't seem to have that level of rigor.
• 3:00
  They just feel like Glob put together, and that felt wrong.
• 3:04
  And that's kind of what inspired VAML.
• 3:07
  OK so you saw a bunch of blank chain examples.
• 3:13
  You thought man we can't Ling chain prompt our way to a better
• 3:17
  future. What besides the linting?
• 3:21
  Was it the linting and ugly code?
• 3:24
  The vibe coding myths about it? What was it that made you think
• 3:29
  that? Take like web development for
• 3:32
  example. When I first started writing
• 3:33
  websites like this was like 2012, 2020, 12 kind of time.
• 3:38
  React didn't really exist. TypeScript didn't really exist.
• 3:42
  The way you wrote a website is you, your back end returned to
• 3:44
  string and like you put it in there and that's what you shit.
• 3:51
  And that was not bad. But there's a reason we all went
• 3:54
  away from that to writing React. Like React does a lot of
• 3:58
  implicit stuff for us automatically that I thought was
• 4:00
  really helpful. Like one, if you forget a
• 4:03
  closing tag, like instead of button, you put slash button at
• 4:06
  the end and you forgot to put slash button, it yells at you.
• 4:09
  You can't even push code. But if you're returning it from
• 4:11
  your back as a string, you can easily forget to put slash
• 4:14
  button. And now your whole website
• 4:16
  breaks building. I remember when I was first
• 4:19
  applying for internships back then, an infinite scrolling news
• 4:24
  feed was a hard interview problem, right?
• 4:29
  And like it was because like interactivity wasn't a thing
• 4:32
  that was built in. You have to do all this code and
• 4:34
  somehow merge jQuery with like a dollar, a bunch of dollar signs
• 4:37
  in your code with your actual HTML.
• 4:40
  I'm like it just looked bad. And then eventually React came
• 4:46
  around and said what if we just pulled the syntax of HTML, pull
• 4:51
  syntax of CSS into your JavaScript and made it native.
• 4:58
  So now not only did it get prettier because it looked
• 5:01
  prettier because it's syntax highlighted button as blue or in
• 5:05
  whatever color you use based on your theme differently from the
• 5:08
  actual content of the button. But it gave you command click,
• 5:12
  it gave you compile time checking, it gave you Dom re
• 5:14
  rendering like the syntax that React exposed allowed it to do a
• 5:19
  lot of novel things. So we kind of have the same
• 5:22
  approach. If you're going to write a
• 5:24
  prompt, you probably don't want to write a bunch of like English
• 5:28
  all three-year code base that is not verifiable in any way.
• 5:32
  So what if we could verify the English?
• 5:35
  What if we could turn the English into something that
• 5:39
  looked a little bit more like code, but preserve the
• 5:42
  flexibility of English? So we do a lot of error
• 5:45
  corrections for you under the hood.
• 5:47
  We do retries and other things minimally as possible.
• 5:51
  They effectively enter your control.
• 5:52
  And most importantly, we can build tooling for you that
• 5:55
  allows you to do things that are probably best seen and not
• 6:00
  heard. But if I were to describe it to
• 6:03
  someone that's just listening, imagine if you could see your
• 6:07
  prompt before you send it out to the model.
• 6:09
  Like a markdown preview. No one writes markdown files
• 6:12
  without actually looking at what it looks like.
• 6:15
  No one ships a website without opening the browser and looking
• 6:17
  what it looks like. Why is it that today, in every
• 6:21
  single framework that exists, the only way to see the prompt
• 6:25
  is to run your code and somehow monitor the web activity that's
• 6:29
  actually being sent out? Because there's like 50 layers
• 6:31
  of abstractions under your prompt.
• 6:34
  So like we just ripped that out and said what if you could just
• 6:36
  see your prompt live in real time in VS Code or Cursor while
• 6:39
  you coded? Small tooling changes like that
• 6:42
  have dramatically helped iteration loops.
• 6:46
  No, you said something there that I would love to understand
• 6:51
  better, which is verifiable English.
• 6:55
  What the hell does that mean? Verifiable.
• 6:58
  Yeah. What does that mean?
• 6:59
  That's a great question. Today everyone treats LLMS like
• 7:04
  this magical box that somehow gets the answer right, and if it
• 7:07
  doesn't get the answer right, it's your fault.
• 7:09
  We kind of like to take a step back and just think of the LLM
• 7:13
  as just a really, really damn good calculator.
• 7:16
  So what is a calculator? A calculator takes 2 numbers and
• 7:20
  an operator like and then based on the operator, it transforms
• 7:23
  those two numbers into a new number, right?
• 7:27
  You have plus sign that adds them.
• 7:28
  You can multiply sign and multiply them, pretty simple.
• 7:31
  Now you might use different tacklers for different jobs,
• 7:34
  like you might use an Abacus if you're in the 1600s or like 12
• 7:39
  hundreds Montessori fishing. Or Montessori.
• 7:44
  Or or a few other things exactly, and then.
• 7:46
  But you might use a scientific calculator today, or you might
• 7:49
  use matplot, MATLAB for something really numerically
• 7:54
  complex, but they're all just calculators under the hood.
• 7:56
  It's just a different calculator has different trade-offs.
• 7:59
  So what if we treat an LM like a calculator?
• 8:02
  You give it some parameters of whatever your data ends up being
• 8:06
  and you just say, I want the LM to guarantee it spits out
• 8:09
  something specific. So if I'm looking for, let's say
• 8:13
  I'm building like a doctor patient transcript, you want to
• 8:17
  ask some audit control before the doctor gave medicine or
• 8:19
  anything that that they have. And all I have is the audio
• 8:23
  transcript. But really what that what that
• 8:26
  looks like is I want a calculator that can take in an
• 8:28
  audio file or a text of the transcript and spit back out a
• 8:33
  list of questions the doctor asked about medical, about
• 8:38
  medicine, which medicine it was and whether or not the patient
• 8:41
  confirmed it. That's a calculator.
• 8:44
  Now I can implement that calculator using a really
• 8:47
  powerful device, which is an LLM.
• 8:50
  And based on the LLM I use, I'll get different levels of
• 8:52
  precision. If I use a small model, I'll get
• 8:55
  good precision, or if I use a big model, I'll get much better
• 9:00
  precision. But it's just a calculator.
• 9:03
  And now the prompt I use kind of is the operator that goes into
• 9:06
  the calculator. So it's like a plus sign.
• 9:10
  And if we view it in that way, then the English in that prompt
• 9:13
  becomes completely verifiable, because all I have to verify is
• 9:17
  that the English I put in will somehow produce the data model
• 9:20
  that I want out. OK.
• 9:24
  So what this I'm, I'm going to show something because I think
• 9:29
  it's going to be easier and then we can talk through it to the
• 9:31
  people that aren't perhaps able to see it.
• 9:35
  So this is like a 15 page Notion dock that we did for like a four
• 9:38
  hour workshop we ran with a bunch of YC founders.
• 9:43
  So I think when we talk about code, it's sometimes easier to
• 9:45
  show code rather than talk about it orally.
• 9:49
  So I'll show a couple snippets of what I mean by leading to
• 9:52
  verifiable English. Nice.
• 9:54
  So in this case what I have is I have a function called class by
• 9:57
  message, and class by message takes in a message which is a
• 10:01
  string. So it could come in from a
• 10:03
  review on Amazon, a tweet, or really anything that I really
• 10:06
  have a text from my girlfriend. It doesn't matter and all I want
• 10:09
  to know is a sentiment of it as one of these three things.
• 10:13
  So I want to build a calculator that can no matter what the
• 10:15
  messages will return one of either positive, neutral, or
• 10:18
  negative. Now the calculator of my choice
• 10:21
  here is going to be opening IGT 4 O, but I could use a llama
• 10:25
  model, I could use a entropic model, Gemini, it doesn't really
• 10:28
  matter. And then like that operator that
• 10:31
  I was talking about is becomes this prompt.
• 10:34
  So now I'm able to verify the English in this prompt very,
• 10:38
  very easily by doing a couple of things because this isn't plain
• 10:42
  English. All I have to do is I can say,
• 10:45
  is this English going to lead this message doing the right
• 10:48
  thing for a couple of different test cases.
• 10:50
  So if my girlfriend sends me a text that says I'm incredibly
• 10:53
  upset with you, it should return negative.
• 10:56
  This calculator should calculate to negative, and when you review
• 11:00
  things like this, it becomes much more easy, much simpler to
• 11:04
  compose them into complex systems and know exactly what
• 11:09
  the contract I've built into the system is.
• 11:11
  Which leads to code that is now usable by static analyzers.
• 11:17
  Code that is like detectable by AI is like understanding when
• 11:20
  things are going wrong. And most important, a strong
• 11:23
  iteration though, because if for whatever reason that message
• 11:26
  returns back as neutral, I know that there's two things that
• 11:29
  could be wrong. It's either I need a different
• 11:32
  calculator, I need to either maybe upgrade the model, click
• 11:34
  Gemini's 2.5 turbo, or I need to go and up, or I need to go and
• 11:39
  update my prompt and change the operate operator that this
• 11:41
  calculator is using. Does that make sense?
• 11:44
  Yeah, yeah, I see that. And now why can't you do that
• 11:49
  today on with the tools that we have in languages that are
• 11:54
  fairly common like Python? Yeah, well, I think there's two
• 11:59
  parts of this. One, not every piece of software
• 12:02
  is Python. So when you go do this in
• 12:06
  Python, that means my sequel database, which is written in C,
• 12:11
  can't support AI pipelines now except by spinning up a Python
• 12:14
  microservice. That just seems wrong.
• 12:17
  All the code in the world that exists in Java there will
• 12:20
  eventually want to use LMS as well because they're, again,
• 12:22
  really powerful calculators. And every computer system, every
• 12:26
  software application will want to take advantage of that
• 12:28
  calculator. So that's like reason #1 like we
• 12:33
  should be able to support all these languages.
• 12:35
  But then you get into really simple things like just like
• 12:39
  what? What would this code even look
• 12:40
  like in Python? So one thing you'll get, you'll
• 12:42
  notice, and what I'm showing you over here is that this code has
• 12:46
  a string which is completely indented.
• 12:49
  Maybe it's easier to see with like how it actually looks in
• 12:51
  static analysis. So you're actually able to
• 12:53
  highlight this in a very easy way, just like in React.
• 12:57
  So I'm looking at Via's code right now, and I'm showing the
• 13:00
  same code that I had over there just written out with the syntax
• 13:04
  highlighting that we offer. And the first thing that you'll
• 13:07
  notice is if I were to do this in Python, then my code will end
• 13:11
  up actually looking something more like this.
• 13:15
  So, and this is like a really, really small NIT that ends up
• 13:22
  happening. Yeah, but it goes back to what
• 13:25
  you were saying earlier on the ugliness of the code and almost
• 13:31
  this readability factor. Exactly.
• 13:34
  So like if I have a prompt now, every single prompt which is a
• 13:37
  long string is going to somehow be de dented all the way back to
• 13:42
  lot line 0. So if it's four layers deep in
• 13:45
  an if statement, suddenly your statement will be pushed all the
• 13:49
  way to the left of your code. And when you scroll, you won't
• 13:52
  know when the if statement exists or doesn't exist simply
• 13:55
  because you have a long string. And most code bases aren't
• 13:59
  designed to have globally available long strings across
• 14:02
  them. And I think that's something
• 14:04
  that people don't really think about is like when I have a lot
• 14:06
  of constant strings, how have we typically used a lot of string
• 14:09
  variables in our code base? We either load them from a
• 14:12
  database or we have them like as very short strings that we put
• 14:17
  in some constants PY file that we then load in.
• 14:21
  But now literally my code, my business logic is embedded into
• 14:25
  these strings and that needs to be Co located in the area that
• 14:30
  they exist in. And even worse, I build these
• 14:34
  strings in really dynamic ways. Sometimes I add if statements to
• 14:38
  conditionally add statements into my string.
• 14:41
  Sometimes I had four loops to dynamically build them.
• 14:44
  Sometimes I want to like I have to change what string I have
• 14:48
  because the model is Gemini 50% of the time and open AI the
• 14:52
  other 50% of the time. That is not what strings are
• 14:57
  designed for. And like that's, that's kind of
• 15:01
  like, and strings aren't designed for that in any
• 15:03
  programming language. So how do you have a string that
• 15:07
  can be both extremely dynamic and extremely flexible?
• 15:11
  The only language that gets close to this is React.
• 15:16
  OK, well it does feel like strings in that regard are
• 15:25
  they're for a different paradigm.
• 15:27
  You never would really use strings to control the computer
• 15:32
  or to to control the outcome. But now with the advent of LLMS,
• 15:37
  you are using strings as a control mechanism and so it is
• 15:42
  part of the code. And so I, I think if I'm
• 15:45
  understanding your philosophy, you're saying more that the
• 15:49
  strings should be treated like a first class citizen in the way
• 15:53
  that we are treating other pieces of code like that.
• 15:57
  Exactly. And and they should be treated
• 16:00
  very like cautiously, almost like like you need caution
• 16:06
  around your strings. If I'm like Delta Airlines and I
• 16:09
  have a bunch of AI chat bots that I'm building on, not just
• 16:11
  one, how do I guarantee that an intern isn't going to come in
• 16:16
  and accidentally forget to add that you work for Delta Airlines
• 16:20
  as a first part of any system message or any chat bot that is.
• 16:25
  That's not a thing we want to leave up to people.
• 16:28
  That's the thing we want to process to Go do, and there's no
• 16:32
  mechanism in any language today to statically analyze the string
• 16:37
  in that way. And so you're saying, hey, this
• 16:40
  kind of process is added in natively with the programming
• 16:46
  language, It's something that only a programming language can
• 16:49
  do. Because I think that's the other
• 16:51
  question, It goes back to this whole idea is why can't we do
• 16:55
  this with the tools we have today and the languages that are
• 16:58
  already out there, right? So we could do it in Python.
• 17:02
  Like what you could do is you could make a special class
• 17:04
  called a prompt string and now everywhere you use strings you
• 17:09
  have to actually use a prompt string, not a string.
• 17:12
  But even if you do did that, you can't de denter strings now and
• 17:17
  you rely on developers everywhere in your company to
• 17:19
  use prompt string instead of string.
• 17:22
  And let's remember, developers are really lazy.
• 17:24
  Well, I'm lazy, so other people are.
• 17:26
  Maybe they don't know, but I'm just lazy.
• 17:28
  Even if I knew I wouldn't do it because I don't want to type in
• 17:31
  extra work. I don't have to.
• 17:32
  And. And then you're exposing
• 17:34
  yourself to risk. Yeah, I can see that.
• 17:35
  Exactly. And I think the whole caveat of
• 17:37
  this whole system is we have to we have to decide where we want
• 17:41
  the burden to be. Do we want the burden to be on
• 17:44
  developers or do you want to be on a process?
• 17:47
  So, and the, and it's, it's fine to put the burden on developers
• 17:50
  sometimes, but I think tangibly what I think the best thing to
• 17:55
  do if an action becomes frequent, then you put the
• 17:59
  burden on the process. And because of these are such
• 18:03
  powerful calculators, it's really powerful to go put it
• 18:06
  into the process rather than a developer.
• 18:09
  And then the other big cabinet that we really say is like, like
• 18:13
  I mentioned earlier, we want every single language to go
• 18:17
  support these calculators, these LMS. So I haven't seen a good
• 18:23
  framework in Java yet. I haven't seen a good framework
• 18:26
  in Go yet. And open the Eye themselves
• 18:29
  barely support those languages. And that's that that says a
• 18:33
  couple of things. Either they strongly believe
• 18:35
  that Python And TypeScript are the only feature that exists
• 18:38
  here and that you know what, screw any of the other ones, or
• 18:42
  they believe that eventually someone will deal with it and we
• 18:46
  don't have to think about it. But I I can have the belief that
• 18:50
  everyone should be able to take advantage of these systems.
• 18:52
  There are some like Kubernetes is built not in Python.
• 18:55
  Some of the most foundational systems that we have are not
• 18:58
  built in Python. And it feels like a pity to me
• 19:03
  if those systems will be the last ones to be able to take
• 19:05
  advantage of AI pipelines because they're the most useful.
• 19:09
  How does Bammel then sync up with all these?
• 19:12
  How does it interplay with all the different languages?
• 19:15
  So Bammel's philosophy as a language is we do some things as
• 19:21
  a language, but then what we do is we do code generation into
• 19:24
  every other language of your choice.
• 19:27
  So you write your code in Bammel and then we do code Gen. to give
• 19:30
  you a Python function. So if you were to function
• 19:32
  called classify message in Bammel, we would make an
• 19:36
  identical function called def classify message in Python that
• 19:40
  would actually call the Bammel code under the hood.
• 19:43
  And the Bammel code is actually written in Rust, so it's
• 19:47
  extremely fast and it runs on every system.
• 19:50
  So for those people that work in Python, it's like Numpy.
• 19:52
  Numpy is not written in Python, it's written in C And that's
• 19:57
  because to do really good numerical processing, you can't
• 20:00
  let Python do it. You write in CMake it way faster
• 20:02
  and expose it to Python via Python interface.
• 20:06
  That's what we do. We take the Bama code and we
• 20:08
  expose it to you via a. Type safe auto completable
• 20:13
  interface in every language of your choice.
• 20:15
  So it feels like you wrote in Python.
• 20:17
  It just happens that you happen to do it in Bammel.
• 20:22
  So did I understand that correctly?
• 20:23
  Because it's basically you're writing in your language of
• 20:27
  choice, yes, but you're writing Bammel code and it will just
• 20:34
  translate it to Bammel on the other side.
• 20:38
  Exactly. That's literally how that works.
• 20:40
  So you write you, you open up dot baml file and then you press
• 20:45
  command S in VS Code or cursor and we run ACLI command called
• 20:49
  baml CLI generate, which will turn the dot baml files into dot
• 20:54
  PY files or dot TS files or dot go files or dot Java files or
• 20:58
  whatever the heck you want. So and then you just use it
• 21:02
  natively. There's no Internet dependency.
• 21:04
  It runs locally in your machine. And then we're able to pack in a
• 21:08
  bunch of algorithms for you automatically.
• 21:11
  And because every language is taking advantage of the same
• 21:14
  exact core under the hood, we don't actually have the same
• 21:17
  type of bugs that a lot of other frameworks have.
• 21:20
  So most other frameworks they implement once in Python And
• 21:23
  they're like, Oh my God, I guess people use TypeScript.
• 21:26
  Then they implement re implement everything in TypeScript again
• 21:31
  and then they kind of forget about it and they keep adding
• 21:33
  features to Python And people like when are we adding this
• 21:35
  type trip. But because Bammel is the same
• 21:38
  everywhere, we actually support every language with every
• 21:42
  feature always by default. Well, OK, yeah, that idea
• 21:47
  reminds me a lot of when you have apps that are specific for
• 21:52
  either Android or iPhone. It's like, what is app going to
• 21:56
  be normally? And they go after iPhone 1st.
• 21:58
  And then it's like, when can I get this app on Android, Right.
• 22:01
  And it's that same kind of idea like implemented in Python.
• 22:05
  And then, Oh yeah, we got to go do this in TypeScript too.
• 22:08
  Yeah. All right, let's see.
• 22:09
  Exactly. And then that's kind of why,
• 22:11
  like I think React Native got a huge push, and that's why Dart
• 22:14
  and Flutter and all these other things had a huge push.
• 22:15
  Because you could implement it once and get it for web, get it
• 22:18
  for Android, get it for iOS all at once.
• 22:21
  Because they did the groundwork so that your team only has
• 22:24
  focused on it once. Yeah, it's a common pattern and
• 22:29
  it is again going back to the persona that you're serving.
• 22:33
  It's going to be much more comfortable if you are just
• 22:36
  building once and it can work wherever it needs to work.
• 22:41
  Yeah. And if you, if you think about
• 22:43
  it from like a larger organization standpoint, most
• 22:46
  large companies use more than one language.
• 22:49
  So now what's nice is we can actually they use many, right.
• 22:52
  So it's just it naturally happens.
• 22:55
  So if you live in that world and you're a large company now, it's
• 22:58
  great because all my LLM code can be shared.
• 23:01
  All the techniques that one team discovers can be used by another
• 23:05
  team, even if they use different languages and they get them for
• 23:08
  free. So that cross pollinate like the
• 23:10
  only thing I it's like Jason. Why is Jason so powerful?
• 23:15
  Because Jason every single language has a Jason dot loads
• 23:18
  Jason dot parse, Jason dot, Jason dot serialize.
• 23:22
  It has those methods built into it because they adapted it.
• 23:26
  Jason is compatible with every language.
• 23:29
  Bammel was designed with the same philosophy.
• 23:31
  We give you all the tooling that you want for LMS, but then we
• 23:36
  build a compat layer with every language so you don't have to
• 23:38
  actually rediscover all that tooling all the time.
• 23:43
  So you must have tried at least to not build another language.
• 23:51
  You must have thought really hard on how you could do this
• 23:55
  without building a language, because as you said earlier, I'm
• 23:58
  lazy and what you are trying to do by building a language is
• 24:03
  probably like playing the game not on a hard mode, but on
• 24:06
  extremely hard mode. Yeah, I think this is probably
• 24:10
  the worst idea, one of the worst ideas for startup ever is what I
• 24:15
  would say not a lot of companies have been able to do this.
• 24:20
  There's a few, but very few. And almost no company has done
• 24:25
  it for business logic ever. But, and I think a lot of
• 24:30
  start-ups have this thing where like you could vibe coded and
• 24:32
  ship something in the hands of your customers in like a day.
• 24:35
  We just sat in a corner and coded for eight months before we
• 24:39
  got our first few users even because like, how would imagine
• 24:44
  I want you to put your shoes in the our first few dabs who used
• 24:48
  ammo and back then it was called glue.
• 24:51
  So it's a different thing. We, that's a funny we got sued
• 24:54
  for a name, so we have to rebrand it.
• 24:56
  And I think we have a much better name now.
• 24:59
  But the first few devs that used ammo used a compiler that was
• 25:04
  really flaky, had a bunch of bugs.
• 25:05
  So we wrote in C++, not Rust. They had no syntax highlighting,
• 25:10
  they had no auto complete. We barely supported Python.
• 25:14
  We definitely didn't support any other language.
• 25:16
  And we had like our type system wasn't very complete yet.
• 25:20
  And then we grew from like from 4.
• 25:23
  I don't mean companies, 4 developers to 5 developers took
• 25:27
  us three months from 5 developers to 8 took another
• 25:32
  three months and then we got 10 like two months later after that
• 25:36
  and then we started booming. Now we have like now we probably
• 25:39
  have like 700 or so companies deploying in production in
• 25:41
  Bammel. Nice.
• 25:45
  Including some like Fortune 5 hundreds, which has been really,
• 25:47
  really surprising. So cool.
• 25:50
  But I think this is one of the think projects in the
• 25:54
  technologies that like the you know, those rides at a roller
• 25:57
  coaster where like you have to be this tall to ride a language.
• 26:01
  As it turns out, you need to be like 6 foot 2 to get in the
• 26:04
  door, maybe 6 foot 8 to even have like a developer consider
• 26:07
  you. And like it was, I don't think I
• 26:12
  we understood the dauntingness of the task we took on at the
• 26:16
  very like we knew it was hard, like programming languages.
• 26:18
  There's not a lot of them, but one thing we under appreciated
• 26:23
  back then was how easy it was to build something stable.
• 26:26
  Like once you got to a point of stability and generality, the
• 26:30
  number of use cases that would be unlocked was really, really
• 26:35
  surprising to us and the way that the tooling could evolve.
• 26:40
  So one of the things that we do is this concept that I was
• 26:45
  talking about, about like being able to see the prompt, for
• 26:47
  example. So typically when people want to
• 26:50
  run a prompt like, they usually have to run some CLI command to
• 26:53
  go run tests. So right now what I'm showing is
• 26:55
  I'm showing VS Code and I'm showing our VS Code extension
• 26:59
  because we own the entire stack. I can legitimately show you the
• 27:04
  prompt in real time and you can watch it edit as I'm typing.
• 27:08
  So if I type something in here, it pops in right over here.
• 27:11
  If I change my test case, it pops in and I can make.
• 27:15
  I can highlight my test case differently than my base prompt
• 27:18
  in the rendering of the prompt. And that matters a lot because
• 27:24
  if I want to quickly glance at this thing, I can easily see
• 27:27
  where the bug is. I can easily read the thing
• 27:30
  that's being sent out to the model.
• 27:32
  I can see the tokens that are actually underlying being sent
• 27:35
  out to the model. Depending on the model, the
• 27:37
  tokenizer changes. And you just did that by you.
• 27:41
  And because with that tokenizer, what you're seeing there is each
• 27:48
  word being highlighted with a different color as a token,
• 27:52
  right? Yeah, because this is what the
• 27:54
  model is actually seeing, right? The model isn't seeing your
• 27:57
  English there's it's seeing the token.
• 27:58
  So like the strawberry question is a very common question and
• 28:00
  people used to talk about like why does the model not?
• 28:04
  Why is the model not able to answer the word strawberry?
• 28:06
  It's because strawberry is like 2 tokens.
• 28:07
  But if you do S space, T space R and like spell out strawberry
• 28:11
  with spaces, it has a really easy time answering how many Rs
• 28:14
  there are in strawberry because a tokenizer can see the Rs,
• 28:18
  right? But that is really hard to
• 28:21
  explain to someone, especially Gene or dev, if you don't have
• 28:23
  the right tooling. But here you can just show it as
• 28:26
  you're debugging and understand what the model is doing better
• 28:29
  with the raw curl mode that we have under the hood.
• 28:32
  We all know every framework is going to at some point put a web
• 28:35
  message together. So if I swap from Open the Eye
• 28:38
  to Entropic my the raw web request itself changes.
• 28:45
  So we try it, We make that like a zero click that you can see
• 28:47
  ahead of time, not after the fact.
• 28:50
  And there's A and I think the biggest unlock is kind of what I
• 28:55
  what I call like the hot reload loop of prompts in React.
• 28:58
  Hot reload. I like that name.
• 29:01
  All right, So like in React, what is your hot reload loop?
• 29:03
  You go to your React file, like your TS file or TSX file, you
• 29:08
  press command S, you look at the browser, if it doesn't match
• 29:10
  what you want, you go back to the file, you edit it, you go
• 29:13
  back. That's really fast hot reload
• 29:15
  loop. And you can't do that without
• 29:17
  React in reality, because a whole bunch of things about how
• 29:20
  web state component works and all that stuff.
• 29:22
  I won't go into that. And prompting, it's kind of
• 29:24
  similar. You want to change your prompt,
• 29:26
  you want to run your test case. And what I did right now is I
• 29:29
  just pressed the run test case button and you can legitimately
• 29:32
  just see exactly what the model did.
• 29:35
  And if it doesn't match your expectations, you just go and
• 29:38
  edit the prompt, edit your data model, edit your input and rerun
• 29:41
  it again. You get into a really fast hot
• 29:44
  reload loop, so you quickly converge on a good prompt for
• 29:48
  your test cases. And because you never leave your
• 29:51
  editor, you don't have to go to a web browser.
• 29:55
  You're using tooling that you're already familiar with.
• 29:57
  You can use cursor, you can use Cloud Code or anything else you
• 29:59
  want without having to really log into a SAS of any kind.
• 30:06
  So this is really, it's helping you speed up that workflow by
• 30:11
  giving you like the rendering on one side of your screen of what
• 30:17
  you're working on on the other side of the screen.
• 30:19
  Exactly. It's it's how I do web
• 30:21
  development. And because web development is
• 30:24
  so experimental, we needed to do that.
• 30:26
  Why is Jupiter Notebook so successful for data science?
• 30:29
  Because data science work is very experimental.
• 30:31
  I need the visual feedback to go do this.
• 30:35
  I don't want to have to run the whole program every single time
• 30:38
  from scratch. A jets are also very
• 30:41
  experimental and you need that. And like the question we have to
• 30:46
  ask is, is every language going to build that experimental
• 30:48
  tooling individually or can we just see something like Bammel
• 30:52
  and then plug into every language with auto complete and
• 30:56
  all the benefits with like a Cogen kind of layer?
• 30:59
  Well, because I was envisioning it as a tool that you would use
• 31:04
  with it like AI know if we're talking about prompting
• 31:07
  specifically, there's a bunch of these different prompting tools,
• 31:12
  I think prompt layers, one of them or OPEC or ML Flow even
• 31:16
  does it these days, right. So how do you see those two
• 31:22
  things playing together? Is it still that I would have ML
• 31:25
  Flow to like version my prompts and have that?
• 31:29
  Why? I think it's just like, why
• 31:32
  would you reinvent version control?
• 31:33
  We have Git. We have like, it's a, it's a
• 31:36
  battle tested version control system that works for decades
• 31:39
  for like the most complex software out there.
• 31:41
  Why reinvent it? It's beautiful.
• 31:44
  It's really, really damn good. There's only two companies that
• 31:47
  reinvented Git and that's Mic and that's Google, Mic Shop and
• 31:51
  Facebook. And that's because their code
• 31:52
  base is massive, that Git was too slow, so they made
• 31:55
  improvements to Git to make it better and they have used
• 31:59
  Material as well. Why would you use anything
• 32:01
  different? But isn't it because you would
• 32:06
  have so many different prompts for certain like what you were
• 32:10
  showing me? There was one test case, but I
• 32:14
  assume that you're going to have a test suite when you're using
• 32:19
  stuff and. Yeah, we have that for regular
• 32:22
  software too. I think if you go back in time
• 32:25
  to a time when software is a lot smaller just in volume of lines
• 32:28
  of code, people would have said, oh, this works for like 10
• 32:31
  functions, but it's get really going to work when you have
• 32:33
  100,000 plus functions. It turns out it works when you
• 32:36
  have 100,000 plus functions. And I think it's the same thing
• 32:41
  with your test cases and everything else too.
• 32:43
  It's just code. And the best place to have code
• 32:46
  is your code base. Now, you might want to load
• 32:48
  tests from a database. That's fine.
• 32:52
  We know how to write tests that load from a database.
• 32:54
  We've done that before. You create a database, call, you
• 32:56
  call it, and then you run the test case.
• 32:58
  We know how to write π tests that does that.
• 33:01
  So it's true, you might store some instances in a database and
• 33:06
  some instances locally. That makes sense.
• 33:08
  But it's just code. Now, as you're working with
• 33:13
  different folks that are using PMO, what are things that
• 33:17
  they're telling you that's surprising you?
• 33:21
  I think the scariest thing that I heard was you're.
• 33:24
  Using it in production. Well, no, that that was fine.
• 33:28
  That that's not being scary like me like 7-8 months ago.
• 33:31
  That was scary or 7-8 months ago.
• 33:32
  Because every time someone ships, I was like, because we
• 33:35
  ship for so many different languages.
• 33:36
  I was like, Oh my God, did we break something?
• 33:38
  Did we break something on like the, what's it called?
• 33:42
  Like a Debbie, what's a slim version of Debian that people
• 33:45
  have like the Alpine? There's like an Ubuntu container
• 33:48
  called Alpine that people use for deployments because it's
• 33:50
  very, very small and we broke it once.
• 33:53
  Or like there's compat layer. So like there's small things
• 33:57
  like that. They used to give me anxiety.
• 33:59
  But most, almost all those bugs have been addressed now.
• 34:03
  But I think the scariest thing is when I heard someone having
• 34:06
  like 25,000 lines of Bama code in the repo.
• 34:10
  Wow. And I was like, that's a real
• 34:12
  code base like that's. Scary to me.
• 34:16
  That's legit, dude. Yeah.
• 34:18
  And I was like, and they're like, can I please have
• 34:20
  namespaces because we don't have namespaces right now in Bevel.
• 34:24
  And I was like, why do you need namespaces?
• 34:25
  And then they showed me that I could.
• 34:27
  And I was like, OK, I guess you need namespaces.
• 34:33
  So we haven't had namespaces yet.
• 34:34
  It takes a while for us to build out some features just because
• 34:38
  they're so corn primitive. Yeah, So features do take a
• 34:41
  little bit, but some features are fast, some are more work.
• 34:45
  But that was probably go ahead. Oh, no, no, sorry.
• 34:48
  Keep going. Keep going.
• 34:49
  No. I was going to say like $25.00
• 34:51
  of code scared me. I think first, first time I met
• 34:53
  someone that wrote 1000 lines of code, that was scary.
• 34:56
  We hired one of the first people that wrote 3000 lines of code of
• 34:58
  Vamil into a company and they ended up joining us.
• 35:04
  But yeah, volume of code is probably the scariest thing.
• 35:09
  It's like oh shit people depend on this for real.
• 35:13
  And how do you look at where to go next?
• 35:18
  Is it by talking to folks and seeing there's namespaces we
• 35:23
  need to do And I'm sure there's a laundry list of other things
• 35:26
  that people are asking you for? I think one of the best things
• 35:30
  about building a programming language is in the end, we're
• 35:32
  building for ourselves. Like there's very, very few
• 35:36
  tools that developers could truly build for themselves.
• 35:40
  One of them is like editors, like Cursor VS Code and you can
• 35:43
  go, you can just go do that. It turns out.
• 35:46
  Another one is a programming language because like, you as a
• 35:50
  developer kind of know what it feels like the right code, and
• 35:53
  you have an idea for what feels right, what is necessary.
• 35:56
  So we do listen to our customers to add different features, but
• 36:00
  we kind of just know what we have to do.
• 36:02
  Like I think if anyone had asked us to make BAML, they would have
• 36:04
  said no, we they would have said, why would I want this?
• 36:06
  And some of our earliest users were like, why the heck are you
• 36:09
  making me write this code? And we were just like, trust us.
• 36:12
  And we hope that we were right. But most features are usually
• 36:16
  just like in our heads. And we're like, we go talk
• 36:18
  internally on team, we write, we write a lot of pseudo code and
• 36:21
  then we go shop test it once we have that concept with our
• 36:24
  community. So we have a community of over
• 36:26
  like 1000 people now. Nice.
• 36:29
  And we, they've been really helpful in helping us guide the
• 36:32
  direction of where we should be going.
• 36:34
  But we usually don't leverage them to like have the inception,
• 36:39
  the first inception of an idea because they're busy thinking
• 36:42
  about how to build the best applications.
• 36:45
  And what we see is sometimes like, I'll give an example of
• 36:48
  streaming a lot of SDKS and frameworks around LLMS don't
• 36:53
  give you a great streaming visualization.
• 36:57
  And what I mean by that? In what way?
• 37:00
  Streaming is a really nuanced concept, so I'm going to show
• 37:02
  you something and maybe we can try describing it to everyone
• 37:05
  else. So let's take this recipe
• 37:09
  generator, for example. So everyone knows we can go to
• 37:12
  Chachi BT and ask it to spit out a recipe.
• 37:15
  It'll dump out something. But when I use streaming, I can
• 37:19
  do something really incredible. What if I can make it
• 37:22
  interactive while I was loading? And if you take a look at my
• 37:28
  screen, you can actually see like the loader icon is telling
• 37:30
  me exactly what it's working on at any given moment versus
• 37:33
  what's done. And this cue of how you build
• 37:38
  this kind of application is it's possible today with today's
• 37:42
  SDKS. It's just really, really damn
• 37:45
  hard. And So what we're seeing here is
• 37:48
  that you've got a little slider button at the top as the recipe
• 37:53
  is rendering. You can move the slider and get
• 37:58
  an interactive experience of that recipe being updated
• 38:03
  depending on the size of or the amount of people that you're
• 38:07
  trying to feed with this recipe. But you also see one of those
• 38:11
  little spinning wheels, so you know exactly which part the LLM
• 38:17
  is. Working on yeah.
• 38:20
  Yeah. And that's just not something
• 38:22
  that I think a lot of people spend effort on.
• 38:25
  Like, can you do this today? Yes, you can do this today.
• 38:29
  With today's alums. We, we remember, we don't modify
• 38:32
  the LM at all. We use every model as is without
• 38:34
  any modification. But the problem that I often see
• 38:39
  is not that you can't do this today, it's just it takes a lot
• 38:43
  of code to go do this. What if you can do this with one
• 38:46
  line of code in Bammel? That is the nuance of what we
• 38:50
  offer and I think that making something that needs to be
• 38:55
  common very easy is an under undervalued thing.
• 38:59
  But I think a perfect example that a lot of people probably
• 39:02
  know that did this is Tailwind. So I think Tailwind
• 39:07
  fundamentally changed the name. It changed the game of how
• 39:09
  styles should be done and things became more standardized once
• 39:12
  used Tailwind because CSS is one of those things where like we
• 39:17
  all want it to be perfect and we all thought like the right way
• 39:19
  to do CSS was to style out CS files with like a bunch of
• 39:22
  classes that we link. But it turned out CSS is very
• 39:26
  hyperlocal. I want to modify just a division
• 39:29
  I'm right at and I don't want to hover over and see what class
• 39:32
  something is. But the problem is I don't want
• 39:35
  to write raw CSS there because raw CSS is hard to read and act
• 39:38
  like there's some attributes in there that are just like what
• 39:42
  the heck is that? I cannot human read that.
• 39:45
  So Tailwind did something very simple, which is they added a
• 39:47
  string that was easy to read and programmatically possible to
• 39:51
  generate and they added it in a spatially local place.
• 39:56
  So they added a new syntax for defining CSS that they then just
• 40:00
  run through React to render the actual CSS under the hood.
• 40:03
  And that allows them to do optimizations as well for free,
• 40:07
  like only include the styles that you actually use in your
• 40:09
  code. Into the rendered formats.
• 40:12
  Your style sheet isn't super long like Bootstrap or other
• 40:15
  systems used to be. But more importantly it became
• 40:18
  more ergonomic and that allowed people to build perfect UIS for
• 40:23
  each part of the website because it's it became easy to do it
• 40:26
  frequently in my code, Bammel tries to do the same thing on
• 40:30
  streaming instead of having to think really hard about do I
• 40:33
  want to add like 100 extra lines here to make this thing stream
• 40:36
  perfectly. You have a lot of code and now
• 40:39
  it just streams the way you want to stream.
• 40:43
  Now what are you seeing users of Bammel create?
• 40:49
  What are some cool projects? There are a couple ones.
• 40:54
  There's kind of things all over the domain.
• 40:56
  So there's some in like the government industry that I
• 40:58
  thought really interesting. We, we have a lot of like
• 41:02
  government RFP generators that are built off Bammel.
• 41:06
  We have some in the medical space like analyzing Dr. patient
• 41:10
  conversations for all sorts of EMR stats, admin work.
• 41:16
  We have agents that operate in like the RFP automation space.
• 41:21
  We've all seen like those Chrome extensions that kind of extract
• 41:24
  data from website web pages into like a spreadsheet like view.
• 41:28
  I've seen those very generic dynamic systems we built in
• 41:31
  Bammel and Sequel chat bots kind of all over the place.
• 41:36
  RAG systems, yeah, a little bit of everything, which has been
• 41:39
  really surprising. And is there stuff that folks
• 41:42
  are asking for besides like the streaming?
• 41:44
  Besides the name spaces that you had said, what else is next on
• 41:50
  your list that you're thinking about tackling?
• 41:52
  More language support is probably like a huge one for us.
• 41:57
  So we technically support every language today using Open API.
• 42:01
  So we support Python, Ruby, Python, Ruby type trip natively
• 42:05
  and then all the rest are available via Open API crew
• 42:08
  bammel. But we're going to add Go
• 42:10
  support soon and then following that will be Java support
• 42:14
  natively as well. So now that we figured out
• 42:16
  static languages, we should be able to unlock every other
• 42:18
  language without having to have like a sidecar kind of system.
• 42:24
  And then the next Big 1 is a is actually an orchestration system
• 42:28
  that we're going to announce pretty soon.
• 42:31
  Oh. Yeah, so not just prompts, but
• 42:33
  like full workflows and orchestrations with a debugging
• 42:37
  experience unlike anything people have ever seen before for
• 42:41
  agents. And when you say orchestration,
• 42:45
  you're talking about what specifically?
• 42:49
  Like full expressive systems like if statements, for loops,
• 42:52
  while loops where you can conditionally call an agent,
• 42:55
  abort to a human worker, Human annotation system, come back to
• 42:59
  it, that kind of system. Wow.
• 43:03
  Yeah, and all that will still be exposed to every language of
• 43:06
  your choice. Oh man.
• 43:09
  So now as you're thinking through a mind share
• 43:16
  perspective, you like to garner more attention or to garner more
• 43:22
  developers using BAML. How are you looking at that?
• 43:25
  Like, how are you think this is ultimately going to be something
• 43:30
  that's not just a flash in the pan?
• 43:32
  Yeah. We spent a lot of time thinking
• 43:33
  about this because I think since we started, which is like
• 43:35
  November or October of 2023, there have been a ton of
• 43:40
  frameworks that have come out since then and a lot of them
• 43:43
  have died in very best. They've all been flash of the
• 43:47
  fan. I think one thing that is good
• 43:48
  is people have sustainably continued.
• 43:52
  Our users of BAML have increased over time without much like
• 43:56
  churn. One thing I was worried about is
• 43:58
  are people going to grow out of BAML?
• 44:01
  But I, as I talked about like the prism of that, the company
• 44:03
  with like 25,000 lines of BAML code, clearly they seem to be
• 44:06
  adding more BAML, not less BAML over time.
• 44:11
  So that sort of stuff I think has a swage.
• 44:13
  A lot of our worries around like will people grow out?
• 44:15
  Like maybe they will, but we'll add namespaces, we'll Add all
• 44:17
  things that people expect out of the language.
• 44:21
  But I have a view on Devtools that I think is very different
• 44:25
  than a lot of people, but I was inspired to by the TypeScript
• 44:27
  team. I think as a company, you have a
• 44:30
  finite set of resources. You choose where you want to
• 44:32
  deploy, your money you can deploy or and your time you
• 44:35
  deploy into marketing or engineering.
• 44:38
  We as a company are just saying screw it, we're just going to
• 44:41
  write a shit ton of code. Like we're just going to keep
• 44:44
  writing code and keep shipping. Our code base is almost like
• 44:47
  half a million. That's code or something.
• 44:49
  And we'll just keep because that's what we as a team love
• 44:52
  doing and we can do some incredible features.
• 44:55
  And what we want to do is we just want to make it so that you
• 44:57
  as a developer aren't thinking about Pamel being a bottleneck.
• 45:01
  We're just shipping. And you're just like, cool, I
• 45:03
  have that feature. Oh, before you even imagine you
• 45:06
  need it, we've already thought of it and added it.
• 45:08
  Then by the time you need it. And what we hope is if people
• 45:12
  keep using BAML and they keep loving it, I hope that they tell
• 45:15
  their friends and I think developers are these really,
• 45:19
  really developers are horrible to sell your product to.
• 45:23
  They're they're just the worst buyers ever, but they are the
• 45:26
  best referral system ever. If a dev loves your tool and
• 45:30
  they abide by it, they will tell every single one of their
• 45:33
  friends how much they love it. Yes, and I want to earn the
• 45:36
  trust of developers that pay we will look out for them and that
• 45:41
  we will do our best to make sure that their problems go away.
• 45:44
  So rather than spending time on like marketing or all these
• 45:46
  other things like let's just ship good code, like shipping is
• 45:50
  the best marketing and that's what we want to keep doing.
• 45:54
  How do you you think developers talk about Bammo when they talk
• 45:58
  to each other? I, I can read some tweets out
• 46:03
  that are like some messages, but like I get like messages almost
• 46:06
  every week now from someone somewhere around the world that
• 46:09
  discovered BAML for the first time.
• 46:10
  And Lily was like, thank you or like, this is amazing.
• 46:14
  I, I don't know how you did it, but it was really good.
• 46:17
  And I had a, someone that messaged me.
• 46:19
  I think the message that suck out was I what they sent me was
• 46:25
  I really, I explored it three months ago and I didn't want to
• 46:28
  go learn it because I didn't have time and I have to go ship.
• 46:31
  But I tried it again this weekend because I was tired of
• 46:34
  seeing your post and I regret not having switched earlier.
• 46:39
  It would have saved me so much time.
• 46:41
  It ended up taking about like 2 1/2 hours to learn.
• 46:45
  But I think it's that sentiment of like the regret that they
• 46:50
  felt was like a really, I felt bad that I could say things
• 46:53
  earlier to save them the time earlier, but I'm really glad
• 46:57
  that they found value and it wasn't a waste of time for them.
• 47:00
  I want to highlight that you said he was tired of seeing your
• 47:03
  post and this is you not in marketing mode.
• 47:06
  I cannot imagine what the world will look like when you are in
• 47:10
  marketing. Yeah, I do.
• 47:11
  I do post about Bama on LinkedIn.
• 47:13
  I, I, I think I'm really proud of what the team has done and
• 47:17
  the things that I mostly post like things that we ship.
• 47:20
  Yeah. So I all should talk about
• 47:21
  features and that's us from marketing knows.
• 47:23
  So more of our users know about the new features we release.
• 47:26
  Like I should probably send an e-mail chain out.
• 47:28
  I feel like people do that thing, but I don't really know
• 47:30
  how to set that up. So we, we collect emails, but we
• 47:33
  have never really, we set up 4 emails in the lifetime of the
• 47:35
  company to people. And I try my best not to do that
• 47:39
  because I as a developer hate emails.
• 47:41
  Yeah. So we do different.
• 47:43
  Things Google Groups going on that feels.
• 47:46
  Like Oh my God I would die. I hope not.
• 47:50
  I mean like I let's talk about like Discord versus Slack.
• 47:53
  Like why do we use discord? I hate discord.
• 47:56
  Like I like I play games and stuff and the discord is for
• 47:58
  social, but Slack is like more commercial and work base.
• 48:02
  But one of the things we thought was I don't want to give up my
• 48:05
  name and identity just to ask a question that feels kind of like
• 48:08
  it's like at increasing A barrier.
• 48:10
  So we just said, what if we just make a discord, no login needed,
• 48:13
  just go and ask a question and it's cool.
• 48:15
  It's kind of the philosophy that we always take like reduce the
• 48:18
  barrier of entry as much as possible because we have this
• 48:21
  huge barrier, which is you got to spend 2 hours on a Saturday
• 48:24
  to play around with it. So it's like every other barrier
• 48:26
  we try and remove out of the way.
• 48:30
  And what are some things that you feel like the folks that are
• 48:35
  using Bammel highlight when they talk about Bammel?
• 48:39
  The biggest 1 is probably our parsing.
• 48:42
  So we have a way to do structured outputs that is
• 48:44
  better than open AI, it's better than Anthropic, it's better than
• 48:47
  every other model out there on almost every benchmark.
• 48:50
  It's a new algorithm we created called Schema line parsing.
• 48:53
  People love that thing and they love it because like it allows
• 48:57
  you to chain of thought in one shot with structured outputs in
• 49:01
  the model and you don't also think about it.
• 49:04
  We just do error correction on the model.
• 49:06
  A lot of other frameworks, what they do is they do retries
• 49:08
  whenever the model doesn't give you the exact thing that you
• 49:10
  asked for. We solve that same problem with
• 49:13
  mills with like milliseconds of work with an algorithm.
• 49:18
  And now it's been battle tested in like millions of calls, 10s
• 49:22
  of like easily over like 10s of millions of calls and all
• 49:27
  different data types. That's one thing people love,
• 49:30
  and the other thing that they really love after they get past
• 49:33
  that is the iteration speed. The iteration speed of BAML is
• 49:41
  unlike anything else. If it takes you, I think the way
• 49:44
  I put it is if it takes you 5 minutes to test one prompt at a
• 49:48
  time, and you need to test 50 prompts to find the right
• 49:52
  answer, it'll take you 250 minutes.
• 49:55
  If it takes you 5 seconds to test the prompt, it will take
• 49:58
  you 250 seconds. And there's just like, no,
• 50:03
  there's just no comparison when it's that much faster.
• 50:06
  That Run test button I showed earlier from our VS Code
• 50:09
  playground is probably the single best feature that we've
• 50:13
  shipped. I am blown away by this.
• 50:17
  No.