Latest / The Neon Show / Why Your AI is Still a Demo: Lessons from Braintrust’s Field CTO

Transcript

Why Your AI is Still a Demo: Lessons from Braintrust’s Field CTO · The Neon Show

• 0:00
  SPEAKER_00: My role at Brainterest is to help our customers who are building Gen AI systems to build their AI in a very predictable, repeatable way such that they can ship high quality AI into production.
• 0:12
  SPEAKER_00: And so the way you do that is by building evaluate data centers.
• 0:15
  SPEAKER_00: And is live coding going through eval?
• 0:17
  SPEAKER_00: It's really important that you have control over the application that you have built by live coding.
• 0:24
  SPEAKER_00: Evals almost become existential.
• 0:26
  SPEAKER_00: There are other agents that are able to manage your subscriptions for you.
• 0:30
  SPEAKER_00: There are agents that can help you set up your itinerary, knowing your preferences, knowing your constraints.
• 0:36
  SPEAKER_00: They can pick the best combination of these flights, these hotels for you.
• 0:40
  SPEAKER_00: So there's lots of interesting agents that are being built out there.
• 0:44
  SPEAKER_01: Where do you think these models and capabilities are heading in 26 and 27?
• 0:54
  SPEAKER_01: Hi, this is Sudha Talu Aliya, your host at Neon Show and managing partner at Neon Fund, a fund that invests in pre-seed and seed stages in the best of enterprise AI companies across US, India, Corridor, like Atomic Work, Spot Draft, CloudSec.
• 1:11
  SPEAKER_01: Today I have with me Ameya.
• 1:13
  SPEAKER_01: Ameya, welcome to the Neon Show.
• 1:14
  SPEAKER_01: Thank you.
• 1:15
  SPEAKER_01: You are at a very unique position, field CTO at Brain Trust.
• 1:19
  SPEAKER_01: So Brain Trust for audience is one of the leading companies in Eval space for AI.
• 1:24
  SPEAKER_01: I'll ask you to explain, you know, first what is a field CTO, and second is what is Evaluation.
• 1:30
  SPEAKER_00: Sounds good.
• 1:32
  SPEAKER_00: So my role at Brain Trust is to help our customers who are building Gen AI systems to build their AI in a very predictable, repeatable way such that they can ship high-quality AI into production.
• 1:47
  SPEAKER_00: And so my role is to work with our customers to build out the technical strategy, the way we deploy and use the Evals platform and the observability systems, how we build out these feedback loops to help the company build the muscle to ship production quality AI.
• 2:07
  SPEAKER_00: And so I lead the post-sales field engineering team as well.
• 2:11
  SPEAKER_00: And so we work with our customers to not only onboard to Brain Trust, but use it in a way that can help them build these AI systems effectively.
• 2:21
  SPEAKER_01: Can you describe now evaluance observability?
• 2:24
  SPEAKER_01: What do they mean in this context?
• 2:26
  SPEAKER_00: Absolutely.
• 2:27
  SPEAKER_00: And so, you know, as we have seen this revolution of Gen AI over the last few years, what has happened is we moved from the world of training models to do very specific things, where we would take large training data sets, build these algorithms, and then train these models to do a certain task in a predictable fashion, you know, with a high degree of accuracy.
• 2:51
  SPEAKER_00: After the Gen AI revolution, like the way now we build intelligent applications is we take models off the shelf.
• 3:01
  SPEAKER_00: These are general purpose models.
• 3:03
  SPEAKER_00: They can reason on a variety of tasks.
• 3:06
  SPEAKER_00: And then we condition the models to work a specific way by doing prompt engineering, by doing context engineering.
• 3:13
  SPEAKER_00: We are really making sure that the models work for a particular use case in a predictable fashion.
• 3:20
  SPEAKER_00: And so when we are not training these models and we're conditioning them, we want to make sure that our instructions work well across the board, you know, for all the anticipated and not yet anticipated ways in which people are going to interact with these systems.
• 3:38
  SPEAKER_00: And so the way you do that is by building evaluate datasets.
• 3:42
  SPEAKER_00: These eval datasets, they describe the system in terms of the expected inputs and the expected outputs.
• 3:51
  SPEAKER_00: And so if your AI is now able to work across those evaluatas, you have a high degree of confidence that it will work well in production.
• 4:01
  SPEAKER_00: That's evals.
• 4:03
  SPEAKER_00: You also want to make sure that the system works as expected in production.
• 4:09
  SPEAKER_00: And so you want to know exactly how people are interacting with your systems, how they're working with your agents.
• 4:16
  SPEAKER_00: In which cases does your system work well and in which cases it doesn't?
• 4:21
  SPEAKER_00: Right.
• 4:22
  SPEAKER_00: And then you want to look at those cases where it didn't quite work as expected and use that insight to continually improve your intelligence system.
• 4:32
  SPEAKER_00: Can you fine-tune your prompts?
• 4:34
  SPEAKER_00: Can you do better context engineering?
• 4:36
  SPEAKER_00: Can you change something upstream in your system to address those shortcomings and not regress your existing capabilities?
• 4:46
  SPEAKER_00: And so that's where observability plays a big part.
• 4:49
  SPEAKER_00: So when you're building Gen AI systems, you really want that feedback loop of observability that helps you build better Evals that helps you ship better AI.
• 5:01
  SPEAKER_01: Can you give an example using one of your customers, like what agents they are shipping, and how does Brainters play a role in them shipping production grade agents?
• 5:11
  SPEAKER_00: Yeah, so we have a lot of customers, a lot of companies that are building some pretty phenomenal AI systems.
• 5:19
  SPEAKER_00: They all are leveraging large language models.
• 5:22
  SPEAKER_00: They are building, you know, um complex agentic systems that are reasoning, that are leveraging a lot of tools and interacting with LLMs to fulfill the user intent.
• 5:35
  SPEAKER_00: And so we have systems where people can work with their agents to perform specific tasks.
• 5:42
  SPEAKER_00: The agents are able to understand the user intent and perform actions to fulfill those user intents.
• 5:49
  SPEAKER_00: Some of these are transactional, some of these are systems that provide answers to queries, some of these are systems that do work on the user's behalf, right?
• 6:02
  SPEAKER_00: An example would be there are systems that are being used by special industries where they are trying to harvest meaningful information from their corpus of work documents to generate new content that is going to help them accelerate certain processes.
• 6:23
  SPEAKER_00: For example, I've seen systems where construction companies are able to now put together effective proposals using complex engineering drawings, architectural plans, specifications to submit proposals for new RFPs.
• 6:43
  SPEAKER_00: There's a very specific format in which these companies are building out that content.
• 6:48
  SPEAKER_00: It needs to be accurate, it needs to be standardized, the certain formats that it needs to follow.
• 6:56
  SPEAKER_00: And the AI systems are able to help these companies fulfill that task at uh with a high degree of automation.
• 7:04
  SPEAKER_00: There are other agents that are able to go in and manage your subscriptions for you.
• 7:10
  SPEAKER_00: They're able to uh make choices on your behalf in terms of uh you know which options uh they should um uh exercise on your behalf.
• 7:23
  SPEAKER_00: So, for example, there are agents that can help you set up your itinerary, you know, come up with uh recommendations for certain flights or certain hotels and help you make that discovery, that discovery cycle far shorter so that like knowing your preferences, knowing your constraints, uh they can pick the best combination of these flights, these hotels for you.
• 7:49
  SPEAKER_00: So there's lots of interesting agents that are being built out there.
• 7:55
  SPEAKER_01: So let's let's take an example of your some of the earlier doctors, like Notion, Zapier, Stripe, Airtable, and Instacart.
• 8:03
  SPEAKER_01: What does eval look like in practice for them?
• 8:06
  SPEAKER_00: Yeah.
• 8:08
  SPEAKER_00: So all of these companies are building agents, intelligent systems that are performing specialized tasks for whatever use cases they have.
• 8:18
  SPEAKER_00: And so all of them build evals that reflect how their systems are expected to behave in production.
• 8:27
  SPEAKER_00: They're all following similar patterns.
• 8:28
  SPEAKER_00: They're building these eval datasets.
• 8:30
  SPEAKER_00: These eval datasets are exercised every time any component in their systems are modified.
• 8:37
  SPEAKER_00: Someone changes a prompt, someone changes some logic in some of the upstream components.
• 8:43
  SPEAKER_00: The eval datasets ensure that like any regressions are caught early in the process.
• 8:51
  SPEAKER_00: These eval datasets, they are not static.
• 8:55
  SPEAKER_00: As the teams look at their logs, at how their systems are working in the real world, in the production use cases.
• 9:05
  SPEAKER_00: Uh, they're able to leverage those insights to continually augment their evaluate datasets.
• 9:11
  SPEAKER_00: And so these evaluate datasets are now highly reflective of how the system is going to perform in the real world and continue to help them improve their AI.
• 9:22
  SPEAKER_00: And so the same pattern is fairly common across all of these companies.
• 9:27
  SPEAKER_00: You can see how they have agents and systems that are specific to the offerings, the products they have in the market, and you can sort of see uh the kind of evals that they would have built to make sure that those systems work as expected.
• 9:45
  SPEAKER_01: Got it.
• 9:45
  SPEAKER_01: And why is observability absolutely necessary for AI when classic monitoring was enough for normal software?
• 9:53
  SPEAKER_00: Yeah.
• 9:54
  SPEAKER_00: I think monitoring gives you a great understanding of how your systems are working in terms of their performance from a latency perspective, from a cost perspective.
• 10:05
  SPEAKER_00: And those are important dimensions to measure.
• 10:08
  SPEAKER_00: I think in the case of uh these probabilistic systems, you also want to have a deep understanding of how your systems are behaving in response to these user intents that are typically being specified in natural language.
• 10:23
  SPEAKER_00: There's a lot of variance in how people can express the same thing.
• 10:27
  SPEAKER_00: And so you want to make sure that your AI system is robust enough to handle all of the various ways in people express their intent and they are able to fulfill the intent accurately.
• 10:39
  SPEAKER_00: And so it's important for AI systems to have the observability that's beyond just these performance metrics.
• 10:50
  SPEAKER_00: These AI systems need to log the entire trace of how the AI reasoned on the initial input, what were the tool calls it made?
• 11:02
  SPEAKER_00: How did it interact with the LLMs?
• 11:05
  SPEAKER_00: How did it sort of ultimately generate the response?
• 11:09
  SPEAKER_00: And did that response actually meet the user intent?
• 11:12
  SPEAKER_00: Was it accurate?
• 11:13
  SPEAKER_00: Was it correct?
• 11:14
  SPEAKER_00: Were there any hallucinations in the generated content or not?
• 11:17
  SPEAKER_00: And so you really need to capture all of that information to be able to effectively evaluate the quality of that response and as a result your entire AI system.
• 11:28
  SPEAKER_00: So that's why observability becomes a pretty key aspect of building these kind of intelligent systems.
• 11:34
  SPEAKER_01: And your title is very interesting, it's Field CTO.
• 11:37
  SPEAKER_01: What does it mean?
• 11:38
  SPEAKER_00: Yeah, so all of my career I've been building AI systems.
• 11:44
  SPEAKER_00: I've I've built internet scale, enterprise scale AI systems at Microsoft, at Facebook, at Dropbox.
• 11:52
  SPEAKER_00: And uh over time, um, you know, I've observed how these AI systems are built, how they're deployed, how they're operationalized.
• 12:04
  SPEAKER_00: And um I started leveraging Brain Trust about a year ago while I was at Dropbox.
• 12:12
  SPEAKER_00: And it was um it was quite the step function change for us where we were able to move from a fairly messy way of building enterprise AI systems powered by Gen AI to a more systematic way of how we were building evals, how we were doing observability.
• 12:32
  SPEAKER_00: And um we were able to go from this chaotic state to a much more systematic way of doing things.
• 12:40
  SPEAKER_00: And um I see a lot of companies, a lot of folks who are building great AI products, they're kind of in the state that I was about a year ago.
• 12:49
  SPEAKER_00: And so I was working closely with the Brain Trust team, and I felt like there would be a great opportunity for me to work with a lot of world-class AI teams to work with them on building production quality AI systems in a way where I could share some of the learnings that I had accrued over my career and help them sort of go from that messy primordial state of building these AI systems to doing something that was going to set them on the path of shipping production quality AI systems fairly quickly.
• 13:26
  SPEAKER_00: And so that was sort of the drop for me.
• 13:28
  SPEAKER_00: So my role at Brain Trust is to work with our AI partners, our AI customers to help them not only onboard the Brain Trust platform and use it effectively, but to systematize their the way in which they build and ship AI in a way that can guarantee high quality AI in a predictable, repeatable fashion.
• 13:52
  SPEAKER_01: Got it.
• 13:53
  SPEAKER_01: And and why did you choose to go from building AI systems to be on the field site where you're not building and selling now?
• 14:02
  SPEAKER_00: Yeah, no, uh as I said, like I've been building AI systems for a long time.
• 14:09
  SPEAKER_00: I started building my first machine learning models that were shipped in the operating system in the browsers in 2008.
• 14:16
  SPEAKER_00: So it's been 18 years of building and shipping AI systems, and that has been a wonderful journey.
• 14:23
  SPEAKER_00: Um I'm at the point where I think I can amplify the impact that I have and not just limit it to one company, but work with many, many, many world-class teams that are doing very exciting things in the space of AI.
• 14:40
  SPEAKER_00: And so the opportunity to work with a lot of different teams across various domains, building a variety of different AI solutions was very appealing to me.
• 14:50
  SPEAKER_00: I was also always on the product side.
• 14:52
  SPEAKER_00: And so this was sort of my opportunity to uh build my muscles and grow a few more folds in my brain as I learned about the sales world.
• 15:02
  SPEAKER_00: And so this was a really interesting opportunity for me.
• 15:06
  SPEAKER_00: So now you work on pre-sales or post-sales?
• 15:09
  SPEAKER_00: I work on post-sales.
• 15:11
  SPEAKER_00: Um, well, I lead the post-sales field engineering team.
• 15:15
  SPEAKER_00: So my team works with our customers to deploy brain trust and to build AI solutions.
• 15:21
  SPEAKER_00: And so we have solutions architects as well as AI engineers who help our customers through the entire journey of onboarding to an Evals platform, operationalizing observability, and then really integrating BrainTrust into their AI SDLC so that they are doing evals in a fairly um systematic way, that they have operationalized how they do error analysis, uh, that they have built all the integrations that they need to automate a lot of this workflow.
• 15:57
  SPEAKER_00: So my team helps our customers go through those phases.
• 16:01
  SPEAKER_00: Uh, but my role is you know talking to interesting teams.
• 16:05
  SPEAKER_00: Like I really enjoy talking to any team that is either considering brain trust or considering you know, onboarding to an Evelse platform, to teams that have built AI systems and are looking at how they can now have uh you know another step function improvement in their processes.
• 16:29
  SPEAKER_00: So I work with everyone.
• 16:35
  SPEAKER_01: When an AI product fails or an AI agent fails, what is what are the few things that could have gone wrong?
• 16:43
  SPEAKER_00: Yeah, so you know, these are all probabilistic systems.
• 16:47
  SPEAKER_00: So when an agent is reasoning on the task, the agent is trying to understand the user intent.
• 16:54
  SPEAKER_00: The agent is trying to understand how it should sequence the set of tasks that it needs to perform.
• 17:00
  SPEAKER_00: It needs to invoke tools.
• 17:03
  SPEAKER_00: The tools are making calls to third-party systems, uh, returning certain results.
• 17:10
  SPEAKER_00: And then these agents are trying to piece together all of that information and then again continue their reasoning to generate the final outcome.
• 17:19
  SPEAKER_00: And so, as you can see, this is a probabilistic system with a long chain of steps that need to be orchestrated and executed correctly to yield the final outcome.
• 17:29
  SPEAKER_00: And so a lot of things could go wrong.
• 17:31
  SPEAKER_00: Maybe the tools don't work out as well as expected.
• 17:34
  SPEAKER_00: Maybe the orchestration got certain things wrong.
• 17:37
  SPEAKER_00: Maybe the sequence of tasks wasn't quite right.
• 17:40
  SPEAKER_00: Maybe there weren't specific enough instructions for the AI to reason in a way that the system was expected to.
• 17:48
  SPEAKER_00: So there's a lot of things that could go wrong.
• 17:50
  SPEAKER_00: There are also a lot of operational things that could go wrong.
• 17:53
  SPEAKER_00: You know, a call to an external system may fail.
• 17:56
  SPEAKER_00: And so you really have a lot of um moving parts when you're building out complex agentic systems, and you could have a failure across any of those.
• 18:06
  SPEAKER_00: And so you really want to have that full observability, that ability to trace exactly how an agent took that user intent, orchestrated the plan, executed it, and then generated the final answer.
• 18:20
  SPEAKER_00: So you want to know exactly where it missed a step or failed to yield the final answer.
• 18:28
  SPEAKER_01: So are you hinting towards that what you are building on the eval and observability side is capturing like this very interesting uh articles by Foundation Capital that got viral on context graphs and context engineering.
• 18:43
  SPEAKER_01: Is that what you're trying to capture in evals and observability, the context of an agent and why agent did what it did?
• 18:51
  SPEAKER_00: Yeah, I mean, capturing the entire context is really important.
• 18:55
  SPEAKER_00: When you want to observe, like when you want to understand exactly what is it that the agent did over the course of its execution, you want to capture all the context that the agent had access to.
• 19:07
  SPEAKER_00: That way, anyone who's debugging and troubleshooting can follow the details to understand exactly how the context was built.
• 19:18
  SPEAKER_00: Was something missing in the context?
• 19:21
  SPEAKER_00: Was it interpreted by the underlying system correctly?
• 19:25
  SPEAKER_00: And so we are building a system where you can capture all of that context within the span, the trace of the conversation.
• 19:34
  SPEAKER_00: So you can imagine a conversation where an agent gets an instruction from the user, then it performs multiple steps, yields an answer, and then that conversation can continue.
• 19:47
  SPEAKER_00: And so you can have these long-running conversations where there's a lot of context that's being generated and built, and all of that is really important to understand how the agent behaved and for aiding with troubleshooting and debugging.
• 19:59
  SPEAKER_01: So, so really rather than going through a log today, a developer or not, even a non-developer, can really talk to an agent why you did what you did through a chat-based mechanism.
• 20:11
  SPEAKER_00: Well, yes, too.
• 20:12
  SPEAKER_00: I mean, if you have all the context, you know, you can have other agents introspect, like look at that context and try to understand how the agent worked out what it did, right?
• 20:23
  SPEAKER_00: And so we are getting into this very interesting phase where you have agents that can analyze and understand and give you insights into the performance of other agents.
• 20:34
  SPEAKER_00: We have an agent built into the tool itself in Braintree, it's called Loop.
• 20:38
  SPEAKER_00: And Loop has access to your entire project context.
• 20:42
  SPEAKER_00: It has, it can access all your production logs, it can look at all your eval datasets, it can look at all the prompts that you have, it can look at all the ways in you are evaluating those uh agentic interactions, where you can have custom scorers that can evaluate the quality of your agentic interactions.
• 21:03
  SPEAKER_00: And because it has all that context, it is able to reason on how your agentic system is behaving.
• 21:09
  SPEAKER_00: It can identify areas for improvement, it can help you generate synthetic data that can be used to augment your eval datasets.
• 21:18
  SPEAKER_00: And so, yeah, you can now have agents analyze other agents and give you recommendations.
• 21:25
  SPEAKER_01: This has never happened before, right, in history of uh engineering like on how do you debug?
• 21:35
  SPEAKER_00: Yeah, I mean debugging is pretty um interesting in these cases because you know you really want to understand how every component of a particular trace of a conversation performed.
• 21:50
  SPEAKER_00: Uh all of that data is logged into your observability system.
• 21:53
  SPEAKER_00: So in Brain Trust, you can see every step that was performed by the agent.
• 22:00
  SPEAKER_00: You know, as a span, and so you can have um you know a full context of the span, but you can also have scorers that can evaluate the execution of a particular span.
• 22:14
  SPEAKER_00: So you can have cus uh uh scorers, which are essentially functions that evaluate the quality of a particular action.
• 22:22
  SPEAKER_00: Uh, you can define those as deterministic functions, you know, implemented in code, or you can use LLM as judges, but then you can evaluate like how did each span perform?
• 22:34
  SPEAKER_00: And that can give you a fairly good way to zero in on problematic areas of your agents.
• 22:41
  SPEAKER_00: So within a particular span, within a particular trace, you can quickly figure out where did the agent go wrong because your scorers can now point you to that particular place fairly effectively.
• 22:53
  SPEAKER_00: But then you can also look at understanding all your production traces holistically.
• 23:00
  SPEAKER_00: You can do clustering analysis to see what are the trends that you're observing in your agents as they work across multiple different interactions.
• 23:11
  SPEAKER_00: You know, where do they get things right?
• 23:14
  SPEAKER_00: Where do they get things wrong?
• 23:16
  SPEAKER_00: How do they get things wrong?
• 23:17
  SPEAKER_00: Why do they get things wrong?
• 23:19
  SPEAKER_00: So you can now do that kind of holistic analysis on all your data to not only do you know debugging and troubleshooting on a on a particular instance, but get uh sort of uh overall understanding of the behavior of your agent across a wide variety of use cases and categories.
• 23:44
  SPEAKER_01: So what do you classify brain trust also as a debugging tool then?
• 23:48
  SPEAKER_01: Not yet.
• 23:50
  SPEAKER_00: It is it is a very effective debugging tool.
• 23:52
  SPEAKER_00: You know, it tells you if your agent made a mistake and it gives you all the context and gives you all the clues to understand exactly where that mistake happened, where did things go wrong?
• 24:08
  SPEAKER_00: And so, yeah, it is a very, very useful debugging tool for folks who are working on building these AI systems or continuing to improve these AI systems.
• 24:17
  SPEAKER_01: And is wipe coding going through evals?
• 24:19
  SPEAKER_01: Like people who are using wipe coding, are they adopting tools like Beinterest?
• 24:24
  SPEAKER_00: I think so.
• 24:24
  SPEAKER_00: Like, I mean, it it almost becomes uh it's it's really important that you have control over the application that you have built by wipe coding.
• 24:38
  SPEAKER_00: Vipe coding makes it really easy for you to take an idea and build an application.
• 24:44
  SPEAKER_00: But how do you know that your application is going to work great?
• 24:47
  SPEAKER_00: Like, especially if your application is encoding agent tick capabilities, right?
• 24:53
  SPEAKER_00: And so when you're building these intelligent agent tick applications using wipe coding, um evals almost become existential.
• 25:03
  SPEAKER_00: You know, that's the only way you have a high degree of confidence that what you've built is going to work well.
• 25:10
  SPEAKER_00: We are starting to see a lot of platforms that provide no-code or low-code ways of building agents.
• 25:18
  SPEAKER_00: They are looking at embedding brain trust in their platform to make it available to these no-code, low-code developers who are building agent systems to test and ensure that whatever they have built will work as expected.
• 25:36
  SPEAKER_00: And so, you know, even if you're a no-code, low-code developer, it's really important that you have the rigor of building out evals, making sure what you've built works well for those evals, and continue updating those evals as you collect more data from the real-world usage of those systems.
• 25:54
  SPEAKER_01: You would have seen uh teams ship fast with weak evals and team move slow with strong evals.
• 26:01
  SPEAKER_01: Which one went in the long-term NY?
• 26:05
  SPEAKER_00: Yeah, I mean you can ship very quickly without doing any evals or by doing a very cursory job at sort of um doing evals.
• 26:15
  SPEAKER_00: Uh, but it's going to result in in some technical debt.
• 26:24
  SPEAKER_00: You're going to see more frequent regressions, things that used to work stop working.
• 26:30
  SPEAKER_00: Uh, any upstream changes can have a dramatic impact on the overall outcome.
• 26:36
  SPEAKER_00: And so while you can start shipping things fast, uh shipping things predictably with high quality becomes really hard, right?
• 26:49
  SPEAKER_00: And so if you want to build something that is going to uh scale, that is going to consistently deliver a high-quality experience, uh then you know you have to invest in high-quality valves.
• 27:04
  SPEAKER_00: Like that, that's just uh non-negotiable.
• 27:07
  SPEAKER_00: And you can see that you know, users are very sensitive to their systems uh and their quality of their output.
• 27:16
  SPEAKER_00: You know, it's very easy for someone to lose faith and credibility into an AI system when they see certain mistakes or they're having these rough experiences due to quality issues in their in their interactions with their systems.
• 27:32
  SPEAKER_00: And so users are not very forgiving.
• 27:36
  SPEAKER_00: And if you make mistakes too often, you're going to see a churn and usage.
• 27:45
  SPEAKER_01: So enterprises, you know, the failure is very expensive.
• 27:50
  SPEAKER_01: So what makes an enterprise trust one AI system and reject another, even if they use the same model underneath?
• 27:59
  SPEAKER_00: So these intelligent systems, these uh AI systems are not just a model, right?
• 28:06
  SPEAKER_00: There's a lot of layering that happens on top of these models.
• 28:11
  SPEAKER_00: Uh, these systems have to really um deliver specific capabilities or specific experiences uh that help people do certain specific tasks, right?
• 28:26
  SPEAKER_00: And so you're looking at fairly complex systems that have a model at the heart of it, but there is a lot of engineering that happens on top of these systems.
• 28:37
  SPEAKER_00: You want to make sure that you're able to collect the right context, that you're able to translate the user queries into rephrased queries that have a higher probability of being successfully executed.
• 28:49
  SPEAKER_00: You want to make sure that the instructions that you're providing to your AI systems are accurate, that are comprehensive, that they ensure that these systems uh don't hallucinate or generate uh unexpected results.
• 29:04
  SPEAKER_00: Uh, you want to have the right guardrails in place.
• 29:07
  SPEAKER_00: And so it's not just the model.
• 29:10
  SPEAKER_00: The model is a part of the system, but it's it's there's a lot of engineering that happens on top of these models, and you have to get it all right in order to have a high-quality system that is at heart a predictable, you know, model.
• 29:25
  SPEAKER_01: Do you think AI startups that are building AI, for example, uh, you know, agentic companies that are providing customer support agents actually know how good or bad their product is or they're guessing?
• 29:38
  SPEAKER_00: Well, if you're developing based on vibes, then you're guessing, right?
• 29:42
  SPEAKER_00: I think uh uh it's going to be very hard to ship something in the real world uh and expect it to work well if you haven't stress tested it.
• 29:53
  SPEAKER_00: You you if you haven't, if you've just tested it um, you know, with vibes, um chances are your system may not work well across the variety of ways in people are going to try and interact with your system.
• 30:09
  SPEAKER_00: And so I do think that systems that are built with a high degree of rigor have a good chance of becoming useful systems that your customers will engage with, otherwise, they end up looking like demos.
• 30:25
  SPEAKER_01: So, can you tell us some of like what if absolutely in layman terms, what are the different ML systems that you worked on in the last 20 years of your career before joining Brain Trust?
• 30:39
  SPEAKER_00: And what was the impact those systems created?
• 30:43
  SPEAKER_00: Absolutely.
• 30:44
  SPEAKER_00: So I'll I'll sort of talk about the three phases of ML that I've experienced in my career.
• 30:49
  SPEAKER_00: I think the first phase was classic ML.
• 30:52
  SPEAKER_00: These are classic algorithms, probabilistic algorithms.
• 30:56
  SPEAKER_00: Um they there are largely um, you know, they're like mostly algorithms that are popular in the 2000s.
• 31:09
  SPEAKER_00: You know, these were uh logistic regression, um, you know decision trees, um uh clustering algorithms.
• 31:19
  SPEAKER_00: Um and these were fairly effective.
• 31:22
  SPEAKER_00: Like for their time, they did a pretty good job in terms of providing uh intelligent systems that could do things like risk management and risk detection, they could do spam detection, they could do um you know uh things like uh churn analysis, uh anomaly detection.
• 31:41
  SPEAKER_00: So they they were like fairly effective.
• 31:43
  SPEAKER_00: Um the way you build these algorithms was you took a lot of data and you worked with the data.
• 31:51
  SPEAKER_00: You did uh you know what you call feature engineering on that data to transform the data into a shape and size that the algorithms could effectively work with to produce high-quality you know predictive systems.
• 32:07
  SPEAKER_00: And um and so at that time, a lot of emphasis was on feature engineering.
• 32:12
  SPEAKER_00: People used to come up with like uh a lot of different ways in which you would you would modify the shape and size of the data.
• 32:21
  SPEAKER_01: Uh I believe search became really popular in like starting 20 years ago with Amazon personalizing search.
• 32:29
  SPEAKER_00: Every company wanted to have a search.
• 32:31
  SPEAKER_00: That search, like with these algorithms, you could implement search algorithms, you could implement personalization systems, recommendation systems.
• 32:38
  SPEAKER_01: Yeah, recommendation systems were very popular and all were based on these old ML models.
• 32:43
  SPEAKER_00: Old ML models, collaborative filtering.
• 32:45
  SPEAKER_00: And so um it was a very interesting time to learn a lot of like how ML worked from first principles, because you could really understand under the covers what was happening, how you were taking these data, uh running them through these algorithms, the kind of uh systematic way in which they were processing the data to generate the final outputs.
• 33:09
  SPEAKER_00: Uh, comparatively, the interpretability of the systems was was much higher.
• 33:14
  SPEAKER_00: Yeah.
• 33:14
  SPEAKER_00: I think in the 2010s, uh, we saw the wave of neural networks.
• 33:18
  SPEAKER_00: Yeah.
• 33:19
  SPEAKER_00: Uh with the abundance of computing and data, we are now able to train much more sophisticated, much more complex algorithms that could do things that the previous generation of algorithms struggled with.
• 33:31
  SPEAKER_00: So they could do a really good job of understanding and working with um visual data.
• 33:37
  SPEAKER_00: They could do a much better job of understanding and working with sequential data.
• 33:42
  SPEAKER_00: So you had different training techniques that came into place where you were now doing representation learning, you were doing reinforcement learning.
• 33:52
  SPEAKER_00: And so, what that meant was the algorithms were now doing a lot of that feature engineering that previously machine learning engineers would do.
• 34:02
  SPEAKER_00: And so the algorithms were now processing data at a much higher volume.
• 34:07
  SPEAKER_00: They were way more complex, and they could now do way more complex tasks.
• 34:11
  SPEAKER_00: So they could do visual reasoning, they could do um machine translation, they could do image detection, you know, facial detection, all kinds of amazing things were now possible.
• 34:21
  SPEAKER_01: Yeah, I've worked on facial recognition using neural networks in 2011, and it was a project for the government of India uh for facial recognition across all airports in India.
• 34:33
  SPEAKER_01: That's phenomenal.
• 34:34
  SPEAKER_01: Yeah.
• 34:35
  SPEAKER_00: And it was a significant breakthrough.
• 34:38
  SPEAKER_00: Like the ability to do facial recognition uh would have been technically uh not quite feasible, like five years before that.
• 34:47
  SPEAKER_01: Yeah, I think the timing was right.
• 34:49
  SPEAKER_01: And um back then gesture recognition became a big thing.
• 34:52
  SPEAKER_00: Yeah.
• 34:53
  SPEAKER_01: So you have Latina recognition became a big thing.
• 34:56
  SPEAKER_01: If you've seen older movies like Angel and Demons, like all those certain labs had Latina recognition.
• 35:02
  SPEAKER_00: Yeah, yeah.
• 35:03
  SPEAKER_00: You could you you saw these uh post-detection algorithms uh become mainstream.
• 35:08
  SPEAKER_00: You had products like Kinect.
• 35:09
  SPEAKER_01: Um Flutter became popular, if you remember.
• 35:12
  SPEAKER_00: Yeah.
• 35:13
  SPEAKER_00: Yeah, yeah, yeah.
• 35:15
  SPEAKER_00: That takes me back quite a few years.
• 35:18
  SPEAKER_00: Yeah.
• 35:18
  SPEAKER_00: But but the whole thing over there was like you now had these, you know, really powerful algorithms, but they were a lot less interpretable.
• 35:26
  SPEAKER_00: These are very complex neural networks, and uh they took a lot of data to train.
• 35:32
  SPEAKER_00: And by that time, a lot of companies had acquired, had amassed a lot of data.
• 35:37
  SPEAKER_00: And these algorithms were more compute intensive, but GPUs were now becoming the way you trained and inferenced these models.
• 35:46
  SPEAKER_01: All the the labs globally, specifically, like Stanford and like many, many even in India, they started sharing their open data sets.
• 35:54
  SPEAKER_00: Yes.
• 35:55
  SPEAKER_00: You started seeing the emergence of these fairly large data sets, right?
• 36:00
  SPEAKER_00: And so now the emphasis was on how do you know do some degree of algorithmic exploration, but you know, a lot of these algorithms were used as is with a few tweaks.
• 36:12
  SPEAKER_01: It was only reserved for specialists, right?
• 36:14
  SPEAKER_00: No nobody the researchers, the machine learning researchers were now shaping and and and you know building these algorithms while the majority of the practitioners were using these algorithms, but they were focusing a lot on building out these data sets and training these models.
• 36:32
  SPEAKER_01: Yeah, I think in the last three to four years, it's been really democratization of AI.
• 36:38
  SPEAKER_00: Absolutely.
• 36:39
  SPEAKER_00: And so what has happened is like in 2017, the famous uh transformer papers uh came out, and transformers that had a seminal and seminal impact on the world of machine level.
• 36:53
  SPEAKER_01: What are transformers?
• 36:54
  SPEAKER_00: Well, transformers was an architecture that allowed these models to understand very long sequences of words of data, right?
• 37:09
  SPEAKER_00: Previously you had these um algorithms that could understand sequences of data, you know, take a long sentence and translate it.
• 37:17
  SPEAKER_00: But as the sentence grew longer, you know, it could probably deal with like 50 words or 100 words well.
• 37:24
  SPEAKER_00: But as you start getting into like, you know, a paragraph or a page of like thousands of words, these algorithms really struggled to understand the entire context and pay attention to all of the information that was presented to them, right?
• 37:39
  SPEAKER_00: They would pay more attention to the most recent words and then forget about the more uh words that occurred in the past.
• 37:47
  SPEAKER_00: And so there were very you know great limitations to the amount of data that these algorithms could process.
• 37:56
  SPEAKER_00: Transformers allowed these models to really work well with large context that was being provided to them.
• 38:04
  SPEAKER_00: So these models could now process a paragraph, a page, a whole chapter, or a whole book to produce the outputs in a meaningful, coherent way.
• 38:16
  SPEAKER_00: And so Transformers was this big seminal moment where the world kind of changed.
• 38:22
  SPEAKER_01: I'll give you an example from one of our portfolio companies, Buddy.
• 38:25
  SPEAKER_01: So they have built AI for senior care living homes, and this is the cambient AI, which exists in all their systems, including CRM, uh, right, and it doesn't have a UI.
• 38:35
  SPEAKER_01: But what had happened is uh, for example, they have built an AI teammate that can tell a salesperson that hey, uh the AI will create you your task on your email or your calendar and tell you that this is a senior that you need to reach out to because they have been considering home home uh there's a child that you need to need to reach out to because they have been exploring home for this grandparents, right?
• 39:00
  SPEAKER_01: Right.
• 39:00
  SPEAKER_01: And this is the right moment because you know, usually people start uh getting anxious after one year or one and a half year, and if they're not able to find anything.
• 39:10
  SPEAKER_01: Right.
• 39:11
  SPEAKER_01: And because AI can pass through their entire records, uh right, so they are able to create these tasks.
• 39:16
  SPEAKER_01: So uh so but uh what what uh happens uh is uh let's say it adapts to the context also.
• 39:25
  SPEAKER_01: For example, there's a book called Never Split the Difference, it's a very popular book in sales.
• 39:30
  SPEAKER_01: So if a senior care living home, the CRO uses terminology from Never Split the Difference culturally, the team uses this.
• 39:39
  SPEAKER_01: Uh the AI is able to absorb the entire and and uh you know the language of the AI is and it uses quotes from Never Split the Difference.
• 39:48
  SPEAKER_01: Yeah.
• 39:48
  SPEAKER_01: So people are amazed, like how did it adopt to our culture?
• 39:52
  SPEAKER_00: Yeah, it's just pretty amazing because ultimately all these algorithms they are essentially trying to predict the next word.
• 40:00
  SPEAKER_00: Yeah, right, given all the previous words that um uh it has been uh given, it's trying to predict the next word.
• 40:08
  SPEAKER_00: And initially, this used to be a highly statistical probabilistic operation.
• 40:14
  SPEAKER_00: You know, these models would look at uh, you know, uh they would be trained on a large corpus of information, you know, like a Wikipedia size data set.
• 40:25
  SPEAKER_00: And the the idea was that these algorithms are learning certain statistical patterns to predict what the next word is likely to be.
• 40:37
  SPEAKER_00: But as these algorithms have been trained on more and more data, with um these algorithms becoming more and more complex in terms of the number of parameters that make up these algorithms has grown exponentially.
• 40:52
  SPEAKER_00: We have seen some very emergent properties in these algorithms, right?
• 40:57
  SPEAKER_00: Uh, you know, at their heart, they are a probabilistic next word prediction system.
• 41:02
  SPEAKER_00: Uh, but they are now, they now seem to encode an understanding of grammar.
• 41:10
  SPEAKER_00: So the probability of the next word, uh, not just based on sort of the statistical occurrence in the training corpus, but they are now also able to apparently reason on whether this next word will make that entire sentence grammatically correct or not, right?
• 41:28
  SPEAKER_00: They are they're starting to understand facts, for example, right?
• 41:32
  SPEAKER_00: They are not only able to say what the next word is going to be, but they are able to now say, like, what is the probability, like, is it going to be factually accurate?
• 41:43
  SPEAKER_00: Because the probability of a sentence of that word being factually accurate makes it uh a higher probability than another word, which which also occurs very frequently in the corpus given that particular sentence, but is factually inaccurate.
• 42:00
  SPEAKER_00: So, for example, if the sentence is the boiling point of water is 100 degrees centigrade, right?
• 42:08
  SPEAKER_00: So the model will likely uh you know predict 100 degrees centigrade instead of 102 degrees centigrade, right?
• 42:16
  SPEAKER_00: And so we are starting to, it looks like the models are starting to learn facts.
• 42:20
  SPEAKER_00: Uh, the model is starting to learn to reason, right?
• 42:24
  SPEAKER_00: Like even if it's not a fact, it looks like the models are starting to understand whether a given proof is correct compared to an incorrect proof.
• 42:36
  SPEAKER_00: And so if you have a statement that looks like a proof, uh, you know, the word that completes that sentence, if it is going to be factually correct compared to another proof, another sentence or variant of the sentence where the word was probably, you know, um highly um probable for completing that sentence, but the proof was wrong, uh, the model will pick the right proof, right?
• 43:04
  SPEAKER_00: And so it seems like the models have built this reasoning capability.
• 43:08
  SPEAKER_00: And uh, you know, there's an argument to be made that, you know, how can models learn these kind of reasoning capabilities?
• 43:19
  SPEAKER_00: But there's also a counter-argument that the internet is big.
• 43:24
  SPEAKER_00: Any question, practically any question that you may have asked has probably been answered on the internet.
• 43:31
  SPEAKER_00: And so, like as these models have grown in size, as they've been trained on a lot more data, you're starting to see these very inter interesting emergent properties of intelligence in these models.
• 43:42
  SPEAKER_01: And as the last, you know, you know, we are coming towards the conclusion of the podcast.
• 43:47
  SPEAKER_01: Where do you think uh uh these models and capabilities are heading in 26 and 27?
• 43:56
  SPEAKER_00: You know, I've stopped trying to predict where these are going to Be uh a year away from now because they seem to be rapidly uh developing impressive capabilities day over day.
• 44:11
  SPEAKER_00: Um, you know, we've already seen models are now getting extremely good at reasoning, they're getting really good at uh orchestrations, they have gotten really good at solving analytical problems like coding, for example.
• 44:26
  SPEAKER_00: These models have become really good at generating code.
• 44:29
  SPEAKER_00: Uh, these models continue to get good at all of these capabilities at an accelerated pace.
• 44:37
  SPEAKER_00: So it's hard for me to say exactly what these models will be able to do a year from now.
• 44:44
  SPEAKER_00: But if you look at sort of the trajectory of these models, there was a lot of work being done maybe about a year ago to uh get these models to orchestrate complex tasks correctly.
• 44:57
  SPEAKER_00: People built these very fancy frameworks and systems that were fairly complex and complicated to improve the orchestration capability of the model.
• 45:09
  SPEAKER_00: And you know, there were some very impressive engineering feats that happened as a as a result of that.
• 45:17
  SPEAKER_00: But now the models are able to do that natively, right?
• 45:21
  SPEAKER_00: And so so it's it's pretty amazing that uh we have just you know built models that are now able to do these kind of planning and execution tasks uh that they would not have, like we would not have imagined about a year ago that they would be capable of doing.
• 45:41
  SPEAKER_00: Yeah.
• 45:41
  SPEAKER_00: You know, like if you were working on the early days of GPT uh 3.5, people spent a lot of time trying to make the models work with a very small context window of 4096 tokens with uh very weak instruction following.
• 46:00
  SPEAKER_00: Uh, and a lot of what people built then is now completely unnecessary.
• 46:06
  SPEAKER_00: The models just do that inherently.
• 46:08
  SPEAKER_00: And so um, you know, who knows what the models are going to be doing a year from now, but I'm sure it'll be impressive.
• 46:14
  SPEAKER_01: Thank you so much, I mean I love that conversation.
• 46:16
  SPEAKER_01: Thank you so much.
• 46:17
  SPEAKER_01: You know, and I love you know how candidness you brought in in the conversation and how simple you made it.
• 46:23
  SPEAKER_00: Thank you so much for having me.