Latest / Convo AI World / Beyond Evaluations: Trinitite on Building Trustworthy AI Agents

Transcript

Beyond Evaluations: Trinitite on Building Trustworthy AI Agents · Convo AI World

• Hermes Frangoudis (00:12.076) Hey everyone, welcome to the Convo AI World Podcast, where we interview the founders and leaders pushing the voice AI space forward. Today I'm very excited for our guest Dustin Allen from Trinitite AI. Thanks for joining us today. Dustin Allen (00:27.842) Thank you, Hermes. It's honestly a pleasure to be here. Really excited to dig in and kind of see where this goes. Hermes Frangoudis (00:35.072) Awesome. Well let's jump into it. One of the ways I really like to start off every episode is kinda like the origin story. So can you tell us a little bit about where Trinitite came from? Dustin Allen (00:45.696) Yeah. So my co-founder, Hearsch Jariwala, gotta give him a shout out. He's he's amazing. we We had been building a productivity app. We had We were really Microsoft Office agent before that was a thing. So we had been working with accountants and finance teams to really make them have AI agents on their machines that were safe, reliable and understood their workflows and you know, we started selling more and more to enterprise and we realized really what we had built to make sure that our agents didn't go off the rails is what these enterprises needed when we really peeled back the onion and saw like, 'Hey, do you know how your AI is doing this? Can you track it? Do you have scopes?'. Like all the things that in traditional kind of engineering we would have said, this is how we want to deliver a product with guardrails, we we realized the the industry was missing that. That was our product from Lingo. Hermes Frangoudis (01:46.892) Okay, so you guys started out kind of as this productivity software and then realize the tools we built are s for our own self usage are actually better suited for for what the market needs versus like the original product, which is a huge change, right? Dustin Allen (02:06.886) You know, it it is and it's funny because a lot of like a lot of it came from like where we found it. Like The first time we screwed up an accountant's Excel spreadsheet, like that was not a fun conversation. And so like we had to figure out how do you replay events, how do you restore data, how do you know what caused this agent to do it. And so we had spent so much time trying to really create determinism around non-deterministic systems that you know, the bulk of the value of our product was really in the reliability and less in how do you, you know, make a spreadsheet do something that Anthropic can do at this point. Hermes Frangoudis (02:48.651) So that like it wasn't so much the experience of building the telehealth infrastructure, like the productivity infrastructure. Sorry. sorry, let me restart this. So it wasn't so much building the productivity tool, it was like when you went to your first customer, realized there was an issue, and then you had to look at your own stack and kind of understand like where did this go off the rails? Dustin Allen (03:16.454) Yeah, I mean, 'cause you know, as an early stage startup, it's all about customer experience. And so you're they're doing a lot of hand holding and yeah, we we really messed up in one of the early days and we were like, 'We have to figure out how to fix this and then how to remediate this. How do you ensure it doesn't happen again?'. And that's where we started just spending so much time and you know, it's out of those lived experiences that I I think that you have some of the best conversations because when you're talking with folks it's it's very deep, it's very personal. It's not having to just listen to them, but it's when you listen, you go, I I can really empathize. I can reach across the aisle and say, I've been there. I know what that's like. Hermes Frangoudis (04:03.015) Makes sense. So from that that idea, that issue that you experience, Trinitite as a platform was born, right? Like so you guys now focus purely on AI governance and trust infrastructure? Or how would you how would you put it in your own words? Dustin Allen (04:20.824) Yeah, you know what, I I think that the industry really is kind of catching up. It's it's interesting. Traditional governance wasn't really connected with the day-to-day delivery of engineering, right? Like you had somebody on the compliance team once a quarter that asked, you know, the team for a bunch of Jira tickets and like, what did we do in change management? Or can I see some kind of like verification of a failover? Like all of these little things, but AI is so fundamentally different that when you deliver an AI agent, governance is built into the product itself. And so what we're really finding is it's a mesh between AI delivery, AI governance, and AI security. And you you really can't separate the three because if you do, you end up with a a patchwork of products that don't really connect, they don't talk with each other, and it makes maintenance and really fixing it on night. Hermes Frangoudis (05:29.035) So that's probably the moment you guys realize, you know, evals alone are not the only piece of it. Like there there's these two other pieces that are really important to make the puzzle work or to make the flow work. Dustin Allen (05:41.422) Yeah, I mean, because we like to dog food. So as we were building these things of how do we evolve for ourselves to try different areas, we're like, 'Well, that's not enough'. You know, it's great to evaluate and understand if something's right or wrong, but then how do you stop it in real time? Or then how do you create a closed loop system that improves the accuracy of the product day-to-day to where you really learned the workflow. behaviors of an enterprise versus just guessing at it through some generic workflow. Hermes Frangoudis (06:17.099) Got you. So a couple things in that statement that I'd love to unpack a little bit. First is you talked about continuous monitoring. So how do you continuously monitor without adding latency? And then what kind of signals are you watching for? Like what are those pieces that you don't just guess on? Dustin Allen (06:37.847) Yeah, no, and and so this is what's really fun because there's two ways to do it. And when we start talking about voice, you know, the Agora world, like this is where latency matters. You can't just like have a a product. It it has to be continuous. And so we we have an asynchronous mode where essentially we're we're kind of like a parallel task. As you know, the audio, the text or the video is playing along, we're gonna sit there and monitor every interaction. And if there's latency, which you know, there's gonna be somewhere between 400 milliseconds to a few seconds at times, usually, it has no impact on the product. And the really sensitive operations, the tool calls, the areas where you're actually gonna expose data or perform you know unwindable actions, it's easier to sit in the middle of that layer because the product experience is less impacted, and it's a lot more about okay, maybe this is an API taking a little longer, maybe the processing it's easier to hide the invisibility of the impact while having a high value. Then the other side of it is, 'Hey, we're just like an enterprise or like a highly regulated environment, and every single action has to be monitored'. You know, we tell people you're gonna have latency, but the trade-off is is if you're not in the audio world, then most of these AI agents are taking 30 seconds to think and respond. And so, you know, 400 milliseconds to a few seconds is really negligible to the impact. That said, we are working on very low parameter models and you know, we've been studying folks like Liquid AI and what Google is doing around some of how they're processing tokens at once and are using recur recursive neural networks to where we can kind of sit in the middle synchronously in the coming year or so around audio. That way, you know, the latency is negligible, but the value chain that we can deliver is there across the space. Hermes Frangoudis (08:59.421) Makes sense. So that's how you keep the latency from growing. In terms of like signals, what what signals are you looking at that maybe more traditional eval systems would would miss? Dustin Allen (09:14.297) Yeah, and so this is where we almost have to like take a tangent to kind of explain what this means is what we do is we look for precision. And all of the signals that I'll talk about afterwards are based on this idea of precision. Because if you're using AI, you're used to the idea, especially generative AI, that you're not gonna get the same output. That it's stochastic or it's probabilistic. The the truth is, is Thinking Machine Labs solved that problem almost a year ago. And so what we do is we implement that math, their libraries combined with whatever inference engines we're using and what models we're using, to ensure that the same input delivers the same output every time. And so what that allows us to do is when we talk about data signals. We actually have a clear signal coming from our side. Instead of, you know, there's a small chance that the model made something up on this side, we get that same level of precision. So when you start talking about data signals, the first thing that we like to talk about is like good data in, good data out, right? That's really what that means on our side for AI. So then when we start looking at you know bias monitoring. Or how does this relate to your policy that is implementing some given control, which is inside of some framework that an auditor is going to audit you on down the road? Or all the way down to is this how marketing wanted the agent to create these designs? All of that is there for the signals that we want to listen to, along with all of the little things like token count, you know how much spend is happening, behavioral analytics, what data is being accessed. But all of that is really table stakes if you get the data right when you're putting it in. You get that lossless experience. If you don't, then everything in your system is kind of muddy, and you don't really know is this because of the AI on my side? Dustin Allen (11:39.225) Is this because of the AI on their side? And it becomes a guessing game to where you're you're kind of playing a little bit of a whack-a-mole with everything downstream. Hermes Frangoudis (11:50.208) Makes sense. So really the heart of it is creating that I think during our our first call together you mentioned this it was like the deterministic output. Every every input along this line always gets that same output. So you can really test against that and make sure like there's a clear breakpoint as to where the hallucination might be happening. Dustin Allen (12:14.98) Yeah, I mean that deterministic inference is so important. I I I've told a lot of customers this. It's kind of like Waze and Google Maps when they first got access to GPS and like mobile signals. So like they could start taking the true data points of where things are actually at and plotting like 'That's a gravel road, that's gonna take you into the middle of the ocean, don't drive there,' versus 'Hey, that's a good highway'. The That's really the way that it allows these models to behave. Once you can make sure the inference creates the same output every time, you can trace back and say, 'Well, what if I changed a word here? What if I change the weights a little bit here?'. And it allows the explainability and for some of the stuff we do behind the scenes to like look at the embedding layers and actually spot where those, you know, risky areas are at and start to really guard around that at the lower level embedding layers. Hermes Frangoudis (13:23.755) Makes sense. So what industries are really like adopting AI governance the fastest today? Where do you see that kind of like growing? Dustin Allen (13:33.873) You know, it's it's kind of the usual players candidly. Audit firms, they're, you know, they're the ones out there having to explain this. And so we have seen a huge push recently with a lot of audit firms trying to shore up their governance programs. But actually, beyond that, we've seen some really cool industries, aviation was one that we've seen recently, which never would have guessed, but you know, with where they're going, chatbots, trying to put things in cockpits, all of these things to improve their efficiency, it's becoming top of mind. Law firms, insurance companies. And then, you know, you see a lot of the AI, not necessarily pre-seed startups, but mid-to-growth to you know growth stage startups that are really starting to say, this is how I can win some of these enterprise deals and prove that my governance layer is better. So it's it really comes down to like, do you pay a fine if you mess something up on the other side? If so, healthcare, for instance, they're probably talking with us or one of our competitors. If you're selling books or you know, doing, you know recommendations for food, it's probably not there yet, but in the next 12 to 18 months, I I expect it to kind of be at that point. Hermes Frangoudis (15:11.849) Makes sense. So s right now, super heavily regulated industries are are really jumping in. Early stage startups that are trying to break into that enterprise layer. Totally makes sense. In terms of Let's see. Dustin Allen (15:31.067) And I, Hermes, I will say, since the OpenAI sandbox escape, where they attacked Hugging Face, and then like a week later, Anthropic is like, "Actually, we attacked three people". And then Meta is like, "Let me one-up you and say that we attacked some people." And then OpenClaw, I think l earlier this week, you know, escaped and put, you know, somebody at the front of the line at a gym. What we're seeing is like these things are escalating. And so I I don't think it's gonna stay just regulated industries for long because I think that we're hitting that inflection point of it is a concern that everyday agents can run amok and how do you just protect yourself? Hermes Frangoudis (16:20.191) Wild. It's like the, it used to be only the big businesses had to worry about it. Now everyone's got to worry about it because there's this level of autonomy that probably wouldn't have been there before if they had a human that they could kind of go to and make sure was doing their job right. Speaking of these failures, like When when you have a customer that That goes live. And what kind of like failures are these enterprises seeing in production? Like like what are the areas that you're catching and and they're able to fix? Dustin Allen (16:55.128) Yeah. Dustin Allen (16:59.844) Yeah, well, it they seem silly, but like tool calls that respond a 400, a 401 you know, unauthorized error or an empty array, you know, completely empty, cause they just the query wasn't right, the data is not there. Well, downstream, it knows that it's supposed to connect to this system as part of this agentic workflow, so it just starts making up whatever data will fit the next task tool calls or what is needed to move on to an ERP system or to explain to an HR. But when you look at the logs, it was an empty array. And so those are some of the areas that it's like low-hanging fruit that you don't realize 5 to 10% of the time or just going through unnoticed. And then, you know, what we see a lot more often after that is really the delta between what they thought they were doing with their policies, not just compliance, but like this is what we wanted the workflow to be, versus this is what's actually happening a thousand times out of a thousand times. What they'll find is there's actually a pretty big delta. You know, it's 30, 40% of the time, it's not that way. And it's, okay, this is where we need to improve either the underlying models instructions and how it's accessing data, or you know, we need to train your guardians a little bit better so that way they can learn your policies and help correct some of that in line. Those are two of the areas that we see just like immediately. And then the third area is just like money. It's like how much are they actually spending? Who's touching what? And then, you know, how does that roll over to ROI? Revenue attachments for those agents versus token cost. That's the kind of like other piece of the puzzle that we kind of push to the business side from all of these data signals we're capturing. So they can go, well, why is Dustin running an agent that's costing us $100,000 a year to like check for flights for our HR people? Like that that doesn't make any sense because it's there's not enough value. Hermes Frangoudis (19:24.107) So there's it sounds like there's a shift from simply being like these are the The Hermes Frangoudis (19:36.499) Sounds like there's a shift. These are like the instead of simply describing the risks, now they're able to like actually quantify them and then like quantify the cost to the business and understanding like, okay, this is what we were worried about. This is what it looks like when it actually materializes. How do we fix that sort of thing? Dustin Allen (19:56.624) Yeah, and you know, the cool thing is, is it when you use deterministic inference, you know, when your AI is the same, you can actually start getting into remediation as well. Because then you know it's not the other system maybe a little bit wrong. We're just kind of trying to take a probability. Hey, this is that variation that we see time and time again. We have another system that's always gonna give us the same output. Let's run our guardrails through a thousand unit test of that one particular area and capture all of our known risk. Is it actually stopping all of those? If not, let's, you know, train it a little bit better, remediate it, and prove now that we're catching all of what we know as known risk for an auditor and for all of the liability, the insurer, the the ones that now the CFO that just got this crappy report about like how much money we're spending can go, well, you know what? At least the risk is covered, so like go do your thing, it should be. Hermes Frangoudis (21:05.429) Fair enough. So a lot of these scenarios that you're talking about, it's really almost comes across as like reactive. Is there a way that this governance and reporting and and all this that you guys are doing, is it something that can make the process more proactive? Like, all right, we saw this over here, let's also fix these other things that might have similar issues. Dustin Allen (21:37.619) I love that you said that. So what we offer is essentially we have the same agent that we've built to kind of be deterministic. We We train that same model or a LoRA, you know, a small little similar model around it on their risk. And what we do is we fan through all of their known logs, everything that we've captured. And then start to kind of surface the risks that we're seeing outside of that, that isn't necessarily captured at runtime. And for enterprises, they already self-host. So at night, they're not running as many agents. So it allows us to do these kind of batched operations, we're getting the most value out of that GPU, running continuous assurance where they start to understand how this fully crosswalks to compliance, risk analysis, gap analysis. They have their own mini auditor with audit rails underneath it that runs through that population, which is 100% coverage, and starts to surface that. And that that isn't just, 'Hey, this is a prompt that we saw'. This is shadow AI. This is threat intelligence reports. You know, we're taking red teaming from MITRE and then putting the latest tests that are available through them and running it behind the scenes and saying, 'Hey, we noticed that there's a gap here'. So what we try to do is say, 'You already got the AI, you already got the context, you need to move beyond evals. You need to do all of these other things'. That way every morning when somebody's waking up, they're going, 'Okay, this is what the latest business is' or in a future world, that agent that you built is then picking it up and then doing something with it from there. That That's really where it is. Hermes Frangoudis (23:36.085) Super interesting. It's like the bleeding edge of this part of the the AI space. Dustin Allen (23:43.121) It And that's why I said, you know, it's hard to separate AI governance from AI delivery, because the reality is, is you can't be reactive. If you're gonna constantly be reactive, AI is constantly gonna find ways to like get around it. So you you have to kind of button up the hatches proactively and say, okay, before something happens, let's let's make sure we're we're covered as much as possible. Hermes Frangoudis (24:13.247) Makes sense and it's like one of those areas you want to definitely be getting ahead. And all of this sounds really good and fine when we're talking about text, right? Like on the text side of it, it's a lot easier because the latency is, you know more forgiving when I'm typing. You can have like a little thing like agents thinking. But when we're getting into the voice AI world, this has to introduce new challenges. So What makes voice AI systems, like different, harder to govern, like w what what are the challenges in that space? Dustin Allen (24:49.874) The real-time governance is a huge challenge because of really a few things. When we start talking about deterministic inference, what we do is we stop truncating, we stop kind of like cutting off part of the math, which means inference by definition does slow down, usually by 30 to 50%. So now you have a slower inference that is sitting in the middle of a highly a highly latent sensitive system where if any interruption occurs, the user experience degrades tenfold comparative to any other system. So this is where you have to have a combination of very low parameter models. I mean, we're talking three, four hundred parameters. And you really have to look at what is the architecture. You can't use a normal, just, you know next token prediction transformer model. You have to use that multi-token prediction. You have to use recursive neural networks. You have to use world models that are, you know, not thinking. You you have to be able to get to the point where you're at an architecture that is delivering multiple answers at once instead of saying 'I'm going to follow that audio encoder and try and keep up with it'. Because that's what that was built was for streaming low latency, you know, text-to-audio, audio-to-text to be able to say, this is a fantastic experience. You're not gonna keep up with that if you try and keep the same architecture. So that's really the the difficult thing is taking that and then bridging the math and saying, 'Okay, you know, Multi-token prediction instead of, you know, next token and next token and next token, we're doing all of it at once and then putting that outcome out there. But that means now, we have no guarantee that the order of operations, the way that you know the layers are going through, may swap because of how multi token predictions are done. So, how do we handle pointers under the hood to be able to make sure that Dustin Allen (27:13.36) we can know what was done, where, while being able to restore that across any device or anybody that replays that. That's where the fun challenges of AI come in, because then it's like really deep at the inference level and saying, 'Okay, how do we figure this out?. Hermes Frangoudis (27:36.182) Seems like the LLM layer just got a little bit more complex. But for this sort of highly regulated systems, I can't imagine simplicity is best. It's it's more so like having all the right tools in place to actually make it, you said deterministic and make sure that the thing is actually staying on the rails because that's the most important piece of it. Dustin Allen (28:01.849) It is, but that's also where it makes it the hardest too, right? If you think about the healthcare sector where folks want to be able to talk to somebody, to be able to get that feedback, not necessarily as a doctor, but as the intermediary to be able to understand what their records are, or to be able to, you know, explain a symptom and make sure that that's really articulated well. So when we see the doctor, it's already available. It's got everything that they would have said, and it's a much healthier experience. But then how do you do that in a way that doesn't violate HIPAA, that doesn't necessarily, you know, introduce the hallucinations that could make up some kind of health record or health data that doesn't actually align with the standard of care that that system wanted to deliver? And so this is where we have a couple of folks that we're, you know, an early stage pilots with. And it's it's not something that we have done in real time and said it's been very flagged and saying, 'Okay, you know, I made a mistake. This is what it should be'. And so there's ways to introduce this into audio where you can be very candid and say, because we have this asynchronous stream going on, we can alert that and say, okay, this was the mistake. This is what we need to clear up. And so it's not perfect, but it does prevent that downstream impact that ultimately is what causes either a bad user experience or some kind of legal financial consequence. Hermes Frangoudis (29:47.18) Makes sense. So it's It's not always about being right there in the real time, but also more so being able to catch those when they do happen, flag them, adjust the system and keep it going because you don't want to degrade that real time experience. It becomes too tough. Dustin Allen (30:05.211) Yeah, I mean, you know, it's a business, right? Like any any security product over the years has realized if they like destroy the user experience of the underlying product, they're getting thrown out the second that it's in there. And it doesn't matter how good the product is, is if it impacts the downstream experience, it's not worthwhile. And I I think that's something that, you know, i you have to keep top of mind because at the end of the day this is about how this impacts somebody's life. Like it's great to have governance, but governance doesn't mean anything if that person now doesn't get the health care that they need, and then you know, downstream their life is literally impacted, all because you wanted to put two rigid of guardrails that never allowed any mistake that maybe wasn't that impactful if you could kind of correct it a few seconds later. Hermes Frangoudis (31:03.787) No, that's the real world impact on people's lives kind of changes the the feeling of the importance of certain pieces. But so when we think about a voice pipeline, traditionally it's more cascading than multimodal. So where would Trinite sit in a traditional cascading pipeline? Like do you sit at the ASR, like the speech-to-text layer, or do you sit at the LLM layer, the text-to-speech layer, or do you sit everywhere? Do you kind of have a piece in everything? Dustin Allen (31:37.905) Yeah, so you know, traditionally what we have tried to do was sit kind of right after the ASR layer. So then we know what text is essentially transcripted before it goes into the LLM. So just like traditional prompt filtering, you can say like, if you're trying to like hack the system, find like prompt instructions, like do the traditional hacking methods, like, yeah, we could just filter that out. The problem with you know, going too deep into that is you have to have a giant model for that. So it's easy to kind of like spot keywords with that or key phrases. But after that, what we really focus on is that LLM to the next layer. So this is where you're gonna see two different areas where we would be injected. One would be between the tool and the LLM because then we can kind of catch anything. If it's calling you know a database to look something up, we can make sure scopes are in session, it's got the right compliance, everything is copacetic in that moment. If it is, then the other area is kind of that asynchronous approach where we don't want to sit in the way of delivery and you know the user experience. So, what we're gonna do is look for those violations behind the scenes and say, 'Hey, hey, Agora, hey, hey, hey. We noticed something that was just said, you know, we need to loop it back and then go through that pipeline again just to make sure that you re-deliver what we found as a violation'. So in that stack you'd see us in three places and it would just kind of depend on like how impactful, how compliant you are. Like maybe we just sit in two places and it's just at the tool call layer, you know, maybe it's one place and it's you know, we're doing such low level risk. We just wanna make sure that if we have a bad experience we can catch it and the model can just say, 'Hey, whoops, screwed up, here's what I missed'. Hermes Frangoudis (33:43.372) I think that's all well and fine when you have these cascading pipelines, but how does this work with something more like a multimodal where you get voice in, voice out? And I've definitely seen these hallucinations where like the voice out doesn't match the text out. So how would you catch stuff like that? Dustin Allen (34:02.45) So this is this is where it's kind of fun. That same deterministic inference does work for those multimodal systems. And so you can take a, you know, a Qwen or you know a model like that that can take an audio/video under the hood and outs, you know, not going through an actual transcription layer, but it's just embedding that in its own vision or audio encoder and then outputting it to text. That's where it's really fun because when you know that it's wrong, but you're precise every time about it being wrong, you can move the needle to what it takes to get it right. Versus right now, if you just have it in a normal state, you can't tell if it's the model, if you didn't instruct it right. Like what is it that is really causing it to kind of slip up in its delivery? It's a lot easier when it's in a fixed state. Hermes Frangoudis (35:02.035) So that deterministic outcome is really what allows you to be very precise on catching these things. Never thought of it that way. Dustin Allen (35:12.648) Yeah, I mean I what I what I like to like think of it as is like shooting an arrow at like a bullseye, you know? Like it's great for the guy that gets the bullseye the first time but then like you know misses the target the second time and then like hits like the outer ring the third time. Like that tends to be what you'll see from an LLM. Like it's the, f you know, that production instance isn't like the demo. The demo is perfect. I don't understand. For us, that's where it's it's better to be precise. If you're missing the target every time, but it's the same shot every time, you you know the difference. You can kind of figure out that delta and understand is it literally the audio encoder and like we need to just sh literally have it take some speech scenarios with perfect text and then retrain part of that audio encoder under the hood. Using deterministic inference, so we we get the same output on the model. Or are our instructions when we just like say something to try and guide it just screwing up how all of it's embedded and it's starting to kind of do some gargly goobly at a certain level? And that's just because that multimodal probably has a fraction for the audio and video layers that they do for the text layers, and so it needs more concrete examples. Hermes Frangoudis (36:39.947) So it's being able to take it back to like what was the training, what was the data that was given into it, and being able to, that's pretty interesting. Shifting a little bit. So we've talked about what your team offers to enterprises and bringing in these solutions. But word on the street is you're you're kind of building your own tools for developers, right? You're building a d You have a developer platform for this. Yeah. And how does that look versus like what you've done traditional? Dustin Allen (37:09.256) Yeah. Dustin Allen (37:14.345) Yeah, what we did is, w we essentially took a lot of our learnings from like the enterprise stuff and said, 'Okay, how do we abstract that away and just like a few lines of code? and And really deliver that to users as just an SDK'. And so from our perspective, that's really what we focused on. How do you say 'I don't need to know what non-human identities are to have a safe and governed agent. I don't need to understand what the 16 different controls that are gonna apply to me as an AI agent across maybe SOC and ISO are gonna be, but I still need to be governed by them. I still need to have you know coverage against it'. That's where we come in and say, 'You don't need to know all of that. Just tell us, you know, do you want SOC? Do you want ISO? Do you want our default best practices built in from what we've seen? And if so, just kind of run through our proxy and we'll connect IDs for you. We'll start calculating that spin. We'll start saying, this is what we recommend for some of your downstream systems. We'll red-team for you. We'll do all of these things behind the scenes'. And what we're really saying is before you even get there, just test the product as a free app on your desktop. It's just a little guardian that like as you're using Claude Code or Codex, it it's gonna pop up and say, 'Hey Hermes, are you sure you want this file deleted? Like just want to pause and make sure. Are you sure you want to send an API key right now that, you know, is to Anthropic and has sensitive data now that we know their chain of thought can be decrypted and you can see everything in a conversation? Like, is that what you want?'. And then introduce folks more and more to what guardian agents really are and say, 'Now that you understand them, it's a few lines of code, it connects to all of your Dustin Allen (39:26.131) governance, it'll help you with delivery downstream and the truth is you're probably gonna put that into cursor or like Codex or Claude Code anyways so just connect our MCP, it's got our docs. Have at it, you're gonna have a good time'. Hermes Frangoudis (39:44.479) So using the AI to make it really simple for developers to be able to embed this within their own applications and kinda use it without having to understand all the intricacies and details that go into being an expert in the space. Dustin Allen (39:58.976) Yeah, it's so w we were fortunate to be forward deployed engineers and one of the big fours, and essentially work with them and we realized very quickly there's just too much going on for an engineer right now to be able to fulfill the business needs. You know, I need to do a new agent that does X, Y and Z. We have to keep up with productivity. We have to deliver this while at the same time understanding how all of these things intersect to their reliability and their governance. We said It there's not really that many humans on the planet that have like the cognitive bandwidth in enterprise to just be like I can focus just on this. And so you you kind of need the AI to start guarding the AI. Hermes Frangoudis (40:54.219) So as the AI becomes more autonomous, right, like how do you see the observability of these AI agents kind of evolving? Dustin Allen (41:04.861) This is where it's it's gotta get in some ways in in multi-staged approaches. Because what you're gonna find is you're gonna have these network layer folks that, you know, we do it, you know, work with a Z scale or a Cisco, you know, take a network packet, run it through and say, this isn't what you set up to use for open AI or whatever the case is, but you can download models onto your machine. That never goes through a network layer. It can go through and do all of the same things and seem like innocuous traffic that would never be detected as AI traffic. And so, you know, you have to both sit kind of at the edge and at the network layer to observe what these AI agents are doing as we see more and more local AI start to spring up. And I think that's where you're gonna see traditional IT start to struggle because everything over the past 20 years has been cloud, cloud, cloud with some mobile device management and you know maybe a VPN installed for zero trust, but like after that they don't have a good idea of what's happening at a daemon level, what's happening at you know some root level that could access files or set up a process that's doing something that the network doesn't necessarily understand. Hermes Frangoudis (42:37.611) To me it sounds like AI governance is almost gonna have to be as ubiquitous as like standard logging or application monitoring. Is that is that how you guys see it? Dustin Allen (42:50.358) Yeah, I mean, Hermes, I think you nailed it. it's It started a lot around compliance because of the EU AI Act and you know, New York and a few others really starting to say, 'Hey, we need to put some laws and regulations'. But it kinda coincided with this shift towards autonomy. And now, to your point, it it's gonna kinda have to be the standard line item that you have to deliver. Like I've got an MFA provider, like you've got to have the AI governance. It And in the few years we'll probably see like insurance just won't cover you if you don't have these prerequisites that you can guarantee are sitting across your network and at the edge. Hermes Frangoudis (43:41.772) Yeah, they're not gonna wanna pay out if there's something that you could have done to protect yourself and didn't, right? Like it's it's like a car insurance. You didn't, your your car wasn't inspected and pass all these other compliance things? Yeah, you know, that's out of policy. So that totally makes sense as like the bigger shift in the overall landscape. And speaking of landscape, I wanna maybe dig down into this. So there's like I feel like Dustin Allen (43:47.392) Yeah. Dustin Allen (43:57.078) Yep. Hermes Frangoudis (44:08.487) AI evals is like a growing field. Like every day I turn around, there's a whole new batch of like startups, frameworks. What do you think a lot of these companies are missing from like the bigger picture? Like what are they failing to cover that your team is actually covering at this point? Dustin Allen (44:32.296) So I like to think of it as like we're the glue, and that's what a lot of people are missing. Like we're the Elmer's glue that, you know, is gonna sit there and connect the different pieces together where they stop moving. One of the things that we've done really well is we've thought about ourselves as not really a platform that we have to be the full delivery layer, but instead we'll meet you where you're at. So if you already have a guardrail set up that you like, that's great. But how are you connecting that to the non-human identities? How are you connecting that to FinOps? How is that connected to your policies? What we try to do is say, okay, you've got some of these areas set up, but you have to have a holistic picture. And if you don't have that connection layer to sit between them. Then you're missing something. And I I think that's what we've seen is players are popping up left and right around point solutions that we do this really well, but the industry is moving so quickly that you you kind of need to just be able to say, I can't figure out how these eight different tools go together. Can like, is there something I can do in between that, that just makes it a seamless process? And I I think that's where we found a pretty good niche. Hermes Frangoudis (46:00.022) Just being able to come in and fit exactly where the customer needs you instead of forcing them to really replace everything. So The flexibility is huge. And you know that from like enterprise. Like once they've already brought in a vendor for something, someone's gonna be really reluctant to pull them out just to be able to bring you as a vendor. So that that flexibility I feel like is paramount for the business. Dustin Allen (46:09.129) Exactly. Dustin Allen (46:26.603) Yeah, and it it it's for us, it's all about AI enablement. So if there's one or two areas that are blocking the business stakeholders from saying yes to enable this, like that's the most critical area. So how do we solve those problems first? And then everything else is kinda like what you spend that crawl walk-run approach for. It's like, 'Okay we have a vision of what we want this to look like, but you know, it doesn't have to be that way at day one'. Just how do we get your general counsel to finally check the box and say, 'Okay, you guys can deploy this to production now?'. Hermes Frangoudis (47:09.323) Speaking of like the industry, what are some of the industry trends that are going on right now that most excite you and your team? Like what do you think is really cool that maybe people aren't talking about? Dustin Allen (47:24.627) So I I I got to speak with this really cool developer. he He works at Thompson Reuters and it was just, you know, like friendly conversations. I'll I'll shout him out. His name's Pankaj, really smart guy, has like a master's in CompSci, and he was doing some papers around how there is this noticeable trend of you know, evals that have focused so much on like, is it right or wrong that we've completely forgotten? That like, is the verbosity of the LLM make any sense? Like, is this actually a cogent, articulate statement versus like, is it right or wrong? And so what he's found is there's actually a huge bias in a lot of these models to be able to over talk, overshare. And it's a huge deviation from like how we would naturally speak and how we would naturally answer. And so you when you combine that with world models, which, I you know, is an area we're very fascinated by, I think you start to kind of like look at some areas where like, okay, we're starting to see where we can take the industry and start to model an output that we want, that we can go from is it right or wrong to is this really how we would have done it ourselves? And really see the next few years of delivery start to be a lot more about an output that is modeled around how humans behave and how we want things done. And a way to really look at that not as just probability, but like literally why did this happen and the explainability behind it, which I I think is really cool. So those are the areas that we've been really fascinated by because it partly touches our work, but then partly gets into where I think that the underlying use of Transformers is headed. Hermes Frangoudis (49:35.445) Speaking of where things are heading, where do you see things for AI trusty like trustworthiness within AI? Where do you see that going in the next like let's say two years because things just move so fast that you can't even predict like three to five? Dustin Allen (49:54.361) I hate to say this, but it kind of feels like it'll be worse before it's gonna get better. it It feels like all of the folks that over the past few years were pointing out, hey, AI is going to get to the point it can do these things that they're starting to become right. And I think that the general public that companies and you know everyone in between are gonna have to experience more pain, more unfortunately like costly mistakes before you start to see an emphasis on trust being done in a way that everybody can come together and kind of agree with it. It's not so much anymore, 'Hey, you know, this is what we think trust is, hey, this is what we think trust is'. You know, Claude has a very specific way of doing trust, whether it's OAuth, whether it's, you know, handling SOC a very specific way, or literally, you know, when we go and set up a you know API, are we going through OWASP and things like that? There are all of these areas to where we've come together through best practices and said, 'This is how you build trust'. in the ecosystem'. I think that the people are in place to do it right now. You have orgs pushing, but I don't think that we've felt enough pain across both enterprise and consumers that you're gonna see everybody agree on something for another year or two. And then I think you're gonna see that just take off like a rocket. Dustin Allen (51:52.428) Hey Hermes, you're on mute, by the way. Hermes Frangoudis (51:58.646) So do you think these systems are as they become more and more autonomous, they're gonna have their own observability layers, right? Like like everyone's gonna have to have one. Do you think that keeps us from getting to those weird like outcomes in the AI world where you know, when they say what's the future of like AI and it's it's gonna take over humanity, we're building this species that's smarter than us? Do you think these guardrails actually are what keep it from being to become too powerful? Because now we're putting business rules around it when we're giving it a lot of power. Dustin Allen (52:10.68) I mean if Dustin Allen (52:40.918) This is an answer that I would say partly yes, but then partly no. Because the truth of the matter is is there's gonna be security vulnerabilities that exist in legacy architecture, in modern libraries, and we're gonna see AI exploit that more and more in ways that some guardrails wouldn't necessarily be set up to solve. We We've done studies where we've shown like a guardian agent can 100% block a or at least a deterministic one, the sandbox escapes that you could otherwise do with these agents pretty quickly. But the reality is, is there's always going to be a hold somewhere. And trying to sell a perfect solution is kind of selling snake oil. And so I am a firm believer that you have to understand that it's not always about getting it right the first time, but how quickly can you remediate those issues so they don't happen again. And I think that's really where you're gonna find a mix of both AI having to be in place and do the observability around it and all of that, but then still the traditional human oversight where, like, is this how we want to approach it going forward? You know, yes, AI can give us all the recommendations, but we still need to be able to understand both it in an engineering and architecture layer, how we want to design these systems to reduce what may have happened and actually create a more scalable, secure system. Hermes Frangoudis (54:29.269) So it's not like you're gonna catch the the model doing this and and stop it before it happens in every case. But there's gonna be the ability to kinda like slow it down from doing it at scale. Dustin Allen (54:43.339) Yeah, I mean most of the time you're gonna catch it. You know, that's why it's there, but there are zero days. There are edge cases, and this is where deterministic inference becomes so important because when you no longer have that kind of moving target with your AI, you can figure out why you had that edge case slip through, why that zero day happened, and you can harden really that model around all of the different variations of what that would be. So it's kind of like, you know, your house may catch on fire as it, you know, and ensure you're you're kind of like looking at all your properties, like these are all my little agents. This is what I gotta make sure works correctly. You know, you may have a fire in one house, but if you study the fire, you can make sure that none of your other houses that are in your system ever have a fire again. And I I think that's the approach you have to take is, this post-mortem analysis, this remediation, these gap analysis, these become so important for the proactive approach that you talked about versus reactive where you're always gonna be behind. Hermes Frangoudis (55:56.852) Yeah, so it's about being able to like as they become more autonomous, being able to like better understand the risks, catch some of these issues. And yeah, like we were saying or you were saying the the proactiveness is really that important element to to keep it in the, keep it within its confines overall. Dustin Allen (56:17.635) Yeah, and the it it does introduce the, a new problem with that, which is now your noise just goes up exponentially. Now you're constantly looking for issues, you're gonna constantly find issues. And so then the next part of the problem is how do we sift real issues from noise? How do we actually know that this is a vulnerability tied to a real risk versus hey, this is just a small issue, and the worst that could ever happen has almost no effect on the business. And that is where the human oversight becomes so important to say this is what truly. Hermes Frangoudis (57:02.389) So what's one thing you think the AI industry is dramatically underestimating or understating today when it comes to observability and evals and this sort of thing? Dustin Allen (57:15.905) No, I mean candidly, I think observability is so table stakes that the industry is treating it like a paid product when it should be a free product. Cause it it it's one of those things that's so vital to the rest of how your AI is going to be handled from evals to delivery that really, the industry is looking at this as like an entire sector of like, my goodness, we need to like pay for this and yeah, it it makes sense that we should have a dashboard that's showing that to us when the reality is is so vital to the rest of what you do that I don't see a reason that the industry should pay for observability. That should literally be baked into some of the systems you're using for the rest of your AI delivery because you you can't do good delivery if you don't have the right data signals around it. And that to me, you know, when you look at the cost and margins associated with operating a model, that's holding the data for that is negligible in between. And so that feels like a, you know, a simple trade off to be able to say, you know, the industry deserves a a better situation than just paying to host logs and run that through a few code snippets that like isolate what this means. Hermes Frangoudis (58:49.195) So the ubiquity, the need for the ubiquity is understated. They they almost treat it like an afterthought when it needs to be a forethought. Dustin Allen (58:58.496) It it really should be. I mean, it's kind of shocking to me that we haven't seen like Anthropic, OpenAI, and others just like deliver observability out of the gate because it's going through their network layer. They know everything that's happening. It you know, it it seems like it'll probably happen based on how they've done all of their acquisitions. And if not an AWS, GCP, you know, the there's folks that should be doing this. Like we give it away for free. We're a we're a startup. So there's no reason that we won't see other infra providers doing that. And I think to your point, the ubiquitous nature of what is inside of those logs are just necessary for the rest of the ecosystem. Hermes Frangoudis (59:46.7) So we're we're coming up here on the hour and I really appreciate your time, Dustin. But I also know as a co-founder, you got a lot better things to be doing. So I'm gonna ask you one last question. This kind of like the the wild card question I ask all the founders and stuff is if you weren't building Trinitite today, right, and you could be in any other area of the AI world. Dustin Allen (01:00:03.77) Yeah. Hermes Frangoudis (01:00:13.909) Do you think you'd still be doing the same thing or is there another area that you see is also emergent that you'd be really keen to jump after? Dustin Allen (01:00:22.604) Yeah, my my co-founder and I both are really interested in the physical AI space. We think that when you start looking at how the world interacts, this kind of modeling of physics into the delivery of real services, real products, and real maintenance and overall efficiency in systems, that physical AI layer is something that not only has so many more problems, so like from the engineer side, it's so much more fun at times because you just constantly are facing new ones. But I think the overall impact on humanity just also skyrockets. Like it's it's great that we can come up with new drug discovery. It's great that we can, you know, deliver quicker healthcare services because we can triage patients quicker and get them into the funnel quicker. But you know, there's still the, you know, 80-year-old sitting at home in the middle of, you know, Kentucky or Mississippi that doesn't have access to the physical you know delivery of that health care or the physical delivery of the chores and other areas that make their life so much easier. And so I think that between the humanoids, between the manufacturing and between really all of what's capable in physical AI, it's probably one of the coolest spaces to kind of look at from afar and see what evolves over the coming decade. Hermes Frangoudis (01:02:02.312) Physical AI is definitely one of those areas that like I think it's capturing everyone's attention right now. 'Cause software has become like table stakes. It's become cheaper to build exactly what you're thinking about. So now how does this manifest into the real world and actually solve like human problems that aren't related to just a computer screen? Yeah. Dustin Allen (01:02:23.994) Exactly. Exactly. Hermes Frangoudis (01:02:27.509) Well Dustin, thank you so much for your time and thank you to our audience for tuning in. like, subscribe. Yeah, of course. Thank you, Dustin, so much. For everyone out there watching, thanks for watching and like, subscribe and follow for more. And definitely check out Trinitite. Dustin Allen (01:02:34.308) Thank you, Hermes. Dustin Allen (01:02:48.634) Really, really appreciate you having me, Hermes. It was just a pleasure. And yeah, if you've got any questions on governance, come see us.