Latest / Practical AI / From AGENTS.md to Enterprise Deployment

Transcript

From AGENTS.md to Enterprise Deployment · Practical AI

• 0:01
  Narrator: Welcome to the Practical AI Podcast, where we break down the real world applications of artificial intelligence and how it's shaping the way we live, work, and create. Our goal is to help make AI technology practical, productive, and accessible to everyone. Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place. Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights. You can learn more at practicalai.fm.
• 0:35
  Now onto the show.
• 0:41
  Daniel: Welcome to another episode of the Practical AI podcast. This is Daniel Whitenack. Am I CEO at Prediction Guard, and I'm joined as always by my cohost, Benson, who is a principal AI and autonomy research engineer. How are doing, Chris?
• 0:56
  Chris: Hey. Doing great today. How's it going?
• 0:58
  Daniel: It's it's going great, and I and I know this is gonna be a great conversation for a few different reasons today. One of those being our our guest is one of the speakers at the upcoming Midwest AI Summit, which is gonna be amazing. There's gonna be a bunch of amazing speakers there. I'm gonna do a bit of emceeing, see if I don't mess that up. Would encourage our encourage our listeners to check that out October 15 in Indianapolis.
• 1:25
  You can, get 20% off with Practical AI 20, so check that out. So, amazing speaker, gonna be at the Midwest AI Summit. Also, a fellow podcaster, which is is great. One of the hosts of Cloud Foundry weekly and, a tech marketing, whiz at Broadcom Tanzu VMware. So welcome, Nick.
• 1:49
  Great to have you.
• 1:50
  Nick: Thanks for having me. Great to be on the show. Excited. Also excited to see y'all in person in Indianapolis.
• 1:57
  Daniel: Yeah. It's gonna be fun. We're representing the good good representation of the Silicon Prairie here. I I like it. So, yeah.
• 2:07
  Nick, I I know you're gonna be talking one of the things I thought was cool about your talk at the Midwest AI Summit is you bring in this concept of taking, like, agents.md and shipping that to a production environment where you run, you know, an actual agent in an, quote, enterprise environment, which seems to be, really fascinating. We recently had on, someone from is kind of stewarding the model context protocol and Agents. Md and those sorts of things. Very relevant kind of carry on from that conversation. But I'm wondering as we set that up, what what exactly does it mean to be operating in an enterprise environment?
• 2:57
  So the customers that you're working with, that you're kind of day to day, what are what makes an enterprise environment an enterprise environment? What are some of the concerns or characteristics of that type of environment? And I know you you're always experimenting. You do a lot of home lab stuff as well, so always have a whole range of experience in working in different sorts of environments. So, yeah, if you could just help us understand from a general perspective, what what what does that mean, when you say kinda enterprise?
• 3:31
  Nick: Right. So, yeah, if we think about the enterprise, we think about, you know, a few things. One, let's just assume you're gonna have not Internet limited or no Internet access. And, you know, I've been at VM Martens here for about five years. And before that, I was at one of these large enterprises for about fourteen years.
• 3:51
  So when we had vendors come in, it would always be like they would always assume that we could just go to the Internet. And it was always like, well, try again because that's not gonna work here. So that's kind of always like the biggest hurdle where you're gonna be regulated and or, you know, you know, you're not gonna your what you can do at home is not gonna be what you can do at work, so to speak. So you're gonna be highly regulated. You're gonna have to deal with, you know, potential, like, you know, PCI or SOX compliance or HIPAA or FIPS or, you know and I could probably go on and on about all the different compliance tiers that you'd have to deal with.
• 4:25
  So it's just a whole different ballgame where, you know, you could potentially do something that has dramatic consequences. And you have a, you know, highly controlled, highly regulated environment where even some of the customers I work with are like, know, they actually have the real air gap. Like, we're carrying in things, you know, physically into the data center because there is no Internet access type of thing. So it's it's it's a wild world, but and it and it frustrates a lot of, I think, new folks to the enterprise space. But if you've been in it a while, you know you know you know how to deal with it and and how how to handle it type of thing.
• 5:00
  Daniel: And I guess, you know, people have been dealing with these types of environments for quite a while. Right? Because there have been enterprises for for quite a while. Maybe just give us a sense of, like, leading up to AI agents, and we'll talk about agents here in a second. But leading up to AI agents, like, what were some of the ways that you could, I I guess, ease you know, put some ointment on those those pains of working in that sort of environment?
• 5:29
  How do you how would you get an application into that sort of environment or or have it be enterprise ready, quote unquote? And then we can shift and talk about, you know, what changes with with agents.
• 5:42
  Nick: Yeah. So, I mean, I think so, you know, I currently work for VMware Tanzu, and we run a a, you know, a platform or a platform as a service, you know, geared for private cloud, whether that be on your, like, on your own bare metal in your data center, or you could deploy it on a private cloud in your own VPC on a hyperscaler, that type of thing. But we think about that. Wanna in the scope of an enterprise, right, you have all these regulations, controls, etcetera. You wanna make that path to production kind of the least resistant and easiest path.
• 6:15
  And if you make that path to production, like all the check boxes and everything good to go, that's gonna be a highly adopted path. So the whole concept of, you know, Teensy platform, and it's actually based on an open source project called Cloud Foundry, which is, I think I mean, it's I mean, it predates Kubernetes and Docker. So, you know, twenty eleven ish is when it kinda started originally out of VMware and spun out and then became part of a open source project. So it's been around and battle tested and seen a lot of things where you have that concept of, like, I just wanna take my code and send it to the platform, and the platform's gonna know, like, the best practices for it's gonna build a container for the developer so the developer doesn't have to worry about, like, you know, securing the container or anything like that. That platform will take care of that.
• 6:57
  It will handle, you know, ingress, like certs, certificates, all of the you know, load balancers, all of those things just handle health monitoring, potentially even kind of like sandboxing the apps from each other. Something that, you know, maybe some of these labs may learn from, but so that the apps can't escape type of thing. And, you know, if it if the app needs services like a database or messaging, middleware, any type of thing, even maybe a large language model, it can what we call bind or it kind of connects to a service on the fly, and that that's all kinda handled under other underneath the covers. So it's just a few commands, like you push, bind, scale your app with just a few simple constructs. And that's that's like that's how you go to production.
• 7:42
  And what we see with Enterprise is once that path like, once they take this, like, this is Tanzu platform, it's been certified through all these different standards. It's just kind of like an unlock for these enterprises because it's so easy and so simple to use, and they don't have to, you know, they don't have to recertify everything. It's it's just like this framework that they can use to get apps into production. And then the biggest thing is that you want these the enterprises or the businesses, the they don't want their developers, you know, handcrafting a a new platform or a new way to deploy apps for every sub team and have, like, a 100 different snowflakes. Right?
• 8:16
  They want the same repeatable pattern across the board so that they can audit it and secure it and have their developers actually spend time writing the business logic versus like fiddling with infrastructure. So that's kinda like the whole premise behind, you know, the the whole platform as a service construct and, you know, attains a platform. As you So, can yeah. Oh, go ahead. Yeah.
• 8:34
  Chris: No, no. Go ahead. Finish up, then I'll follow-up. All good.
• 8:37
  Nick: Okay. As as you could imagine, those principles when we're talking about apps could probably play nicely into this agent world that we're living in.
• 8:45
  Chris: So it sounds like you're drawing a lot of commonality between kind of traditional app development and all of the structure that we're all used to, that we've been doing forever, and kind of now we're into this agentic world, and maybe people have been thinking about that in a different way, but it sounds like you've really kind of said, and correct me if I'm wrong, I'm gonna throw something out here, like it's kind of the same thing in the sense of there are differences, but agents and apps should sort of be treated the same way. Am I getting that correctly in terms of how you're structuring that, and that maybe people need to look at what they've been doing for years and make that work for agents in the same way. Is that fair?
• 9:33
  Nick: Yeah. Mean, I think it's a pretty fair assessment, and we're starting to see that more and more just within our customer base. You know, even like, if you think about, like, apps back in the day, like, someone would write some app on their laptop and, like, run it on their laptop. But it's like, we can't just run it. You know, like, you gotta get it off your laptop somewhere.
• 9:50
  And the same thing with agents. You're like, I got this I've got all these agents run on my laptop. If I shut my lid, everything stops. Like, I I don't really want that to happen. Right?
• 9:57
  I don't want them to run twenty four seven, you know, three sixty five, and do all the work I give them. So, yeah. I think that's kinda precisely what we're going with.
• 10:04
  Daniel: So so I guess then it begs the question, why why are people kind of throwing out what maybe their intuition? Like, in in certain ways, people are saying, oh, we have agents now. We need to do something totally different, which is maybe driven by some things that make an agent not an app not just a sort of simple application, but also it doesn't necessarily mean you know, obviously, we have to throw out everything we've learned about platform engineering and and and how to deploy things. So from your from your mindset, what where does the kind of agent equals just another app? Where where might that fall apart, or what are the kind of unique characteristics that you're dealing with on the Agents side
• 10:53
  Nick: Yeah.
• 10:54
  Daniel: On on the deployment pathway that you might not have had to consider before or at least are considering in a new way?
• 11:00
  Nick: Well, I mean, I I think some of the biggest differences, right, where, like, you talk about Cloud Foundry and Teensy platform, it was kinda based on this premise of a 12 factor application where storage and state are a little decoupled cleanly from the actual apps. You could I can scale up to a thousand instances and be fine, and you handle session state and things. The traditional harnesses or agents were kinda built originally just to be like, oh, I've got file system access, and I can just write a bunch of MD files, and that's like my memory. Everything's great. And they're like, well, you start to get into a cloud cloud world that things spin up and down and are ephemeral, you're gonna wanna save those MD files somewhere.
• 11:37
  They're gonna go off into the ether. So that's probably the, I think, the biggest challenge when we start talking about how do we take the app deployment methodology, right, and then push that into an agent running on a of a cloud platform per se. It's probably the biggest challenge. And then just kinda getting the fact that there's some valid use cases for your laptop or whatever. But then also, as we're starting to see more and more, like enterprise was Enterprises want agents to be on demand and usable kind of in a, like, say in a CICD pipeline or like on you know, within part of their, you know, ecommerce suite or just normal applications.
• 12:14
  And you want those to be kinda really close to the apps. Right? Because if you're if you have them way off, you know, even just from a pure networking perspective, if you have, you know, the agent or the LLM or whatever that's like, you know, 30 hops away from the the microservices try to call it, there's, you know, physics that are gonna add on to all of that latency and potentially at scale even be problematic. So they they kinda wanna collate all the intel co locate the intelligence with the app and the data as close as possible. That's what we've seen too.
• 12:40
  That that's something that I didn't expect, really, because some of these some of these responses are kinda long, but it's like you what we've seen with our customers at scale. Like, we wanna get the LLMs and the agents closer to us and our actual apps using them and the people using them as well. And not to mention, if you're using some of these SaaS providers, they're maybe not the most reliable in terms of what traditional enterprises expect and demand. So you can't just have them, you know, going offline for hours. Because, you know, even even at my past job, the fur the very first thing I did there was work on a warehouse inventory system.
• 13:11
  And now let's just say it was for a large retailer that was a grocery retailer in The US. And I was an intern out of college, and the app was older than me at the time, still older than me. And it was a bunch of C and shell scripts on Unix systems. And if that thing went down, like, within five minutes, like, the warehouse was backing up trucks, like, on the interstate. So it's like, you have to like, the scale of enterprise versus just, you know, have you know, like, consumer is a completely different scale.
• 13:43
  So that's why some of all the things we're trying to make them a little bit more enterprise grade and, more more used to what, these enterprises expect from software.
• 13:55
  Daniel: Some mornings, as I'm drinking my morning coffee, I listen to live news updates to figure out what's going on in the world, and it seems like recently everyone is talking about AI kill switches. But the reality is that there is no one AI kill switch. We've moved from models to agents. Those agents are connected to multiple models. They have an agent harness.
• 14:18
  Sponsor: They're connected to MCP servers and tools. And those agents are acting within a fleet, delegating to one another and interacting with one another. In that environment, what you don't have is a single kill switch that solves all of your problems, but that doesn't mean that you can't exert your control. You just have to exert that control across various layers of that stack through, of of course, component input output safeguards, but also controls on agents tied to their agent identity, things that track tool misuse or memory poisoning or goal drift, etcetera. And what we're providing at Prediction Guard, the company that I lead personally, is an AI control plane that you self host in your own infrastructure that gives you that control plane or control layer for the relevant stack that agents operate on top of.
• 15:17
  I'd encourage you to check out what we're doing at predictionguard.com/practicalai. Go ahead and book a call with myself and the team to learn more about how you can gain control of the agents operating in your environment and exert zero trust principles in the way that you operate those agents. So check us out at predictionguard.com/practicalai. That's predictionguard.com/practicalai.
• 15:49
  Chris: So Nick, I would love to dive back. I wanna kinda build on what been sharing so far. And part of that is that we're all diving into this multi agentic world where late last year you'd have an agent, then for most people I think getting into multi agents it was probably developers in the environments that the AI providers were making available, and you had multiple agents working on a project. But now we're really seeing, with these work products and other things that have come out, we're seeing agents really multiplying across many different contexts and with a lot of different utility, it's moved out of being just frontier services, and we're now doing that with our hosted agents as well that we're running. This multiegenthic world has just taken off in a very short amount of time.
• 16:53
  I guess I'm asking if you can paint a picture as you start scaling this out in the enterprise and what that looks like. Could you describe And we have analogies, as you've pointed out, with the traditional software development world, the containers, and how you're structuring deployment and stuff. Could you talk a little bit about what that looks like? Kind of paint a picture so that somebody can really get that in their head, me first of all, about what that multi agent world looks like when you were doing all these right things that you need to do in the Enterprise.
• 17:33
  Nick: Yeah, so I think the first thing that we wanna focus on is that the enterprise, you know, needs to get whatever technology approved throughout their system. And then we so we are providing a harness, called the, Tainzu Agent Build Pack, and that helps to that process. So our existing customers already have our customer you know, we're already approved so that we can help provide them those tools. And we just make that process just another app deployment on the on the platform. So I probably haven't covered this, but I'll explain what a buildpack is.
• 18:05
  Right? So when you when
• 18:05
  Chris: you Yeah. I was about to I was about to ask you that. You could kinda dive into what the specifics of each of those things does.
• 18:11
  Nick: Sure. So when I would do demos before AI and before, you know, a year or two ago or whatever, we would I would always take, like, a Java app, and it was like a simple, like, kind of, like, you know, music album called a spring music or whatever. But we take that Java code to be like a JAR file, like compiled Java code, and we're like, alright. Now we're gonna take this. We have a little simple manifest file that tells us what the what the name of the app is, how many how much resources to give it.
• 18:33
  And we just run a command called c f push. And that would then just take that that code, the JAR file, and upload it to platform. And the platform would be like, oh, it's it's a Java app. So I'm gonna use the Java buildpack, and the buildpack is a it's basically a kind of a command set to make a best practice container based off that use case. So the Java buildpack knows to do all the JVM memory calculators, do the JDK and certificates, and all everything that's best practice to run Java in a container on a cloud platform.
• 19:01
  And that would be great. And then we would, you know, we would attach a database and then, you know, all these things and show how that that scales. Now we've taken that same concept and then said, hey. We're gonna have this agent. We're gonna just see if push an agent.
• 19:13
  And instead of, like, you know, machine readable code, we have a human readable language, the AgentsMD. So, basically, you tell what the agent it's gonna do, and then you just push it to the platform. And it starts up with, you know, within a minute type of thing. And you can scale that up, and, you know, push as many agents as you want. And I think the the coolest thing that I've seen is that once we're on the platform, then we we can run models on the platform.
• 19:35
  So, you know, obviously, a lot of our customers are air gapped, and some of them were smart enough to buy GPUs a few years ago. So some of them have lots of GPUs that can run all these local models and have a great experience. Some are wishing that they bought GPUs and, you know, hardware as as the pricing of of all this hardware goes through the roof. But then they can run local models and then tie the agent to that. Or if they don't have, you know, they might have a contract with one of their cloud providers or one of their, you know, one of their approved LLMs of choice, and we can to register that.
• 20:05
  Then that kinda gave us a simple, like, here's your out of box agent with chat experience. And then, you know, along the way, like, we've done a lot of things with MCP, so, you know, the model context protocol. You guys covered that on the show before, but it's, know, basically a way for an app to talk to an LLM. Like, very, very high level. Right?
• 20:22
  So we that's become, you know, useful as well as we then give the agent tools via MCP servers. And even at Broadcom, like, the only way approved way to use for our developers is to deploy MCP servers on Tansy platform so that we have a whole set of MCP servers that are approved and hosted, and they're kind of secured and maintained. Because they are very similar to applications as well. And then we have a a concept of like the MCP gateway service that kinda registers that and secures all those the servers. So then we can then extend the gateway to that agent.
• 20:53
  So then the gateway this agent now has, like, secure access to a runtime, you know, battle test runtime. It can talk to an LLM that's approved and, get tools via and gateway. And then we're starting to add in like the memory service as well to have a kind of a shared memory service. So we're kind of expanding all of that out where we're seeing all those best practice app lifecycle go to, you know, see a push, go. Here's an agent.
• 21:18
  Here here's all the things it wants to do. And then, you know, it can be a it can be a security review agent. And like one of the things I I showed, I was at a I think a week or two ago. I can't quite remember. I was at was in Las Vegas for a conference VM or Explorer.
• 21:30
  And I had like a five phase demo where I walked through this. Maybe I'll show a little bit of this at the at the Midwest Summit here. But the you know, push the agent, bind bind all these services. But the end the end goal was like the the boss level was like, alright. Now we have these these agents sitting in there, and I can hook up hook them up to a repo with like an event listener or webhook and say, alright.
• 21:49
  Now do a sick like, you know, issue five on my repo. Do a security review on this repository. And then like, it sends it off to the agent via webhook and, oh, the Tainzu agent. And, like, it just fires up, gets it. It, you know, fires up from the platform.
• 22:03
  It gets a request, and it just immediately pulls down the repo and starts doing a a security review and then publishes or pushes up to the issue that I created on GitHub and says, this this is security review. So you could see how that flow would be really useful in Enterprise where, like, maybe on every Git commit, you can just have all these agents spinning up on the fly that could be a security review agent, you know, potentially like a, you know, a style or, you know, code quality review agent or potentially even like a, I don't know, like maybe a design review agent, all firing up on the fly as things like call. So it's kinda like and then they all have their they're all in their own little sandbox and everything's proper. Right? Because part of, you know, the platform, for a while, it's been this very secured container runtime where they can't talk to each other unless you enable them.
• 22:48
  So they can't go to the internet. Unless it's enabled, they can't escalate. Otherwise, they only get access to what you give them. So it's kind of that benefit as well. And then obviously they're containerized, so they don't have access to the underlying host system as well.
• 23:02
  So that, you know, as we've seen, you know, some of these things of uncontrolled agent forms, I think we have some remedies here that some of these AI labs could maybe take into account and implement some best practices that's learned throughout the past two decades.
• 23:17
  Chris: The unplanned swarms, swarms that you don't necessarily want. I actually have a two second follow-up, and that is it's for my benefit, and we've talked a lot on the show over various episodes about agents using MCP to get access to different apps and services and tools and stuff We like that out haven't really talked about what an MCP gateway is, and I was wondering if you could share that for a moment just as a detail because that's kind of the one little new piece of architecture that we've never touched on on the show.
• 23:49
  Nick: Sure. Yeah. So if you think about an MCP gateway as a way to easily control and scale access to MCP servers, especially within you're in a cloud, I guess, platform. So we may have, like, 30 or 40 MCP servers that we run, or I don't even I don't even know how many it is. Right?
• 24:06
  And then those get bound those get registered and and bound to certain specific MCP gateways. So maybe there's like a you know? And the MCP gateway is something we can spit up on the fly on on the platform. There are many different MCP gateways out there. There's many different AI gateways out there.
• 24:21
  Some of some of them are the same, you know, the one in the same. But you basically register MCP servers to this gateway, and the gateway kinda controls access, so to speak, and regulates that. So it may say, like, you can bind to this gateway, you only get these these MCP servers. You only get these MCP tools within it, that type of thing. So it's a way to easily abstract all the different MCP servers that may be in an organization and kinda direct them into a certain either a certain agent, or we even have a flow where, like, you can take we'll we'll take those MCP gateways that we run that front all of our MCP servers.
• 24:55
  And then you can say, hey, go. Give me a give me a way to connect this to my cursor or my cloud code or my whatever. And then it just injects it in there. And then then your local agent has all the tools that are approved as well for your organization. And then it's just like like a simple click, and then it just registers.
• 25:11
  And then, like, on the MCP server helps handle the auth as well. Because some of them are like, well, it's my if I'm using the GitHub MCP server, I wanna pass my end user credentials all the way through to that MCP server because I don't want, you know, I don't want it just to be like, well, here's our general generic service account for the entire organization, and every GitHub action is is gonna be, you know, seen by that one account. And you have to you have to have the individual identity pass through. So you have, like, the concept of, like, SSO and sign in as well for an identity for agents or humans running their agents on their their desktop. So that's kind of a quick level view.
• 25:45
  And then a great way about that too is because it's all flowing through that gateway, we get metrics out of it. So we can see all the tool calls and all the all the events happening coming from the agents to the MCP servers and back and forth. So you kinda kinda see what's happening, especially if maybe if there's something unexpected. Like, well, this one agent, you know, made 200,000 tool calls to, you know, delete repo. Like, oh, like, maybe we should alert on that.
• 26:07
  Like, that's probably not good. Yeah. So, you know, another visibility or metric to what agents are doing and what tool calls are doing as well. So I guess, hopefully, that made sense because that was that's kinda how I understand it. Right?
• 26:21
  Daniel: That yeah. That was great. And and there were a couple things that you mentioned in there I think are additionally worth digging into. And you you kind of have this, if I'm understanding right, in part of the the build pack that you're working on. Mhmm.
• 26:35
  Part of that centers around kind of pushing an agents dot m d file. You mentioned, like, people have these m d files hanging around. You mentioned memory. You mentioned identity. I'm wondering if you could kind of help us understand, like, if I have if I if I'm working on my laptop, right, and I'm using a certain agent harness, I may have an MD file that is kind of generic, and then I have files that are created by the agent that it uses that are only corresponding to my single agent.
• 27:11
  Mhmm. Then I have, you know, memory and then, you know, calls into other things. As you're pushing this, like, off the laptop now, and I imagine taking that off, like, what belongs in, like, an agent's dot m d file? And what is, like, as you're giving that example, right, you may have one agent in the platform that is a security reviewer for, that that does security reviews. Right?
• 27:37
  Mhmm. When you spin that agent up, does each instance of that, each session have a unique identity? What is the what's unique about that session and the data that that is generated by that session versus maybe the things that are always static? Like, I I'm not sure if the Agents. Md is is always static.
• 27:57
  So could you help us understand, like, which are the static elements? What's developing over time? How is that, like, state of the agent spread across Right. You know, the implementation, I guess?
• 28:09
  Nick: Sure. Yeah. So if you think about it, like, in, I guess, in in in this specific implementation, the the agent's build pack is basically like the the boiler plate code of what you want the agent or app to do. So, like, I'll do demos, and I'll have it be like, you're a pirate. Like like like, one of my demos is like, you're a you're a you're a Jira.
• 28:27
  Like, you're a senior software engineer. Know, you're gonna watch this Jira page, or I forgot the terminology here. But basically, watch this Jira queue and look for any inbound tickets. As inbound tickets come in, review them, and make sure that they're of good quality. And here's all these different things that you wanna do, and don't do any of this.
• 28:47
  So it's more of a very simple AgentsMD where it's just telling this agent what it should do how it should act. And then you you can call that and have, like, alright. Well, like, you know, ticket 500 came in. It was poorly documented and, like, update that Jira ticket and, you know, whatever. And then you can add fun things like talk like a pirate.
• 29:04
  So it's like, alright, matey. Your, you know, ticket 500 is is terrible. Right? Like, go back and, you know, update it, that type of thing. So that's kinda where the the static part of it is.
• 29:13
  And then, like, the memory service we have, where you can attach it. You can have, you know you know, potentially memory of a certain project. So, you know, we kind of envision, like, each set of teams to have their own multiple set of agents. Right? And it could be like, well, you know, team A's agents have the access to team A's memory.
• 29:29
  So as they come up, they know, like, all of this historical data of, like, what like, the architecture of the application, the what's been done before, the roadmap, that type of thing. So as it spins up and does a security review, it can make proper decisions and proper guidance. So that as you kind of build your local agents and kind of give them memory and they become smarter, that that memory service as you have agents spin up and down, they just immediately they immediately know what what's happened before, know how to how to proceed. As
• 30:02
  Sponsor: you heard at the beginning of this episode, some amazing people like Nick, who's the guest on this episode, are joining us at the Midwest AI Summit, October 15 in Indianapolis, Indiana. Nick is gonna come talk about what it looks like to deploy AI agents in real world environments, and we've got a bunch of other amazing speakers talking about how AI enabled teams work, how to make decisions about what to say yes to and what to say no to in relation to AI and much more super practical and amazing talks. Plus, there's gonna be an AI engineering lounge where you can actually sit down with folks like myself. I'll be there. Get get review on your AI architecture, your stack, your road map, etcetera, towards the rollout of things like Agents or other AI technology.
• 30:59
  This is a practical, event that you don't wanna miss. There'll be great food, great people, great networking, and practical value. Do not miss this. It's October 15 in Indianapolis, Indiana, and you can use the code Practical AI 20 to get 20% off registration. So go to midwestaisummit.com and use the code Practical AI 20 to get 20% off registration.
• 31:32
  Chris: Nick, before we were going into break, I think I may have cut you off when you were just starting to say something. You wanna dive back into that real quick?
• 31:41
  Nick: No, I think I covered Hopefully I answered the question again, but That's no problem. Okay.
• 31:48
  Chris: So I really appreciate This has been really good for helping me trying to conceptualize how all this fits together. I think I had some understanding, you've gotten a level of detail. I want to ask you, we are looking forward, I have several questions that are really kind of driving my thinking. You've got me thinking creatively based on what you've said. So as you look at We've talked about MCP.
• 32:18
  There are new things coming out, things like there's the A2A, which is the agent to agent protocol, and organizations are starting to look at that, and they already have their MCP servers maybe, but they're trying to get the structure and kind of bring the sanity around it that you've been talking about. Then they kind of go, Oh, okay, it's not so different from what we already know. But as this is evolving very fast and you have things like these new protocols, do you have any guidance on how to start integrating this constant flow of change in, terms of First on the technology, and I'll hit the other half of that afterwards. As you're trying to bring things in and make it work, have you guys kind of thought about how to structure that?
• 33:07
  Nick: I guess we'll just say take baby steps from everything, right? Especially when we talk about the Enterprise scope, they can be a lot slower to adopt some of these newer technologies. So you kinda layer that on. And you don't have to necessarily adopt all of these things at once. You can maybe just get a simple agent running.
• 33:28
  And, you know, for most most places, you know, just getting access to an LLM is still the biggest hurdle. Right? A secured, you know, approved. So and then potentially, you know, going through the going through the AI AI council, right, their committee. Right?
• 33:42
  Like, everything must run through the AI council and be approved. If, you know, like, most you know, if you're not if you've never worked in an enterprise, most large enterprises have a council
• 33:50
  Chris: or committee So I know
• 33:52
  Daniel: for everything. Right? Yes.
• 33:53
  Nick: So it's like, we shall go to the committee and we shall present our idea. We will wait six months and see what has the result been. Right? So you just kinda have to work through the councils and the committees and and all that fun. But once you finally get something approved, just just just iterate.
• 34:07
  And you're like, don't don't try to boil the ocean, make baby steps, you know, to and and get progress. And then as you'll like, and you'll learn the technologies, and then the next step will just kinda be obvious. Like, well, I I can make the agent better if I get some MCP servers, get them tools. Alright. And then we gotta get these tools approved and then add on to that.
• 34:25
  And then, you know, you know, and some of these guardrails help inner you know, of Tainzee platform help enterprises bring these tools to to to light, because it's not just, you know, all this random stuff downloaded from the Internet. Right? Which that can be its own danger nowadays. So having the you know, having some some guardrails in place is is a good thing. If you've had you know, if you have experience with some of these platforms or apps running before, take that experience and that knowledge and kind of bring it forward.
• 34:55
  And utilize your existing people and skills, I think, too. I think a lot of times too, I see in organizations, like a new technology will come up, and a new team will be built, and that new team will decree. But there's so much other, like, really skilled people that could help, and it's always kind of a a friction too. So how try to try to get the teams to work together too, and not not just have the new team doing all the new things. And then the the teams the teams that have been there all the while have that.
• 35:18
  Like, that's always another constant flow. So try to break those silos down as well to help get people up to speed and get through that process.
• 35:27
  Chris: Having worked in multiple enterprises myself, I think, that last point, I think I've seen that multiple times across different organizations. How do you bridge that? I think part of it, kind of have Every organization has its internal politics. They all have the different structures. It's really common to say new tech, new team there, but then I think people recognize why.
• 35:56
  You'll see that, and sometimes you'll see a reconciliation, but it always seems like a little bit of a struggle when that happens to try to Do you have any thoughts around, as you're trying to do that, you may have A lot of big organizations have They're very one business unit, maybe almost like in a different universe from another business unit And so how do you get that kind of cross adoption where you're taking advantage of maybe both the new teams that may be there, but also some of the old structured teams, and you're trying to spread the capability across the larger organization in a useful way? Guidance? Do you have I imagine it can't be that different from the more traditional app dev side, but I think this is a struggle that a lot of orgs are facing right now.
• 36:56
  Nick: Yeah, well, think, yeah, so I mean, it goes, I think, a lot both ways. So if you have the new team, right, the new team can like, the responsibility of the new team is, like, engage every other team because they have a lot of people that can help you out and achieve your goal. Right? Because the new team probably most likely has a very stressful and, like like, you know, like, adopt AI everywhere. Or before it was like, go cloud everywhere.
• 37:18
  Right? Like, those those goals are, like, not quite easy to to get going in enterprise. The the new team needs to, like, you know, potentially run, you know, office hours, weekly office hours, reach out to their other teams, you know, maybe before, you know, like a lunch and learn if you're in the office and have people in the office, that type of thing. And just have, like, a forum for people to come in and learn and get started and, like, ask questions and not, you know, like, hey, like, I I saw this, you know, this post about, you know, this new platform or this new AI thing. How do I get started?
• 37:51
  Is there any documentation, or what can I do to get started? And just kinda have an open and very welcoming place to get started to bring in the folks. And then if you're on some of the, let's say, not new teams or teams that have been there for a while, be engaging and and and reach out to those teams. Because a lot of the the new people are gonna need help. Like, they, you know, they're maybe under stress.
• 38:12
  So it's it's just breaking that down. And then I think with this iteration, right, it's gonna happen a lot faster. And I think people just naturally wanna work together. Because I I think I I was at VMware Explorer, I I did a few talks. And I started every talk with like, alright.
• 38:27
  Let's let's get a a show of hands in the room. Who has used one of these tools like in at work? Like cursor, Claude code, whatever. And pretty much the entire room and all of them went up. And I was like, well, that's a lot more than I expected.
• 38:38
  So like, the adoption rate even at, you know, the traditional, like, you know, platform or developer engineer, and, you know, or just infrastructure admin at some of these, personas is is well into that that scope. So the entire enterprise, I think, is picking up on this really fast. And the, you know, the more collaboration, I think it just will help out. So, you know, just office hours, you know, reach out, be proactive. And I guess if you're on if don't be be pro be be mindful or be open to change.
• 39:07
  I just this is just in general. If what I've seen before and and if you're if you're resistant and resist the change, and, you know, that's that's gonna kinda get you labeled and kinda put off the side. But if you're you may be at a you may have your own opinions, but if you present them in a way that kinda is, let's say, maybe open minded per se, you have a better chance at some of those key learnings that you've dealt with throughout your career to actually be applied in the new way as well.
• 39:32
  Daniel: That makes sense.
• 39:33
  Chris: Yeah. I'm curious. I wanna circle back for a moment on something you said. As we're talking about kinda spreading across, my mind goes to security, because I know in my industry security is pretty big deal. We do have air gap things and stuff like that.
• 39:52
  That's what we do in defense. But as you're thinking about that, one of the topics that is a really common topic now is kind of going back a few weeks. We covered it in-depth on a previous episode. The whole OpenAI, the swarms got out, hugging face, that whole thing. I'll refer people back if they're not familiar with it to our episode recently that covered that, but that is top of mind.
• 40:21
  It's not only top of mind for security, but it's top of mind for legal. It's top of mind for comms as they're thinking about what do we do if something like this were to happen in our org, whatever that org is. And can you talk a little bit about, as you've kind of brought the enterprise structure up to date to deal with agents, and we're beyond just the traditional software and app world, and we're taking some of the same lessons and reapplying them in a new context here. Can you talk a little bit about how that helps on the security side? For the folks out there that are really That's their job is to keep things from really going awry.
• 41:04
  Can you talk for a moment about keeping things in the box, if you will?
• 41:08
  Nick: Yes, sure. And then I guess I'll put a disclaimer. These are all could be suggestions to help out with. If you really dig into some of the hugging face stuff, it's a little scary. So I'm not to say that we're, like, better than all of the AI labs or anything like that.
• 41:23
  So but if you if you look at some of the the things where how that kinda got started is where they, you know, they started getting out of the sandbox. Right? And then they they found things that were, like, they could get access to. Or, you know, having monitoring control around what those apps or agents can do is is, you know, something that's been been in mind for all enterprise apps. Right?
• 41:43
  So, like, having, you know, kinda guardrails, even, like, network zones. Hey. This is a highly regulated zone. It's it's super locked down. Like, even just like traditional firewall would have potentially blocked a lot of this stuff.
• 41:53
  Right? So it it it's kinda key. Like, you know, a lot of the core enterprise learnings that kind of maybe take that for, like, you know, the the old team versus the new team. It's like, well, guys, you could've just, you know, I don't know, had some basic controls. And, like, I think they even talked about, like, their monitoring wasn't working, and they weren't really monitoring it.
• 42:09
  And, like, you had all these agent swarms. And then, of course, then then they got to the Artifactory, which then they found the weak link, and the Artifactory had the Internet access, and somehow they magically, you know, zeroed the Artifactory instance multiple times. So not to say that we could prevent everything, but it's just you take those, at least, core enterprise technologies that maybe would've helped out a little bit of just agent control, like hardened sandbox type con constructs, very locked down network controls. That would help a little bit potentially. You know, just, you know, monitoring what's in the in the agent, Make sure that the agents aren't, you know, somehow injecting tools that they're not supposed to have.
• 42:48
  I don't I'm not sure if that was an issue or not. But, you know, when you talk about an application, the whole dependency stack is always up to up to grabs. You wanna make sure that the dependency stack, the container, everything is and the whole OS and the whole platform is trying to patch and reliable as possible. So it's a I I think the the key thing is obviously, you know, we had the whole mind bending effect of that they somehow made this messaging board on Artifactory, and then it got deleted, they recreated it. So there there's that factor, which I think everybody's a little like and it goes into the whole AI is gonna kill us thing that happened over the weekend.
• 43:17
  Right? Versus just do some basic security, basic stuff. Right?
• 43:22
  Chris: That that was a a social media post. Less people misinterpret That
• 43:28
  Nick: that's that's a quite yeah. If you're not on AI Twitter or AIX, there's a lot of posts around, you know, somebody left Anthropic, and a bunch of Anthropic people started posting around that they're afraid that there's like greater than 10% chance that AI is gonna take out humanity over the weekend slash Friday. And then that stirred up a lot of conversation, let's just say. So that's what I was referring to. A bit.
• 43:51
  Yeah. Yeah.
• 43:53
  Chris: Okay. I guess that's useful. I appreciate that. We like to finish off by kind of giving you kind of a free form question, and that is kinda toward the future. As you are doing your job and running your podcast and talking to people and kind of building up your perspective on where things are going with this, We like to ask guests to go ahead and get crazy and prognosticate just a bit, and we don't hold you to it.
• 44:30
  Really the way we do this is when you're kind of just letting your mind wander and you're thinking about these things in the back of your mind, going to bed at night, having a glass of wine, whatever it is you do to chill out, where do you think things are going? And you can choose the timeframe that you like, but I really like to think, what do you think is coming? What's next? What should orgs be having a thought toward for the future, even if we're not there yet today? Mhmm.
• 45:03
  Any any thoughts on on what that future looks like as we close out?
• 45:07
  Nick: Oh oh, boy. It could be it could be many things. But I I think from a a enterprise perspective, I I think we'll see a lot of, like, team change and role change. Not even just an enterprise. Like, the whole, like, from the business side, things are gonna change a lot.
• 45:20
  Right? Because you're starting to see and I I was going through old episodes of my podcast this year and just some of the old just the start of this year, and there's just now. Was like, wow, a lot has changed. And I like, can't imagine, like, can you, like, you know, imagine a couple years from now how much change the rate of changes now if the L
• 45:38
  Chris: It's accelerating.
• 45:38
  Nick: Perceived, So it's it's hard to imagine that, but I I definitely think from a business perspective, we're gonna see, like, a radical change in, like, how how, you know, development and, you know, product management and everything changes or everything kind of, you know there's gonna be, new roles get created. You know, orgs are gonna change, I think. And I think I think for the good too, because I think, like, there there's always a fear of job loss, but I I find that I've and I've heard this echoed across many personal experiences. I am, like, working harder than I ever have because I have all these agents, and they, like, they need stuff from me. And I'm constantly, like and I'm doing building things that I always wanted to build, but just never had the time to.
• 46:14
  So I feel like we're all gonna be busy, and it maybe maybe we need to, you know, not not get too crazy with the agents. But it's it's hard for me because I it's just it's it's fascinating. It's just it's really enjoyable. So doing a lot of things with automation, I can only imagine what these agents are gonna be like in the, you know, the next six months in terms of, you know, what they can do for you. I think we'll see a lot with voice come come into play and maybe even a new style of interaction with the computer as well.
• 46:41
  So that will be interesting as well. And then I I obviously, I think, you know did you see the the post, like, Hugging Face had little robot things on sale?
• 46:50
  Chris: Yes. Yeah.
• 46:51
  Nick: Yeah. So like the little little micro duct things or whatever.
• 46:53
  Chris: I did. I was like I was like to my wife, was like, maybe we should get one. And she's like she's like, you have enough toys. You just stop.
• 46:59
  Nick: Well, I I may have ordered a few for the family, and I'm like, those things be awesome. So, like like, robotics and things, like, you know, I I think the LLM is is great and wonderful, but there's so many more possibilities where we could go that I guess, like, for humanity. Right? As long as you don't we prevent the Terminator scenario. It'll be great.
• 47:18
  Right? So Totally. Absolutely.
• 47:21
  Chris: Great. I appreciate you painting that. That that helps a lot. Yeah. The rate of change is just insane now, I agree with you.
• 47:28
  I think it's going to get faster and faster, but great way to end the show. A lot of fun. I always love to hear what people say we're asking about kind of where they think things are going. Thank you for coming on the show today. It was a great I learned so much.
• 47:46
  Really appreciate that. And I hope people tune in to Cloud Foundry weekly hear you talking more about these topics. And they should definitely go to the Midwest Summit and hear your talk. I was there last year, did a talk. I won't be there this year, so I'll have to see if I can get ahold of the video of it or something.
• 48:07
  But anyway, thanks for coming on Practical AI today.
• 48:10
  Nick: Thanks for having me.
• 48:15
  Narrator: Alright. That's our show for this week. If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X, or Blue Sky. You'll see us posting insights related to the latest AI developments, and we would love for you to join the conversation. Thanks to our partner, Prediction Guard, for providing operational support for the show.
• 48:36
  Check them out at predictionguard.com. Also, thanks to Break Master Cylinder for the beats and to you for listening. That's all for now, but you'll hear from us again next week.