Latest / The MonkCast / When Agents Become Users: Rethinking Developer Portals with Balaji Sivasubramanian

Transcript

When Agents Become Users: Rethinking Developer Portals with Balaji Sivasubramanian · The MonkCast

• speaker-0: Hello and welcome to this Red Monk Conversation. My name is Kate Holterhoff. I'm a senior analyst at Red Monk, and with me today I've got Balaji Sivra Supermanian. He's the Senior Director, Product Management, Agentic AI Developer Platform at Red Hat. Balaji, thanks so much for joining me on the Monkcast. speaker-1: Thanks, Kate. Thanks for inviting me. Happy to see you at the Red Hat Summit last month and ⁓ good to see you in person and looking forward to this conversation. speaker-0: Yes, I know. That was such a great event. I did a quick take as part of it. ⁓ so got to collect my thoughts, but ⁓ it was a lot of fun to have it in my own backyard here in Atlanta, but also just to be able to to see so many interesting folks and have a lot of really exciting conversations here. So yes, and and Beloggy and I got to go out to dinner, which was awesome. ⁓ so it's always great to get out of just ⁓ talking via Zoom. Yeah, absolutely. Well, let's just begin by having you ⁓ introduce yourself. Can you tell me a little bit about what you do at Red Hat? speaker-1: Yeah, man. Manage the the developer ⁓ tools and experiences product portfolio for Red Hat. ⁓ I joined Red Hat three years ago. ⁓ at that time Chat GPD was just introduced and the AI was revolutionizing the coding and this was a great place to be in because I touched different parts of it from desktop tools to cloud ⁓ developer environments through a portal. So it's it's a great place to be and it's it's been a great journey last three years. with developers is center of AI, everything they're doing. The world has completely changed. So what an exciting place to be. speaker-0: I have noticed that as well. Yes, the the world has completely changed, especially for developers here. Okay, so you know, I invited Balaji on here to talk about a blog post that he wrote recently. I mean, he maintains his own personal blog, which which is very active, super interesting content there, but also published one on Red Hat's blog. And the title was Why Developer Portals Matter More in the Age of AI agents, which I I I can't think of a more pertinent question and really ties back to actually a lot of the conversations that we were having at Summit. So ⁓ so this post opens up ⁓ by saying that there's ⁓ a whole bunch of folks who are are declaring that internal developer portals are dead here. So you know, talk to me about the history of this. I mean we're we love talking about things being dead, right? You know, DevOps is dead. This is the the ⁓ sort of exaggerated language to try to get attention common in the the tech industry here, but where did you first ⁓ hear that? speaker-1: one of the analysts ⁓ reposted somebody else's post echoing the same sentiment. And I'm like I think it misses a point. So definitely you could potentially use cloud code or things to deploy in certain scenarios. For example, I maintain my own little website and I just use cloud code to deploy via versal and deploy everything without going anywhere. It goes into portal of course, the small one person team. But yes in in a large regular enterprise it's not right. So I'm gonna try to actually explain why it's not the right approach. speaker-0: Yeah. Okay. So some thought leaders were coming out with these things. And the idea is that we're all just gonna be using our individual clods, codexes, et cetera, as as personal portals. But you you're arguing that in the ⁓ enterprise that just isn't the case. And it's it doesn't make sense. I d I tend to agree with you here. ⁓ but I I would say just taking a step out that that sort of larger conversation around using a multipurpose agentic platform versus using really targeted one is ongoing. And I I feel like your piece sort of contributes to that, where we're trying to work out what is what's that going to look like in the future here. So you give those skeptics ⁓ of the the ⁓ IDPs some credit up front, but how much of their case do you actually buy here? Is the portals a place you visit Genuinely finished versus unfashionable, maybe is a another way of framing it. speaker-1: Yeah, I mean before a few years ago, I used to Google to search for things. I go to the Google portal essentially to search for things. Now I don't. I just go to Chat GPT and or Cloud to ask the same question. Anything I want, I ask Chat GPT because it's already giving me the right answer. So I think the same way applies to portal also. At the end of the day, humans going to a website is definitely diminishing and people are essentially using developers particularly want to be in their workspace, in their IDE in there, ⁓ Cloud Core, for example, and be able to get the information. I think the part that's sort of maybe true in that's that's truth to every argument. And there's always something right about it. That's why it's actually an interesting point. Definitely if you look over time, today let's say people are going to a portal more percent than ⁓ agents. Over time it's gonna shift. The agent will be the primary consumption of the information in the portal ⁓ versus a human going and clicking through. five clicks to get something. So the real argument is missing the system to the UI. UI is just an interface to that data, the context that's in there. And so that's what I was like saying the portal, the UI part maybe used less, but the information in there is still highly relevant. And we'll talk more about that as we go along in our conversation, why UI is still relevant in some cases. speaker-0: Yeah, yeah. And it it the impression that I got was that your own approach to UIs and agents as users has been evolving. That you're you're still trying to figure out what that's gonna look like here in the future or even t today, right? Across multiple verticals and and, you know, different ways that folks are approaching these tools. ⁓ so you said that the agent ⁓ as a consumer came to you later, that it's been evolving. Maybe eighteen months ago you hadn't thought of it. of it in this particular way. So talk to me about what that evolution looked like. What how have you I I guess maybe what proof points have occurred in your thinking that have made you change the way that you think through this consumer model of agent behavior as it relates to to portals? speaker-1: Yeah, eighteen months ago when I wrote the blog in March was before MCP was even discovered. Netnet was I was thinking the portals are a great place for humans to discover, learn. Because that was a s phase bold of AI was. In an enterprise context, people didn't know how what is it and how do I learn it, how do I discover it, and how do I know what models and what things are appropriate for my enterprise? It's still a debate even going on right now. But back then it was more like I don't know what to run. Am I supposed to run this or not? So for me, ⁓ Portal was a great already it has a catalog and tech talks and API, which sort of supposedly show what is the right way right day to do other things. Portal could be an AI distribution. It still is true. For example, it's evolving now. We're gonna soon release something. AI catalog essentially. Models, agents, skills that are relevant for developers to discover. Here's the approved skills, here are the approved agents. So that These still remains either the platform or the portal as a distribution layer of AI content, enabling that. I think the part that sort of changed obviously the introduction of the MCP and agent as a consumer. The first citation was for me was add on a chatbot to the portal and ask questions. Instead of clicking five links, I can just ask a question and then chatbot will answer for me instead of me trying to figure out where to click the link. If you look at a typical portal, there's lots of links, lots of catalog items, lots of information. Still you have to go through search, maybe other things, but let's use chatbot as a board intelligence. So that was sort of the evolution where the AI is enabling better use of the portal. What flipped was essentially agent as a consumer. And that really was enabled by with MCP. The world has evolved now agents are essentially how people are wanting to access data. And that's why this whole thesis came about, essentially people are saying, hey, you know, if I don't need the portal anymore because now I'm sitting in a agent plot core and be able to get the information. But where are you going to get the information from? speaker-0: Yeah. I it is you you're so right about the eighteen months thing and and so much changes so quickly. I feel like we're all just sprinting to keep up, thinking of terms like vibe coding, which have isn't even been around for two years at this point. ⁓ yes, so much moving so fast, ⁓ we all kind of have to revisit our prior assumptions in this ongoing way that maybe unprecedented isn't the right word, but certainly sped up, accelerated, absolutely. So I yeah, I I I love these ideas. I want to hear From your privileged place of expertise within Red Hat about what you're seeing in the enterprise specifically here. So you've got this great line about agents being confident hallucination machines, which, you know, from one writer to another is just a great line. Love that. And so I, you know, I I I want to hear maybe about where you're you've seen that play out. Can you give us some concrete examples of of hallucinations that you've encountered, either with ⁓ some of the folks that you work with at Red Hat or maybe even in some of your own hobby projects, kicking the tires. speaker-1: Yeah, I think this happens almost every day. You ask an agent to scaffold a new Python project, ⁓ Python ⁓ service, for example, and it does a great job. And it it pulls our image, it pulls it from the Docker Hub, the Python three eleven ⁓ image, it actually even writes the Kubernetes YAML file and and actually deploys. That's great. So if you look at it, it does the unit testing, it it does every testing, it passes and it actually is right. But the problem was if you're an enterprise context. Now you have a certified or or validated images that you want to use in your organizations. For example, you have UBI image of the same Python 3.11 that you should be using. If you pull from Docker Hub, it has the forty-seven vulnerabilities. But we have images at zero vulnerability that the enterprise already curated for the developer. But the developer, when you use a hardcore, for example, this has actually happened in one case. The agent doesn't even know that I have an enterprise policy. The problem is the agent is doing the right thing from his perspective and it actually is okay. But in an enterprise context, you do want to ground it to the enterprise approved images. And that's where the problem comes. Essentially, it's great at hallucinating something and it's not trying to be malicious. It's actually trying to do the right thing. But it doesn't know the enterprise context. For me in a real enterprise ⁓ production use cases, these things really matter, right? A human would know that there's a policy at the company they work at. But the agents needs to have that information. speaker-0: Right, right. And in terms of what I'm hearing from buyers and and folks who are in the industry, everybody's thrown around the word determinism now because we're all trying to look for replicable systems that but still ⁓ have the benefits of AI. And I think that that's that's still a tough thing to achieve. And so trying to get past that hallucination. speaker-1: Yeah. So go back to your that you had is that, hey, why do I need a portal in this context? The agent, you would have a big vulnerability problem machine up there. You would think, ⁓ it worked, of course it worked. You didn't use a portal, but it worked. But then if you had a portal, you would use a scaffolder, you'd you would use a approved path, golden path, it'd already make sure that you have the right image before you go to the production. That way the developer does not have to know, the agent does not have to know what is the right way to go about it. It is already done for you. And that's where I'm saying is you can't just like throw away the golden path and the the concept of a portal. Why is one part of it? The the actual value of portals is what is matter here. This is a good example of how you would avoid a problem in a production. speaker-0: Okay. Yeah. ⁓ so let's dive into the UI part of this a little bit. So you argue that human friendly UIs are not necessarily agent friendly APIs. So is serving agents genuinely different then in terms of building a portal or is it like the same sort of, you know, portal with the bolted on protocols just with with under a new name? speaker-1: The agents can definitely leverage the existing scaffolders, for example. Like that's why the MCP allows them to expose the same ways. But agents can use other protocols. I think I've seen people wanting to use CLI or even better documentation to serve the agent. Agents don't have eyeballs, so you don't have to worry about a UI way of providing information. definitely use existing APIs and any of these things work. API, CLI, documentation, MCP, any of them work. to to answer agents that the what's available inside the portal basically, compared to humans. Humans typically require a UI based solutions. speaker-0: So you point out that agents break old machinery. So permission models and they don't know what an agent is, audit logs can't capture, you know, what agent white, that the brand X number of actions, right? And all of this leads to bottlenecks. And we've written a little bit about this at Red Monk. I guess I'm interested in ⁓ which of these you consider to be the most difficult to address when it comes to portals. speaker-1: Yeah, I think the the main point is ⁓ if you look at permissions, agent acting on behalf of the humans, and that's delegation, that's understandable. But the problem within that is that humans, let's say I have an approval to create a database, but then agent can create thirty databases. Is it right? I mean, I do have permission to create a database, but the problem with agents are that they're non deterministic, right? So for example, I can write a script right now that creates a database. It's a deterministic script, it goes through steps and it creates a database. It is acting on behalf of me. And that's a reasonable thing. But the problem with the agents is obviously now you have an agent that could hallucinate something. An agent could delegate to another agent. So basically, I am the user. I just said do something and I walked away. And then it could you have four agents down and it could do something that I don't know whether I actually even did it to it. So it could be based on a context. You could say go delete the database or something. You have access to doing that. So that's the problem. So the real question is not Like, do we need a different permission model? Definitely I think there's work going on around creating a non human identity. So it's a human identity versus a non human identity. And then you can have policies for that differently. So you don't wanna have the same policy. Humans, you can say this, but then the agent acting on behalf of humans, you probably have to have some additional controls. You don't want that 30 database creation problem because that's our normal human behavior. So humans will stop doing that. So I think that's where the the permission models needs to be probably ⁓ have a policy attached to it. Or I think there's some work going on with AWS, Octa and Microsoft and Tra. There are work going on around this. Essentially you don't treat agent and humans the same way. That's that's one one part of it. When it comes to audit, it's a volume problem. With humans, you could see one action done by the humans. Now you have two hundred actions done in a such a rapid space. How it happened. you'll probably have a lot of time figuring it out. What called what because if you just looking at actions by me in a real world, you'll see, okay, I clicked this scaffolder and it created a bunch of things. Very simple, very easy to follow. But if I look at the same laudit log later, Balogy called like two hundred things. And why did he do that? I can't even answer the question because I didn't do it. The agent didn't. So we need to figure out how do I sort this out? How do I solve this problem? For example, if you look at SOC analysts or like compliance people. They still wanna h hold it right because at the end of the day it doesn't matter whether agent did it. If you do financial violation, you can't say well sorry, agent did it. No, you still pay the price. So I think yeah, it becomes a challenge. It's a different problem. The last part I think you asked about like what is still hard. Approvals is hard as well. Like you you can see already in coding, humans create a code and then humans approve the code. It's a lot easier. One to one. I can deal with it because there's only one PR coming in once a month. from a the developer. Now it's a hundred PRs in a day or an hour. So who's gonna review it? So if agent approves everything and AI reviewing an AI, it becomes a bigger problem. Essentially now hundred PRs came in, hundred PRs got approved. You could obviously have a level of approvals, something very simple, go approve it. But then it still creates a lot of human tax at the end of it. You were doing everything slowly. Everything was built for slow. But now everything is so fast. Like it created such a problem on certain areas and it created another set of problems on other areas. And since it's all probabilistic, it's put AI to solve AI. But all layers are probabilistic. In the real world you don't wanna shoot a nuclear bomb because of some AI mistakes. So yeah. speaker-0: Man, yeah, so much interesting stuff you brought up there. It's it's almost in the sci-fi realm of ethics in terms of like who's responsible when the agent is the one who is creating these problems. Because at the end of the day, of course, it comes back on the human who at an enterprise organization, ⁓ it is a little bit difficult to assign accountability to to to say during a postmortem, you know, what actually went wrong here and how do we keep this from happening again? And with the the acceleration. of CI C D and all of these important parts of code, right? Code is cheap now. All all of us I think are are are reckoning with what this is gonna look like in the future. So I love the idea of putting portals in that. speaker-1: Beyond even coding Yeah, go ahead. So it's even beyond coding. I think the real challenge happens in sort of the non-coding situation. There's a report from DX. The number of pull requests is like thirty to sixty percent higher. The business outcome is only eight to fifteen percent higher. So the thing is you created a you know huge ⁓ PR, but that doesn't mean you know you can actually take it to production. There's a definite is more to this problem. Mm-hmm. In a non coding situation, that's even more higher stakes. Approving expense reports. You've seen examples where Chatbot creates a refund policy which is different from what the company says. And there's there's a lot of what we real problems could happen if humans are not in the loop or humans couldn't review in time. There's a policy has to be put in place, like a tight guard rails. It's an evolving problem. speaker-0: I I think it's worthwhile to to pause on that. So thank you for just drawing our attention to that that macro idea because I agree. It's it's beyond coding. ⁓ it goes outside of that. I most of the conversations I have around security and acceleration actually have to do with supply chain security. ⁓ so what you're saying about PRs is absolutely part of that. Okay, so the next part of your blog that I found very interesting, again, you're ⁓ quite the wordsmith, is when you talk about ticket cues ⁓ with a portal on top. And just to begin like what does that mean? When when you when you talk about that, like h help help contextualize that idea. speaker-1: The platform teams if they're still using like a service now ticketing system, they created a interface to it. But at the end of the day it still creates a ticket and then you're still solving it manually. There's a large bank that used to work with they have ten thousand tickets per month because they have a lot of developers and then they create a lot of tickets, but they're still doing it manually. So the platform engineering team essentially is solving a vacuum all basically. So I think the the real ⁓ thing is to be able to build these scaffolding so that self service can be enabled. And that's really how they solve the problem of doing that essentially so that developers can self serve themselves rather than the platform team essentially solving the ticket queue. So the the platform needs to be treated as a product so that people can self service, for example. And so anyway that was my point so that the platform needs love to make it worthy. And now with AI, it's even more important because if you don't have a good process, AI cannot solve your problem. Just because you have Cloud, Cloud cannot solve a badly designed ⁓ system. If it's gonna create a ticket, then you didn't solve anything. You have Cloud create a ticket for you. That's that's all you did. You haven't done anything to to really solve anything. speaker-0: Right, right, right. ⁓ which is making me ⁓ again think about this at a more organizational level. And so I'm wondering because these b these bottlenecks are are shifting, right? ⁓ and some things are easier to do, some things are harder, ⁓ would you argue that resources within an organization then need to shift? Do we need to invest more in some that maybe didn't need that sort of investment in the past? speaker-1: Yeah, absolutely. Like I said, from the team I was talking to, they have twenty people so called platform engineering, but they have to do a lot of things, right? And if the volume of tickets is like ten thousand tickets, they can't do anything else. So they need to hear away some time to invest in building those platforms so that you can ⁓ scale. And so absolutely the I think the investment has to be ⁓ the thing is obviously if you're getting that many volume of tickets, you're never gonna solve you're never gonna get out of the jail on that because it's like already overwhelmed. But you need to figure out a way to high value ⁓ things you can automate, you can make it self service and then try to get more time, more and more time so that you can actually do more things. I mean obviously they can use agents and other things to automate some of the decisioning themselves. But ideally you wanna have everything bolted down to the platform. So the platform itself has the guardrails. I have the ability to to do it for it, for the developers. Okay. speaker-0: So the sense that I get is that it's enterprise complexity that is this deciding factor. So does that mean that you are implicitly conceding that the skeptics are right for the small single cloud team ⁓ that all sits on one Slack channel, you know? So I I guess w where's the line where this matters? speaker-1: Yeah. If you're a small team, like I was saying, I was the only dev in my own project and I could do it without any any portal but it's I don't need a portal. I everything is in my head, so I can do it. And I think if you have a small team where you can just slack somebody and ask a question. But it obviously breaks down in a larger company. I would say maybe you're our less than fifty sort of company and now you're still maybe a single cloud, very constrained ⁓ environments or very deterministic. You're still in the early stages. I think it makes sense. But where I work, I don't even see anybody because there are distributions all around the world and there's no way that I could actually know what is right. I don't want to do it myself either. I don't want to make a mistake and cause the production down. So I think you need to build something that is available twenty four by seven and for humans and agents. And that requires some work. And so I would say that would be the distinction. Multi team, multi cloud, complex applications, higher security risks, like compliant like a regulated industries. I mean you can't just live code yourself. There's no way. speaker-0: Yeah, it's funny. I ⁓ part of me as an inveterate hobbyist and and a a vibe coder, I part part of me does feel like I would benefit, even though it's just me, from having that sort of guardrail, because I ⁓ when I come back to my projects, I forget. Like old Kate is not today Kate. I I do I forget what I've done, I forget why I did it. And of course who wants to write documentation for themselves? But that you know, th there's no one that needs it more than you. ⁓ so than me at least. ⁓ so but I speaker-1: Having the skills, for example, we can create our own skills to guardrail ourselves so that you don't have to tell Claude every time a new session starts what is right what what I want to do and how do I push something to production or how do I do certain things. I think so you can solve with sharing of skills. Here's a bunch of skills, all of us share the skills. And that's reasonable. But I think that's just one part of solving the problem. How many skills would you create? And ⁓ and How do you audit? If you have a scaffolder, you can audit better than a skills auditing. Skills auditing essentially is knowledge you're giving to the agent at the runtime to do something. Again, you can't audit that and and he could still go wrong because it is still a probabilistic approach. I was talking to CEO's ACA in one of my podcasts. You have to assume it's never going to be a hundred percent correct. AI is neural networks. It's like humans also. Humans also are not ever ha never hundred percent right. You have to assume it's gonna go wrong. The question is, when you go wrong, What is your blast radius? And the more grounded you are, the more closer you can get to less a blast radius. speaker-0: Right. Yeah, limiting that. Okay. That makes sense. So I wanna talk about another project that you've written about. And I mean, my main takeaway from it was that you spun up a Redis project, you were kinda kicking the tires there, and you were debating like what is the the line between the plumbing that you outsource and the things that you can't? Like w what how do we discover where judgment fits in here? So instead of me trying to summarize what that project looked like, would you mind sharing with us like what is it that you built and and what were some of your takeaways? speaker-1: Yeah. The models are really awesome. The models are not the bottleneck anymore. ⁓ we just got fable, the cloud models. And ⁓ so I think the models are not the problem. The real problem essentially is the context. So when you want to build a production ⁓ agents, we do that for our own products. How does it r remember the conversation? For example, you're talking to a chat GPT or a cloud, you know that the agent is very smart because he remembers who you are. He doesn't answer the same way for you versus for me because he understands who you are. So that kind of the short-term memory, session memory, long-term memory, these are really valuable aspects of production agents. And so that it doesn't start cold every time it starts up. I don't want to ask the same question. I know what your preferences are. I know what your background is. ⁓ these are some of the sort of like a components of production agents. There's lots of like context memory, like the organizational context memory. The biggest thing you're hearing nowadays is everybody's running out of tokens in one month or two months or three months. So cash is very important part where you're if five developers ask the same question, you're gonna have five different times a fable is gonna be used to to generate some some output and that'll be very expensive. So how do you store some of the cash? I think the biggest other thing is that obviously those are nice, you know, where you can lower your API cost, you can have short-term, long-term memory. But the real thing is there's a lot of data in the enterprise that needs to be brought to the agent to for it to make sense. And then the freshness of that. information is also relevant. You don't want data that's not right, that's not updated recently, for example. So that is the whole thing is a is a challenge. So there are different aspects. So the agent need all of these things. Like every time every agent needs the same information like memory and you know caching and and contextual data. I think these are infrastructure problem. It has got nothing to do whether you're working on a customer service chat bar or some other things. This information, this infra to get that information. The context varies for obviously for different use cases. But this whole infra is just infra. And what my idea was that let's just outsource that work. You don't need to do that work because that there's no like value in it. Particularly for platform engineers, I don't want to figure all of these things out. I just want to worry about my other problem I just talked about. What is the scaffold I should be building? Last thing I want do is like figure out this this basic one layer down infra layer. Right. So that's what I was saying is that Redis is just one example there More where you just wanna buy them. Vector D B or caching or retrieval. These are what you call commodity. Not commodity, we'll just say that. It's a base thing as a as an application team or a platform team. I would rather get it, buy it from somebody. speaker-0: Think what you're pointing to is is something that I saw you flag in a number of your your blog posts here. And that's the problem that ⁓ you know, keeping context current and authoritative as systems change is is an extreme challenge right now. So you know, in the Redis test, you know, the freshness piece seemed to be the component that you couldn't even evaluate. So if I guess if freshness is the soft spot under the the whole thing. Are we just teaching agents to act confidently on stale data? I mean, perhaps I'm paying playing devil's advocate here, but like ha wh where where does that f speaker-1: You're on to something. That is actually, I wrote a blog about it. It will come out in an hour. But basically, the rag and other things we've built, industry has built, is good at retrieving the content. So it's using semantic search to find the right content. He doesn't know it's right. So you have agent call up backstage or portal and say, hey, who owns the service? It tells you it's in catalog. But the problem is it gives you the answer confidently. And then agent says, Okay, the service is down, let me open up a ticket to TMI business. But that's not the right team anymore. It got the right information. It got what it thought it was the right information. It just retrieved it right now from a system of record, for example, and it is wrong. Or it is stale, basically. So it acted confidently. That's the problem, right? Essentially as a humans, we have a better sense of things. So you will look at when it was updated. What is the freshness of the data? So this was updated two years ago. But if you look at the service, Bob from Team B is the one who's maintaining the core right now. Or Alice from Team C is the one who's on Call of Duty. So it is multiple data points we can correlate as humans and and say, okay, you know what? There's something wrong here. Because I don't think team A is the right one because it's stale, first of all. And so that's the challenge that we need to solve as a industry. The verification. So today it's more like retrieved content you give to L L ⁓ And then you try to make a decision. Basically the idea is that you have better retrieval. I have great semantically accurate retrieval. Which is true. But it's not you don't assume that just because you have the better retrieval doesn't mean it's better agent. So what you're saying is we need to retrieve. That's essentially the evidence. In fact, it will be multiple evidence in some cases. And you need to verify that information. So then you get the better context. That is the right information that you give to the elder. So if you do that, then you're you're much better off. And so basically you look at the source authority, who where is the source coming from? And what is the freshness? When was the last updated, for example? And then you want to validate with other sources and you want to see obviously the source the freshness as well. And then you get a confidence score. So based on that, you give it to Lemon and say here are three ways I found the data. Team A, team B, team C, and here's my confidence level. Team A, most likely not because it was updated two years ago. Team B and C more likely. Team B is more likely because Bob is maintaining that code and maybe he's the one who's gonna respond. So that is what we need to get into. So it's not like I'm gonna shoot a ⁓ C P tool call, get an answer, go. I think the world hasn't gotten there. And it's almost like a Google search ranking, if you think about it that way. How did Google kind of like do when you do a search, it it ranks based on is it from The actual website or some blog that wrote about something. It actually tries to use the main website as a source of truth. And but agents today don't have that kind of a sense. It just gets something and then acts on it. So I think we need to figure out a way to go from sort of just a retrieved context to like a verified context. So that's a blog about you would see more of the writing I tagged users. Awesome. speaker-0: Okay. Can't wait to read it. So you mentioned retrieval and we've also talked about 18 months ago a lot of the IDP story as it related to AI had to do with like having a chatbot feature for for the search functionality, right? And so RAG was a big part of that in those early days. But now MCP seems to be the thing. You've mentioned it earlier, ⁓ but it's still young. Are you bullish about MCP? Do you think that that's where things are trending or ⁓ do you think it still lacks some of the functionality that you want in order to connect these disparate systems and in ensure that they are fresh. speaker-1: Yeah, exactly. So th that you already answered my question a little bit, right? So de definitely I'm definitely bullish on a standardized agent access, which is MCP is obviously the first protocol doing it. And I think it has gotten quite a bit of traction. Every major vendor is creating an MCP for their for their products. I think it's definitely on the way to becoming sort of the standard, at least for that side of the house. You still have other protocols like A to A and and A C P recently. So there are different protocols coming to solve different aspects of the problem. But a and by the way, MCP by itself, like we just talked about, an MCP agent to a catalog that gets the stale information, it doesn't solve the problem. I mean, it's good that it got the information and it could go to Confluence and and go to GitHub and go to PagerDuty and get different information for the same problem. So it is just a retrieval method. It's just a way to connect to something and get information back. It is not solving the problem I just talked to earlier, which is the verification of the information. So In some sense, is MCP the winner? I've seen like examples like one of the A sixteen Z V C was saying, CLI is the right way. And we have seen multiple, even for our products, we are thinking about doing obviously the better documentation was a helpful thing. Obviously API already existed. CLI could be another one we could do to make it better. so these things will evolve. I'm not worried about it. I think MCP is a definitely ⁓ is a obviously accepted protocol at this point. It doesn't solve your governance or trust or the quality issues that we talked about. speaker-0: Yeah, that resonates. ⁓ so I want to talk about ⁓ again some of these organizational issues that are newly important in in our agentic era. And that has to do with like gatekeeping and making sure that Golden Paths support folks like product managers and these, you know, team leads who are now shipping agents. How do we make sure that they do it in a safe way? What sort of advice are you giving to the teams at at Red Hat and what other organizations you're working with to to make sure that we're doing this ⁓ i in a way that ⁓ ensures that these are secure, compliant, etcetera. speaker-1: Like anything else, you want to do a crawl, walk, run sort of model. Obviously there's a spectrum basically of what you can do initially and and obviously start with the easier ones. Summarization kind of ⁓ use cases, maybe read only agents, and then you're slowly getting into autonomous activities. I mean human initiated versus ⁓ agent initiated. is another thing. Human initiator is maybe the first one. Humans can make a decision on when to initiate something and obviously make sure it works correctly. I do believe that you're gonna have lots of agents created by a lot of people. Either personal agents they create for themselves and which acts on some data. Also I'm seeing is we talked about this portal where people bring your own agents. These are specialized agents that are created from people of authority. Let's say I want to do migration for example. And I know exactly how the migration framework works and somebody creates it, tests it over a period of time. But they don't just give it a production agent without testing it, validating it like small key. Gradually making it available to a larger audience. So I think it's anything else, adopting new technologies, you do this the crawl, walk and run approach. speaker-0: Okay. And then applying this to how you a lot of these lessons that you've learned and and sort of your argument around like where the the future of portals is heading, that they're not dead, they just need to to maybe adapt. How does Red Hat's developer hub factor into all these things that you've noticed around platforms today? speaker-1: Dopro just an example of a portal. I'm sort of all the things I talked about, I'm trying to make it happen within that portal. ⁓ so we ⁓ talked about we started with enabling just the access to information because it's in the catalog and obviously templates and other things one can do as well. And people want to have access to agents and the ability to enable MCP for everything. So the humans doesn't have to come to the portal at all and trigger something. They could just trigger it from their cloud code, et cetera. I think the other things that we're looking at we I alluded to already is that people want to build specialized agents and have them connect to the portal. So why would developers still come to a portal? Today they used to come there or they're coming there right now to look at certain information. The portal for example has a plugins that allows you to see five different tools in one place. That's really powerful if you think about it. You let's say I'm running an application, I'm using GitHub, GIR tickets, I'm using different pipelines or deployment technologies, I'm monitoring Quay or a Docker Hub to store stuff. That's like ten, fifteen tools. There's so many tools you can have. So today the portal already gives you all the information in one spot. Humans are better at looking at this thing quickly. Like if I have to ask an agent and have it pull all this data over a long period of time, it's gonna cost me like ten dollars. I could have just clicked it and gotten all the information. It may not be right because I don't know if it's right. So I could have just gone to the portal and just click it and be done with it. So in some ways it's still gonna remain. So Now the same information can be shared from the agents also. Let's say instead of agent going to these ten different tools, I could be at the agent and I could be talking to Developer Hub and Developer Hub it already has the curated data from the plugins. So it could serve that information to the agent. So the portal becomes sort of a tier layer. I've already have curated data of that application. So you agent don't have to go look around for it. I already have it. It's called plugins. Already have it. So I'll go and give it to you. Don't waste your time. So don't waste your tokens. So we can actually give it to you. So anyway, the point is that's a useful, interesting use case where you say, why portal anymore? Because I gave the data for you in an aggregated format. And that's a potential value. People are moving from I want to come to a portal to look at things to I want to take actions. So that's where this integration with agents, your own agents makes sense. For example, I see a problem, let's say if the service is down or having errors, today I can only look at it. I'm like, okay, the service is down. Now I have to hop off to a data dog or Planet Trace portal and figure out what the errors are. Instead, I could just be on the portal and say, the service is down. Can you troubleshoot this? Then the portal is not trained to troubleshoot in your organization. So you could create an SRE agent that's customized for you, that knows your organizational context, that knows what run books I can run to remediate the problem. From a developer point of view though, he goes there, he sees something. Today you can only see it and do something else later. Now you can take your actions. So you can actually say, go fix it. And then it immediately goes and fixes it. So in a way, now portal becomes interesting. No, I go to portal because hey, it actually does things for me now. So I mean, I think these are all different ways to skin the cat. You could obviously argue, I could turn a clot and the same thing. That's fine. I believe that the world will not be like one way for everything. I think there'll be a long tail of people who wanna still have a UI way to go about it. And there'll be people want just to ⁓ agentic way. Those are all fair games and our job is to be neutral and serve you whichever way you want. speaker-0: Fair enough. ⁓ I got one last question for you, and then we will wrap this up. ⁓ so much good content here. I think I want to end on thinking about the future here and where you see portals going in the next five, 10 years. ⁓ so I mean, we've already discussed that there is an agentic user for these portals, and that's causing thought leaders like yourself to to really have to reevaluate what it is that portals even can do here. Do you think that in the future it the the balance is gonna tip toward toward agent users? And if that's the case then, are we gonna get rid of this whole single pane of glass sort of approach to the UI and and we'll just let that, you know, crumble, right? We we don't need that anymore. That's that's not important. We don't we d maybe we'll just have this in the CLI like you you had mentioned as like a an access route. ⁓ are you anticipating that use that like human use users are not even going to be accessing these portals at some point in the in the future and that it'll be completely a computer resource that that we just maintain and and keep tabs on. speaker-1: Yeah, of all, ⁓ tokens cost money. And sure it's not free that you can just do agents and I'm sure that the subsidization will go away. I try to look up ⁓ who owns a certain product within my ⁓ company and I I ask a lot and it takes me like five minutes to get back an answer. I could have just go into the directory page and looked at it myself. In some ways, token costs money. Humans can make better judgments in some of the cases and ⁓ humans are faster from any of the tasks. I can look at a large UI dashboard with all kinds of services and where the problems are happening like that because there's a red button there where it says everything else is green. If you ask ⁓ agent to figure the same information out, it's gonna take you ten minutes and it's gonna burn a lot of tokens and then come back and say the same answer. I'm not saying it won't say the same answer. But then it took ten minutes to do something that I could have just looked at it visually and okay, there's a red butt there. So I think humans are also smarter for some tasks. I think we can quickly ⁓ triage thing. The AI is essentially trying to become a human, obviously a better human and faster human in some cases. But we are actually the smartest brain in the world, so to say. And so I think we can do things faster too. So I think the right way to look anything, actually it's not just a portal issue. It's basically anything, is what's the fastest way to get the things I want? I think people would tend to do that. And I would argue that it's not always C L I or agents. There's a place for it a hundred percent and it's accelerating lot of work, but there are certain cases where UI based solution. I think we're arguing for UI based solutions now, not just a portal. You know, like it still has a value. It still has a reason to exist. And so I think portal would would be fine. Like I said, we're gonna evolve into a more useful context layer, verified context layer if it can be, that would be the best way to essentially solve a real enterprise problem. Agents needs that and we can be the layer of that. Okay. speaker-0: So things are evolving, absolutely, but the humans are still necessary. We still need taste judgment, all of these things that our brains are well equipped to accomplish. ⁓ you're you're not worried about us being replaced ⁓ anytime soon. speaker-1: I hope not. We've still been doing podcasts. Well, we're already getting replaced. Google podcast already does it itself. You know just drop the notes and it will do a great ⁓ podcast. speaker-0: Yeah, the horror. Yeah, no. Well, I I am one hundred percent with you. Let's let's make sure that the humans are ⁓ still doing the podcast at least, because I I I can't take it. ⁓ okay, well, ⁓ Belagi, it's been an absolute pleasure speaking with you today. ⁓ looking forward to reading your future blog posts on this subject. It's super important, extremely interesting. The folks at Red Hat are doing a lot of really important work around this and and just kind of thinking through what this ⁓ a future is gonna look like because my goodness ⁓ You know, there's there's so much at play right now. Things are moving so fast. So yeah, again, I appreciate you coming on here to to give me your perspective here. speaker-1: It was great talking to you. Appreciate you inviting me and ⁓ glad we could do it. Yeah. speaker-0: Me too. Where are you directing folks in terms of ⁓ your social media presence or your blog? How how can folks ⁓ you know get a taste of your writing for themselves? speaker-1: I think ⁓ medium is a way to go or ⁓ I have a podcast called David AI Podcast. I try to bring a thought leaders into the conversation as well. So maybe you can link it in. speaker-0: I'll include those in the show notes. Okay, so we've got ⁓ your blog, we've got your podcast here, ⁓ the blog post that's from Red Hat, I'll include all those in the show notes. And let's go ahead and wrap up then. Again, my name is Kate Holterhoff, I'm a senior analyst at Red Monk. My guest today has been Balaji from Red Hat. If you enjoyed this conversation, please like, subscribe, and review the monk cast on your podcast platform of choice. If you're watching us on Red Monk's YouTube channel, please like, subscribe, and engage with us in the comments.