Latest / CXOTalk / Agentic AI in the Enterprise 2026 | CXOTalk #916
Transcript
- 0:00Was a Goldman Sachs study that came out.
- 0:01Enterprises are blowing through their token budgets a years
- 0:05worth of AI budget in three months.
- 0:07This is the notion of token maxing, which is as many tokens
- 0:10as you provide will get consumed as quickly as possible.
- 0:13Naveen Akiraju, an Insight partner, invests across the
- 0:17Agentic AI stack. Praveen, What is the state of
- 0:21agent technology today? So if you kind of take a step
- 0:24back and and look at the evolution of just the models,
- 0:28the end of 2024 was like one of the most Seminole moments in the
- 0:32in the evolution of models, right?
- 0:35It was when the four O model came out, which essentially
- 0:38introduced the concept of reasoning.
- 0:40And I think that was the first step towards truly making AI
- 0:45less of a chatbot and more of an agent, right.
- 0:49And so from there, we saw the introduction of DeepSeek and
- 0:54sort of early or late, late 24, early 25.
- 0:58And that fundamentally changed the economics of of large
- 1:01language models. And later part of last year was
- 1:05when we started to see the true sort of agentic capabilities in
- 1:10the model start to start to emerge.
- 1:12Coding became a huge use case. And in all of a sudden we, we
- 1:15start now realize like, wow, you know, you could actually use
- 1:18models to fundamentally, you know, write code.
- 1:22And beginning of this year, I think was sort of the the real
- 1:25inflection point, both in terms of, you know, the, the
- 1:30capability that came forward from coding perspective, but
- 1:34also the ability for open source frameworks like Open Claw to be
- 1:40able to use models to actually show us that work can be done.
- 1:44Not just, you know, technical work like coding, but also just
- 1:49basic work, right. So you see that arc of models
- 1:52and the evolution of these things, the pace has been
- 1:55dizzying towards unlocking the true capability of these large
- 2:00language models to actually, you know, do work for us.
- 2:03So what is the state of AI agents today?
- 2:09Think about what agents is an agency is right, which is
- 2:13essentially the notion of being able to take a task, understand
- 2:18the context of the task, break it down into a plan, and then be
- 2:22able to use a set of tools to execute the task.
- 2:25That essentially what the definition of of agents and
- 2:28agency is. Now, the core of these agents is
- 2:30the large language models that we, we've talked about, you
- 2:33know, the evolution of these things.
- 2:34But fundamentally, you know, you think of large language models
- 2:38as these kind of stateless, stateless machines, right?
- 2:42In the sense that they absorb all the information you give
- 2:45them. You know, it's a compression
- 2:47algorithm. They essentially take all the
- 2:48world's information, compress them into weights, right, and
- 2:52are able to sort of reproduce that or respond to prompts based
- 2:57on those compressed weights. Now the issue here is that
- 3:02because they're stateless, they don't remember or like humans do
- 3:06when we do tasks, the context of what the work that has been done
- 3:10before. So the real sort of evolution of
- 3:14agents has been the combination of these really powerful large
- 3:18language models with these kind of things we call harnesses,
- 3:22right? And so harness essentially is,
- 3:27you know, is a set of tools, It's context, it's memory that
- 3:31is given to the model in order for it to perform a particular
- 3:34task. So, you know, we can think of
- 3:37these harnesses in the context of applications, right?
- 3:40So an application like, you know, Cloud code, right, has a
- 3:44very well defined harness that gives us the, gives it the
- 3:47appropriate tools, the memory, the, the kind of guardrails, if
- 3:52you will, in terms of coding practices and such like that,
- 3:55allow it to then deliver code, right?
- 3:58So combine the model with the harness, give it a set of tools,
- 4:02and that's how you kind of do work, right?
- 4:04And more more recently, I think you started to see, you know,
- 4:09applications beyond coding. A ChatGPT in and of itself is an
- 4:12application and open Claw essentially is a personal agent
- 4:16which created its own model, right?
- 4:19It's own, sorry, it's own harness that attached to the
- 4:22model. And So what you can think of
- 4:24these as if large language models are stateless, you know,
- 4:28powerful AI engines, you combine that with harnesses to give it
- 4:34sort of things like in context learning so you can actually do
- 4:39work. Remember, you know the context
- 4:42of work you're doing in a long horizon and actually execute
- 4:46tasks. We have all of this opportunity
- 4:50and there's been evolution among the, the, the tools and we all
- 4:56see the, the promise, or at least we're told there's all of
- 5:00this promise out there. So what's real versus what is
- 5:06the hype in agentic AI right now?
- 5:10Can you help sort that out? When you think about just the
- 5:13models itself being going back to because that's the core of a
- 5:16lot of what agents are, you know, they have what's called
- 5:20jagged intelligence in the sense that they are state-of-the-art,
- 5:24like amazing, you know, surprise you with the capability that
- 5:29they have in terms of coding or in terms of being able to
- 5:32synthesize information. But at the same time, they make
- 5:35very common silly mistakes, right?
- 5:38Or they're easy to be tripped up when you when you change
- 5:42prompts. And so we're still on that
- 5:44frontier of jagged intelligence. So I think when you think about
- 5:49agents, you know, 2025 was sort of the, the, the year of agents
- 5:53and rightly so because the models advanced.
- 5:56I think we had, you know, huge advancement in terms of computer
- 6:00use agents, you know, browser use things like that, that, that
- 6:04are that enable these agents to be real.
- 6:08And so you you saw this significant inflection in terms
- 6:12of deployment of agents. So, you know, sophisticated
- 6:15enterprises today have deployed over 1000 agents in production
- 6:19for various different tasks, right?
- 6:21You know, at the same time, this jagged intelligence essentially
- 6:24means that you really need to do your homework in terms of
- 6:29defining this harness, right, for your particular use case.
- 6:33Which essentially means that you have to make sure that the agent
- 6:37has the appropriate context. And that context is essentially
- 6:41mean in in an enterprise's use case, access to the appropriate
- 6:44data, the appropriate sort of guardrails, compliance right and
- 6:49and security framework that enables it to then function
- 6:53within that that sort of that particular use case.
- 6:58And today, most agents still do incorporate a human in the loop
- 7:02for most tasks as a way for from a from a final sign off
- 7:06perspective. So I think, you know, the the
- 7:08hype versus reality is that are we seeing agents in production?
- 7:13Yes, right. Are they perfect?
- 7:15Are they completely autonomous? You know, fully agentic?
- 7:18No right. You need to do a ton of work in
- 7:22terms of defining these harnesses, defining these sort
- 7:24of guardrails, if you will, in order for you to get the
- 7:29appropriate, you know, output with a human in the loop.
- 7:34And they're also, you know, significant areas of of concern,
- 7:37right? I mean, you know, security being
- 7:39one of them, but but also how do you actually regulate how token
- 7:47use happens, right? Are these agents doing useful
- 7:50work? You know, do you, do you
- 7:53actually have the right sort of governance of how you manage all
- 7:58these agents? If you have like 1000 agents in
- 8:00an, in an organization, you know, how do you kind of how do
- 8:04you ensure that they are, they have access to the right tools,
- 8:08they've been governed properly. So there's there's a lot of
- 8:10these open questions, which I think a lot of enterprises are
- 8:15still working through. But there's no denying the fact
- 8:18that just from a pure capability perspective, models, harnesses
- 8:23and such like, as well as from a, you know, deployment in the
- 8:27field, we've seen a huge step forward over the past year.
- 8:31You have just laid out the agent world.
- 8:37Let's start unpacking this. Let's begin with applications.
- 8:42So where are you seeing real applications for autonomous
- 8:50agents that are effective, that are actually delivering real
- 8:55value? Let's start there.
- 8:57There's a spectrum here, right? So you can think the spectrum is
- 9:011 end is what we as individuals experience in terms of agentic
- 9:06capabilities. And you can think of that as
- 9:09you're, you know, you might be using a tool like, you know, you
- 9:16maybe you built your own open claw or something like Manus.
- 9:18On the other end of the spectrum, you have enterprise
- 9:21class agents, right? And, and there's a whole set of
- 9:25things in the middle where you have vendors building these very
- 9:29specific agents for specific kind of use cases.
- 9:32So I think if you kind of look at look across the spectrum, I
- 9:36think one thing that obviously was like a huge Seminole moment
- 9:39was open claw, you know, earlier this year.
- 9:43And the reason why Open Claw kind of burst out was, and if
- 9:46you're going to go back in history of AI, there's been a
- 9:49lot of companies trying to build these personal agents, you know,
- 9:52agents that you and I could basically deploy, give access to
- 9:55our, our contacts, you know, our e-mail, calendaring, etcetera,
- 9:59and have them do work for us. And Open Claw was the first kind
- 10:03of open source framework that actually started to work because
- 10:07it was able to bring the, the appropriate sort of ability to
- 10:14understand the work, ability to remember work, right, and
- 10:17ability to take the right context and plug it back into
- 10:21the model in order for it to execute a task.
- 10:24And I think that truly blew people's mind because it was so
- 10:28it was so effective, right and simple to, to, to execute.
- 10:32So you saw on the personal side and I think all of us have been
- 10:36playing with, you know, cloud code either on the CLI or in the
- 10:40application itself and and doing, you know, interesting
- 10:43work flows in a very seamless way.
- 10:46So I think the so we've seen huge amount of like personal
- 10:49productivity improvements. Most of these tend to be like
- 10:55single use case or more single user oriented agents, right?
- 11:01The other end of the complexity is where you actually have an
- 11:04enterprise that's trying to build or, or, or buy an agent
- 11:08that needs to be deployed across thousands of employees.
- 11:12So here you're kind of seeing there's this two aspects to it.
- 11:16So the first aspect is there's certain functional sort of
- 11:21agents, you know, you can think of customer service, for
- 11:24example, where you you clearly have, you know, well defined
- 11:30agents that are effective, that are able to deflect a lot of
- 11:37first line kind of support calls and and be able to kind of
- 11:41demonstrate productivity, right in that context.
- 11:44And so you're seeing the the use cases where you're seeing clear
- 11:48fit is customer support. Obviously coding.
- 11:52Coding I think has been the ultimate sort of AI use case.
- 11:56I think it's almost like it's estimated about 3 billion in
- 11:58revenue right now from just coding agents, you know,
- 12:01customer support agents, I think it's close to about 500 million,
- 12:05you know, in terms of revenue generated just for that, for
- 12:08that sort of use case. I think search has been another
- 12:12one that's it's, it's less of an agent, I think it's more of a
- 12:16tool. But you know, you've seen good
- 12:18traction enterprise search, right, as a, as a fundamental
- 12:23use case, right. And if you think about sort of
- 12:26the specific sort of verticals where you're seeing good, good
- 12:30adoption and tools, you know, clearly there's been a lot of,
- 12:33lot of momentum in the legal space in, in healthcare, in, in
- 12:38a tech forward, companies are essentially deploying AI agents
- 12:42in a significant way, right? So I think when you look at the
- 12:46spectrum, you, you see significant advancements in the
- 12:51personal sort of single user, single use case kind of agents
- 12:55where you can build your own because the interfaces become
- 12:58much more simpler. At the same time, enterprises
- 13:00are focusing on these specific sort of use cases where there's
- 13:03clear product market fit, clear value and measurable ROI for
- 13:09these deployments. We have a very good question,
- 13:12interesting question from Anthony Scrifignano on Twitter,
- 13:18who is the former Chief data scientist of Dun and Bradstreet.
- 13:22He's been a guest on CXO Talk a number of times.
- 13:25And he says this one of the big challenges with agency in the
- 13:30real world is negligence. Can you talk about the danger of
- 13:35AI agents acting in concert in a negligent way and the impact on
- 13:41the enabler? When do you think about models
- 13:46fundamentally, I think they are prompt driven.
- 13:48And you know, when you think about sort of the construct of
- 13:51an agent, you know, you give it a task, it creates a plan and it
- 13:56goes out and uses a bunch of tools to execute that task.
- 13:59So you're in that. There are many, many areas where
- 14:05you could potentially have this notion of in, in the plan could
- 14:08be wrong, it could be using wrong tools.
- 14:11It could take the input from the tools and and and go down a
- 14:15different path, right, where it might actually do something
- 14:19malicious. Or there might be a prompt
- 14:20injection attack that makes it do something malicious.
- 14:23So I think this is the reason why this notion of a harness and
- 14:28guardrails is so, so critical, right?
- 14:32And almost say the the agent is actually the harness.
- 14:36So the ability for you to create, define clear guardrails,
- 14:41right, clear constraints and clear, you know, evaluation
- 14:46criteria for the agent is extremely important.
- 14:50And that's a fundamental aspect where you need to spend a lot of
- 14:53time when you build these agents because, you know, essentially
- 14:58these large language models are steerable.
- 15:01I mean, they're very sensitive to prompts and they react very,
- 15:05very, very immediately to the prompt that they're that they're
- 15:09given, right? So the actual sort of
- 15:12constraints and the the steering of this agent has to happen in
- 15:17the harness. So, you know, observability is,
- 15:21is extremely critical to be able to incorporate into these.
- 15:24So 11 design aspect here that we we're seeing is this notion of
- 15:28sandboxes, right? You know, if your agent is
- 15:32essentially, let's say, writing code to execute specific tasks,
- 15:37what we're seeing from a design pattern perspective is, you
- 15:40know, developers are spinning up these sort of ephemeral
- 15:43sandboxes to first kind of make sure that the the code is that
- 15:48the AI agent has written is sandboxed, executed is is
- 15:52essentially measured against the guardrails to ensure that it
- 15:56confirms before you actually execute and push it out into
- 16:00production. So there there are ways to
- 16:02address this, but you know, it's absolutely a critical concern.
- 16:06We have a great question from Chris Peterson and he says
- 16:10giving agents access to our quote UN quote context is
- 16:15interesting, but how do we maintain boundaries between the
- 16:20context we own as individuals versus what belongs to others,
- 16:25friends, family, employer and so on?
- 16:28This is sort of one of those things where from a data access
- 16:32perspective, you know, being able to design this thoughtfully
- 16:36is critically important. Again, when you look at sort of
- 16:41the way an agent accesses data, you either giving it an API or
- 16:48you're giving it access to an MCP server, or maybe you're
- 16:51using CLI. If you essentially, you know,
- 16:55getting giving it access to something that's running on on
- 16:58your computer. So an API app gives you the most
- 17:03granular ability to create, like, you know, an Oauth
- 17:08architecture that allows you to be very granular about the kind
- 17:11of information the API can access.
- 17:14MCP tends to be a little bit more abstracted in terms of what
- 17:18what it returns because it's more of a generalized sort of
- 17:21access paradigm. So when you give an agent access
- 17:26to data, I think you want to pick the right sort of interface
- 17:31that it uses in order for it to be able to get access to this
- 17:35this particular data. And again, you know, I'll, I'll
- 17:38go back to this notion of creating the right guard rails.
- 17:43You know, you have these tools, as I said, either using an API
- 17:46or, or the appropriate choice there.
- 17:48But also just in terms of policies, you know, you can
- 17:51define these dot MD files that effectively tell the agent, hey,
- 17:56check to see if this data is PII, right?
- 18:00Or check to see if this data is sensitive before you actually
- 18:04use it. You can define what the agent
- 18:07does in in these kind of dot MD files that are given as a
- 18:11specification for the agent when it gets started.
- 18:13Yeah, sure. Except when the agent ignores
- 18:16the instruction and gives you back something altogether
- 18:21different. And then you look at it and it's
- 18:23not right. And you say, well, you didn't do
- 18:25what I said. And the agent comes back and
- 18:27says, yes, you're right, I didn't do that.
- 18:29Here's what I should have done. And this is totally, and this is
- 18:32seamless. So, you know, there's no way to
- 18:35know unless you know something about the answer that you've
- 18:39been given. This is very dangerous.
- 18:41You give it the appropriate guardrails.
- 18:43You give it the appropriate data, it still produces a
- 18:47result. That's, that's not, that's
- 18:49incorrect, right? And so, you know, what we are
- 18:53seeing again, from a design perspective is you, you need to
- 18:58create the right observability framework.
- 18:59So you're not just waiting for the output at the end of the
- 19:03agent's execution, but rather you're looking at the traces in
- 19:06at every single step, right? And being able to sort of
- 19:09understand that. And again, combining this with
- 19:11sandboxes, you know, we have a company in this space called E
- 19:13to B that that that's been, you know, tremendously the
- 19:17successful kind of helping developers with this right to
- 19:21create observability at different stages of the agent's
- 19:24execution. So you can catch it because, you
- 19:27know, look, this is an important point, right?
- 19:29Like errors compound. So if you have a large language,
- 19:34you know, you have a multi agent architecture and an agent makes
- 19:36an error and feeds it into another agent, you essentially
- 19:39have a compounding effect of error.
- 19:41So it's really critical for you to be able to have this kind of
- 19:44granular observability and design the evals.
- 19:47Now it's easy for me to say to design the evals, but that is
- 19:51actually turns out to be one of the more challenging aspects of
- 19:55agentic design, right? And I think you see the
- 20:01sophistication of something like Cloud Code, right?
- 20:04Or a customer support agent? It's actually the brilliance of
- 20:07the harness because they've actually done exactly this.
- 20:10They've created the observability.
- 20:11So it checks it's answers, they create the eval so it actually
- 20:15knows what is correct and what is wrong.
- 20:17And, and that's kind of how you get to and it's a fine tuning
- 20:21process. And I think a lot of enterprises
- 20:23are actually going through this, which is to essentially start
- 20:26this process of billing the agent and, you know, continually
- 20:29tweak it till you get to a point where you feel like, OK, you,
- 20:32you've got this, you know, you've got the, the, the agent
- 20:37executing the way you want it. Preeti Narayan, who asks a very
- 20:41interesting one. And she says most enterprises
- 20:45are built for AI assistance, that is, tools that help humans
- 20:49decide, but agents actually make decisions and act on their own.
- 20:55That's a fundamentally different problem.
- 20:58So when the shakeout comes, and parenthetically, it's
- 21:03interesting that she's making the assumption that a shakeout
- 21:06will come, but when she But she says, when the shakeout comes,
- 21:12do the survivors rebuild? Or do the ones who bolted agents
- 21:16onto old infrastructure simply not survive?
- 21:21The rate at which model capabilities advancing, right is
- 21:25making a lot of the assumptions or sort of these notion of you
- 21:30can wrap things around a model and and deliver value obsolete
- 21:34because that's been absorbed directly into the model.
- 21:37So I think the the so there's if you kind of like take about this
- 21:42question, I did a couple of things here.
- 21:44So the first thing is how, you know, all the legacy software or
- 21:49the existing software that we have, you know, how does that
- 21:53transition into in, in this agentic era, right.
- 21:57And the second question is, you know, agents have autonomy and,
- 22:02you know, what does the future look like when, when you know,
- 22:05effectively, we do get all of these things figured out and,
- 22:08and get fully autonomous agents, right?
- 22:11I, you know, I kind of see the world as a bit of a hybrid and,
- 22:15and that's primarily because the challenge of actually allowing
- 22:20an agent to be fully autonomous is, is a fairly challenging
- 22:25task. I mean, in a bounded problem
- 22:27space, like let's say coding where you know what the output
- 22:30needs to look like, right? You can measure it, you can test
- 22:34it, right? And you can certify it.
- 22:37You, the convergence towards autonomy is likely to happen
- 22:41faster. And we see, we see this today.
- 22:43You know, I think if you talk to some of the large labs, they'll
- 22:46tell you that they feel like coding is at AGI already in the
- 22:49sense that it is a level about human capability and is able to
- 22:54actually work as as well as humans do, right?
- 22:58But in, you know, more sort of unbounded tasks or I think a
- 23:01task where there's a lot of a lot of dynamicity.
- 23:04Like, you know, let's say you're manning a supply chain in an
- 23:07enterprise and you know, you have different skews, maybe a
- 23:10different Jio, maybe a different sort of supplier onboarding with
- 23:14various different criteria. The ability to kind of
- 23:17generalize what the agent does across these new capabilities
- 23:22pretty much again comes down to how well your harness behaves.
- 23:25And so designing that is a lot more challenging than, you know,
- 23:29a bounded problem like solving math, right, or being able to,
- 23:33we need to write good code. So I think that's the way I
- 23:37would kind of think about this in terms of this, you know, how
- 23:39do you spread these across bounded and unbounded?
- 23:42So for existing software, you know, if you're just a bolt on
- 23:46agent on top of software without fundamentally changing the the
- 23:51way you know, software interacts with data with with users and
- 23:58being able to respond dynamically.
- 24:00Then I think you clearly are going to lose that battle to
- 24:03potentially a newer version of of software that's a lot, lot
- 24:08more sort of agentic and dynamic, right Ability to
- 24:11respond, take input, you know, be able to do work and work and
- 24:17be able to offload the human from a lot of the work that an
- 24:22existing platform would do. However, I think I do see a
- 24:26future where you, we and we are seeing this and we have a number
- 24:31of companies obviously that are doing, you know, great work,
- 24:35being able to sort of understand how an application works and
- 24:38figure out where can they insert an agentic step that accelerates
- 24:45the time to value for the customer.
- 24:46I'll give you one example. You know, Stanley, which is one
- 24:50of our companies, they do accounts payable that are
- 24:53procured to pay a company, right.
- 24:54So one of the challenging aspects of of accounts payable
- 24:58workflow is reconciling invoices.
- 25:01And you know, it essentially is a manual task because unless you
- 25:05reconcile the invoices, you know, you can't approve the
- 25:08payment. And So what the team's done is
- 25:10basically said like, hey, let's, let's understand the entire
- 25:12workflow of accounts payable and figure out where are the steps
- 25:17like, you know, potentially this form filling or there's invoice
- 25:20reconciliation where we can insert an agentic step to
- 25:23accelerate the time to value for the customer.
- 25:26I think that's a thoughtful way where you can actually take an
- 25:31existing software platform, right?
- 25:33And of course, it does require some surgery, right, to be able
- 25:36to insert these steps in the middle.
- 25:38But then you can accelerate dramatically the value for the
- 25:41customer because all of a sudden now you, you've eliminated, you
- 25:44know, days worth of work of invoice reconciliation.
- 25:47The the AI does it for you and then the human approves it.
- 25:51So you know, that is real value in an existing software
- 25:54paradigm, right? At the same time, there are
- 25:57other examples where it makes sense to kind of like bottoms up
- 26:02reconstruct sort of the the, the workflow.
- 26:05So a lot of these sort of operations oriented software,
- 26:09right? You know, whether it's security
- 26:10operations, IT operations, etcetera, customer service
- 26:13operations where you have a, a strong data platform that you're
- 26:19then able to now build an agent on top to essentially, you know,
- 26:23use one of the LLMS super powers, which is the Sonic
- 26:27signal from noise. So I think that's the way, you
- 26:30know, that's why I, I believe this is a hybrid world where
- 26:33they will be, you know, companies that are doing things
- 26:37in a thoughtful manner to transform the existing software.
- 26:39At the same time you have bottoms up AI native approaches
- 26:42which you know clearly will will either create new surface areas
- 26:45like coding. Nobody thought coding engines
- 26:48would be a a a brand new kind of market space.
- 26:50It's $3 billion worth of revenue in that space right now.
- 26:53We have a question from LinkedIn on LinkedIn from Swami Vai Vai
- 27:00Jonathan. And Swami says, how do you
- 27:04envision the target operating model #1 enterprise AI to be
- 27:09dominated by dominated by human in the loop by design or
- 27:15autonomous agentic enterprise with guardrails and governance,
- 27:21especially with the questions around AI reliability.
- 27:27And obviously the the human in the loop is what everybody is
- 27:32doing now because it's a more conservative approach.
- 27:34But how do you see this shaking out?
- 27:36Human in the loop is the defined paradigm and in a lot of ways I
- 27:41think that I go back to sort of my reference of bounded
- 27:47unbounded problems. The the more deterministic a
- 27:53particular workflow is, right, the more bounded, more well
- 27:57defined the outcome is, I think the faster the opportunity for
- 28:01us to get to autonomy will be, right?
- 28:05I think I, I don't believe that we will at least in the near
- 28:09term, be able to get to complete autonomy across the entire
- 28:12enterprise architecture. The enterprise is just way too
- 28:14complicated. You know, you have a lot of very
- 28:18specific workflows that are unique to an enterprise.
- 28:22So you can, you know, divide the spectrum of use cases.
- 28:26You know, there's some standardized workflows, you
- 28:28know, maybe front end facing workflows, which are fairly
- 28:31standardized across multiple enterprises where you could
- 28:34potentially get to a higher level of, of autonomy.
- 28:39But the, it's, it's a combination, I think of three
- 28:42things. So the first thing is the nature
- 28:44of the workflow itself, right? And the, the, the, where you lie
- 28:48in the spectrum of, you know, bounded deterministic workflows
- 28:53versus dynamic sort of more unbounded workflows.
- 28:56The, the second aspect I think is there is a regulatory
- 29:00compliance aspect to a lot of these industries.
- 29:04So if you're in healthcare, for example, where there's been a
- 29:07lot of AI adoption in a space that typically has been
- 29:10challenging in terms of technology ramps, you see a lot
- 29:14more rapid adoption. But there is significant amount
- 29:17of regulatory and compliance based requirements essentially
- 29:23requires a human in the work, human in the loop, because you
- 29:27do need that, that sign off and that accountability that is,
- 29:30that is mandated, right? And the third part of this is,
- 29:33again, it's going back to the previous conversation we're
- 29:35having is how you know how well designed is your guardrail,
- 29:42right? How well designed is your
- 29:44governance framework, right? How well designed is your
- 29:47security architecture in order for you to have that confidence
- 29:51to let the model function, if you can go back to the human
- 29:55way, listing coding as an example, you know, you always
- 29:59had this notion of you, you write code, there's a there's
- 30:02APR and then there's a code review in order for you to be
- 30:05able to before you push code to production, right.
- 30:08And you know, so there's always sort of even in your
- 30:12traditional, you can take any workflow.
- 30:14I mean, it could be a legal workflow, it could be a finance
- 30:17workflow. There's sort of there's amount
- 30:18of work done and then there's like one step of approval could
- 30:22be a senior in a legal professional or finance
- 30:25professional who signs off on the work before it goes out.
- 30:28So that step of verification has always been built into a lot of
- 30:32these work flows. And so the question is, you
- 30:37know, is that step, you know, is it worth eliminating that step,
- 30:40right? Or is, you know, can, if you're
- 30:44able to get to increasing amount of accuracy on those, you know,
- 30:49that is essentially what determines whether you can get
- 30:51there. So the quality of the harness
- 30:53design and the quality of the governance.
- 30:56And that'll be I think an experiential process.
- 30:58We're going to keep experimenting till we get to a
- 31:00point where we get more confidence and you know, it's
- 31:03very much going to be use case dependent.
- 31:05We have a really interesting question from Arsalan Khan on
- 31:10Twitter. Are you aware of any libraries
- 31:14of AI agents that are accessible to all organizations?
- 31:19So organizations, all organizations have to do is
- 31:23provide the data. And could industries work
- 31:26together to create this? Personally, I'm not aware of a
- 31:31Public Library of AI agents per SE, other than, you know, those
- 31:35that might be, you know, available.
- 31:39I guess you, you could kind of, there's probably a library of
- 31:42different open claw agents that, you know, folks have built and
- 31:46published that you might be able to access to.
- 31:49But again, that's one of those things where you, you really
- 31:52want to be very careful about, you know, deploying a third
- 31:57party agent into your into your environment unless you you truly
- 32:01have confidence. So, but what we are saying
- 32:03obviously is within an enterprise, the the notion of,
- 32:08you know, building agents and publishing those agents and
- 32:11making sure that the teams across an enterprise have access
- 32:14to the agencies is clearly the way, you know, enterprise are
- 32:20encouraging the transition to energetic architecture, right?
- 32:25You know, I think the broader question of, you know, are there
- 32:28ways you can collaborate? And I think you're starting to
- 32:31see this in particular areas such as security, for example,
- 32:35you know, you can think of the Mitos release, whenever it's
- 32:40going to be publicly released, could be a a very powerful tool
- 32:44for the community to be able to sort of figure out how to
- 32:48protect themselves. Of course, it's a double edged
- 32:50sword. And that's, I think the
- 32:51conundrum that we're we're dealing with, with this really
- 32:55capable model. You know, I think I just saw an
- 32:58article from Mozilla, the the latest version of Firefox, they
- 33:02found 271 bugs using Mitos, which is kind of amazing.
- 33:07I think only three of them are probably CV ES, like truly high
- 33:10severity. But still, you know, the surface
- 33:13area of the power of that tool is obvious.
- 33:15So that is an example, maybe an imperfect example because it's a
- 33:19double sword, but I do think that you, you will see some of
- 33:23these general purpose agents that that will be out there that
- 33:26help the broader community. But I think given the non
- 33:30determinism and and so many, you know, kind of open questions
- 33:35around agenting architectures that we just talked about as
- 33:38governance and guardrails and such, like it's, you know, you
- 33:42should be very careful about just taking third party agents
- 33:44and, and, and just using them, right.
- 33:46So there's we're still not at that level of maturity.
- 33:49We have an interesting question from Syeda Zeenath and she asks,
- 33:57do we need to ensure that organizations or do
- 34:00organizations need to ensure that they fix their processes
- 34:04that are not working properly before they integrate with
- 34:07agents or can to what extent can they work with existing
- 34:12processes? And I'm going to ask you to keep
- 34:14your answer pretty short, Praveen.
- 34:16I'll kind of take this as an example of the data
- 34:19architecture, right? So this is a typical conundrum
- 34:21because most data today in an enterprise is sort of pretty
- 34:25fragmented. Not everything has a nice clean
- 34:27little API that you can access it through.
- 34:30So the conundrum has always been, should I first do a big
- 34:34data transformation project before I get started with my
- 34:38agentic transition? And I think what we're seeing
- 34:40enterprises do is they can't afford to to wait, right?
- 34:43So, and you know, you have interesting tools now like
- 34:46computer use and browser use that actually enable agents to
- 34:51use your processes and your data and access your data like a
- 34:56human would, right? So it's sort of a, you know, I
- 34:59think we've found these tools so you don't have to wait.
- 35:02And I don't think you can afford to, to wait to kind of drive
- 35:05this huge transformation project before you go to AI.
- 35:08You know, I think you'll need to kind of do these things in
- 35:10parallel. So from that standpoint, it's
- 35:12somewhat different than historically implementing
- 35:16something like ERP or CRM where you really did have to clean up
- 35:21the data and or clean up clean up your processes it it's what
- 35:25made sense. When you say processes, you
- 35:27know, obviously the way you know, you have to rethink human
- 35:32centric versus agent centric policies, right?
- 35:34These are completely different sort of workflows.
- 35:37So there is a level of sort of architectural design of
- 35:41processes that does need to happen, right.
- 35:44And so you know, you start off with a few workflows and you you
- 35:48assume that there's an agent in the loop and you know, design
- 35:51your process for that and you learn and you go along.
- 35:54So, you know, it, it, the, the answer here is really sort of,
- 36:00it will be experimental, experimental, right?
- 36:03You're going to learn as you go along, right?
- 36:05But my broader point is that you, I don't think you can wait
- 36:09to kind of do a process transformation or a data
- 36:12transformation. I think you need to pick a
- 36:14surface area and start to work on that, right?
- 36:17And ideally, use the tools that that these AI models provide you
- 36:22in order for you to accelerate that process.
- 36:24It's a really, really good point because the technology is
- 36:28changing so much more rapidly than anything in the past.
- 36:32I mean, ERP systems didn't change from one week to the
- 36:38next. They were stable.
- 36:40And so you had it was not a moving target in the way that
- 36:44the current AI technology is. It's almost feels like we're
- 36:48building on quicksand right now, right?
- 36:50Because just given the rate at which model capability is
- 36:55advancing or you know, new capabilities are being unlocked,
- 37:00you you need to be able to be more dynamic.
- 37:03And that's part of I think the challenge.
- 37:06I think this is called the Red Queen effect, right?
- 37:08Which is effectively every enterprise, every knowledge
- 37:12worker, every professional today, we all feel the same
- 37:15sense of urgency that if we don't change, we don't evolve.
- 37:19You know, we're not going to keep up.
- 37:20We're feel about getting ahead, right?
- 37:22Just to kind of keep up, you have to, you have to keep, keep
- 37:26changing, keep evolving. And that's that's the state of
- 37:28things where we are. Another question, again from
- 37:33LinkedIn, and this is from Yaw N, who says you've watched
- 37:38dozens of agentic AI companies hire.
- 37:41What's a skill that mattered a lot in new grad hiring two years
- 37:45ago that you now see hiring managers have stopped caring
- 37:50about? If you go back two years, they
- 37:53were clearly defined roles, right?
- 37:55You had software engineer essentially proficiency on code
- 38:00and all these like elaborate coding exams that you have to
- 38:03get through, you know, designing walk a tree, you know, these
- 38:07these kind of things that that were kind of bars for you to get
- 38:11through that They were product manager roles.
- 38:14They were, you know, they were sort of field roles in a in well
- 38:18defined sort of, you know, architecture of what an
- 38:21organization looks like today. If you actually look at a lot of
- 38:25startups, all these roles are mashed together, right.
- 38:29There is no clear product manager.
- 38:31A software engineer today is on site with the customer, actually
- 38:35working with them, deploying code, learning from them,
- 38:39developing almost, you know, be the customer in the morning,
- 38:42develop code at night, you know, go back to the customer, right.
- 38:45The iteration loop has gotten much faster and it's broken down
- 38:49the silos. So, you know, if you are a new
- 38:51grad coming out, essentially that is what you're expected to
- 38:55do. You're not, you know, you're not
- 38:56going to be given a project and say, OK, here's the project.
- 38:59OK, learn the code base and then, you know, produce
- 39:02something or fix a few bugs before you write code.
- 39:04You're essentially going to be dropped into a customer side and
- 39:06say, OK, let's let's go build a product and iterate fast.
- 39:09And I think that means that you have to be sort of a a software
- 39:14engineer, a product manager, a forward deployed engineer, all
- 39:17of this stuff rolled into one. I'm talking about a startup
- 39:20context. Of course, you know, if you're a
- 39:21large organization, some of these roles might be further,
- 39:24more well defined, given sort of the complexity of what that that
- 39:29particular company might deliver.
- 39:31But I think that's what we're seeing, you know, these these
- 39:33roles in, in startups are, are hybrid, right?
- 39:36And close to the customer and close to the code is kind of
- 39:40what what differentiates, you know, engineers now.
- 39:44I had conversations during the past week with CIOs from 2 very,
- 39:49very large insurance companies, you know, with thousands or 10s
- 39:53of thousands of employees, separate conversations and one
- 39:57of the things that became clear is this melding of roles, the.
- 40:03The dropping of boundaries between product design, software
- 40:09development, DevOps, these are all merging together now.
- 40:13These particular folks are highly, highly advanced and
- 40:16highly innovative. But it's a glimpse into the
- 40:19future where Praveen, what you just described, will take place
- 40:24inside very large companies as well.
- 40:26It's not just the startups, at least going into the future.
- 40:29If you think about what we saw with Claude Co work, right, you
- 40:34know, cloud Co work essentially was a realization in anthropic
- 40:38as well as other labs as well that, hey, this coding harness
- 40:42that we built to write code can actually do work, right, can
- 40:45actually do non coding work as well.
- 40:50And So what that means is that if you're a business analyst or
- 40:54a, a financial analyst or, you know, support engineer,
- 40:57whatever, you now have a very powerful model that you can use
- 41:03to define your workflow and create your own sort of, you
- 41:07know, application agent, if you will, to, to get your work done.
- 41:11And, and that's sort of the democratization of that, that AI
- 41:15is enabled, right? In the past, you probably needed
- 41:17to go to an IT, you know, team and say, like, Hey, here's what
- 41:22I need to get built. And there's a certain amount of
- 41:24time. So now if you're a business
- 41:25analyst, you just say like, OK, I just need to like whatever,
- 41:28you know, get all this information, synthesize this,
- 41:30put this into a particular spreadsheet, run this particular
- 41:32analysis, and I could create that entire workflow myself.
- 41:36And the AI would be writing code in order for me to do that.
- 41:39So that is actually the true power of of agents.
- 41:43It's democratized coding. It's Democrats, the ability for
- 41:47knowledge workers, whether you are, you know, an engineer or an
- 41:51analyst, to be able to get your work done.
- 41:54Let's talk about the SAS apocalypse.
- 41:57The SAS apocalypse. Do you believe that AI agents
- 42:03are going to replace software like Salesforce, Like Work Day?
- 42:09Where do you fall down on the SAS apocalypse argument?
- 42:13We talked about this a little bit in in the previous question
- 42:16as well, where we were sort of talking about how you would take
- 42:19existing software and be able to transform it, right, in a
- 42:23thoughtful way, right? Essentially looking at where can
- 42:27AI accelerate work that's that that is being done and
- 42:32accelerate value to the customer, right.
- 42:33So I gave the example of invoice reconciliation in an, in an
- 42:37accounts payable or procure to pay platform like Stanley.
- 42:41You know, I think look, system of records are are not going
- 42:44anywhere in my opinion, right? And because fundamentally, you
- 42:48know, the one question we haven't talked about is there's
- 42:51a real cost to agentic software, right?
- 42:54And this is basically there's a variable cost to software.
- 42:58So every time you are writing code or using a tool or doing
- 43:03reasoning, you're burning tokens, right?
- 43:05And sure, I mean, the token costs have gone down
- 43:08dramatically, you know, over the years as these model
- 43:12capabilities have advanced. But you know, it is still
- 43:16fundamental cost that you have to now, you have to now, you
- 43:21know, think about when you start initiating a project.
- 43:24So does it make sense for an enterprise to say, you know
- 43:27what, I'm just going to bottoms up, build a brand new CRM or
- 43:31bottoms up build a new ERP, right?
- 43:34You know, is that worth the amount of tokens that you would
- 43:37spend to go do that versus, you know, where is the real value
- 43:41for you as an enterprise, right? You know, which is basically
- 43:45accelerating your business, right?
- 43:47Your use case, you know, if you're whatever insurance
- 43:49provider, you know, bank, you know, whatever manufacturer,
- 43:53right? That's kind of where you want to
- 43:55spend your tokens as opposed to going down to the infrastructure
- 43:58level and saying like, well, let me just go reinvent the wheel
- 44:01just because I can, right? Can you build it?
- 44:04Of course you can, right? And, you know, it's interesting
- 44:06because in a previous cycle, you know, companies like Tesla
- 44:09essentially rebuilt their ER PS because they felt like the
- 44:13existing ER PS did not fit the kind of model that, you know,
- 44:18Tesla wanted. I mean, the fully vertically
- 44:20integrated car production systems.
- 44:23And so that makes sense. I mean, there might be certain
- 44:24use cases where it logically makes sense, but I, I just, I
- 44:29don't buy the fact that you're, you're going to go and reinvent
- 44:31the wheel here and try to take some of these really complex,
- 44:35must be super reliable kind of infrastructure pieces and
- 44:39software and, and redo them. Oh, I totally agree.
- 44:42I mean, you know, I personally, I think it would be insane.
- 44:44Who's going to try to rebuild their Oracle or SAP or work day
- 44:50systems now? There's going to be nibbling at
- 44:53the edges for sure. But we have a question relating
- 44:57to specifically this from Elizabeth Shaw on Twitter who
- 45:02says how is agentic AI pressuring existing enterprise
- 45:07software pricing and business models, which is which is really
- 45:12the the question. We clearly are seeing budget
- 45:16shifting in, in enterprises right from traditional SAS, you
- 45:21know, proceed pricing to, to more sort of agentic tools.
- 45:27And and a big part of that is in some ways just sort of a clean
- 45:29up if you will. There's a lot of sort of excess,
- 45:34you know, seats or excess sort of software buys that have
- 45:36happened, which are being kind of reconciled now based on usage
- 45:40and that budgets being redeployed for AI experiments,
- 45:43right? The one thing I'll say, and it's
- 45:46kind of important, right? A lot of this is we're observing
- 45:49this in real time. None of this is none of what I'm
- 45:52sharing or, you know, the conversation we're having is
- 45:56absolute truth, right? It's kind of a point in time.
- 45:59This is kind of where things are and and it's a big part of where
- 46:01we are as an industry right now, which is given the rate of
- 46:04evolution, we're trying to connect the dots and
- 46:08extrapolate. But we are still early in this
- 46:11journey towards sort of agentic architectures.
- 46:14So I believe that today where enterprises are, they are very
- 46:20aggressively leaning into this notion of an agentic
- 46:23transformation for the whole architecture.
- 46:25At the same time, I think what we are also seeing is that most
- 46:30enterprises are running experiments right around where
- 46:34does the an agentic solution make sense, how much is it going
- 46:40to cost us, what is the true ROI, right?
- 46:43How do we govern these things? So these are all questions that
- 46:46are being answered in real time. In certain use cases, obviously,
- 46:51you know, the, the, the answer is obvious, you know, customer
- 46:54support, obviously we're seeing huge amount of traction there
- 46:57writing, you know, marketing collateral.
- 47:00Those kind of use cases are clearly demonstrating a lot of
- 47:02value, lot of operations use cases where you can, you know,
- 47:06you have a ton of logs or a ton of data and you're trying to,
- 47:09you know, disown signal from noise.
- 47:12Those use cases kind of make sense.
- 47:14But I, I still kind of, we're still sort of in this early
- 47:18stage of adoption or agentic architectures and you know,
- 47:22still have a number of significant questions to be
- 47:24answered, right. So we're not quite, you know,
- 47:27across the line yet. How should CIOs make the build
- 47:31versus buy decision? Regarding agents, you know in
- 47:36talking to a lot of enterprises, what we see is sort of this the
- 47:40build versus buy is kind of breaking down around the front
- 47:42end versus back end kind of line.
- 47:45So a lot of like front end customer facing tools where the
- 47:49workflow is perhaps a little bit more standardized, right?
- 47:53You, you are, you know, enterprise are leaning towards a
- 47:57buy kind of decision. So it could be something like
- 47:59customer support. It could be something that's,
- 48:01you know, a standardized workflow.
- 48:03Like, you know, if you have a finance workflow, you know,
- 48:05you're, you're, you're running your, your finance statements
- 48:09kind of done the same way in most organizations in the back
- 48:12end, there's a lot more variability depending on your
- 48:14industry, depending on so the compliance and regulations that
- 48:18you have to, you have to abide by.
- 48:21And also, you know, there's a lot of data that you have.
- 48:23So bringing those things together, it's kind of where
- 48:26we're seeing the buy, you know, lean, you know, enterprises
- 48:29leaning towards towards the buy side of decision.
- 48:32So using frameworks, right, using these model capabilities
- 48:35to kind of build their own sort of customized agents.
- 48:38So at a high level, that's kind of where the the line is.
- 48:40But again, you know, we're early in this journey, so it's
- 48:43something that we need to keep, you know, following.
- 48:46Where can CIO safely remove humans from the loop, and where
- 48:52not? In other words, where does
- 48:54autonomous operation actually make sense?
- 48:59It really comes down to sort of the harness that you're able to
- 49:04you, you're able to build around this agent, right?
- 49:07And what I mean by that again is the ability to create, you know,
- 49:13a, a well defined reasoning and a well defined sort of
- 49:17decisioning process for the agent.
- 49:19And so the the context that you provide to the agent in terms of
- 49:23the data it has access to the guard rails, the tools it can
- 49:26use, the kind of the, I guess the parameters of how those
- 49:31tools can be used. I think defining that being able
- 49:36to have very granular observability and evaluation
- 49:40frameworks in order for you to make sure that as the agents
- 49:43progressing, you're able to understand, you know, and catch
- 49:47any, you know, drift in in it's performance and and being able
- 49:51to sort of have a well defined set of criteria of what good
- 49:56looks like, right? This sort of again, the bounded
- 50:00unbounded sort of spectrum, if you will, of workflows that give
- 50:04you confidence to allow the agent to be able to execute on
- 50:09it's own right. So the answer to that question
- 50:12again, is there's no, you know, there's this, there's the
- 50:16ability for the organization, for, you know, for the team to
- 50:20be able to define these criteria.
- 50:22I think will is what is what enables, you know, that dial of
- 50:27autonomy, if you will. I I think we are we're still at
- 50:34least what we see today is still sort of significant amount of
- 50:37human in the loop. And that's probably going to be
- 50:39the case till we get to an inflection point in, you know,
- 50:42the way in the accuracy of these harnesses.
- 50:45What practical advice do you have for CIOs on managing cost
- 50:51risk, unpredictability, The variability of inference costs,
- 50:59your token costs going out of control?
- 51:02What did? What can CI OS do?
- 51:04What should they do? Let me focus on kind of the cost
- 51:07aspect of things we've talked about to the governance and you
- 51:10know the the ability to deal with the unpredictability of
- 51:14large language models, right. I think as we said earlier, you
- 51:18have to, you know, software now has a variable cost and, and I
- 51:21think it's important to understand what that means.
- 51:25So you know, you've seen a lot of things in the press or, or
- 51:28you know, on, on, on acts where you say like, oh, I'm measuring
- 51:31my team by how many tokens they've used, right?
- 51:34And, and that sort of, you know, engineers are measured by what
- 51:37their token consumption is. The one thing we've clearly seen
- 51:40is in organizations and there's some data pointing.
- 51:43There was a Goldman Sachs study that came out that basically
- 51:46said that enterprises are blowing through their token
- 51:49budgets. A year's worth of AI budget in
- 51:52three months. You know, it, it's, you know,
- 51:55this is the notion of token maxing, which is as many tokens
- 51:59as you provide will get consumed as quickly as possible.
- 52:02So we're still in the stage where I think CI is going to
- 52:06take a step back and say, OK, I have X number of projects,
- 52:09right? Because we've unleashed the
- 52:11beast. You know, everybody now has
- 52:13access to build their own agents.
- 52:15How do you prioritize these, right?
- 52:17And you know, just old school sort of making sure that you're
- 52:20investing your tokens in the most highest priority projects.
- 52:23And I think that's, you know, what we've always done
- 52:26historically in terms of defining software projects is
- 52:29just because you're able to build your own agent doesn't
- 52:32mean that you should. Again, going back to the example
- 52:35of do you want to go build your own ERP or your CRM?
- 52:38Does is it worth it? So I think prioritization,
- 52:42clearly understanding set of token costs, because now you
- 52:44have multiple agents. It's not just token costs.
- 52:47By the way, you might be using APIs to go access different
- 52:49tools. So you know, those tools may
- 52:51have their own particular cost. So for an agent to actually
- 52:54execute, you have to now break breakdown the cost of the the
- 52:58platform, the the tokens, the tool uses, all of these things
- 53:02going to compose it to figure out and then weigh that against
- 53:05your ROI criteria. Are you actually getting better
- 53:08productivity? How do you measure productivity
- 53:11in granular terms? Is it calls deflected if you're
- 53:13in a customer support area? Or is it the speed to
- 53:17reconciling accounts payable in the previous example?
- 53:21So creating those kind of business level metrics, matching
- 53:26them to the cost of what it takes for an agent to execute, I
- 53:30think is important. Now, I do think that models are
- 53:33going to get better, more efficient from a token use
- 53:36perspective. And you already see that
- 53:37happening, you know, all all the way from like Opus 4.5 to the
- 53:414.7 or even the GPT 5.5. They are getting much, much more
- 53:45efficient in terms of token use. And I think that's important,
- 53:49but also, you know, being creative in the way you use
- 53:52memory and being able to sort of use smart ways to kind of ensure
- 53:57that you're putting the right context in.
- 53:59These are the kind of things that are design principles that
- 54:01help you optimize token cost, but you kind of weighing the
- 54:04token cost versus the ROII think is a critical kind of lens in
- 54:07addition to the other things we talked about.
- 54:09Bridget Ejegy on LinkedIn says how can selling companies charge
- 54:14for their AI solutions just in a sentence.
- 54:18There's a spectrum of do you go to consumption based pricing?
- 54:22You know, I think we've actually done some work at Insight, our
- 54:25pricing team. And I think one way we're
- 54:28thinking about this is perhaps you think of this as a
- 54:30fractional FTE, right or a fractional headcount.
- 54:34Because I think we are going to this notion of a hybrid human
- 54:38agent workforce where, you know, each of us as, as employees will
- 54:42have our own agent. And maybe the way to think about
- 54:45this even from a pricing perspective is to price it as a
- 54:49partial FTE, particularly if it's doing specific work, right.
- 54:54But this is a point of experimentation.
- 54:56Again, this is a longer answer, but you said 10 seconds, so let
- 55:00me just leave it at that. All right, Praveen Akiraju,
- 55:03Managing Director of Insight Partners, thank you so much for
- 55:08being a guest today, Praveen, on CXO Talk.
- 55:11I'm very grateful to you once again, and thank you to
- 55:16everybody who watched. Before you go, subscribe to the
- 55:20CXO Talk newsletter, go to cxotalk.com and we have great
- 55:25shows coming up. Our next show is with the Chief
- 55:28Technology Officer of Mozilla, so check it out.
- 55:31Thanks everybody. Have a great day.