Latest / In The Loop / How Enterprise CIOs Build & Buy AI In 2025 (a16z Report)

Transcript

How Enterprise CIOs Build & Buy AI In 2025 (a16z Report) · In The Loop

0:00
Today I want to deep dive into the amazing piece of research
0:03
but the very famous VC firm A16Z created by the legendary
0:08
entrepreneur Andreas and Horovitz.
0:11
I think this piece of research gives us potentially the best
0:13
insight into how things have changed when it comes to
0:16
enterprise. So those with the big money
0:18
pockets buying AI products today, specifically in today's
0:22
episode, I'm going to talk about three things that I think I
0:24
learned from reading their paper.
0:26
The research was based on interviews and surveys with over
0:28
100 different CIOs, so chief information officers across 5
0:33
different industries. Now, these are the people that
0:35
buy big AI products for their companies.
0:39
And what it shows us is how things have really changed when
0:42
it comes to AI purchasing, as you'll see in today's episode.
0:46
One thing for certain, scale and speed of this adoption within an
0:49
enterprise. So if you're building or selling
0:51
AI, if you're working at a big company or just trying to figure
0:54
out how the world of AI is changing, you're going to want
0:58
to listen to the research findings in today's episode.
1:01
This is in the leap with Jackal. I hope you enjoy the show.
1:15
So I think we'll start with something that I find
1:18
particularly interesting and really resonated with me.
1:20
So two years ago, if you talked to any big, big companies about
1:24
AI, typically the conversation was about how they can build
1:27
these AI solutions internally. So they all have IT teams with
1:31
developers and a lot of the leaders were thinking that we
1:35
need to build our own AI capabilities or we can't rely on
1:38
third parties for something that's strategic or the desire
1:42
to maintain control over, I guess the AI destiny of that
1:45
company. And these are the sort of things
1:47
you'd hear a lot. The conversation has completely
1:49
changed now, and I mean completely.
1:52
The research found that 90% of enterprise are now testing third
1:56
party applications for things that customer support.
1:58
And what's really funny is that customer support was one of the
2:01
areas where companies started to build an internal solution
2:04
because obviously it seems straightforward.
2:06
The use case was clear, the ROI was really obvious.
2:08
Yet 9 out of 10 companies now are actively looking to buy
2:13
rather than build. And it makes sense.
2:14
Previously there wasn't solutions that were ready made
2:17
for people to buy, and instead people were piercing together
2:20
different pieces of technology and infrastructure to create a
2:23
solution. And companies like Mindset have
2:26
provided basically all of that infrastructure inside an AI
2:29
development platform that's all integrated and works really
2:32
easily to make it really simple for companies to build new
2:35
features and things on top of it.
2:37
But again, a couple of years ago, this just didn't exist.
2:40
People had to innovate, had to really figure out which parts of
2:43
the technology they should pick and how to glue it together and
2:46
what to build on top of it. For example, the research said
2:49
that in financial services, over 75% of customer support
2:53
implementations are now just using third party applications,
2:56
even in industries like manufacturing.
2:58
So traditionally quite a conservative industry because of
3:02
compliance has now gone to about 60% of them buying AI
3:05
applications. And in the research, it actually
3:07
cited a financial technology company discussing this build
3:10
versus buy dilemma. And now this financial
3:12
technology companies had a dedicated team.
3:15
Apparently they were making really good progress and they
3:17
said they were about 60% done. But when it came to new
3:21
solutions popping up, they did a competitive analysis and decide
3:23
we're going to build, we're now going to buy this and not build
3:26
it. And basically because the AI
3:28
native applications that they were buying are just better.
3:31
And this is evident in some AI native applications in the world
3:35
right now because, you know, some CTOS of big companies are
3:38
saying that 90% of their code is AI generated from platforms like
3:43
Cursor or Clod. And 90% is just a crazy number.
3:46
You know, just 12 months ago, that was about 10 to 15% of code
3:51
all generated by AI. And the fact that it's now gone
3:54
from one in 10 lines of code to 9 in 10 lines of code in just 12
3:57
months is kind of madness. It blows my mind a little bit.
4:00
And now this is all data that's in that research report.
4:03
Now you may ask then, OK, so people are now buying versus
4:07
building. But I guess what, what is
4:09
driving this? So when researchers ask these
4:11
big enterprise CACIOS why they preferred AI native applications
4:16
and they said it was just they were faster to innovate.
4:19
And also the fact that they're building this from the ground up
4:21
around AI makes just a fundamentally better experience.
4:25
It's like the early days maybe of cloud computing, where
4:28
initially every enterprise was trying to build their own cloud
4:31
infrastructure. And then obviously AWS and many
4:33
other big providers popped along and they just did it so much
4:37
better. And it obviously became
4:39
ridiculous to build your own. And I think we're seeing the
4:41
same pattern play out with AI applications at the enterprise
4:44
level. So if you're a technology
4:47
vendor, so you're a SAS provider right now, I think you need to
4:50
move really, really fast and move like your life depends on
4:54
it. Because clearly you've got an
4:56
opportunity to grab whether it's market share, do really
5:00
incredible things because enterprise now want to buy best
5:03
in class solutions. And the data on user
5:05
satisfaction in the report backs all of this up.
5:08
The research found that AI native solutions averaging an
5:10
MPs or a Net Promoter score from their customers.
5:13
So when customers vote on whether they like something or
5:15
not are at 40 to 50 points higher than a traditional SAS
5:19
company. That's a huge number.
5:21
Now, I promise this isn't just a happy coincidence.
5:23
But you know, our proposition to customers who are SAS companies
5:26
and technology providers is essentially we're a development
5:29
platform that helps them become an AI native technology
5:32
provider. And we're seeing all of these
5:34
same trends play out in real time because essentially we
5:37
allow product management and dev teams to build customer facing
5:41
AI features on our platform and then plug it into their product.
5:44
So they become magical and AI driven.
5:46
And what we're seeing at the moment is a huge number of
5:49
people, I guess retrofit chat bots into their product.
5:53
But an AI native company, this AI native technology provider,
5:57
everything's controlled by AI. Everything's just controlled by
6:01
natural language. You just, you tell the
6:03
application what to do and then it does things for you.
6:06
And I think over time, we'll start to see all interfaces melt
6:09
away because really people just want to have a conversation and
6:12
something happens. And then you'd given the results
6:14
of that. And I think that's where all
6:15
technology providers are going to be moving to over the next
6:18
probably. I mean, I'm always ambitious and
6:20
probably overly optimistic. I want to say 18 months.
6:23
In reality, this is a journey that's going to go on for many
6:25
years. But yeah, I think the research
6:27
was just spot on here. So again, that build versus buy
6:31
dilemma is ending an enterprise buying cycle and it's being
6:34
driven by IAI native platforms and products just being so much
6:38
better. Now let's move to another key
6:52
thread. I think that was quite apparent
6:54
in the research, which is, I guess, just this huge budget
6:57
explosion. So when it comes to new
6:58
technologies, typically you see gradual adoption.
7:01
So you see pilot programs like we've seen then people start
7:05
small and then a slow budget growth as technology becomes
7:09
really useful and helpful and accepted everywhere.
7:11
And it was only last year that a lot of Wall Street, for example,
7:15
was asking, you know, will AI ever be helpful?
7:18
Will it ever be accepted and bought by enterprise on a large
7:21
scale outside of pilots? And so far, again, more
7:25
predictions have been shown to be wildly false because AI has
7:29
broken all of the rules. The research found that
7:32
enterprise AI budgets have now grown 75% beyond what companies
7:36
had actually projected spending. So they're spending 75% more
7:40
than they thought they would be spending.
7:41
And these were already quite aggressive projections.
7:46
And what's even crazier about is this is the enterprise leaders
7:49
are now predicting another 75% growth over the next year.
7:53
To put that into context, typical enterprise software
7:55
budgets increase maybe at 10 to 15% every year.
7:58
So it's just a massive amount of spend and growth here in the AI
8:02
sector. And the numbers aren't just part
8:04
of this kind of insane story, because what's really
8:07
interesting is where the money's coming from, because last year
8:11
all the money was coming from innovation budgets, which are
8:13
basically R&D. And essentially what that means
8:16
is experiments. And that made-up about 25% of
8:18
total AI spending. Yet this year, in 2025, that's
8:22
dropped to 7%. So that dropped to 7% means that
8:25
AI is, I guess, graduated from this experimental technology to
8:31
now something that's obviously core to the business strategy
8:34
because companies aren't funding this from R&D budgets, they're
8:37
funding it out of their core IT budgets.
8:40
You know, 77% from the research or according to the research
8:43
comes from IT budgets. Now 16% from a business unit and
8:47
only 7% from that innovation fund.
8:49
Now, this shift I think has just huge implications that most
8:52
people probably don't fully appreciate yet.
8:54
When you're spending innovation budgets, you can just afford to
8:57
experiment. You can pilot things, you can
8:58
accept fairly, you can try crazy ideas.
9:01
But when you're spending caller IT budgets, you need
9:03
reliability, approving and guaranteed return on investment.
9:07
You need vendors who can deliver an enterprise scale.
9:10
And this is exactly why I think the budget shift is reversing
9:14
that build versus buy. When you have an innovation
9:16
budget, you could justify building things internally just
9:20
because learning is part of the value.
9:21
But when you have that core IT budget, you need solutions that
9:25
work and you need them today. So in terms of the reasoning
9:28
behind this, I guess budget explosion, the research
9:31
highlighted 2 core things. First, it says enterprises are
9:35
discovering loads of new use cases that they never really
9:38
expected. Things like internal enterprise
9:41
search, data analysis, processing documents, and
9:45
they're seeing rates of between 60 and 80% adoption across a
9:48
company, which is a huge number. I mean, if anyone's ever been
9:51
given a new HR software, how many people ever log into those
9:55
things? Second, which is really
9:56
interesting, which is customer facing use cases and now
9:59
starting to emerge at companies at scale.
10:01
So one of the large technology providers that were interviewed
10:05
as part of the research said that they've been mostly focused
10:08
on internal use cases so far, but now what they're going to be
10:11
doing is shifting budget and effort into customer facing AI
10:14
solutions. I promise this isn't a terrible
10:17
sponsored ad. It is just a happy coincidence
10:19
because this is exactly what mindset is seeing.
10:21
This is what we do. We allow technology companies to
10:24
build customer facing AI into their products.
10:27
And we're seeing this at scale, you know, agentic experiences
10:31
and AI automations happening inside technology providers
10:35
platforms is a quite a magical experience and something that's
10:39
really difficult to achieve. You know, one of our customers
10:41
actually had 20 pages of customer facing use cases that
10:44
they were launching. And what's really funny here is
10:46
internal use cases has limits. You know, an internal tool will
10:50
always have that natural limit. You can only automate so many
10:53
internal processes. But when it comes to customer
10:55
facing applications, well, there's just almost unlimited
11:00
numbers of them. So yeah, obviously this trend is
11:03
particularly good for a mindset as well as many LLM providers,
11:06
you know, which obviously it make a lot of money based on
11:08
anyone ever using any form of their application.
11:24
Anyway, let's move on to the third and final key trend or
11:28
theme that I think came from this research, which is
11:31
absolutely how wrong most of us were when it came to large
11:34
language model providers. And this is where I personally
11:38
was wrong. I predicted something that and
11:40
now I have egg on my face. So conventional wisdom says that
11:44
as a technology matures, they consolidate so enterprise will
11:48
pick a winner. Big companies buy the smaller
11:50
ones, everyone standardizes on typical platforms, and they
11:55
drive down costs through volume. Now the AI market is actually,
11:59
funny enough doing the exact opposite of that.
12:01
And last year it felt like the large language model space was
12:04
following the same pattern. They seem to be converging and
12:08
becoming more and more similar and hard to tell, you know,
12:11
which was better or why would I use this one over that one.
12:14
And everyone thinks that the benchmarks are still silly.
12:18
And really, we were assuming that that space was becoming
12:20
incredibly commoditized, so it was only ever going to become
12:23
cheaper and cheaper, which it is.
12:25
But there's no differentiation on models anymore.
12:29
Well, we were all wrong because now 37% of enterprises are now
12:33
using 5 or more models in production.
12:36
So not in testing, not in pilots, in full production.
12:40
So that's up by 29% in the last 12 months alone.
12:42
And when you look at experimentation, so companies
12:46
experiment of different models, that number jumps to 52% and now
12:51
testing over 5 models. Now maybe at first branch you
12:53
might think, obviously that's just because no company wants to
12:56
be locked into using a single large language model because
12:59
enterprises have obviously learned that putting all your
13:01
eggs in one basket and that basket being a single technology
13:04
provider is always quite dangerous.
13:06
But the research showed that this isn't the case.
13:09
The main driver is actually otimizing for erformance based
13:13
on specific use cases. Now, for those that listen to
13:16
this podcast, every single week, I've covered how certain large
13:19
language models have really specialized in certain use
13:22
cases, and they've done incredibly well for it.
13:25
And what I think this research shows people is that most of us
13:27
have been missing a key fact, which is that the enterprise
13:30
large language model provider layer has not been commoditized.
13:34
Instead, it's actually specialized, which is again, the
13:36
exact opposite of what everyone thought would happen just 12
13:39
months ago. So some examples from the
13:41
research where the Anthropics models excelled at coding tasks.
13:44
We'll discuss that on the podcast before.
13:46
But what's really interesting is actually nuance and
13:48
specialization even within coding tasks.
13:51
So Claude is seen to perform better on very detailed fine
13:55
grained coding tasks, while Google Gemini is actually
13:57
supposed to be better at high level system design and
14:01
architecture conversations. So it's really interesting how
14:04
even within a use case, there is sub use cases that models are
14:07
now specializing towards. However, what is very clear from
14:10
the research is that when it comes to dominance, Open AI has
14:13
dominated everything. They have overall market share
14:16
leadership. 67% of Open AI users are deployed non frontier models
14:20
in production compared to just 41% for Google and only 27% of
14:24
Anthropic. So it just shows that even
14:27
though many of us criticize I guess the number of language
14:30
models that Open AI have and gosh, how do I choose between
14:33
this one or that one? Well, it's worked very well for
14:37
them, but I don't know how long that will last.
14:40
Google is rising really quickly. And according to the research,
14:43
especially with big companies with minimum of 10,000 employees
14:47
and I guess part of that is they've got a lot of
14:49
relationships with big enterprise, lots of brand trust.
14:53
And also most importantly is cost.
14:56
For example, Gemini 2.5 cost $0.26 per 1,000,000 tokens,
15:01
while GPT for one costs $0.70. So this means that Google is
15:05
nearly three times cheaper. So price could become a major
15:08
factor for these big companies when deciding which models they
15:11
choose. But the challenge for these big
15:14
companies is that switching costs become higher.
15:17
So we've talked a lot on this podcast about what an agent
15:20
actually is, which is just a set of configuration, which
15:23
essentially is a way of organizing different prompts to
15:27
an LLM on how it should behave in certain circumstances.
15:30
And 2nd, it's a set of integrations and ability to
15:34
orchestrate those different tools.
15:36
But here's where things get quite complex because as big
15:39
enterprises are now building more and more sophisticated
15:42
agentic workflow. So we talked about this, the
15:45
ability to tell the agent to do this, then that, and then that
15:48
actually makes it very difficult to switch between models.
15:52
So the research shows that 68% of enterprise now find it either
15:56
difficult or very difficult to switch models, which is up from
15:59
43% last year. And a good example of this is
16:01
one of the CIO quotes they referenced, which is that the
16:04
CIO said all of our prompts and our systems have been worked and
16:07
built around open AI. Each one has a specific set of
16:11
instructions and details. And so it's a very big task to
16:14
now swap that out. So this creates quite a
16:17
fascinating dynamic which this multi model approach gives
16:20
enterprises lots of flexibility and ability to optimize based on
16:24
performance in the short term. But as these individual
16:26
workflows become more complex and sophisticated, switching
16:31
becomes more and more expensive. So if I'm a large language model
16:35
provider, essentially lots of different models and a strong
16:40
performance and specific tasks is going to do really well long
16:43
term. So, yeah, really, really
16:55
interesting themes. But if I take a step back about
16:58
what I think that means for I guess business leaders,
17:01
technology leaders, people in this space, I think first and
17:05
foremost for those that are building technology, it's the
17:09
best time in the world to be building technology because
17:12
actually there's build versus buy shift signals that the
17:14
application layer is maturing so fast.
17:18
We're moving to a world where those are the best products are
17:20
starting to win. And this is following the exact
17:24
same pattern as you saw with cloud computing or mobile app
17:28
development and pretty much every single major technology
17:30
platform. But the speed is just something
17:32
that's never been seen before. You know, these transitions
17:34
typically take 567 years, but this is happening in 18 months.
17:39
And 2nd, this budget explosion really tells us this isn't a
17:43
bubble or a fad. You know, where where
17:45
enterprises are actually spending their core budgets on
17:48
AI applications. It shows that if you are in this
17:50
space you have an opportunity to really take massive leaps
17:54
forward, but the challenge is you need to really be a
17:57
genuinely best in class product to win.
17:59
So yeah, if you are in the space, I hope this research acts
18:03
as another kick up the bum to move faster.
18:06
Anyway, thanks for today. I hope you found that helpful
18:09
and I will see you next week.