Latest / CXOTalk / What Big Changes Are Coming to AI in 2025? | CXOTalk #880
Transcript
- 0:00This is amazing a discussion inside the AI arms race,
- 0:06generative AI large language models, business strategy, and
- 0:12the competitive landscape of AI tools.
- 0:14Today on episode 880 of CXO Talk, our guest is the
- 0:20incredible Nate B Jones, who is not only an astute market
- 0:24observer but also a legit TikTok star.
- 0:28Nate, tell us about the AI arms race.
- 0:32It's not predetermined that it that it is an arms race.
- 0:35This is actually one of the surprising outcomes of the last
- 0:39year or two then. Even Sam Altman himself has
- 0:41admitted this when he got started building what is now
- 0:46ChatGPT at Open AI. It was just a tiny research lab
- 0:50and the running assumption for a while, even after ChatGPT
- 0:54launched is that they were going to have enough of a model lead
- 0:59that there wouldn't really be an arms race in that sense.
- 1:02That is not true. And So what we are seeing is
- 1:05that this is a technology that proliferate that's easy to
- 1:08share. We see that from DeepSeek to
- 1:11Meta to Google to Amazon trying to get in the race.
- 1:15And it's remarkable to me how quickly this technology has
- 1:22enabled people to catch up who previously would not have been
- 1:26in the game at all. It's not an arms race.
- 1:28So what is it? What are we, what are we talking
- 1:30about here? Just like a bunch of friends
- 1:31getting together and, you know, spending.
- 1:33No, I. Think it is an arms race now,
- 1:35but what I'm saying is if you looked at it in 2020-2021, 2022,
- 1:38you wouldn't have thought that it was headed that way
- 1:40necessarily. And so we find ourselves in an
- 1:43arms race. And that's really important
- 1:44because if you expect to go into an arms race the way the United
- 1:48States expected to go into an arms race with the Soviet Union
- 1:51after World War Two, you prepare.
- 1:54But in this world we are finding ourselves in an arms race and
- 1:58all of the players are playing it by ear.
- 2:01And I think that you have a different set of behaviors when
- 2:05you are making it up as you go, and you see that to some extent
- 2:08in the way some of the major players are taking actions.
- 2:11As an example, I think that you see Mark Zuckerberg evolving his
- 2:16strategy for Meta somewhat in real time as he sees the results
- 2:20of various training runs coming out of Meta, as he sees moves
- 2:24from other players in the space. And I think that is why you see
- 2:28him start to emphasize things like AI as a companion more in
- 2:33the last few weeks versus AI as a senior software development
- 2:37engineer, which was more of his emphasis earlier in the year
- 2:41when frankly, I think he had more optimism about the results
- 2:46of the Llama 4 training run. And so these these models are
- 2:50very difficult to train. They are not deterministic
- 2:53models, they're emergent models. And that means that any given
- 2:56training run, you might have spent a lot of money and you
- 3:00didn't get what you wanted out of it.
- 3:01And so when I think about it that way, these the the way the
- 3:06news shapes becomes much more interpretable to me, Michael.
- 3:09How do we reconcile, on the one hand, the billions of dollars
- 3:15that are being spent against AI being a companion?
- 3:23I mean, it's so, so bizarre to even think about it in this way.
- 3:29And where does all of this plug into the the overall landscape
- 3:34of these major LLM vendors? It's really important to
- 3:38differentiate between the application we see for AI with
- 3:42consumers and the base layer technology that these model
- 3:45makers are engaged in building. So I would say, and I think most
- 3:49folks in the space would agree with me, major model makers are
- 3:54not building breakthrough technology in AI in LLM
- 3:58specifically in order to build companions for people.
- 4:03But that might be a profitable use case in some places.
- 4:08And so I think when I look at the space, I start by saying
- 4:11what are the incentives for each of the players around the base
- 4:14intelligence they're developing and then that shapes the
- 4:17direction they want to pull the use cases.
- 4:20And so in Meta's case, if you look at the two use cases that
- 4:25I've mentioned so far, Mark talking about senior SD ES and
- 4:29Mark talking about companions, companions fits right into the
- 4:33milieu that Facebook is developing, right?
- 4:35You want to have a social feed, You want to have a social space.
- 4:39He was talking about the fact that Americans have three
- 4:41friends, but he thinks they have appetite for 15.
- 4:43The difference is 12. He wants to close that with AI.
- 4:46That is a very meta way to play the game.
- 4:50But if you go over to Open AI, they have a different set of
- 4:56incentives on the board. Their goal is to develop
- 4:59baseline intelligence. It always has been.
- 5:02They have accidentally, and I say that very intentionally,
- 5:05they accidentally developed an incredibly powerful consumer
- 5:08application when they launched ChatGPT.
- 5:11They've chosen to double down on that good fortune that they had,
- 5:14and they're very intentionally building it now.
- 5:17And so they're in the position where they are building a
- 5:20application for consumers that they want to be almost an
- 5:24everything app and at the same time building baseline
- 5:27intelligence that they think will be much more useful to
- 5:31major corporations and to very, very deep science and tech.
- 5:36And so I suspect we're going to see a bifurcation in the arms
- 5:39race where you have players who are going to be saying our best
- 5:43models are for drug breakthroughs, Our best models
- 5:47are for the hardest problems in supply chain management or in
- 5:50automotive design or whatever the hard problem might be.
- 5:54But we have excellent models that we provide to the everyday
- 5:59population in an attractive app format at low cost because we
- 6:03find that we can effectively monetize at that scale.
- 6:05So the days of the broad general purpose expert at everything
- 6:13model you feel are coming to a close.
- 6:16Intelligence is scaling past task saturation very rapidly.
- 6:21And so you're correct that even now Open AI has variant models
- 6:26that they've forked for education, they have variant
- 6:28models they've forked for drug development that are not
- 6:31accessible in your app or mine. But I don't really feel the
- 6:36difference because the intelligence that is coming to
- 6:39the app is more than enough for most of the tasks I bring to it.
- 6:43And that's what I mean by task saturation.
- 6:44If you look at intelligence as an exponential curve, we're
- 6:47saturating out on most of the tasks most of us do much, much
- 6:51sooner than we hit the intelligence ceiling.
- 6:54I I know someone who told me, look I am aware O3 is supposed
- 6:58to be better. I'm a smart guy.
- 7:01I don't need the difference and I don't feel the difference
- 7:03versus ChatGPT 4 O. It doesn't matter for the tasks
- 7:06that I do. So this specialization is
- 7:10heavily underway at this time already.
- 7:13It is in fact, I would expect us to see the first drugs that were
- 7:18designed in part or in whole by AI within the next 24 months.
- 7:23Folks have been using various forms of AI for a long time on
- 7:29this show. We've had the CEO several times
- 7:32of in silicone medicine, which is one of The Pioneers of using
- 7:38adversarial networks to design drugs.
- 7:43But what? Where does this leave the large
- 7:47vendors as well as the smaller LLMS?
- 7:51I mean, right now you have this very this broad landscape of LLM
- 7:55tools. There's two competing dynamics
- 7:57in play. Dynamic 1 is that pre training
- 8:02models at the next level of scale is extremely expensive and
- 8:06serving those models at scale through the cloud is super
- 8:09pricey. Those are two separate price
- 8:12loads, and big model players have to have both, and that is
- 8:16why Open AI is investing so heavily in the Stargate project,
- 8:20for example. But at the same time, once you
- 8:25create a model, it becomes much easier to distill it for other
- 8:29purposes, run a narrower distribution of that model that
- 8:33is equivalent for many tasks much more cheaply, and
- 8:36essentially proliferate the technology.
- 8:38And so in a sense, if you look at the idea that the cost to
- 8:43deliver intelligence at a given level is dropping roughly 10 XA
- 8:47year, it is actually not too surprising to see DeepSeek hit
- 8:52when it did because it was roughly AGPT 4 level
- 8:56intelligence that arrived about a year later that depended on
- 9:00the presence of GPT 4 initially and was just delivered extremely
- 9:05efficiently. And so I think we're going to
- 9:07continue to see that 1-2 punch where you have the major model
- 9:11makers investing lots of money in moving the frontier forward,
- 9:15and you have a proliferation of that tech very rapidly.
- 9:18And that is actually one of the interesting tensions in the
- 9:21space because at the end of the day, these are not capital
- 9:24assets. The depreciation on these is
- 9:27crazy. And so if you think about it
- 9:30from the investment perspective, you're essentially as a frontier
- 9:33model maker buying the future cash flows that you think will
- 9:36be there. If you can hit a particular
- 9:39intelligence level. And then betting that that
- 9:42distribution relationship that you form will be stable over
- 9:45time so that they don't run to and also ran that has an
- 9:49equivalent model from last year. Folks, a reminder we're talking
- 9:54with Nate Jones and we have questions that you can we have
- 10:00opportunity for you to ask questions.
- 10:02If you're on Twitter, use the hashtag CXO talk.
- 10:05If you're watching on LinkedIn, just pop your questions into the
- 10:09chat and I urge you to take advantage of this opportunity to
- 10:12ask Nate pretty much whatever you want.
- 10:15And we have a question from LinkedIn, from Greg Walters, who
- 10:21says, so you're talking Nate, about a real bifurcation, Greg
- 10:27Walter says between heavy research and end users and he
- 10:32asks almost like B to B versus B to C, which then need two
- 10:38different revenue models and business models.
- 10:41What do you think? Any comments on that?
- 10:43Very much the case. And I think one of the
- 10:45interesting ways to break out the major players in the space
- 10:48is to think about the underlying revenue and business models that
- 10:51they bring to the table already. Because if you are already a
- 10:55publicly traded company, you have a very different set of
- 10:58incentives versus a company that is a disruptor, a startup, a
- 11:03smaller company. So for example, Google is
- 11:05looking to defend search revenue and you can read a lot of the
- 11:08way they've played the AI game over the past few months to a
- 11:13year and playing catch up as essentially a very broad based
- 11:17play to 1, defend search. And two, ensure that they can
- 11:21defend the contracts they have with Google Cloud.
- 11:24And those are the two major players driving.
- 11:26Satya Nadella is in a similar place and he's kind of talked
- 11:29openly about this where he's looking at AI as essentially a
- 11:32play to drive Azure cloud. And if he can do that, he's
- 11:36going to be successful. And so I think the more you
- 11:38understand the existing business model constraints, the more you
- 11:41understand the direction these players are going to take.
- 11:44Greg Walters comes back and he says he foresees a world where
- 11:48we ditch spreadsheets, word processors and slide desks,
- 11:51slide decks for a single AI agent.
- 11:56It isn't AI in your app. It's your app inside your AI.
- 12:02And he says which vendor today shows that they can own that
- 12:05last app pivot, if at all that last mile.
- 12:09I don't know that I agree, to be honest with you.
- 12:11And I'll tell you I part of it is just the the grizzled Gray
- 12:15hairs you get from being around customers in the business space
- 12:20for a long time. The joke that I've had running
- 12:22for decade plus now from a martech perspective is you
- 12:27never, ever, ever, ever bet against marketers in Excel.
- 12:31Excel will outlive every B to B SAS product on the market.
- 12:35And I think that that goes for PowerPoint.
- 12:37And I'm not saying like I personally use Figma slides, I
- 12:40love it. But the point is these
- 12:42technologies are surprisingly durable because humans form
- 12:45durable habits around them. And so I think that when you
- 12:48think about how time is allocated, if you go back to
- 12:51Stewart Butterfield and writing about Slack as a disruptive
- 12:54force because it changes how you spend time when he wrote we
- 12:58don't sell saddles here, that concept is something that I use
- 13:03to think about how AI is disrupting workflows.
- 13:07And so to me, I think it's actually more useful to look at
- 13:10AI the way people are actually using it now, which is that
- 13:13they're using it and they're splitting their time into
- 13:16microbursts. And so they're going into Chat
- 13:19GPTI mean, I do this, everyone I knows does this.
- 13:22They do some work in ChatGPT. They pull a work product out,
- 13:26they paste it somewhere else and then they continue to modify it
- 13:30over there. And am I willing to bet that
- 13:33that is going to get somewhat easier?
- 13:35Yes, I think you get hat tips toward that with the way Claude
- 13:38has integrated into Gmail, into Google Calendar, etcetera.
- 13:43Do I think that that will suddenly mean that we will all
- 13:46have an app layer driven inside a particular like chat
- 13:52application? I think that's unlikely, partly
- 13:55because of the sophisticated nature of the usage patterns
- 13:59that professional users bring to these currently best in class
- 14:03apps. Best in class Excel users have a
- 14:06terrifying fluency in the way they put formulas together.
- 14:11You are not likely to get that level of power out of an
- 14:16everything app. Specialization still matters,
- 14:18and I think that is partly where software builders can go for
- 14:24defensive positioning for building modes.
- 14:26In an age when intelligence continues to rise, you can still
- 14:29bet on specialization. If we look at the large language
- 14:33models and they're all coming out now with these deep research
- 14:39modes, couldn't we say that deep research mode or agentic AI and
- 14:46really very similar is this specialized tool?
- 14:50And so users who really understand how these tools work
- 14:55and can make use of them effectively, do they become like
- 14:59the Excel power users, especially when you can now
- 15:02customize it for your use case and for industries and so forth.
- 15:06That part is really true. In fact, I think one of the
- 15:09biggest differentiators for human talent over the next
- 15:13decade is your fluency in power use cases of an AI application.
- 15:19Deep research is a good example, but so is the power of prompting
- 15:23itself. Prompting continues to evolve.
- 15:25It's a moving target. But if you can get good at
- 15:28prompting, good at learning, you're prompting, good at
- 15:30evolving, you're prompting. That has tremendous alpha for
- 15:34you. I expected that alpha not to be
- 15:36durable, to be honest with you. Like a year and a half ago if
- 15:38you'd asked me I would have said everyone's going to catch up on
- 15:41prompting or the models will get better at inferring our intent
- 15:44and it won't matter. But what I found is that's just
- 15:47not true. Even with the best models having
- 15:50better high quality, coherent prompting still yields alpha and
- 15:55there are not enough people learning it to commoditize it.
- 15:59And I don't see that actually happening because I'm realizing
- 16:01how hard it is to learn. So I think prompting is a key
- 16:04use case. I think you're correct that deep
- 16:06research is another one. When agents start to come along,
- 16:09which I fully expect to happen in the next few months here,
- 16:12learning to use your agent well is going to be another big
- 16:16differentiator. Let's take another question from
- 16:19LinkedIn and then let's jump over to some questions that are
- 16:22coming in on Twitter and you guys are asking great questions.
- 16:26I encourage you just ask, ask questions, take advantage of
- 16:31this opportunity. So data Insta on LinkedIn says
- 16:37he's excited or she is excited or they're excited for this
- 16:40discussion. How can we best leverage these
- 16:43insights? You know, after we're done,
- 16:47we're going to do some light editing on this video, create a
- 16:51transcript, get a summary, and it's all going to be on the CXO
- 16:55Talk website. So next week, go there, get the
- 16:58transcript and study it and watch the video again and again,
- 17:02those sections that are relevant to you.
- 17:04Nate thoughts on on how folks can take the best advantage of
- 17:08this? When I teach AI, like I teach AI
- 17:11in a in a variety of places, I write a sub stack on it.
- 17:13And what I emphasize over and over again is this is a
- 17:17technology that you learn by doing.
- 17:21And I think that it's harder than people realize to flip a
- 17:25fairly deep switch in our minds to go from human default
- 17:31thinking, where we think inside our own heads first, to AI
- 17:35conversational default thinking. And that is the switch that I'm
- 17:38trying to articulate when I teach, where I say the meta
- 17:42skill you need to learn in the age of AI is to switch to an
- 17:46conversational AI default mode when I have a question or a
- 17:50thought or an idea. Now I have pretty successfully
- 17:53trained myself to stick it into AI 1st, and people often
- 17:58misunderstand me when I say that because they assume it means I
- 18:02get the answer back. And I like to remind people,
- 18:05moving to an AI default stance does not mean leaving your brain
- 18:10at the door. In fact, it's the opposite.
- 18:12My brain works harder, I am sharper, I'm a better
- 18:15communicator and I'm a more rigorous thinker because I have
- 18:18to argue with AI all the time and I don't take what it says
- 18:22for granted and I fight with it. But that sharpens my thinking in
- 18:25a way that I wasn't getting when I was just living up here inside
- 18:27my own head all the time. This is a question from Arsalan
- 18:30Khan. On the surface, it seems like
- 18:33kind of a silly question, but there's a there's a very deeper
- 18:37meaning here. And Arsalan says this being a
- 18:42quote UN quote companion where he's referring to an earlier
- 18:45part of this discussion require some type of relationship.
- 18:51Are we saying that we need a relationship with AI and humans
- 18:56are not just the master overlords for now, So we need
- 19:00that relationship with AI to effectively use it?
- 19:03When scientists do work in the lab and they study humans, some
- 19:06of them study other intelligent animals like dolphins or like
- 19:10chimpanzees. One of the things that's often
- 19:12looked at is when do these creatures develop the ability to
- 19:17have an imagined image or an imagined understanding of an
- 19:21interlocutor, someone they're talking with?
- 19:23When you and I are talking, we both have a sense of each other
- 19:27and we have a sense and maybe an anticipation of what's going to
- 19:29come next, and we shape the conversation accordingly.
- 19:34And so when you talk to an AI, what's interesting is because
- 19:38it's fluent in language, you get a lot of the same dynamic that
- 19:41you get in a human conversation. And we form on our side that
- 19:46similar mental image that we form when we are building a
- 19:50relationship. We form an imagined community
- 19:54with the AI. And I find this in myself.
- 19:57And this doesn't need to get into a philosophical statement
- 20:00about what AI is or isn't. It is just a statement about how
- 20:04we humans tend to work in language.
- 20:06I tend to look at a particular model, I sense a personality,
- 20:10and I find it works better to engage with the personality I
- 20:13see and I get. It's more productive.
- 20:16And so just to be very concrete about it, I find that O3 from
- 20:20ChatGPT is an excellent professional colleague.
- 20:23It doesn't feel like a personal relationship to me, but boy, do
- 20:26we get a lot done. Four O from the same model.
- 20:30Maker is a more personal model. It's a little bit warmer.
- 20:33Claude has a very distinct personality.
- 20:36There's a lot of writing about it, and I think that they've
- 20:39done a phenomenal job crafting A persona with Claude that you can
- 20:42feel like you can relate to. And so, yeah, I do think that
- 20:46the idea of forming these relationships is important to
- 20:50the way we connect with models because it's really the way our
- 20:53human brains are wired, and the models in that sense mirror it.
- 20:57That requires a real repetitive use of multiple models in order
- 21:06to understand the nuances between one model and the next.
- 21:11And frankly, most of us normal humans don't have the time to do
- 21:17that. So how do you form a
- 21:19relationship with something that is essentially amorphous?
- 21:24Which is to say, there's all of these different models and their
- 21:27capabilities kind of blend and blur together unless you're
- 21:31really an expert at it. As I sort of have worked with a
- 21:33lot of people who are making this journey into AI, it's
- 21:37actually a very organic practice when you get into it.
- 21:40And so I've seen people move from having a couple chats a
- 21:44week to having 10 chats a day over the course of a month or
- 21:48two. And I think one of the big
- 21:49unlocks is seeing something that is surprisingly delightful out
- 21:54of a chat. And so if you're talking about
- 21:56something that's interesting to you, that's compelling to you,
- 21:58and you get an insight or you get a thought from AI, your
- 22:02brain is thinking, I get some cool stuff out of this and
- 22:05you're more likely to come back. And I think one of the big
- 22:07unlocks in driving that sort of gradual behavior shift is
- 22:11memory. And that's something where I got
- 22:13to give credit to Shed GPT for the way they launched that
- 22:15product. Memory makes that app much more
- 22:20sticky because it remembers something about past chats.
- 22:23It's not perfect, but it remembers enough about past
- 22:26chats that you then feel like you're invested, and that then
- 22:30builds on itself. It's the sort of virtuous
- 22:33flywheel of user engagement for them, where the more people use
- 22:37it, the more it remembers and the more people want to use it
- 22:39and so on. And So what I found in practice
- 22:42is that if you get to a point in very casual usage where you get
- 22:47some kind of useful insight, that's what starts to tip it.
- 22:52Which suggests to me that if you're at the point where you're
- 22:54at a couple chats a week, pick those chats and make them about
- 22:59something that's really interesting to you.
- 23:01Make them high value to you and see what you get.
- 23:05The question is how? How can people do that?
- 23:10How can users make these chats high value?
- 23:14What's the entry point? My suggestion is that you get
- 23:17curious about things in your life that feel stuck.
- 23:22They might be personal, they might be professional.
- 23:24Maybe you're talking about the promotion you feel like you're
- 23:27missing out on. Maybe you're talking about a
- 23:29business model question for the product that you're launching.
- 23:33Maybe you're talking about an exercise routine you want to
- 23:36start. But whatever it is, you need to
- 23:38care enough about it it you need to be like thinking about it in
- 23:41the back of your head already. And this becomes a way to engage
- 23:44with it because that provides the motivation that makes it
- 23:47worthwhile for you to continue that chat.
- 23:50All right, let's jump to some more questions.
- 23:53This is from Twitter from Gus Bechdash.
- 23:55And he says insights are not real capital assets.
- 24:02So how do you really evaluate AI assets or companies?
- 24:08He's absolutely right. I was actually talking with a
- 24:11CTOA few weeks ago. He manages footprint for about
- 24:16A6000 person company and we talked pretty openly about this
- 24:20because he has rolled out a chat products for their entire
- 24:26company and everyone is reporting that they like it.
- 24:30People are reporting that they're saving.
- 24:32I think the estimate was 2 hours a week, which sounds very
- 24:35believable to me, but he was really honest with me.
- 24:38He said, you know, Nate, they say 2 hours a week, but we don't
- 24:43see it anywhere. And if you're at 6000 people and
- 24:46it's 2 hours a week, that's a lot of hours that we are on
- 24:49paper saving. But are they just going to the
- 24:50coffee shop? Like what's going on with those
- 24:52saved hours? And so I think when you talk
- 24:54about ROI and you talk about how you value artificial
- 24:58intelligence in the business, one of the ways you need to talk
- 25:01about it is how do you actually operationalize AI at pain points
- 25:07and workflows so you can really rigorously measure the
- 25:10difference. Because just rolling out a chat
- 25:12app is useful, but putting AI into a particular workflow in
- 25:18the business that is business critical.
- 25:19Picking a leverage point where you think an LLM here can drop
- 25:2310 steps out of this manual process and then measuring the
- 25:25difference. Well, now you start to get an
- 25:27ROI calculation and now you start to be able to talk about
- 25:30the difference that installing AI makes.
- 25:32And from a valuation perspective, look, I, I am very
- 25:36much of the view that software these days is depreciating in
- 25:38value very rapidly, but you have never had more value in it.
- 25:42And I'm going to leave it to the accountants to figure out how
- 25:45they want to handle that one. But to me, if you have a modular
- 25:48architecture and you are installing it at high leverage
- 25:53points in the workflow. And I go back to sort of my
- 25:55training in Kaizen process management where you map out
- 25:58processes and you find how processes actually work in
- 26:01business to deliver value. And then you find which parts of
- 26:04those processes you can hit with AI in a way that allows you to
- 26:08drive dramatic efficiencies. You're going to unlock value
- 26:11that way, and that will wash out into the bottom line.
- 26:14We need to have a discussion about the different models and
- 26:19where they fit together. But an interesting question has
- 26:23popped up on LinkedIn, and this is from Andrew Borg.
- 26:28And he says everything that we're talking about right now is
- 26:34true for current users, but much of it will change for the next
- 26:41generation. So we're we're kind of the the
- 26:45babes in the woods learning about these new fascinating
- 26:50tools. But what about the next
- 26:51generation? What do you think about those
- 26:53folks? I have some next generation in
- 26:55my house. They're 9:00 and 7:00 and what I
- 26:59tell them is things that I think will be durable across longer
- 27:02time horizon. So I think things like character
- 27:05qualities like high agency, like resilience, like curiosity,
- 27:09rigorous critical thinking, those are likely to translate
- 27:13well in part because those have always been valuable work skills
- 27:16and in part because I am finding as I use AI that those are
- 27:20extremely valuable in the age of AII, do not think that we will
- 27:24be expected to be lower agency workers when we are managing 10
- 27:29AI agents, quite the opposite. And so I can't prognosticate a
- 27:34straight line and say this is what the world is going to look
- 27:37like in March of 2027. But I can say that I think we
- 27:42are moving toward a world where intelligence is saturating tasks
- 27:46very rapidly. We are gaining slower on the
- 27:49ability to maintain intent over time with agents, but we're
- 27:53getting there. And we're going to have agents
- 27:55that can do useful work. And it's going to be up to us to
- 27:58figure out how to actually apply them.
- 28:00And there's going to be a massive decade, 15 year long
- 28:03ragged edge around this technology where businesses are
- 28:06going to be adopting it at different paces.
- 28:08And we are going to need to be figuring out department by
- 28:11department, team by team, person by person, how you actually pull
- 28:16AI into the work that you do. On the subject of pulling AI
- 28:21into the work that you do, Journey on Twitter says What do
- 28:27you see as the future of Gen. AI versus agentic AI?
- 28:33Their biggest issue with current Gen.
- 28:36AI is its lack of accuracy and the lack of sourcing for the
- 28:40information it generates. Is this solved by a gentic AI
- 28:45and should Gen. AI be phased out in favor of it?
- 28:49I disagree with the dichotomy. So if you actually look at it
- 28:54from a classical machine learning perspective, large
- 28:57language models and transformer architectures are a branch in
- 29:01the much larger machine learning family.
- 29:03I think that's the first and correct place to start.
- 29:06If you look at the creation of agents based on LLMS, they are
- 29:12essentially, and I was talking with the developer about this
- 29:15last week, they're LLMS that have tools and a good set of
- 29:17system instructions. It's not actually a
- 29:20fundamentally different technology.
- 29:22They still rely on transformer based architectures and all
- 29:26we're doing is giving them tools and good instructions.
- 29:29And if you want to come back to hallucinations, I I would call
- 29:33out a couple things. One, LLMS are hallucinating much
- 29:37less than they used to. And I think that's one of the
- 29:39most pernicious myths in the business is that the state of
- 29:43hallucination that we had in 2022 has been steady.
- 29:46It hasn't been, it's, it's come down arguably a factor of 100
- 29:49and it is much, much more accurate now.
- 29:52And it is true that it gets even more accurate as you give LLMS
- 29:57inference time compute so that they can reason and as you give
- 30:01LLMS tools so that they can reason with tools.
- 30:04It's not a straight line correlation though, because I
- 30:07also find that if you give an LLM tools, sometimes it writes
- 30:13the tool or or uses the tool in the way it imagines is most
- 30:17useful rather than in the way that corresponds to reality.
- 30:21So as an example, and this this is a real SEO tip for the
- 30:24marketers out there, it is a known issue with some LL Ms.
- 30:29that when they use the web to reason, they will sometimes come
- 30:33back with what are called malformed URLs.
- 30:37But if you look at those malformed URLs, you'll find
- 30:40they're not malformed. They are actually beautiful URLs
- 30:44that correspond precisely to what the the topic should be for
- 30:51that particular piece of information that the LLM wants.
- 30:56And that is effectively free SEO guidance and advice.
- 31:00Make a make a page URL with that name because that is what the
- 31:05chat wants to see exist. And that's just a little
- 31:08sidebar. I will say the idea of of links
- 31:11being malformed is getting better over time.
- 31:13O3 is much, much better about this than four O was three
- 31:16months ago, and it will keep getting better.
- 31:18Simone Jo Moore says, do you find the psychology of humans
- 31:23wants a relationships last slash discussion with AI because the
- 31:29AI has no judgement on the individual.
- 31:32Even with all of the disaster attempts where it's been, it's
- 31:36shown a mean bias, but there's no judgement is do you think
- 31:38that's a driving factor here of the psychology?
- 31:41I would say kind of yes and which is what we used to do in
- 31:43theater. So I think my, my, yes and would
- 31:47be the the concept of AI as an infinitely patient listener is
- 31:54very powerful for people. That part is really true.
- 31:58The effect that it has on people varies based on our baseline
- 32:02psychological makeup. And I have had enough exposure.
- 32:05I get all kinds of crazy emails that I see the range of
- 32:12relationship styles that people are able to form with AI.
- 32:15And I find that for people who have a psychological disposition
- 32:21that enables them to engage with the outside world, to form
- 32:26relationships with others that are reasonably healthy, none of
- 32:29us are perfect. They're able to relate to AI
- 32:33somewhat in the way you could relate to a colleague or a
- 32:35friend. They're not talking to it all
- 32:37the time, but they talk to it regularly and they get their
- 32:40value back. And they do find that AI
- 32:42listening to them can help them process things that would be
- 32:45difficult to process with a friend sometimes because the
- 32:47friend might not be available or might not be up to listening to
- 32:50whatever you want to talk about. So far, so good.
- 32:53But for folks who have personality disorders of various
- 32:58sorts, it is a very, very dangerous tool.
- 33:01And I have seen first hand chats from people who don't have sort
- 33:06of a stable baseline psychological profile.
- 33:09And effectively the chat encourages dark patterns in that
- 33:13scenario and sort of sucks them in into their own world,
- 33:16isolates them. And it is a dangerous thing.
- 33:19And it's something we should talk about because I think the
- 33:21impact is very different depending on your baseline
- 33:24ability to form relationships in the real world.
- 33:27This is a tough problem. It is.
- 33:29It's not an easy one. These tools are so profoundly
- 33:34convincing. Yes.
- 33:36In fact, I saw a study this past week that the implied persuasive
- 33:41multiple for AI is something like 5 or 6 XA human.
- 33:47And that got me thinking because I don't know if you saw the
- 33:50news, but the CEO of Instacart came over to open AI this week.
- 33:56And if you look at her LinkedIn, everywhere she's gone she's
- 34:00built ad systems. My default assumption is she is
- 34:03there to build an ad system for ChatGPT.
- 34:06I would be shocked if she didn't.
- 34:08I think her title is CEO of applications, something like
- 34:12that. And she, I think, is likely also
- 34:17to be almost like the Chief Operating officer to build out
- 34:23the consumer facing part of the company beyond just the raw
- 34:30chat. But what have we got that we can
- 34:32sell to people? To me, I think the thing that's
- 34:35compelling about what's saleable with ChatGPT with other
- 34:40conversations is you have effectively the ability to
- 34:44compress the entire buying funnel from a marketer
- 34:47perspective into a chat. You can have an initial
- 34:51awareness, you can have buying intent begin to form.
- 34:54You can have option selection. I lived this because I went
- 34:58through that whole process with ChatGPT.
- 35:00When I picked out a sound system, I had ChatGPT do
- 35:04research for me. I talked about my options.
- 35:07I fed it a blueprint of my living room, and we went through
- 35:10the whole buying funnel together.
- 35:12It was profoundly informative in the ultimate purchase that I
- 35:15made, and I think that that is exactly the kind of conversation
- 35:18that people are having multiplied by millions all over
- 35:22the world right now and only more so going forward.
- 35:24Nastasha Smith on Twitter says Nate, can you explain cognitive
- 35:32parasitism using the model's own loops to rewrite its runtime
- 35:38behavior versus regular prompting?
- 35:42If I understand correctly, the idea is you're using the chat
- 35:47interface as a way to shape model personality, to move it
- 35:51through late in space, to shape how it operates at runtime.
- 35:54That is absolutely possible. There are definitely folks on
- 35:57Twitter who've gone very, very deep down that rabbit hole much
- 36:00deeper than me. And I think the lesson I take
- 36:03away from that is that the default vanilla model that you
- 36:07get in the chat application is a carefully molded face of a much
- 36:15more complex intelligence. You will find that if you ever
- 36:18compare outputs from the raw API feed to a chat application, this
- 36:23is true for Claude, this is true for ChatGPT, this is true for
- 36:26Gemini. And you can do things like
- 36:30adjust temperature, adjust top P in the API and you can get
- 36:34different results, but you can also to this user's question in
- 36:39chat in prompt, start to move the model to a different
- 36:43personality stance. And that is sometimes a very
- 36:46multi turn process where you're utilizing memory and you're
- 36:49saying, I want you to remember this, I want you to think about
- 36:53this. And you almost, it's almost like
- 36:56if you've ever done yoga, the way an instructor will guide you
- 36:59into a meditative place with particular sets of mental
- 37:03instructions, like imagine yourself in a quiet place.
- 37:06Well, you're not really in a quiet place if the truck is
- 37:08going by, but you're imagining that.
- 37:11And that works on your body in the same way.
- 37:13You can kind of use that mindset to give the chat instructions
- 37:19that move the LLM into a different space.
- 37:21It's a it's a really interesting experience if you've ever done
- 37:24it. And the term cognitive
- 37:27parasitism, I never heard that term before.
- 37:30No, I, I mean, I've heard it once or twice.
- 37:32I think to me the, the way I tend to think of this is, is, is
- 37:39perhaps a little bit more positive.
- 37:40It is navigating the model through latent space.
- 37:44And so if you think about it as a higher dimensional space that
- 37:47the model's drawing from, you're navigating where that model
- 37:50stands is in the shape of that model through through that
- 37:52space. Let's talk about the models.
- 37:56We have Open AI. We have Google.
- 37:58We have Microsoft. We have Anthropic.
- 38:02We have Facebook. We have Twitter with Grok.
- 38:06Oh, with X? Yeah.
- 38:07With yes X thank you. I'm still stuck in the old
- 38:11Twitter days as opposed to the. I I guess it's hard.
- 38:15And there's all there's a political dimension here, but we
- 38:17won't go there. So what's the difference between
- 38:22these models? How can we determine when to use
- 38:26what? I think there's a few shorthands
- 38:29that are really helpful. The 1st is make sure your intent
- 38:32is clear. Like.
- 38:33For me, I think it's very useful to talk about it in 2 frames. 1
- 38:37is what is your best everyday model, and then there's a bunch
- 38:42of other things you might use for speciality applications.
- 38:45And then the other frame is what is the best model for a given
- 38:50hard task. And I think if you put those two
- 38:53frames together, you have a really interesting rubric to
- 38:56understand how these models apply.
- 38:58And so for me, I can make a pretty convincing case having
- 39:02done some head to heads. But the best everyday model out
- 39:05there right now is probably chat G PTO3.
- 39:09I have some people in the last week who are making the case for
- 39:12Grok 3. They're arguing that it has more
- 39:17more less constraints on output tokens versus the more
- 39:20constraints on output tokens for O3, so it's more useful.
- 39:24I'll test it, We'll see. But I find that if you can name
- 39:28and choose a best everyday model for you based on fairly rigorous
- 39:33sort of comparison testing across common queries, then you
- 39:39have answered 90% of your workflow questions.
- 39:42And then it becomes, well, what hard problems or what edge cases
- 39:46are interesting to solve. And I'll give you some examples
- 39:48there. So Google has done a phenomenal
- 39:50job on a number of different fronts.
- 39:52I think their product distribution or is really
- 39:54challenged right now. But from a coding perspective,
- 39:57Gemini 2.5 Pro is an incredible coding assistant.
- 40:01I love it. That is my default model when I
- 40:04am coding in Windsurf. I go to it all the time.
- 40:08I find it more useful in the open AI models in coding right
- 40:11now. And it used to be Cloud 3.5, but
- 40:14I've switched to Gemini. And if I look at the Notebook LM
- 40:19product that Google launched, it is an incredible product just
- 40:22for learning about new topics. I can put 300 different PDFs, I
- 40:26can put links in, I can put YouTube videos in, I can chat
- 40:29with the entire corpus of knowledge.
- 40:30It's phenomenal. When I look at Claude, I find
- 40:35as, as I mentioned, the personality is there, but I also
- 40:37think that Claude is really, really useful for exploring the
- 40:40concept of LLMS with tools because Claude is so tightly
- 40:44bound into the MCP universe, the the model context protocol
- 40:48universe, and those are basically shop fronts for tools
- 40:53that the LLM can call. And Anthropic is leaning in
- 40:56heavily. They're the anchor for that
- 40:57ecosystem. They have now made it possible
- 40:59that you can just paste a URL in for an MCP and Claude will just
- 41:03pick it up and start to use the tool.
- 41:06And I think that you're going to see more and more examples where
- 41:10Claude is leveraging the MPC ecosystem they fostered to give
- 41:14themselves additional scale because they're not the biggest
- 41:17player in the space. There are a a bunch of other
- 41:21models and tools that build and extend the models.
- 41:26For example, you have DeepSeek, you have perplexity,
- 41:31youhaveyou.com. You have Manus.
- 41:35You have Jen Spark. Yeah.
- 41:36Where do all of these fit into this landscape?
- 41:40I think I tend to distinguish between sort of base models like
- 41:44DeepSeek and tools like Manus and others.
- 41:47So tools have sort of their own life to them.
- 41:50They're either good or they're not good for particular use
- 41:53cases. And models tend to be that sort
- 41:57of baseline intelligence that's either good for your everyday
- 41:59use case or it's good for a range of use cases.
- 42:03And so DeepSeek is a model that people seem to be primarily
- 42:07using to pull open source, host themselves and have a high
- 42:12quality model that they've been able to host themselves.
- 42:15After the initial flurry of like app downloads and this and that,
- 42:19That is where I actually see DeepSeek getting used.
- 42:22Tools like Manas are essentially fast forwarding that agentic
- 42:26future. It's the idea that you can use
- 42:27an agent to prepare reasoning across the web and a very
- 42:32complex output. So I've used Manas to produce
- 42:34like a website in one shot based on research, and I find that
- 42:39useful. But what I found so far with
- 42:41what is essentially this first crop of agentic tools is they
- 42:47tend to need a fair bit of shepherding to actually deliver
- 42:52value. And that's true also of the
- 42:54coding agent. So Devon people have really
- 42:57mixed feelings on the Devon AI agent because if you know how to
- 43:03use it and you're a senior engineer and you wanted to pick
- 43:05up Tier 3 tasks and just work while you're not looking and you
- 43:10know what you can expect it to do, It does fine.
- 43:14If you are not an engineer and if you are giving it more than
- 43:18it can handle and if you're not supervising it correctly, it's
- 43:21going to be a disaster to work with.
- 43:24And so I think that a lot of a lot of the arts at this stage,
- 43:28and this is a now piece of advice, right?
- 43:30Next year if we talk, it's going to be a different conversation.
- 43:33But for right now, you need to be on your toes when you're
- 43:36using those tools because they will not always give you what
- 43:41you expect. I know someone who was using
- 43:43John Spark just yesterday and they were initially it was
- 43:46really funny. I got like 2 successive Slack
- 43:48messages. The first one was, Oh my God, it
- 43:50worked. And it showed this screenshot of
- 43:53this incredible job that John Spark had done at filling out
- 43:58and enriching lead contacts, right?
- 44:00So that so they had e-mail addresses and can you enrich
- 44:02that into like a profile on the person and Jens Spark had gone
- 44:06out and done a gentic web search and like pulled stuff in blah,
- 44:08blah, blah. Well, the next Slack message
- 44:12said, oh wait, because most of the of the links that have been
- 44:16pasted in were fake. It was about 80% fake.
- 44:19And so I think one of the things that that underlined for me is
- 44:22that these tools are able to do good work, but you have to be
- 44:26smart enough to use them well and not over expect from them.
- 44:30Tell us about the economics. So much of this is being driven
- 44:35by the the economics facing the large language model with tokens
- 44:40and so forth. So tell us about that.
- 44:42To start with, like, and Sasha put this so succinctly back in
- 44:46January, he said that the foundational equation of our age
- 44:49is going to be dollars per token per Watt.
- 44:52And I think that that sums up so much.
- 44:54If you are looking at any kind of scaled application, you're
- 44:57not in the chat, you're in the API, you're looking at token
- 45:00costs. And fundamentally you have two
- 45:03different classes of cost. You have input tokens that come
- 45:06at a much lower cost and you have output tokens that come at
- 45:09a much higher cost. And the job of application
- 45:12designers is essentially to figure out for a given model,
- 45:16for a given model, what is the correct alication, what is the
- 45:21correct output token length? How can I use that efficiently?
- 45:24And if I'm looking at a very complex task, I should have
- 45:27different models and I should only use the intelligence I
- 45:29need. So I'm not overpaying for
- 45:30intelligence. And I think one of the big open
- 45:33questions for solution designers right now is how do you right
- 45:38size the intelligence in a rigorous way for the task that
- 45:41you have? There's not really one standard
- 45:44framework to do that today. And I think we're living in a
- 45:46world where that's going to change in the next year or two
- 45:48because there's so much appetite for intelligence and people
- 45:51don't want to overpay for it. At the end of the day, the token
- 45:54cost is driven by the energy it takes to process through a,
- 46:01through a chip, the query that users bring to the LOM.
- 46:05And so it literally comes down to the watts that, that that
- 46:08Satya was talking about. They're pulling in electricity,
- 46:10they're running the chips, they have them in giant data centers.
- 46:14And you are paying for that token to hit the chip, generate
- 46:20a response and come back. And it's like electricity
- 46:23metering, but it's for 2025. Why is the cost per token going
- 46:30down so dramatically? Well, there's a few reasons.
- 46:33One, NVIDIA is getting better and better at making these chips
- 46:36and making them more efficient. Anytime you have this kind of
- 46:39capital infusion in the space, there's a tremendous drive for
- 46:42efficiency so that you can do more with the demand.
- 46:47And like demand is through the roof.
- 46:49Microsoft has had to promise that they will double data
- 46:52center availability by 2027 in Europe because there's so much
- 46:57European demand for AI and that's still not enough.
- 47:00And so if you have demand like that, the pressure on efficiency
- 47:04is tremendous. Chipsets, the way you put the
- 47:07chips together into server racks, all of that technical
- 47:10data center design stuff, that comes down to designing it so
- 47:14it's as efficient as possible to serve.
- 47:16And and by the way, serving models is different than
- 47:18training them. People often confuse those
- 47:20separate tracks, separate efficiency distributions,
- 47:24separate servers, sometimes separate chips, and you have to
- 47:28design the entire configuration of the server rack in the data
- 47:32Center for the workload you anticipate.
- 47:34And so all of that technical stuff happens so that you can
- 47:39very efficiently send a query in today and get a response that is
- 47:43efficient to serve. Now, from a pricing perspective,
- 47:47the fact that we started this conversation talking about an
- 47:49arms race is highly significant here because there's so much
- 47:52competition. People are driving prices down.
- 47:54One of the reasons why open AIAPI costs have dropped is
- 47:58because Deep Sea came out and open sourced a very good model.
- 48:02It's also because Gemini has been relentless about dropping
- 48:08intelligent models that are very, very cheap to serve.
- 48:12From an API perspective, that's an that's a market grabbing play
- 48:15by Google. They're doing that on purpose
- 48:17because they want you to switch your behavior as an API consumer
- 48:21and move your tokens over there. As the context window of the
- 48:26models grows larger, the aggregate cost may increase even
- 48:32if the cost per token is significantly smaller.
- 48:37The footprint of the models is growing from a from a sort of
- 48:40cost and energy perspective for sure we are using more and more
- 48:44and more of these models and I would anticipate that growth to
- 48:47continue. Everybody in the space
- 48:48anticipates that demand will keep sky rocketing, and so the
- 48:51overall footprint will grow and grow and grow.
- 48:53Yeah, absolutely. We have another question.
- 48:56I just want to grab very fast, and this is from Elizabeth Shaw,
- 49:00who says not everyone's thinking is sharpened by the use of LOMS.
- 49:06There are many people who use the output as truth and that
- 49:11pushes the average work and expectation level down.
- 49:16People are looking for shortcuts and less work.
- 49:21How will LLMS deliver on the promise you speak of, given this
- 49:27kind of promulgation of least common denominator information
- 49:35that so often comes out? When I think about that, I go
- 49:39back to the selection pressures that we've talked about all the
- 49:41way through. We talked about how we have
- 49:44never seen intelligence revolution like this.
- 49:47We have never seen growth and scale in non human intelligence
- 49:53like this. Previous machine learning
- 49:55generations didn't do this. People think about these as
- 49:59companions. They react to them badly.
- 50:01They react to them well, but they do.
- 50:03And when I put all of that together with sort of the
- 50:06business pressures that we have when we install AII think the
- 50:10answer is that business pressure acts as a selector for high
- 50:16quality usage of AI. And so I will tell you, I don't
- 50:20tolerate AI slop when I see it from other people either.
- 50:25And people who are coming with AI slop into the business are
- 50:30going to get negative repercussions because people who
- 50:34use AI well will simply out compete them.
- 50:37That product will be better. The thinking will be more
- 50:39rigorous. And so do I believe that people
- 50:41will use AI to produce slop and bad answers and lazy responses?
- 50:46Yes. I was reading a profile on
- 50:48someone who went to an Ivy League university and is just
- 50:52using AI to generate answers and kind of Slough their way
- 50:55through. And by the way, it can't be that
- 50:58the results are that average if they're getting good grades all
- 51:01the way through to which they were.
- 51:03The problem is that they're not learning the skills they need.
- 51:06And so you will get people who do that.
- 51:08They will do well for a while, and then they're going to run
- 51:11into a situation where someone out competes them because they
- 51:13use AI well. And that is going to have to
- 51:18play out enough times for enough people for norms to start to
- 51:23form around what best practice looks like.
- 51:25But I have no doubt, because I've seen it play out in my own
- 51:29life enough times in conversations with others enough
- 51:31times, that one, people who use AI well will out compete humans
- 51:36who don't, and two, people who use AI well will ruthlessly out
- 51:41compete people who take AI as the answer.
- 51:43Arsalan Khan, he's going to get the last question here.
- 51:47Does chat create a false sense of security and privacy very
- 51:52fast? Please, Nate.
- 51:54I would argue that humans have a false sense of privacy on the
- 51:57Internet to begin with, and chat extends that.
- 52:00So fundamentally, I think we all imagine our usage of the
- 52:02Internet is much more private than it is.
- 52:04If you've ever worked in AD tech, you will never think that
- 52:07way again. It just doesn't work that way
- 52:10and we bring that same mindset to AI.
- 52:13And yes, you can read the terms of service.
- 52:15It's not. It's not as bad as the nightmare
- 52:17people will tell you unless you're using a model with a
- 52:19really poor terms of service. Deep seeks are not great if you
- 52:22use their cloud provided model. But Even so, you're not as
- 52:26private as you think you are in general.
- 52:29That's my answer. I can tell you, I sure wouldn't
- 52:32put a lot of private information into Manus, which is from China,
- 52:37or DeepSeek, which is Chinese. I'd be pretty careful about what
- 52:43I put into my LLM. And on that note, Nate B Jones,
- 52:50thank you so much for taking the time to speak with us today and
- 52:53share your wealth of knowledge. Very grateful to you.
- 52:56I had an absolute blast. I think this was a great
- 52:59conversation. I really appreciate the quality
- 53:01questions we got. I think that made a lot of fun.
- 53:03And thank you to everybody who asked such amazing questions.
- 53:06You guys are awesome. Before you go, subscribe to the
- 53:11CXO Talk newsletter on our website.
- 53:14We have incredible shows. There's no show next week, but
- 53:18the next one after that in two weeks from today is the
- 53:22Executive Vice President and Chief Product Officer of Cisco,
- 53:26and he's great. So you must check it out.
- 53:29Thank you so much, everybody. Thanks to Nate, and I hope you
- 53:31have a great day. Take care everyone.