Latest / KAJ Business & Money LIVE / Why Your AI Investment Is Failing (And How to Fix It) | Brenn Hill
Transcript
- 0:23Speaker 1: God, God, God, Welcome to the kJ Masterclass today, Own
- 0:37Speaker 1: Business and Money.
- 0:38Speaker 2: We are joined by brain Hill AI data science and
- 0:42Speaker 2: engineering leaders specializing in transforming raw AI capabilities into secure
- 0:48Speaker 2: high ROI enterprise systems, and we will discuss with him
- 0:53Speaker 2: why even capable companies fail to get value from AI,
- 0:57Speaker 2: the guardrails that allow sale, safe deployment at scale, and
- 1:01Speaker 2: what engineering leaders miss when they focus on capability instead
- 1:06Speaker 2: of delivery.
- 1:08Speaker 3: Welcome to the show.
- 1:09Speaker 4: Brand, Thank you, thank you for having me. Really good
- 1:13Speaker 4: to be here.
- 1:14Speaker 2: Well, welcome, Welcome to the show. Welcome to India in
- 1:17Speaker 2: this online form brand, and I'm sure not just in India,
- 1:20Speaker 2: but a lot of companies, a lot of leaders across
- 1:23Speaker 2: the global benefit from what we are going to talk
- 1:27Speaker 2: right now. We'll talk about why companies don't see AI
- 1:30Speaker 2: value and things around that and much more. But before that,
- 1:34Speaker 2: let's talk a bit about your journey, your background, so
- 1:38Speaker 2: that the audience knows who they.
- 1:39Speaker 3: Are listening to.
- 1:41Speaker 4: So I've been in engineering and software for about seventeen
- 1:45Speaker 4: eighteen years. First is a hands on software engineer full stack,
- 1:49Speaker 4: and then later in leadership. I was the CEO of
- 1:53Speaker 4: a venture funded company in Vietnam and then part of
- 1:57Speaker 4: Vietnam's first unicorn, and now I'm in Germany is part
- 2:01Speaker 4: of a publicly traded firm here in Germany, despite being Americans.
- 2:06Speaker 4: So I've been all over the world. I've seen people
- 2:08Speaker 4: building things all over the world at every possible size
- 2:11Speaker 4: of company, and I've worked across pretty much every possible stack.
- 2:18Speaker 3: Absolutely absolutely thank you for that.
- 2:21Speaker 2: So to understand the topic itself, you know why companies
- 2:25Speaker 2: don't see AI value capable companies failing to get value
- 2:30Speaker 2: from AI miss for a common person. You know, it's
- 2:34Speaker 2: difficult to understand it that way. That with the amount
- 2:38Speaker 2: of adoption and the amount of talk, you know that
- 2:42Speaker 2: one cannot do without AI. And the companies are also
- 2:46Speaker 2: adopting it in whatever ways they can, and still they
- 2:49Speaker 2: are not able to see value or will not get
- 2:52Speaker 2: the value that they think they should be. It seems
- 2:56Speaker 2: to like, you know, running in the dark. Help us
- 2:59Speaker 2: understand what happening, why it's happening.
- 3:02Speaker 4: So, you know, when you use AI, it seems like magic.
- 3:07Speaker 4: You type some stuff into the box and then you say, hey,
- 3:11Speaker 4: create something, and you see something that's really polished, and
- 3:14Speaker 4: we're used to something that looks polished being polished if
- 3:18Speaker 4: we are. If you're part of a company and you
- 3:20Speaker 4: do a demo for an executive, if it looks really good,
- 3:23Speaker 4: people have spent a lot of time making it really
- 3:25Speaker 4: good end to end. But with AI, it is designed
- 3:28Speaker 4: to make you happy. It's very good at the surface stuff.
- 3:30Speaker 4: The deeper stuff is a more difficult part and often
- 3:35Speaker 4: more complicated to specify. And so what happens is the
- 3:40Speaker 4: basic root of the problem is that when you produce
- 3:43Speaker 4: software this way, or you do agents or whatever, getting
- 3:47Speaker 4: something that looks good and something that is good is
- 3:49Speaker 4: a hard thing to determine. How do you know that
- 3:52Speaker 4: what you've created is safe, that it will work across
- 3:55Speaker 4: all the different things that your customers need. Let me
- 3:58Speaker 4: give you an example. An executive came to me and
- 4:01Speaker 4: he said, hey, brend I want to put this thing
- 4:04Speaker 4: I built with AI into production. I said, sure, you know,
- 4:07Speaker 4: we can do that. First show it to me and
- 4:08Speaker 4: tell me how it works and what it does. And
- 4:11Speaker 4: he described it, and he loaded it up on his
- 4:13Speaker 4: screen and I said, ask the AI, you know, some questions,
- 4:17Speaker 4: and it came back and said, oh, yeah, I'm not
- 4:19Speaker 4: actually doing these things that you thought I was. I
- 4:21Speaker 4: was just simulating it, so if we put it into production,
- 4:24Speaker 4: it wouldn't work because really it was a wrapper around
- 4:28Speaker 4: just calling the AI some more, not the vector database
- 4:31Speaker 4: and all these other things that he thought were part
- 4:33Speaker 4: of this deployment. So we were able to get it working,
- 4:36Speaker 4: but it wasn't what he thought it was. And as
- 4:39Speaker 4: a company, especially as a campaign company, if you have
- 4:42Speaker 4: a lot of customers writing on certain specific behavior and
- 4:45Speaker 4: the AI says it's good, you don't really know it's good,
- 4:48Speaker 4: so you end up spending a lot of time verifying
- 4:51Speaker 4: and testing instead of building. The other thing is that
- 4:55Speaker 4: a lot of times it's not the creation of code
- 4:59Speaker 4: that really eats up a lot of time. So, for instance,
- 5:02Speaker 4: in a larger company, you may spend a little bit
- 5:04Speaker 4: of time or even a lot of time writing code,
- 5:07Speaker 4: But in the background, while people are writing code, there's
- 5:09Speaker 4: a tremendous amount of political alignment, customer alignment, customer testing,
- 5:14Speaker 4: all these things going on in the background that informs
- 5:17Speaker 4: the development. But if the development happens in a day
- 5:20Speaker 4: and that process still takes a month before you can
- 5:23Speaker 4: safely release, then the fact that the software took a
- 5:26Speaker 4: day to write a bunch of code doesn't matter because
- 5:28Speaker 4: it's a holistic process. So there are ways you can
- 5:33Speaker 4: use AI to deliver value, but you have to use
- 5:37Speaker 4: it the right way, and people do not do that.
- 5:40Speaker 4: In fact, the intuitive way, which is kind of to
- 5:42Speaker 4: yolo as I call it, into writing a bunch of
- 5:44Speaker 4: prompts and taking what you get tends to result in
- 5:47Speaker 4: very buggy software and outcomes, and then people feel they
- 5:50Speaker 4: can't safely release it.
- 5:53Speaker 3: Right.
- 5:54Speaker 2: Right, So, if I look at it from my audience perspective,
- 5:59Speaker 2: common person's perspectives, small businesses perspective, and people who want
- 6:05Speaker 2: to understand you know, this new, this thing called artificial intelligence.
- 6:09Speaker 3: It seems to be on a very fast track of growth.
- 6:13Speaker 2: If you look at it from AI company's point of view,
- 6:16Speaker 2: they are you know, since twenty twenty two, they are
- 6:20Speaker 2: in you know, we are talking about Fable five, my
- 6:23Speaker 2: house five, and so many other things. At the same time,
- 6:27Speaker 2: you know, governments are talking about cybersecurity and risks for
- 6:32Speaker 2: them and all those things. And when you talk about
- 6:36Speaker 2: small companies which are using you know, advanced models, it
- 6:42Speaker 2: doesn't seem to be solving a problem which can solve
- 6:46Speaker 2: a customer or client's problem. Again, it looks like to
- 6:50Speaker 2: be difficult to believe. It's almost like an illusion that
- 6:54Speaker 2: we are chasing. How do you want us to understand this?
- 6:59Speaker 4: Sure, so I can explain part of what goes on
- 7:01Speaker 4: under the hood, the technical way llms work when it's probabilistic.
- 7:06Speaker 4: So let's say you use an AI and it solves
- 7:09Speaker 4: your customer's problem. The first time. The next customer comes
- 7:12Speaker 4: along has the same problem, the AI gives them a
- 7:14Speaker 4: different answer. Then they complain, why is my answer different
- 7:17Speaker 4: from the other person's answer. It's not as good or whatever,
- 7:19Speaker 4: So now you have a problem even if it works right,
- 7:22Speaker 4: which may which it may not. But to understand what's
- 7:25Speaker 4: going on behind that is that AI are trained to
- 7:30Speaker 4: basically make people happy. They go through a process called
- 7:32Speaker 4: reinforcement learning. You ask them a question, they give you
- 7:36Speaker 4: an answer. If it makes people happy, they click yes.
- 7:38Speaker 4: So this is why AI are psycho fantic that the
- 7:42Speaker 4: nature of how they are trained makes them try to
- 7:45Speaker 4: give you answers it will make you happy, which is
- 7:47Speaker 4: not the same as answers that are actually good or
- 7:51Speaker 4: thorough or fully correct or whatever. And then the other
- 7:55Speaker 4: part of this that happens under the hood for how
- 7:57Speaker 4: AI work is they're Essentially the compressed knowledge of humanity
- 8:03Speaker 4: and compression is never perfect. So if you compress a
- 8:06Speaker 4: bunch of knowledge and then you ask them a question
- 8:08Speaker 4: that they don't know the absolute answer to, they will
- 8:12Speaker 4: make their best guess. And this is just how they work.
- 8:15Speaker 4: And also added to the psycho fancy part, they will
- 8:18Speaker 4: tell you something very confidently that is wrong but sounds good. Now,
- 8:22Speaker 4: this works on the code level, works on the answers
- 8:24Speaker 4: you give them. That's just how they're built. And now
- 8:28Speaker 4: they're trying to learn how to train them to have
- 8:30Speaker 4: this be less true. But it's sort of intrinsic in
- 8:34Speaker 4: the nature of the technology to have this misalignment problem.
- 8:38Speaker 4: Even if we make it better, it's unlikely to completely
- 8:41Speaker 4: go away at least anytime soon. So it's very very
- 8:44Speaker 4: powerful and making you feel like, wow, this is amazing,
- 8:47Speaker 4: I'm getting so much value. But then in the end,
- 8:50Speaker 4: a lot of times the value becomes more mercurial and
- 8:53Speaker 4: difficult to actually realize as a business. That's not to
- 8:56Speaker 4: say you can't, it's just that the road to get
- 8:58Speaker 4: there isn't. And that's what the book is really about,
- 9:02Speaker 4: which is I go through why this happens and what
- 9:06Speaker 4: companies need to invest in to be able to do
- 9:10Speaker 4: AI driven development or agentic work successfully. And that's really
- 9:14Speaker 4: the hard part is that the early stage, that generation stage,
- 9:19Speaker 4: that's what they're good at. But making sure that it's
- 9:21Speaker 4: right and right consistently requires infrastructure and work and development
- 9:26Speaker 4: that most companies don't have because they've never had a
- 9:29Speaker 4: reason to do it. The fact that humans were doing
- 9:31Speaker 4: the process meant that the verification and making sure it
- 9:34Speaker 4: was good and making sure it was reliable and consistent
- 9:37Speaker 4: was baked into all that time that humans spent. So
- 9:41Speaker 4: even if you have a human pigly an AI like
- 9:44Speaker 4: part of how if you've ever been a developer, when
- 9:46Speaker 4: you're building something for a customer or client internally whatever
- 9:50Speaker 4: that process that may take a week or month or
- 9:53Speaker 4: however long, you're asking a lot of questions, you're getting feedback,
- 9:56Speaker 4: you're iterating, you're thinking about not only the problem and
- 9:59Speaker 4: how you're solving it, but also who you're solving it for.
- 10:02Speaker 4: And that process behind the scenes, not just the code,
- 10:05Speaker 4: is what drives a lot of the final value. And
- 10:07Speaker 4: if that all gets short circuited by just having the
- 10:10Speaker 4: AI pump out something that looks good without people thinking
- 10:13Speaker 4: about it a lot, then you don't have the same results.
- 10:16Speaker 4: So I give companies a process and a method that
- 10:20Speaker 4: they can and some tools they can use to understand
- 10:24Speaker 4: how to approach AI and to make sure they get
- 10:26Speaker 4: ROI and I mean, this is a tough problem. You've
- 10:28Speaker 4: seen in the news recently Uber Microsoft both pulling back,
- 10:32Speaker 4: both saying, hey, this is hard to justify on a
- 10:35Speaker 4: financial basis. One other way of looking at that, if
- 10:39Speaker 4: you have say some engineers, and you spend thirty percent
- 10:43Speaker 4: of their salary on tokens and you go thirty percent faster,
- 10:46Speaker 4: you're not actually more efficient, You're just spending more money.
- 10:49Speaker 4: You could just hired more developers. And so finding the
- 10:53Speaker 4: actual return on investment finding the efficiency is not as
- 10:56Speaker 4: simple as people want it to be, which is to
- 10:58Speaker 4: just throw AI at the problem. That doesn't work. And
- 11:01Speaker 4: I think that's pretty clear.
- 11:04Speaker 3: Absolutely will come to your book also.
- 11:07Speaker 2: But then in this sense, it almost looks like that
- 11:10Speaker 2: AI is in beta stage itself for the companies to
- 11:15Speaker 2: actually implement it and to bring it out to the
- 11:19Speaker 2: implementation stage. Actually it has. It is a lot of
- 11:23Speaker 2: work for companies. So whose job is it Whose job
- 11:26Speaker 2: is it to make it implementable. Is it the companies
- 11:30Speaker 2: who are paying money or is it the AI creators
- 11:33Speaker 2: who have to actually make sure because see, we haven't
- 11:36Speaker 2: still come out of this whole thing called hallucination.
- 11:39Speaker 3: That has come down, The talk has come down a bit,
- 11:41Speaker 3: but that's.
- 11:42Speaker 2: Still I guess is there. Then there is whole talk
- 11:45Speaker 2: about you know, incorrectness. They themselves keep on saying AI
- 11:49Speaker 2: can be incorrect. And then the psycho fancy part is
- 11:52Speaker 2: already very much you know, dominant in terms of every
- 11:56Speaker 2: evening a simple person wants to do that search. AI
- 12:00Speaker 2: can lead you to a garden, you know, path and
- 12:03Speaker 2: lead you to nowhere. So keep us understand from that point,
- 12:07Speaker 2: how do you solve this problem? Because for companies it's
- 12:10Speaker 2: an extra amount.
- 12:11Speaker 4: Of what that's correct. And the companies that have had
- 12:16Speaker 4: tremendous success with AI, I view it as they were
- 12:20Speaker 4: kind of in the right place at the right time
- 12:22Speaker 4: for other reasons. They had built the infrastructure that they
- 12:26Speaker 4: basically needed already without knowing AI was coming because of
- 12:30Speaker 4: regulatory or other concerns, something intrinsic to their business model,
- 12:33Speaker 4: or their philosophy or their culture, and they were able
- 12:36Speaker 4: to get results. So it's like the top couple percent
- 12:38Speaker 4: of companies have great results. For everyone else, it's flat.
- 12:42Speaker 4: So in order to get the results you need from AI,
- 12:45Speaker 4: you need to do that infrastructure for your business now
- 12:48Speaker 4: to answer your question about should it be like anthropic
- 12:50Speaker 4: or open AI. Providing this, they can improve their models,
- 12:54Speaker 4: but anthropic and open AI, at least right now, can't
- 12:58Speaker 4: tell you exactly what is correct for your business. If
- 13:01Speaker 4: they could do that, then you wouldn't have a business.
- 13:03Speaker 4: There would just be opening eye and anthropic. It would
- 13:05Speaker 4: be the end of everything, which is not going to happen.
- 13:09Speaker 4: It's just it's too much to do to all for everyone,
- 13:12Speaker 4: so that's not going to happen. They will keep making
- 13:14Speaker 4: their AI better as much as they can, making it
- 13:18Speaker 4: less psychophantic, making it less accurate, making it hallucinate less,
- 13:21Speaker 4: but making that problem less bad. While it limits the
- 13:25Speaker 4: problem somewhat, doesn't make it go away, and it doesn't
- 13:29Speaker 4: guarantee and that's the important way. It doesn't guarantee that
- 13:32Speaker 4: it goes away for your business and your life. So
- 13:35Speaker 4: the people who have to be responsible for that infrastructure
- 13:38Speaker 4: are the people who are using AI and their business,
- 13:41Speaker 4: and that what really surprises them is the need for
- 13:44Speaker 4: that infrastructure. And the second surprise is that it's actually
- 13:47Speaker 4: quite It's not something you can currently get off the shelf.
- 13:50Speaker 4: You can't just be like, yo, give me, give me
- 13:52Speaker 4: a magic AI verification infrastructure for my business, whether it's banking, healthcare, whatever.
- 13:58Speaker 4: And that's part of why you're seeing things like legal
- 14:01Speaker 4: AI getting a lot of funding, because they're saying, I'm
- 14:04Speaker 4: going to go attack this problem in this one little
- 14:07Speaker 4: tiny space and fix it because you have to. While
- 14:10Speaker 4: there are generic things you can bring to bear, you
- 14:12Speaker 4: still need to learn how to apply them. And most
- 14:15Speaker 4: companies don't have any of the experience with the AI
- 14:18Speaker 4: or all the technologies attached to AI like retrieval, augmented
- 14:21Speaker 4: generation to know how to do that properly. And of
- 14:24Speaker 4: course the people with that experience are rare and very
- 14:26Speaker 4: expensive and may not be available and may not be
- 14:29Speaker 4: affordable even if they.
- 14:30Speaker 3: Are absolutely so.
- 14:34Speaker 2: How do companies, then, you know, get the present version
- 14:38Speaker 2: of so called best versions of AI to work for them?
- 14:42Speaker 2: How have you dealt this part in the book? And
- 14:45Speaker 2: is it a lot of problem or is it a
- 14:48Speaker 2: small bit of problem for companies?
- 14:50Speaker 3: Is it very costly for them?
- 14:53Speaker 4: Because extremely costly I mean, if you look at like
- 14:56Speaker 4: Uber right, they have a tremendous engineering culture, a huge team,
- 14:59Speaker 4: and the recently the COO was in the news saying
- 15:01Speaker 4: that he wasn't seeing impact, he wasn't seeing a value
- 15:05Speaker 4: from the AI. So if and Microsoft is dial backed,
- 15:09Speaker 4: So these are people with tremendous resources who are still
- 15:12Speaker 4: sometimes struggling. And if you're a smaller company then you
- 15:15Speaker 4: can struggle. But there are ways forward the putting. So
- 15:21Speaker 4: my book does have ways to go about this. There's
- 15:24Speaker 4: a measurement of how to actually check ROI, which is
- 15:26Speaker 4: a problem, but I also propose a model where you
- 15:30Speaker 4: use the AI the right way and then has three vertexes.
- 15:33Speaker 4: There is the one where you look at the costing,
- 15:35Speaker 4: which is a cost for accepted change, and this is
- 15:37Speaker 4: for doing software development with AI. But there's a similar
- 15:40Speaker 4: model for agentic work. But if you can't do just
- 15:42Speaker 4: software development, you don't try to do agentic work. That's
- 15:45Speaker 4: like another level of difficulty. And then the other vertex
- 15:48Speaker 4: is intent clarity. So you know you can use AI
- 15:51Speaker 4: to build something, how do you know you're building the
- 15:53Speaker 4: right thing? You need to use the AI to maximize
- 15:57Speaker 4: the value of what you're building. By making sure you
- 16:00Speaker 4: build the right thing by leveraging its ability to prototype,
- 16:03Speaker 4: so you acknowledge that AI does not build a final
- 16:05Speaker 4: product magically, but you can rapidly prototype to find out
- 16:10Speaker 4: what you want and lock down that clarity. And this
- 16:14Speaker 4: is actually being kind of formalized to some extent, this
- 16:17Speaker 4: concept of loop engineering, where you're no longer even prompting
- 16:19Speaker 4: the I. You're trying to specify sort of a target
- 16:22Speaker 4: state and adapt and then have the AI achieve that
- 16:25Speaker 4: state as you iterate. And then I have a verification
- 16:28Speaker 4: quality vertex, which is, once you've figured out what you
- 16:31Speaker 4: want to build and you can verify with yourself that
- 16:34Speaker 4: it's the correct thing to build, then it's how well
- 16:36Speaker 4: did you build it? How do you know it's safe
- 16:38Speaker 4: to give to your customers? And you know, if you're
- 16:41Speaker 4: in a regulated industry like healthcare or banking or something,
- 16:43Speaker 4: then that bar is very very high. If you're building
- 16:46Speaker 4: an internal dashboard, maybe not so much. But I give
- 16:49Speaker 4: some metrics there that can be used, and some approaches
- 16:51Speaker 4: and some technology, but all of these are evolving over time.
- 16:56Speaker 4: So but for most small businesses, if they approach this
- 17:00Speaker 4: knowing that they're going to have to invest in the infrastructure.
- 17:02Speaker 4: It is a solvable problem, even though it will take
- 17:05Speaker 4: some time. The other advice I give to companies is
- 17:08Speaker 4: to kind of walk before you run. A lot of
- 17:10Speaker 4: times there's been people who've gone to companies and said,
- 17:13Speaker 4: everyone uses AI. Now here's some money, go burn tokens,
- 17:17Speaker 4: and then they spend too much money and then say,
- 17:19Speaker 4: oh god, stop stop, stop, stop burning all the tokens,
- 17:22Speaker 4: and that neither of those is the right way forward.
- 17:25Speaker 4: The right way forward is to take a small unit
- 17:30Speaker 4: inside your company with a low blast radius where the
- 17:33Speaker 4: damage if AI does something wrong is small. Use people
- 17:36Speaker 4: who are passionate about that technology, roll it out, learn
- 17:40Speaker 4: as an organization how to roll this out, build the
- 17:43Speaker 4: verification infrastructure you need, start measuring all the metrics I
- 17:47Speaker 4: give in my book, and then do the cost analysis
- 17:50Speaker 4: so you know where you're weak because you can use
- 17:52Speaker 4: that to verify that, and then build the infrastructure for
- 17:56Speaker 4: yourself at the small scale, test it out, and start
- 17:59Speaker 4: scaling out with those lessons learned. And this is what
- 18:02Speaker 4: I suggest and tell companies, because I've had executives that
- 18:05Speaker 4: large companies come to me and say, oh my god,
- 18:07Speaker 4: I read your book. This is exactly what's happening. We've
- 18:10Speaker 4: got hundreds of open pull requests and things for code,
- 18:13Speaker 4: but I don't know if I can release any of
- 18:15Speaker 4: these production. What do I do? And I have to
- 18:18Speaker 4: tell them, Okay, well you need to slow down, like
- 18:20Speaker 4: pick one part of your company, pick those pull requests,
- 18:23Speaker 4: pick you know, how do you start verifying those? Figure
- 18:26Speaker 4: out how you can do that without using a lot
- 18:29Speaker 4: of human time. And there's a bunch of technologies and
- 18:31Speaker 4: suggestions in the book on how you can do that,
- 18:33Speaker 4: and then just do the work to build that infrastructure
- 18:37Speaker 4: for yourself. You cannot. This is one of the things
- 18:39Speaker 4: I say in the book. You can't really skip to
- 18:41Speaker 4: the end. People want to, but that's not really there yet.
- 18:44Speaker 4: I mean, I'm thinking the future technology will develop. I'm
- 18:48Speaker 4: looking at a lot of things that I'm working on
- 18:50Speaker 4: right now to make that better, but it's not there yet.
- 18:54Speaker 2: Absolutely. What about the psycho fancy part? Can that be
- 18:57Speaker 2: taken care of? Because if you say I try to
- 19:00Speaker 2: be strict, then they will try to be strict and
- 19:03Speaker 2: make you again be acting like a psycho fed How
- 19:06Speaker 2: that is.
- 19:08Speaker 4: A fantastic question, And I'm really glad you asked because
- 19:11Speaker 4: it will make you happy either way, but you can
- 19:13Speaker 4: turn it against itself. So if you say build this,
- 19:16Speaker 4: and then you ask a completely fresh agent context, hey,
- 19:19Speaker 4: I got this thing, tell me what's wrong with it?
- 19:22Speaker 4: Attack it help me find out all the things that
- 19:24Speaker 4: are bad and wrong. Right, Well, it wants to make
- 19:27Speaker 4: you happy, so it's going to find all the things
- 19:28Speaker 4: that are bad and wrong. This is actually kind of
- 19:31Speaker 4: how AI is good at cybersecurity. It's good at finding
- 19:33Speaker 4: these complicated connections and it will tell you to make
- 19:36Speaker 4: you happy. And then not all of those are always accurate.
- 19:39Speaker 4: But let's say eighty ninety percent of the things that
- 19:42Speaker 4: finds that are bad are bad, right, and then you
- 19:45Speaker 4: can fix them. And you can do this over and
- 19:48Speaker 4: over again. And so what I've noticed in my own
- 19:50Speaker 4: development is if I'm using AI for development for software engineering,
- 19:55Speaker 4: maybe ten to thirty percent of the tokens I spend
- 19:58Speaker 4: and the time I spend is actually building the software.
- 20:00Speaker 4: The other sixty seventy percent is doing exactly this. It's
- 20:03Speaker 4: running those red teams and running those verification loops and
- 20:08Speaker 4: building infrastructure. So I'll build out all these all this
- 20:12Speaker 4: extra code that acts as a test harness to make
- 20:14Speaker 4: sure that I can keep things from aggressing, not just
- 20:16Speaker 4: unit tests but much more sophisticated smoke tests and things
- 20:19Speaker 4: to know that nothing broke. So with the psycho fancy,
- 20:23Speaker 4: you can have it attack its own ideas and then
- 20:25Speaker 4: you scale that same concept of habit attack itself to
- 20:29Speaker 4: the rest of those stack. But then that does cause costs. Right,
- 20:33Speaker 4: You're going to burn a lot more tokens than just
- 20:36Speaker 4: build me something. Absolutely absolutely balance, Yeah.
- 20:43Speaker 3: Absolutely, Brent.
- 20:44Speaker 2: One more thing you see now this the more AI
- 20:48Speaker 2: has become very political. So one country can say, because
- 20:52Speaker 2: I own a lot of these companies are in my country,
- 20:55Speaker 2: so I will you know because of security results right now,
- 20:59Speaker 2: which is happen with my thoughts and fable five and
- 21:02Speaker 2: all those things.
- 21:03Speaker 1: Yep.
- 21:03Speaker 2: Then for businesses it, how should they look at it?
- 21:07Speaker 2: Because even though it seems to be a global world,
- 21:09Speaker 2: it is also a growing increasingly political world.
- 21:13Speaker 3: In terms of AI. How do businesses get.
- 21:17Speaker 2: Rid of this this thing you know called politics in
- 21:21Speaker 2: artificial intelligence?
- 21:23Speaker 4: I don't think you can get rid of it, but
- 21:24Speaker 4: you can hedge. I personally have subscriptions to Claude, to
- 21:30Speaker 4: open AI codex and to the Chinese GLM models, and
- 21:36Speaker 4: you know I haven't been denied those models because I'm
- 21:38Speaker 4: an American, But I keep all three and I run
- 21:41Speaker 4: workloads against all three, and I regularly switch between them
- 21:45Speaker 4: just to keep you know, a clear view of where
- 21:47Speaker 4: they're good and where they're bad, because they can have
- 21:49Speaker 4: a different vibe, and if you're a business building on AI,
- 21:53Speaker 4: it's worth building the extra infrastructure again to switch out.
- 21:57Speaker 4: So if you've built stuff on top of Claude, pay
- 22:01Speaker 4: the few tokens to regularly test with Codex and make
- 22:04Speaker 4: sure that you can still get what you need out
- 22:07Speaker 4: of Codex, and then pay the few you know, the
- 22:09Speaker 4: few dollars go to GLM or use deep seek or something.
- 22:12Speaker 4: Make sure that you can still survive without any of
- 22:14Speaker 4: these models. More and more the model capabilities, while they
- 22:18Speaker 4: keep getting better, they've also gotten good enough for most people.
- 22:22Speaker 4: And there's increasing talk that the agent harness and the
- 22:26Speaker 4: infrastructure around the agent is becoming more relevant than the
- 22:30Speaker 4: agent itself. So if you carefully test whatever you build
- 22:33Speaker 4: on top of these agentic solutions so that you know
- 22:36Speaker 4: that it can run across multiple providers, then you're in
- 22:40Speaker 4: a safe spot. And while it's very expensive with GLM
- 22:45Speaker 4: five two, it is plausible while expensive, Like I said,
- 22:49Speaker 4: to build if you needed to, your own cluster of
- 22:53Speaker 4: like H one hundreds or B one hundreds and run
- 22:55Speaker 4: it locally if you needed to. And that's if you're
- 22:59Speaker 4: assuming you need a tier model. If you don't need
- 23:01Speaker 4: a frontier model for your work, which is even better
- 23:04Speaker 4: and much cheaper than that can be run on increasingly
- 23:06Speaker 4: affordable machines that you own, and then you can ignore politics.
- 23:11Speaker 4: You own the machine, the models on your machine, You
- 23:14Speaker 4: control everything about it. If you're willing to take on
- 23:17Speaker 4: the infrastructure overhead and the monitoring and management overhead, then
- 23:22Speaker 4: that's yours completely, yours, maybe even on prem and you
- 23:25Speaker 4: don't have to worry about that stuff anymore. I think
- 23:27Speaker 4: it's going to happen more and more going forward.
- 23:29Speaker 3: Honestly, absolutely, Brent. Absolutely.
- 23:32Speaker 2: One last question I have for you for the audience
- 23:35Speaker 2: is you know, in terms of you know, a bubble,
- 23:39Speaker 2: A lot of people aren't now talking about all these
- 23:43Speaker 2: things you know more of AI as a bubble, not
- 23:47Speaker 2: maybe perhaps in terms of technology, but in terms of
- 23:50Speaker 2: financing and all those things and trillion dollar companies and
- 23:54Speaker 2: all those things out you So somewhere's this butt bubble
- 23:58Speaker 2: in one way or the other can burst or will burst,
- 24:03Speaker 2: and if it does. If it does, the impact will
- 24:07Speaker 2: be huge on financial systems because of this whole financing
- 24:11Speaker 2: and all those things. Or even in terms of companies
- 24:14Speaker 2: who have you know, implemented and and and and may
- 24:17Speaker 2: not be able to use it in the same way
- 24:20Speaker 2: that they are looking at it. Do you think companies
- 24:23Speaker 2: should have have some sort of a system where one
- 24:28Speaker 2: day they may have to just reterace and and look
- 24:31Speaker 2: look a bit back, you know, backwards, and see if
- 24:35Speaker 2: they can you know, find more solace there and other
- 24:38Speaker 2: than just running with the artificial intelligence of today.
- 24:42Speaker 4: Well, I don't think companies have to worry about, even
- 24:46Speaker 4: if there's a financial crest, losing access to the technology
- 24:51Speaker 4: in general, because of the open weight models, this isn't
- 24:55Speaker 4: going to go away. So companies that are building with AI,
- 24:58Speaker 4: if they follow my and test it against multiple providers
- 25:02Speaker 4: and especially the open weight providers, and they can make
- 25:06Speaker 4: their business processes work with those, then those won't go away.
- 25:11Speaker 4: The way I would look at the AI market in general,
- 25:13Speaker 4: and I'm not an investment professional, so this is not advice,
- 25:16Speaker 4: but the way I see it is there's two or
- 25:20Speaker 4: three pieces to that ecosystem. One is the model providers
- 25:24Speaker 4: infeloves like Anthropick and open Ai. They may struggle if
- 25:29Speaker 4: China continues to produce these open weight models that are
- 25:31Speaker 4: a tenth to price and good enough, and then it's
- 25:33Speaker 4: much harder for them to defend their valuation. What's a
- 25:37Speaker 4: little bit different, though, are the hyper scalers and the
- 25:39Speaker 4: people making hardware, Because whether you're running open weight models
- 25:43Speaker 4: or or anthrowpick or whoever, you still need a lot
- 25:46Speaker 4: of GPU power. Like so, like GLM five point two
- 25:49Speaker 4: takes a terabyte of RAM to run. That is a lot.
- 25:52Speaker 4: That is hundreds of thousands of dollars of hardware to
- 25:56Speaker 4: run one instance of GLM five two at max. So
- 25:59Speaker 4: if open AI and Anthropic vanish tomorrow and everyone moved
- 26:02Speaker 4: to that, you're still talking about millions or billions of
- 26:05Speaker 4: dollars of infrastructure spend to scale those models up as
- 26:10Speaker 4: everyone uses those. But at a certain point we will
- 26:13Speaker 4: hit saturation and then the hardware won't make sense either.
- 26:16Speaker 4: But no one really knows when that comes. But I
- 26:18Speaker 4: don't see an imminent crash of that, but you know,
- 26:24Speaker 4: whoever does, right, That's the thing about these bubbles. But
- 26:27Speaker 4: the technology won't go away. I don't think anyone has
- 26:29Speaker 4: to worry about the tech going away. So if you're
- 26:31Speaker 4: not writing your retirement on this, then it'll be fine.
- 26:36Speaker 3: Absolutely, absolutely, Brent.
- 26:38Speaker 2: There's so much to learn about all this from you,
- 26:41Speaker 2: and I'm sure a lot of people would want to
- 26:42Speaker 2: learn more about you from your book. What's the best
- 26:47Speaker 2: way for them to do so?
- 26:50Speaker 4: So my book is on Amazon. You can just search
- 26:53Speaker 4: for my name and that it'll come up pretty quick. Also,
- 26:56Speaker 4: you can find me on LinkedIn. I'm very accessible. If
- 26:59Speaker 4: your company is ruggling or you're trying to learn, just
- 27:01Speaker 4: reach out. I'm glad to.
- 27:02Speaker 3: Talk wonderful with this.
- 27:06Speaker 2: It's set up on this very special edition of the
- 27:09Speaker 2: kJ Masterclass Life. Thank you so much indeed for joining us.
- 27:13Speaker 4: Thanks for having me