Latest / First Trust ROI Podcast / Ep 73 | Mandeep Singh | Is the SaaS-Pocalypse Over, and Where Are We in the AI Capex Cycle? | ROI Podcast
Transcript
- 0:13Ryan: Where are we in the AI capex cycle?
- 0:16Ryan: Is this apocalypse over?
- 0:18Ryan: And what's the next industry that will be boosted or perhaps disintermediated by artificial intelligence?
- 0:25Ryan: Joining me to make sense of these questions and more is Mandeep Singh, global head of technology research at Bloomberg Intelligence.
- 0:32Ryan: Thanks for joining us on this episode of the First Trust ROI podcast.
- 0:36Ryan: Mendeep, thank you again for joining us on the podcast.
- 0:40Ryan: I really enjoyed uh our conversation.
- 0:41Ryan: I think it was about a year and a half ago that you and I first chatted.
- 0:45Ryan: Um but there's a lot that's happened since then.
- 0:48Ryan: Um there's been literally hundreds of billions of dollars of CapEx announced, and um it seems all of the AI companies are pursuing what they call uh AGI or general intelligence.
- 1:03Ryan: And so that's my first question for you.
- 1:05Ryan: Um, have we reached a point where AGI has been reached?
- 1:09Ryan: Um are we going to reach that point?
- 1:11Ryan: What's your perspective on that?
- 1:12Mandeep: Well, I don't think there is any official definition of AGI yet, so uh I'll leave it to the frontier labs to agree on, you know, uh some something that they can say, these are the parameters, this is how we accomplish AGI, and we haven't heard that yet.
- 1:31Mandeep: But what has transpired over the past, I would say, 12 months or so is one, we have seen this huge inflection point when it comes to adoption of coding agents, and it feels like uh, you know, if I had to come up with my own definition of AGI, it would definitely include functionality like that where you can code anything with a prompt.
- 1:59Mandeep: And you know, the computer can understand your instructions, you can be very descriptive about what you uh want to do, and it's all about uh ideas and creativity and and uh these tools, which is what I think would be part of that AGI definition, facilitate that creativity.
- 2:19Mandeep: And to me, uh that is where the industry will keep improving.
- 2:24Mandeep: These models are constantly improving, and at some point we'll say, okay, they can do X, Y, and Z, and this is the definition of AGI.
- 2:32Ryan: Yeah.
- 2:32Ryan: Well, that brings me to the SAS pocalypse, right?
- 2:35Ryan: The software as a service companies were all re-rated this year because those coding agents are so good that uh I think a lot of the fear was that every time um Anthropic made a new announcement of a version of Claude that um that consumers, enterprises were just going to replace their um subscriptions to with vibe-coded software.
- 2:59Ryan: Um is is uh that process uh over?
- 3:03Ryan: Is the SAS the SAS apocalypse, as it's been coined, um, done?
- 3:07Ryan: Are we gonna we have begun to see a bit of a rebound in some of those software companies?
- 3:11Ryan: Um where do we stand now?
- 3:14Ryan: Is is are we through that?
- 3:15Ryan: Are we going to see a uh rebound of these companies?
- 3:18Ryan: I'm I'm asking to predict the future there, but where are we in the SaaS Apocalypse?
- 3:22Mandeep: I mean, look, this is one of the biggest technology shifts that we are witnessing, and we have witnessed quite a few over the past two, three decades, you know, when you think about the on-prem to cloud shift, the desktop to mobile shift.
- 3:37Mandeep: Like these were big changes.
- 3:38Mandeep: And the biggest probably was the shift to internet, you know, the uh fact that we have come such a long way with just uh deploying internet for the masses across the globe.
- 3:50Mandeep: And uh technology shifts always take, you know, a very long time to play out.
- 3:57Mandeep: So to answer your question, uh, you know, we haven't really seen uh what are the software segments that would be uh impacted in a big way versus the ones that will uh be, you know, whether the system of records that will maintain their value over time.
- 4:15Mandeep: I think the jury is yet to be out in terms of you know uh what will truly get disrupted and you know it will change forever versus where these companies may add some incremental functionality using the tools like coding agents.
- 4:32Mandeep: And I think that's where uh there will be a lot of back and forth.
- 4:37Mandeep: I I don't think we will have a moment anytime uh in the near term where we can ascertain that okay, this segment is disrupted forever.
- 4:47Mandeep: I mean, a lot of these software companies, uh especially you know, when you think about horizontal versus uh vertical software, on the vertical software side, they have so much domain knowledge, which is very proprietary to their software, that these models cannot train on that data.
- 5:05Mandeep: Whereas on the horizontal side, yes, we have a lot of samples of code, you know, open internet data includes a lot of that.
- 5:13Mandeep: So these models have become good at that.
- 5:15Mandeep: And and so we have to think in terms of where is the data available for these models to train on, whereas uh, you know, the things where there is no data available for everyone to train on.
- 5:28Mandeep: I mean, these models are still brute force in nature.
- 5:31Mandeep: They're trained on data, they don't have their own intelligence.
- 5:35Mandeep: We're not talking about, you know, continuous intelligence like human beings have.
- 5:39Mandeep: They have to, these models have to be constantly trained.
- 5:42Mandeep: You need more and more data, and that's how I would uh you know go about figuring out which uh software segments would truly get disrupted versus where the incumbents may have a very good shot at adding AI features and and they may turn out to be fine.
- 5:59Ryan: So there's some competitive moat, is what I'm hearing you say, um, with maybe some of the the systems of record that are necessary and things like cybersecurity, for example, is what I've um sort of ascertained, but also um the value of data, uh maybe some network effects.
- 6:17Ryan: Um is is that um is that what will maybe prevent certain companies from being being disintermediated?
- 6:26Ryan: And I guess the question on the minds of a lot of investors is you know, what's what's next to real what's the next shoe to drop?
- 6:32Ryan: What's the next area, whether it's software or not, that will be disintermediated by it?
- 6:37Mandeep: I mean, look, right now we are looking at a lot of back office work when it comes to business processes, you know, how things are done in the accounting world, payroll world, uh, you know, even CRM, supply chain management.
- 6:53Mandeep: So everything is on the table in terms of reimagining uh, you know, business processes using these new tools, which certainly make you efficient in certain aspects of the job, but I don't think they can do things on an end-to-end basis.
- 7:10Mandeep: And this is just the B2B side of things, even on the consumer side, when you look at you know uh what people are using these tools for, it's great for search.
- 7:21Mandeep: I I think the fact that you can do long-form searches now has been just revolutionary in terms of driving engagement to a lot of internet platforms.
- 7:33Mandeep: I would imagine you know, some parts of e-commerce would change because of our ability to search in such uh a descriptive way.
- 7:42Mandeep: So, from that perspective, it's still early days in terms of reimagining things, using the capabilities of these LLMs.
- 7:51Mandeep: But uh I I think uh the end-to-end aspect is where it will take much, much longer because at the end of the day, uh, you know, existing software works, and it works very well, you know, using the way currently things are done.
- 8:09Mandeep: It's digital in nature, and that's the basis for these LLMs to really train agentic AI or whatever new concept they come up with.
- 8:20Mandeep: At the end of the day, they have these models have to be trained, and they are trained on data that is generated by these systems that have been in place over the last two decades.
- 8:30Ryan: I think um there's also the concern or the fear that agentic AI will displace labor.
- 8:36Ryan: Maybe there'll be layoffs, and there's been some announcements of layoffs, but on the other side of the coin, I've also seen um some job postings have have risen for software engineers.
- 8:48Ryan: So where do you where is that where do we shake out?
- 8:50Ryan: Is that just a replacement of one job function for another?
- 8:54Mandeep: I think clearly everyone is focused on productivity when it comes to the use of these tools, and uh we are seeing those productivity benefits show up when it comes to you know developers or anybody who who can code finds themselves to be more productive as a user of these tools.
- 9:14Mandeep: Customer service is another big use case, you know, whether it's uh just providing the plain old, you know, uh voice agent type of service or any other type of chatbot service that involves uh reading documentation from a bunch of databases and then answering promptly.
- 9:34Mandeep: And so all these are ways to make existing folks uh you know productive at what they are doing.
- 9:42Mandeep: Previously, they probably uh had to do a lot of manual work just in terms of copy-pasting stuff.
- 9:50Mandeep: Now the LLMs can do that with a prompt and and you know, pull in data from different databases, for example.
- 9:57Mandeep: So I would focus more on those efficiency benefits uh compared to the true end-to-end automation, which we will get over time.
- 10:07Mandeep: But as I said, AI needs to be trained, and for us to be able to capture the edge cases using LLMs, it's gonna take a while.
- 10:19Mandeep: I mean, you're not gonna get that in the first instance of training an LLM that it covers all the edge cases.
- 10:25Mandeep: Those have to be done in a very iterative way, which is what we are seeing with uh you know how the frontier labs are approaching this.
- 10:33Ryan: Yeah.
- 10:34Ryan: So we're hearing more and more um from the potential for potential um IPOs for some of these AI-related companies, major IPOs.
- 10:45Ryan: And from an investor standpoint, you know, a lot of the growth has already taken place in in scaling up in the private markets, but now these companies are going public.
- 10:54Ryan: Um what does that mean from a liquidity standpoint?
- 10:57Ryan: Is this going to pull capital away from some of the existing companies?
- 11:00Ryan: Um, and is that a short-term phenomenon?
- 11:03Ryan: I'm curious if you have any thoughts on the the impact of some of those large IPOs that are coming.
- 11:08Mandeep: I mean, you're right.
- 11:09Mandeep: Uh I I think we have never seen IPOs of this scale when it comes to you know the capital markets and what we are seeing now with uh SpaceX and then potentially with OpenAI and Enthropic, all of them uh likely to be trillion-dollar companies.
- 11:27Mandeep: Yes, you need to find capital, you know, uh even though uh these companies have done a great job of raising money in the private market.
- 11:35Mandeep: But uh, you know, when it comes to the growth uh in terms of what Enthropic and OpenAI have done over the past uh you know six to twelve months, it feels like this is the true hypergrowth phase of these companies.
- 11:53Mandeep: And that's where uh I mean the valuation seems rich, but at the same time, you are seeing that top-line growth to justify, you know, some of the high uh multiples that these companies are likely to get when they uh, you know, uh go public.
- 12:09Ryan: Yeah, it seems like when I was starting out as an analyst, we we talked about the law of large numbers, yeah, and that doesn't really exist anymore.
- 12:17Ryan: You you see a company like NVIDIA, which is expected to continue to have these growth rates of revenue and earnings that are like a small cap.
- 12:27Ryan: Um is the law of l large numbers is that not a thing anymore?
- 12:32Mandeep: Well, so you have to look at it from the standpoint of uh, you know, a huge infrastructure build-out.
- 12:38Mandeep: And we have had uh, you know, a few occurrences of that when it comes to industrial revolution, the build out of the railroads.
- 12:46Mandeep: So that's where it is of that magnitude in terms of you know being part of a country's infrastructure and you know, this data center build-out and onshoring of some of these assets.
- 13:00Mandeep: This is uh, you know, a trend that is led by governments, sovereign entities, and then as part of that, you know, you've got bellwether companies like Nvidia and the hyperscalers here who have the deep pockets and the balance sheet to finance some of this build-out.
- 13:20Mandeep: I I think what's hugely different from uh the build-out of the internet uh is that back then you didn't have companies like the what we have with the hyperscalers right now.
- 13:32Mandeep: And so these companies are putting their own capital in terms of building the infrastructure, whereas back then it was mostly government sponsored, and you know, telecom uh companies had to take a lot of debt, you know, to do that.
- 13:46Mandeep: And and so that's where this build-out is somewhat different.
- 13:50Mandeep: And at the same time, it makes sense that you know these companies feel there is a massive growth opportunity, they are uh willing to put their own capital at risk.
- 14:02Mandeep: The shareholders seem to be supportive so far, even though you know everyone sees uh how much these companies are investing in buying NVIDIA chips, for example.
- 14:13Mandeep: So there is one company, or you know, if you include Micron and some of the memory companies, two, three companies that are really benefiting from this huge spend uh which is coming from all the hyperscalers, but at the same time, everyone sees growth on the other side when it comes to the AI products that will come out of it, whether it's the model companies or even the cloud growth for the hyperscalers.
- 14:37Mandeep: So from that perspective, there is a flow through of that capex that you can see in the ecosystem.
- 14:43Ryan: Do you have a sense of where we are in that cycle?
- 14:46Ryan: Are we midpoint?
- 14:48Ryan: Are we late?
- 14:49Ryan: Are we early in the the capex cycle for building up the data centers in that that ecosystem?
- 14:56Mandeep: I think uh I mean we have uh an estimate.
- 14:59Mandeep: Uh we at BI uh Bloomberg Intelligence, we do our own estimates, and uh so we've been uh modeling the CapEx from the hyperscalers to hit uh $1 trillion over the next two years.
- 15:13Mandeep: Right now they are somewhere around $750 billion expected for 2026, and we have had three years of 70% growth.
- 15:22Mandeep: So we were, you know, the normalized levels previously was around $200 to $250 billion.
- 15:28Mandeep: Right now we are at $750.
- 15:30Mandeep: We expect it to go to $1 trillion.
- 15:33Mandeep: And look, maybe the normalized CapEx after we have this huge build out will be closer to $500 billion.
- 15:41Mandeep: Because when you think about traditional cloud, that's how much you know companies used to invest.
- 15:47Mandeep: The the three major hyperscalers, Microsoft, Amazon, and Google used to invest with traditional cloud.
- 15:53Mandeep: With uh AI infrastructure, we have gotten a lot more players in the neo clouds as well as the hyperscale, the traditional cloud hyperscalers.
- 16:04Mandeep: And so it's a much bigger market now.
- 16:07Mandeep: And the normalized levels of CapEx, uh, if I model that you know five to ten years out, would be closer to $500 billion as opposed to $200 to $250 billion where we were with the traditional cloud.
- 16:20Mandeep: So, I mean, look, at the end of the day, AI is a lot more compute intensive than the traditional server, a website or you know, mobile workloads we used to have.
- 16:31Mandeep: And so from that perspective, there is a justification for higher capex for longer.
- 16:37Mandeep: But you're right, at some point we will hit a digestion period.
- 16:41Mandeep: We probably uh uh will see double ordering when it comes to, you know, when you see memory prices going up like this, uh parabolic, I'm sure people are uh, you know, double ordering with the hope that you they don't have to buy at that elevated price.
- 16:57Mandeep: So there will be capacity that will catch up to demand over the next two years, and that's when we'll talk uh about normalized capex.
- 17:07Mandeep: But for now, everyone wants to invest in this, and the supply shortages and the tight supply of components is is what's driving a lot of this capex increase.
- 17:17Mandeep: I don't think we would have had you know another 70% year of capex increase had it not been for the memory prices going up, you know, 30-40 percent over the past six months.
- 17:27Mandeep: So that's what's driving some of these capex increases.
- 17:31Ryan: So is that the the key constraint from um just the the fact that there's lack of supply, the supply hasn't caught up to the demand, um, or is it related to supplying power for some of the data centers?
- 17:45Ryan: I've that's the other thing that you hear quite a bit, that you know, all the electricity demand that's coming for these data centers, which is the more important bottleneck?
- 17:55Mandeep: I mean, look, everything is in short supply right now, and and it's pretty obvious that you know there are a lot of uh things that have to line up for these companies uh to be able to add supply, not just you know, uh for a specific component, but really to set up these data centers, which you think about the supply chain for these data centers, it's pretty complex.
- 18:19Mandeep: You know, it requires a lot of parts, and then everything has to be lined up properly for these data centers to go live.
- 18:27Mandeep: Because once you procure the GPUs, you want to make sure you're running the GPUs at 100% utilization, you get them live as soon as possible because guess what?
- 18:37Mandeep: Nvidia will release another architecture in you know nine to twelve months.
- 18:41Mandeep: So uh given the pace at which things are going, you really want everything to be done quickly, but obviously we know, you know, whether it's getting regulatory approvals for some of these data centers or just getting sourcing the power, a lot of that power is off-grid.
- 19:00Mandeep: I mean, we know the backlog for gas turbines is like 15 years.
- 19:04Mandeep: So all these are sort of adding up when it comes to the tight supply that I mentioned, you know, when it comes to onboarding this AI infrastructure.
- 19:15Ryan: Do you think that data centers have kind of developed a PR problem?
- 19:20Ryan: People see their utility prices going up, and it might not be related to the data centers, but that's the narrative.
- 19:28Ryan: And I'm I'm I'm always curious about what impact that might have if there is a PR problem.
- 19:33Ryan: It what's your perspective?
- 19:34Mandeep: Well, I mean, clearly uh suddenly we are talking about, you know, uh data centers consuming uh 10% of overall power, whereas previously it used to be low single digit.
- 19:47Mandeep: So you can see an inflection in power demand from data centers because of the fact that AI is a lot more compute intensive, and you know, traditional servers required uh, you know, 5 to 10 kilowatt uh per rack.
- 20:05Mandeep: The new AI servers require almost 10 times power.
- 20:09Mandeep: You know, the NVIDIA latest architecture, whether it's Blackwell or Rubin, it requires almost 100 uh plus kilowatt power per rack.
- 20:16Mandeep: So from that perspective, there is a clear correlation between uh you know AI requiring more power.
- 20:24Mandeep: But I would say these companies have been cognizant about, especially the hyperscalers, they have been talking about off-grid power for a very long time, and they have uh, as part of their CapEx investments, also invested in sourcing their own power, off-grid power.
- 20:44Mandeep: So from that perspective, I mean, uh obviously regulators will uh probably get involved in a much bigger way in terms of making sure the impact on consumers when it comes to electricity prices is not being felt uh because of the AI build out.
- 21:00Mandeep: But I I do think, you know, uh the high the US hyperscalers uh have done a good job in terms of just making sure that they allocate some portion of the CapEx when it comes to uh sourcing their own power.
- 21:14Ryan: Yeah.
- 21:14Ryan: So the other um the other power related, but but maybe not the most important thing that um we've heard more and more about with the IPO of SpaceX, people focused on what payload you can launch into space, and maybe you can put data centers in space because they'd have uh more access to solar power, and uh I'm curious what what you think.
- 21:37Ryan: I I I don't know the engineering well enough to know uh how far down the road, how far off on the horizon that that possibility is.
- 21:46Ryan: Um I'm curious, where do you think do we end up with orbital data centers, or is that something that's pie in the sky?
- 21:54Mandeep: I mean, look, uh whenever we have constraints, and you know, we talked about out a lot of constraints in terms of building these data centers, the terrestrial data centers, you know, whether it's power or component related.
- 22:08Mandeep: And that is where humans have been very creative through the history, you know, and we know Elon Musk has pioneered a lot of, you know, new concepts, whether it's electric car or in this case, you know, just the orbital data centers and where he thinks it could scale up.
- 22:30Mandeep: It's just a result of all the tight supply and you know the fact that we have a lot more demand than supply when it comes to the terrestrial data centers.
- 22:41Mandeep: And look, as a concept, I I think it it makes sense that we are talking about orbital data centers because it does solve for you know land where you require a very lengthy regulatory approval here.
- 22:54Mandeep: It it uh solves for cooling you don't have to spend as much on cooling infrastructure.
- 23:01Mandeep: The uh challenges are if you're talking about you know building a large size cluster for uh you know training these LLMs or even inferencing requires a much larger cluster and if something goes wrong with one chip how do you replace it?
- 23:19Mandeep: Right now this is uh being done you know by folks who are managing these data centers in a very efficient way so that it doesn't really slow you down and we know latency matters for a lot of these workloads.
- 23:33Mandeep: So that's where the maintenance problem is where I see the biggest challenge when it comes to deploying orbital data centers at scale.
- 23:42Mandeep: I do think you know it will be something that again I know SpaceX is investing a lot of their own money in that.
- 23:52Mandeep: They are doing their CapEx investments in that area.
- 23:56Mandeep: But it will be interesting to see if there are other companies like Google has also launched a pro uh project around orbital data centers.
- 24:05Mandeep: I just would want to see more dollars being allocated by multiple companies before we truly see an inflection when it comes to you know the deployment of orbital data centers.
- 24:17Ryan: So there's some hurdles but maybe don't bet against Elon Musk and his uh ability to be an entrepreneur and generate innovation and solve problems.
- 24:26Mandeep: Yeah I would never bet against the American entrepreneur.
- 24:29Mandeep: Yes.
- 24:31Ryan: So another big announcement that that uh happened earlier this year from Anthropic was um the the Mythos preview model that um that led to the project Glasswing and there's concerns about what does this mean for national security?
- 24:48Ryan: This is a really advanced model that can maybe um you know solve all the cryptographic uh issues and so I'm curious what what you think on where we are with respect to these advanced models.
- 25:04Ryan: Do they do they require more regulation?
- 25:07Ryan: If so how do you avoid um you know killing off the innovation with all that regulation any perspectives on Mythos preview project glasswing other advanced models?
- 25:21Mandeep: Yeah look what these companies have shown the frontier labs like Anthropic and OpenAI have shown is uh you know the scaling laws are still intact which basically means if you have a larger compute cluster if you've got more data to ingest these models would get better and better in terms of solving for different use cases across domains.
- 25:44Mandeep: And in this case you know we were talking about coding agents before well cybersecurity is another potential use case in terms of discovering vulnerabilities with all sorts of software that exists in the world today.
- 25:59Mandeep: And these models because of their innate knowledge of code and you know the ability that they can code so well they can also detect where are the vulnerabilities in the code.
- 26:12Mandeep: And so it doesn't surprise me that you know a Frontier lab has capability like that.
- 26:18Mandeep: I don't think it can be deployed at scale in terms of inferencing just because of the compute costs involved.
- 26:25Mandeep: In fact training the Mythos model based on some estimates costed more than a billion dollars just for one single training run.
- 26:34Mandeep: So deploying this you know for inferencing use cases would require a very hefty spend and so I I think they'll be very selective in terms of the use cases and look it does pose a lot of risk in terms of the geopolitical climate we are in and what it means in terms of sophisticated cyber attacks because once you expose all the vulnerabilities then you don't really have control on how hackers would end up using it.
- 27:06Mandeep: So that's where I think they did the right thing in terms of creating this industry consortium, the project Glasswing and they are uh you know working with uh folks to patch all the vulnerabilities the known vulnerabilities before this this gets released and I think that's the right approach.
- 27:24Mandeep: Yeah so you mentioned the high cost of um the the mythos um platform and other some of the more advanced AI platforms are uh also pretty expensive are are they ex expensive to not just train but also run and I I guess I'm curious about how the cost of tokens has evolved um do we reach at some point a cost that's maybe too high to actually um see the adoption rate that people expect well so the thing about uh the LLMs that have been released you know let's say a year back is their costs have come down almost 85 to 90 percent in a lot of the cases so think of you know when openai released GPT-4 and 01 if you look at the cost of serving those models the tokens uh per minute uh that has come down drastically and so from that perspective you do see uh that curve uh kind of go down in a way where if you use a model that's six to twelve months old you can serve it a lot cheaper it's just for the latest frontier model like Mythos which was just previewed that's where the costs are the highest and so it depends on the use case.
- 28:42Mandeep: If you see a high ROI use case where you can generate a piece of code that is worth a lot more, then it's worth paying up you know for for and that's why these companies charge by tokens and the whole model has shifted from you know subscriptions to consumption based on tokens because it's the quality of the tokens that really determines you know the output.
- 29:06Mandeep: And and so I I think uh you will see a lot of logic being applied to how does a query get routed to a particular model, whether we use an older model or a newer model, depending on what's required in terms of processing and the quality of tokens needed.
- 29:25Ryan: So for the consumer um how do they know which of those models they should be using to get the most efficient uh cost of tokens?
- 29:33Mandeep: I I don't think this would apply more at the consumer level this would be more uh an enterprise tiering whereas for consumers yeah you can say there could be a a version of open claw like agentic framework that they may want to use as their personal agent and then uh the you know the properties of that agent would be determined by which model that agent is using.
- 29:59Mandeep: So I'll give you an example if you're trying to use these LLMs as your personal scheduler you know managing your calendar and your appointments and whatnot maybe there is a tiering of LLMs that could work there because the basic search and summary can be done by you know an LLM that was released a year back.
- 30:22Mandeep: But when it comes to finding you know a lot more nuanced information and then curating something uh that hasn't been done in the past uh where you require the newer models that's when uh you know the feature can be turned on for an add-on you know that uh these consumers may have to pay so I do think there is tiering that's possible but right now the enterprises are the ones who are willing to pay for that tier token usage because how do you uh develop a subscription where you encapsulate all the model usage it's not possible I think companies have realized that even a flat $200 subscription is not enough to you know make it a profitable uh subscription so uh that's where companies have been shifting away from subscriptions both on the consumer and the enterprise side and it's more a consumption and uh you know the tiering based model when it comes to uh the use of these LLMs thinking about the physical application of some of the AI um whether it's robotics or uh EVs last time we spoke um at the beginning of last year uh it was right after the uh the big uh consumer electronics show and that was all the buzz was kind of surrounding some of the humanoid robots and other robotic applications.
- 31:53Ryan: Where do we stand now with respect to some of the physical AI have there been any big advancements?
- 32:00Mandeep: I think uh the natural foam factor would be you know voice-based interaction when it comes to uh even these physical devices and we are already seeing that in cars you know I mean uh uh Tesla has deployed their Grok LLM inside their cars we'll see more of that in terms of chatbot usage uh we're talking about a new version of Siri now and so if if uh Apple enhances Siri you will see that being deployed more broadly uh but overall I I think when it comes to humanoids I still think uh and robotics overall you need more time because the hardware is not ready it's not because uh the you know the physical device can understand uh the instructions it can because the LLMs have become so good and I'm guessing they will be able to distill it uh to uh a level where it can run locally on these devices the question is can the hardware execute the instructions that the LLM is able to uh provide and and that's where you know the form factor will probably evolve the you know the fine finer tasks that our hands are able to do I don't think these humanoids can do it yet and and look that's a hardware evolution that uh we'll have to go through uh to see a lot more of these as we see more of that physical part of AI do you do you expect that that would have more workloads kind of edge workloads um related to that and what would that mean from a hardware standpoint?
- 33:41Mandeep: I mean look right now everyone is looking to deploy these LLMs um on their laptops or even using Mac minis and you know what they're finding is one these LLMs require a lot of compute again AI is compute intensive and and then also the power side look uh when you are processing this amount of data and and you require this much compute there is a reason why cooling demand has gone up so much because uh you know when you are using this on the server side you need to cool these servers because they're using a lot more power.
- 34:20Mandeep: So how do you do that on your laptop?
- 34:23Mandeep: I mean what sort of fan do you have uh you know on your edge device or your phone to be able to cool the device in real time and and those those are practical I would say considerations when it comes to uh deploying LLMs on edge devices okay so as we think about the different um I mean we talked about disruption disintermediation whatever you want to talk about it uh what what industry what segment are we not thinking about that is um maybe we're underestimating the impact uh is there any any that you think that maybe the market's just got wrong at this point or maybe um you know when we talk again next year uh that we'll be looking back and saying oh my gosh we should have seen that coming I I think the impact of code uh and and coding agent is huge.
- 35:18Mandeep: So when I try to you know expand my aperture in terms of what that could do to a number of big end markets like e-commerce I mean it shouldn't surprise anyone that you know agent tick AI will have an impact on how we consume stuff, how we order stuff online and you know uh how some of uh the friction points can be removed with the help of technology.
- 35:50Mandeep: And so from that perspective I do see a lot of opportunities on that side uh and and we know e-commerce is probably the biggest addressable market uh out there when it comes to you know the uh tie-up with consumers so that's where I I do think of there will be pockets that will emerge where someone will find a big use case uh built on LLMs and that will just uh go viral and and uh you'll see very rapid adoption.
- 36:23Ryan: Okay.
- 36:23Ryan: Final question for you Mendy um so there will be a lot of people that watch this episode that are lay people.
- 36:31Ryan: They're not they're not specialists they're not um they don't understand the technology.
- 36:35Ryan: But for someone who's a lay person who really wants to gain maybe a basic understanding of what they actually are talking about with artificial intelligence.
- 36:44Ryan: Is there a resource or a book or you know some other resource that you would recommend for someone who just wants to get that basic sort of foundational level of understanding?
- 36:55Mandeep: I mean look uh the good thing with these chatbot interfaces now is they will personalize you know the education on any topic depending on how you are thinking about it, what is your level of understanding.
- 37:12Mandeep: So just engaging even with the chatbot interface is a great way to actually uh acquire knowledge.
- 37:20Mandeep: And you know as opposed to reading a certain prescribed text where certain concepts or terms may be completely foreign to you.
- 37:30Mandeep: You can uh ask that chatbot and engage in a way where you can have it unpack the terminology according to the way you want.
- 37:40Mandeep: And I find that to be very engaging for my personal use where there are times when you know certain terms are completely foreign to me.
- 37:49Mandeep: And I really want to like you uh it's one of the chatbot queries explain to me like a five year old or a 10 year old yeah that's perfectly fine way to engage and and you know uh really uh understand the basics which you may find to hard uh you may find it hard to get a book that really uh describes it in the way you want and I I feel the uh chatbot interfaces can do that for you.
- 38:16Ryan: That's that's a really great point.
- 38:18Ryan: You can and you can ask follow-up questions and it has unlimited patience with you.
- 38:23Ryan: It doesn't get exasperated when you don't understand.
- 38:25Ryan: I I actually asked uh one of the I forget which which one it was one of the uh chatbots to explain uh quantum computing and boy I still don't quite say I can understand quantum computing but I understand it better um and and so um yeah I think that's a great recommendation well as always uh the time has flown by and I have probably an hour and a half more questions for you so let's do this again sometime for next time uh we'll save it for next time and deep sing thank you for joining us and thanks to all of you as well for joining us on this episode of the First Trust ROI podcast we'll see you next time