Latest / The Adults In The AI Room / Earley AI Podcast - Episode 81 : Building AI That Works in the Real World with Krishna Rangasayee

Transcript

Earley AI Podcast - Episode 81 : Building AI That Works in the Real World with Krishna Rangasayee · The Adults In The AI Room

• 0:01
  SPEAKER_00: This is the Early AI Podcast hosted by Steph Earl to us as detailed deep into the passions, expertise, and experiences of thought leaders and practitioners to talk about what's possible with artificial intelligence.
• 0:14
  SPEAKER_00: The Early AI Podcast is sponsored by Early Information Science, for a digital transformation journey with design and deployment of innovative technology solutions, as well as Better.
• 0:26
  SPEAKER_00: Now enjoy the talk.
• 0:28
  SPEAKER_01: Welcome to the Early AI Podcast.
• 0:31
  SPEAKER_01: My name is Steph Early.
• 0:32
  SPEAKER_01: I'm your host, and each episode explores how artificial intelligence is shaping how organizations manage information, make decisions, turn knowledge into real business outcomes.
• 0:42
  SPEAKER_01: Today we're talking about what it takes to scale AI in the physical world, not just in labs and demos, but in production systems at the edge, robots, cars, medical devices, and other intelligent infrastructure where performance efficiency and reliability are really critical.
• 0:57
  SPEAKER_01: And as AI expands beyond centralized computing, questions around cost, power consumption, latency, operational readiness, those are all becoming very important.
• 1:07
  SPEAKER_01: Joining me today is Krishna Rangastadi, founder and CEO of SIMA.ai.
• 1:12
  SPEAKER_01: That's S-I-M-A.
• 1:13
  SPEAKER_01: Krishna works at the intersection of AI software, specialized hardware, systems architecture, and his focus is on delivering efficient high performance AI at the edge, enabling intelligent systems to operate closer to where the data is generated and where decisions are made.
• 1:27
  SPEAKER_01: Krishna, welcome to the show.
• 1:29
  SPEAKER_01: Thank you, Seth.
• 1:30
  SPEAKER_01: Such a pleasure.
• 1:30
  SPEAKER_03: Thanks for the great intro too.
• 1:32
  SPEAKER_03: Wow.
• 1:32
  SPEAKER_03: I need to have that for later use.
• 1:34
  SPEAKER_01: I know it's always when people introduce you, they're like, oh wow, did I write that?
• 1:38
  SPEAKER_01: Anyway, so let's talk about, I'd like to start off with.
• 1:42
  SPEAKER_01: Well, let's start off with when people, you know, when people think about AI, they're thinking about chatbots, they're thinking about the cloud models.
• 1:47
  SPEAKER_01: What do you mean when you talk about when when we talked about AI in the physical world, how is that fundamentally different?
• 1:54
  SPEAKER_01: And what does it really mean?
• 1:55
  SPEAKER_03: No, and I think I mean AI has gone through its evolution where I think the footprint was mostly consumer along the lines of Chat GPT or LLM agents.
• 2:04
  SPEAKER_03: And in the cloud concept, we have derived benefits of AI for 10 years, 15 years.
• 2:09
  SPEAKER_03: And what what I think we're gonna, and that's taken, if you will, an evolution for the last 10, 15 years, AI in the cloud, AI in the consumer persona.
• 2:17
  SPEAKER_03: I'm totally convinced that where AI will really bring value to human beings is when it touches our lives every day.
• 2:24
  SPEAKER_03: So for cars, for medical devices, for home appliances, things we live and breathe and touch every day, as and when AI starts coming into our lifestyle is when I think we're gonna truly see the benefit of AI.
• 2:37
  SPEAKER_03: And so it's not that the cloud is gonna go away.
• 2:40
  SPEAKER_03: I mean, there is a reason and there's a value that the cloud's still gonna retain, but increasingly you're gonna find a balance between the physical world we live in, benefiting from it, and coexist with cloud infrastructure.
• 2:53
  SPEAKER_01: So, so you know, we you said that the industry is moving from rules-based computing to machine learning.
• 3:00
  SPEAKER_01: And what has really changed in the last couple of years?
• 3:02
  SPEAKER_01: I mean, we know that that this has been emergent, we know it's been around for a long time, we know that you know the AI in lots of incarnations, but what what do you consider as the real inflection point?
• 3:14
  SPEAKER_01: Obviously, large language models are critically important, but what else would you say is is really causing it?
• 3:19
  SPEAKER_03: So if you really take a step back, neural networks have been around forever.
• 3:24
  SPEAKER_03: So this is not a new concept.
• 3:25
  SPEAKER_03: What's really, I think, been interesting is our ability to be able to deploy neural networks in a commercially scalable way is the big difference that chips are enabling today.
• 3:35
  SPEAKER_03: So we have seen the benefit of what was CNN, convolutional neural networks.
• 3:40
  SPEAKER_03: We saw the evolution of transformer-based architectures.
• 3:43
  SPEAKER_03: And broadly, though LLMs are capturing the mind of everybody today, reasoning-based architectures was for the first time in humanity ever being able to be commercially deployed.
• 3:57
  SPEAKER_03: And we are seeing this in the cloud along the lines of chat agents or chat bots, but you're gonna see reasoning-based architectures and the compute paradigm shift from rule-based to analytical to now context and memory aware, like human beings think.
• 4:14
  SPEAKER_03: This is for the first time in the history of humanity, right?
• 4:17
  SPEAKER_03: And we're all looking at this big, big tipping point.
• 4:20
  SPEAKER_03: I'm sure historians are gonna write about this tipping point for a long, long time to come.
• 4:24
  SPEAKER_03: So this is a renaissance age for computing.
• 4:26
  SPEAKER_03: So so we're in an amazing time.
• 4:28
  SPEAKER_01: Yeah, yeah, it really, it really is.
• 4:30
  SPEAKER_01: And every day, every time I use uh a large language model, I'm astounded.
• 4:35
  SPEAKER_01: You know, and you start taking that for granted.
• 4:37
  SPEAKER_01: You start, you know, then then you start getting angry when it misses something, you know.
• 4:41
  SPEAKER_01: When it's you know you're giving these complex documents and it's and you're asking all these questions.
• 4:46
  SPEAKER_01: It's like, you missed this point over here.
• 4:48
  SPEAKER_01: What's wrong with you?
• 4:49
  SPEAKER_01: Oh, I'm so sorry.
• 4:50
  SPEAKER_01: It'll apologize.
• 4:51
  SPEAKER_01: And yeah, yes, you made a good point.
• 4:53
  SPEAKER_01: I, you know, I'll go back and rethink this.
• 4:55
  SPEAKER_01: And it's like you're really interacting with something that is kind of mind-boggling.
• 5:00
  SPEAKER_01: Absolutely.
• 5:00
  SPEAKER_03: No, I I I would say, I mean, I think LLMs are the reasoning architecture flavor of the year.
• 5:07
  SPEAKER_03: I promise you that every year there's gonna be an evolution.
• 5:10
  SPEAKER_03: We are gonna get better and better at it, where models will get more accurate, more personalized, and more context aware, and be able to really be nimble and quick.
• 5:19
  SPEAKER_03: And we are seeing so many implementations of it, even in the physical AI world.
• 5:24
  SPEAKER_01: So, so let's talk a little bit.
• 5:26
  SPEAKER_01: You mentioned um we we talked a little bit about uh inference and and a lot of times people blur training and inference, right?
• 5:34
  SPEAKER_01: They think about just AI and training these models, but but they're very different.
• 5:39
  SPEAKER_01: What is important, especially with physical AI systems, in the terms of the difference between training and inference?
• 5:46
  SPEAKER_03: Sure.
• 5:46
  SPEAKER_03: Training is a very different universe than inference.
• 5:49
  SPEAKER_03: And so I'll take a step back.
• 5:51
  SPEAKER_03: So training is where you're you have a set of data, you're training a model to really recognize the data and really learn along with the data set.
• 6:00
  SPEAKER_03: So when you're given a new data point, you know what to recognize and what probably to place that at, right?
• 6:06
  SPEAKER_03: So in the early days of AI, training from a revenue and from a market focus has taken on the biggest footprint.
• 6:14
  SPEAKER_03: So inference is relatively a smaller market compared to training.
• 6:18
  SPEAKER_03: But once you've trained these models and once you start deploying them in the cloud, and increasingly now in the physical AI world, it's gonna be more and more an inference world.
• 6:27
  SPEAKER_03: And you're gonna see that the inference market is gonna be 10x, maybe even an order of 100x the size of the training market going forward, because you don't need training everywhere.
• 6:36
  SPEAKER_03: Once you have really trained a model, it's now scale of deployment, and it's gonna be more and more an inference world, particularly in the physical AI world.
• 6:44
  SPEAKER_03: And SIM as a company, if you will, we are an inference company.
• 6:47
  SPEAKER_01: And inference is really uh uh giving the the AI model the problems to solve.
• 6:53
  SPEAKER_01: It's actually doing the work once you have the training completed, and that requires a really different way of thinking about power and different uh way of thinking about architecture and so on.
• 7:04
  SPEAKER_01: What you know, when you when you um think about that, you know, what is what's particularly important in physical AI and AI at the edge, and what makes it a different problem when it lives in uh a robot or car or medical device?
• 7:20
  SPEAKER_03: Absolutely.
• 7:21
  SPEAKER_03: So let's start with the basic of problems.
• 7:23
  SPEAKER_03: While cloud really doesn't care for power, they should.
• 7:30
  SPEAKER_03: But at this point, people are having a fear of missing out and a mad rush to enable capability without thinking through the power implications.
• 7:38
  SPEAKER_03: Right.
• 7:39
  SPEAKER_03: At the physical AI world, you actually have no option at all but to really think through power.
• 7:45
  SPEAKER_03: Tend to be five watts or ten watts, and that's the limit of what you really have, right?
• 7:50
  SPEAKER_03: So power and thermal are huge limiters.
• 7:53
  SPEAKER_03: So you cannot take a GPU that you use for training or a GPU that you're using in the cloud and assume the same architecture will hold its same value at the edge.
• 8:03
  SPEAKER_03: And at the edge, power and thermal are unpergated.
• 8:06
  SPEAKER_03: You almost go the other way, saying, Hey, I have five watts, I have 10 watts.
• 8:09
  SPEAKER_03: What capability can I fit into five, 10 watts?
• 8:12
  SPEAKER_03: So that's problem number one.
• 8:13
  SPEAKER_03: Problem number two is the sheer variability and the sheer scale of it.
• 8:18
  SPEAKER_03: In the cloud, hyperscalers, it's seven customers.
• 8:22
  SPEAKER_03: The physical AI world is thousands, tens of thousands of customers.
• 8:26
  SPEAKER_03: So the scalability of the problem in being able to be cost efficient and also really software flexibility is really at the very key, right?
• 8:35
  SPEAKER_03: So the needs of the edge or the physical AI world are very different.
• 8:39
  SPEAKER_03: The third large variable is in the physical AI world, you and I are living with robots, you and I are living with cars.
• 8:46
  SPEAKER_03: Safety, security, privacy, and if you will, human-machine interface and the real world has huge implications architecturally that you don't need to worry about in the cloud.
• 8:57
  SPEAKER_03: If your chat GPT or your chat LLM agent made a mistake, nobody died.
• 9:03
  SPEAKER_03: Right.
• 9:04
  SPEAKER_03: Right, right.
• 9:05
  SPEAKER_03: That's not the case in a car.
• 9:08
  SPEAKER_03: So you're living through safety, privacy, security constraints that are radically different than in the cloud.
• 9:14
  SPEAKER_03: So our outlook is that I think there will be a new class of architectures that will have to be built for the needs of scaling for the physical world that are quite divergent, if not quite different, than what's being used.
• 9:25
  SPEAKER_03: And that's our this is behind why we started our company too.
• 9:28
  SPEAKER_01: Yeah.
• 9:29
  SPEAKER_01: And so what are the misconceptions?
• 9:31
  SPEAKER_01: What do executives not understand?
• 9:33
  SPEAKER_01: Uh uh when they uh look at deploying AI into real world system real world systems, what are they not understanding?
• 9:40
  SPEAKER_01: What are they missing?
• 9:40
  SPEAKER_01: Or what are the what are the misconceptions?
• 9:42
  SPEAKER_03: So I would say the touching on them here, but I just wanted to kind of all new technologies introduced into traditional industries.
• 9:51
  SPEAKER_03: The learning curve of what it takes to scale something in production has a longer timeline than everybody expects.
• 9:58
  SPEAKER_03: One popular misconception people have is they think AI is magic, and that you just show up, and as somebody said, it's like a Betty Crocker moment.
• 10:07
  SPEAKER_03: You just add water and it works.
• 10:08
  SPEAKER_03: Far be it from the truth.
• 10:10
  SPEAKER_03: AI is really a large predicator on the quality of the data that you train this on and the quality of the data that you have.
• 10:17
  SPEAKER_03: Um, what's the other adage?
• 10:19
  SPEAKER_03: Garbage in, garbage out.
• 10:21
  SPEAKER_03: So people overlook the fact that a model is only as good as how well it's trained, a model is only as good as how well it's deployed.
• 10:29
  SPEAKER_03: So that's one popular misconception.
• 10:32
  SPEAKER_03: The second one is people don't understand the life cycle of AI in that you it's not about showing a proof point and getting it done.
• 10:40
  SPEAKER_03: In rule-based compute, you get the problem right once, you can then scale that as long as you want.
• 10:45
  SPEAKER_03: And the rule-based construct doesn't change.
• 10:48
  SPEAKER_03: AI, there's every day a revisit to where's my model, what's my drift, what's my false positive, what's my false negative, am I learning from the mistakes that I'm looking to?
• 10:58
  SPEAKER_03: And there's an ML operations infrastructure, ML ops infrastructure that helps life cycle maintenance.
• 11:04
  SPEAKER_03: And so AI is a perennial, everyday learning, breathing entity.
• 11:10
  SPEAKER_03: Nurturing that infrastructure to scale is radically different than what people are used to.
• 11:15
  SPEAKER_03: So these are new paradigms.
• 11:17
  SPEAKER_03: I have no doubts that we'll look back in this phase and laugh at some point.
• 11:20
  SPEAKER_03: I joked in some other forum that when we discovered fire, we went through the same hazing ritual too.
• 11:28
  SPEAKER_01: Exuberance will fit pragmatism, it'll settle in a fire stuff is really useful, but wow, it's dangerous.
• 11:36
  SPEAKER_03: So it's no different, but AI is overly hyped to be very simple, particularly the physical world, it's a lot more complicated.
• 11:45
  SPEAKER_01: Yeah, yeah.
• 11:45
  SPEAKER_01: And what did what do people underestimate?
• 11:48
  SPEAKER_01: I mean, you did you, you know, you're you're alluding to some of these things.
• 11:51
  SPEAKER_01: What do they really miss out?
• 11:52
  SPEAKER_01: What specifically do they not understand?
• 11:55
  SPEAKER_01: What do they underestimate when they're trying to build this out in the physical world?
• 11:59
  SPEAKER_03: No, I I think they need to think through the life cycle consequences.
• 12:03
  SPEAKER_03: So that's number one.
• 12:04
  SPEAKER_03: Number two, I mean, less in the physical AI world, but we see a lot of companies getting into AI for the sake of AI.
• 12:11
  SPEAKER_03: True.
• 12:11
  SPEAKER_03: And what really matters at the end of the day is two things.
• 12:14
  SPEAKER_03: What is my use case?
• 12:16
  SPEAKER_03: What's the tangible delta I'm gonna bring with an AI-based infrastructure?
• 12:20
  SPEAKER_03: And third, no wonder.
• 12:21
  SPEAKER_03: I mean, no doubts the capitalistic element of will this make revenue and will make will it make me profit?
• 12:27
  SPEAKER_03: And so these usual suspect things cannot be just saved by a magic wand of AI.
• 12:32
  SPEAKER_03: I mean, AI is a tool, but it doesn't take away for use cases, doesn't take away for deliberate lifecycle maintenance and management outlooks, nor does it take away for revenue and profit.
• 12:43
  SPEAKER_03: And take the consumer world today.
• 12:45
  SPEAKER_03: I promise you, 80, 90% of the companies that are building wrappers around LLM agents may not be around in a long time because another day it's a capitalistic world.
• 12:55
  SPEAKER_03: Sure.
• 12:56
  SPEAKER_03: Revenue and profit matter.
• 12:57
  SPEAKER_01: Right.
• 12:57
  SPEAKER_01: Can you talk a little bit about life cycle management?
• 13:00
  SPEAKER_01: What are the stages that you need to think about when you're looking at AI and lifecycle management?
• 13:04
  SPEAKER_03: Sure.
• 13:05
  SPEAKER_03: So it I think it's in some ways no different than what we've traditionally done, but there's devils in the detail of the amount of variability and the amount of AI that you need to face.
• 13:14
  SPEAKER_03: So I would say no doubts everybody starts off with a POC or proof of concept phase and saying, hey, I've done this all my life with this outlook, or it's a new problem solving.
• 13:24
  SPEAKER_03: Let me prove out the capability of a proof of concept.
• 13:27
  SPEAKER_03: What will I get out of it?
• 13:29
  SPEAKER_03: What cost, what power, what accuracy, what capabilities will I get out of it?
• 13:33
  SPEAKER_03: So that's one phase where it's a very different set of dynamics and needs, even a very different set of an organization that does that.
• 13:40
  SPEAKER_03: Then you have a pilot phase where you say, okay, I'm going to deploy it over 10 systems, and I'm going to now get an aggregate outlook of if I do it over 10 systems or a projected 100,000 systems one day, how would it look like and how would I deploy?
• 13:53
  SPEAKER_03: The third one is I'm now in volume production and I've deployed it.
• 13:57
  SPEAKER_03: How do I maintain it?
• 13:58
  SPEAKER_03: How do I upgrade my models?
• 14:00
  SPEAKER_03: How do I get better data?
• 14:01
  SPEAKER_03: How do I make my system more accurate?
• 14:03
  SPEAKER_03: How do I make it more feature capable or feature-rich?
• 14:06
  SPEAKER_03: And then the last one is maybe this has reached its final stage of value and I need a totally different foundation for it.
• 14:13
  SPEAKER_03: How do you phase out a existing production program and how do you phase in?
• 14:18
  SPEAKER_03: What I'm describing to you is a standard engineering deployment phase that we have lived with for 100 years, or maybe even more than that.
• 14:25
  SPEAKER_03: But the AI-centric implications of it bring an entirely different outlook, an entirely different set of organizational capability that not every company in the world is sitting with a commensurate.
• 14:36
  SPEAKER_03: Hey, I'm ready to go.
• 14:38
  SPEAKER_03: So we're all practicing and preaching and learning simultaneously, as we can.
• 14:42
  SPEAKER_03: Yeah, true.
• 14:43
  SPEAKER_03: Yeah.
• 14:43
  SPEAKER_03: So, so and and and the world's moving so fast.
• 14:46
  SPEAKER_03: The pace of innovation is so radically different.
• 14:49
  SPEAKER_03: And most of the physical AI customers have been around for 50, 100 years.
• 14:54
  SPEAKER_03: And they built their strength and competencies in different areas.
• 14:57
  SPEAKER_03: And AI is a new muscle that they have to gain.
• 15:00
  SPEAKER_03: And so expectedly it's going to take time, but none of us are patient, and we all want results tomorrow morning.
• 15:07
  SPEAKER_03: So, human beings, as long as we remain who we are, we'll struggle through these every day.
• 15:12
  SPEAKER_03: But if history is a proof point, we'll all get across the finish line and be excited for the new world that we're getting into.
• 15:19
  SPEAKER_01: And where are you seeing the greatest impact?
• 15:22
  SPEAKER_01: And give us some examples of where you know the efficient edge inference has changed what systems can do.
• 15:30
  SPEAKER_01: Sure.
• 15:30
  SPEAKER_01: Kind of some examples of what that look like and what's possible.
• 15:34
  SPEAKER_03: Absolutely.
• 15:35
  SPEAKER_03: And so I would say um I'll park it into things that we've already seen already, and there are things that I think we're going to see.
• 15:44
  SPEAKER_03: Simple area where things have changed a lot is drones.
• 15:48
  SPEAKER_03: So these used to be pretty mostly consumer-ish toy elements.
• 15:52
  SPEAKER_03: Right.
• 15:53
  SPEAKER_03: And you're going to see that drones are going to be a huge component of our life in mobility.
• 15:59
  SPEAKER_03: In two, three years, very rarely is somebody going to drive to your place to hand off a DoorDash or a food delivery or a grocery delivery.
• 16:06
  SPEAKER_03: Majority of it is going to be drone-based.
• 16:08
  SPEAKER_03: And so this is one area where power matters a lot.
• 16:11
  SPEAKER_03: There's a lot of mobility and mapping and targeting and safety that needs to come in.
• 16:16
  SPEAKER_03: But this is something that's already at scale today.
• 16:19
  SPEAKER_03: And you're going to see more of it.
• 16:21
  SPEAKER_03: That's that's one simple example of it.
• 16:23
  SPEAKER_03: The other area where you're going to already see AI and really kick in is really industry floor automation.
• 16:29
  SPEAKER_03: So you're really seeing capabilities or accuracy elements or productivity elements that were impossible previously with traditional rule-based compute.
• 16:38
  SPEAKER_03: And so as our supply chains are getting robust, and given the challenges we've had and labor shortages, you're seeing a huge deployment of robotics infrastructure and also in capabilities in industry floor automation.
• 16:51
  SPEAKER_03: These are two examples of things already in production scale and well behind us from an AI capability.
• 16:57
  SPEAKER_03: Some of the new areas where you're going to see is if I were to pick a horizontal that's just going to pervade all markets, conversational AI is really going to be huge.
• 17:07
  SPEAKER_03: So where you could talk to machines like you talk to a human being.
• 17:11
  SPEAKER_03: Machines reason along with you like we do with human beings.
• 17:15
  SPEAKER_03: No longer is it, hey Siri, hey Alexa.
• 17:18
  SPEAKER_03: No longer is it, hey Mercedes, wake up.
• 17:21
  SPEAKER_03: You're just going to be context aware, memory aware, and everything that you do in the cloud is going to be locally embedded on the device, car.
• 17:28
  SPEAKER_03: You're driving in your car and your no 5G access, you do not want to be handicapped.
• 17:33
  SPEAKER_03: And so you're going to see a huge paradigm in edge-based appliance.
• 17:36
  SPEAKER_03: And power matters a lot.
• 17:38
  SPEAKER_03: Most cars are going to be EV-based and running one kilowatt just for compute and AI is not tenable.
• 17:43
  SPEAKER_03: You're going to see this in robotics in a very big way.
• 17:46
  SPEAKER_03: Power is at the very core of it, power efficiency.
• 17:49
  SPEAKER_03: And I'll leave you with the last vineyard, which I think I'm sure everybody's going through.
• 17:53
  SPEAKER_03: This is data centers are already 4.5%, 5% of the global power consumption.
• 17:57
  SPEAKER_03: And we are poised in 2030 or 2032 to double that.
• 18:02
  SPEAKER_03: 10% of the world is going to be just on data centers in a very, very few years.
• 18:07
  SPEAKER_03: But contrast, the physical AI world is 28% of the global power consumption today already.
• 18:12
  SPEAKER_03: Pre-AI.
• 18:13
  SPEAKER_03: If that pivots on AI and has no control on power, I don't know what we're doing.
• 18:20
  SPEAKER_03: So getting ahead on power at the edge is really, really critical.
• 18:24
  SPEAKER_01: Yeah.
• 18:24
  SPEAKER_01: Yeah.
• 18:25
  SPEAKER_01: And so where do you so tell me a little bit more about where you play in terms of that space?
• 18:31
  SPEAKER_01: So you are offering software and hardware.
• 18:34
  SPEAKER_01: Right.
• 18:34
  SPEAKER_01: And talk a little bit about your kind of philosophy.
• 18:37
  SPEAKER_01: If you're not not meant to be a sales pitch about your company, but just to kind of talk about conceptually where you are and how you address these problems, and then you know uh how how organizations kind of engage with you, who are your yeah.
• 18:52
  SPEAKER_03: So I I've been a student of the physical world for all my career.
• 18:56
  SPEAKER_03: And so this is an area where I've spent a lot of time and in hindsight built good businesses and technology behind it.
• 19:03
  SPEAKER_03: My observation is that I think there are two things needed for physical AI to scale meaningfully.
• 19:09
  SPEAKER_03: Two things.
• 19:10
  SPEAKER_03: One is performance per watt, not just performance alone.
• 19:13
  SPEAKER_03: Everybody wants amazing performance, but performance per watt is really, really critical because you're either limited by power or you're limited by the ability to dissipate heat, which is thermal.
• 19:25
  SPEAKER_03: So that's table sticks.
• 19:26
  SPEAKER_03: Number two, this is a world that's diffused.
• 19:29
  SPEAKER_03: You don't have five customers, you have 40,000, 50,000 customers globally.
• 19:34
  SPEAKER_03: They need to be self-managed and they need to be able to derive the benefit of AI without the learning curve of AI.
• 19:40
  SPEAKER_03: So scalability and ease of use in software is the second vector that's really very critical.
• 19:45
  SPEAKER_03: People need to be self-managed, and you need to make AI very digestible.
• 19:49
  SPEAKER_03: The analogy I draw is before iPhone, we used to have thick manuals that came with every one of our phones.
• 19:55
  SPEAKER_03: You had to press five buttons to get a function coding.
• 19:58
  SPEAKER_03: We lived with our Blackberries, we lived with our Nokia phones.
• 20:01
  SPEAKER_03: Comes iPhone, no manual, you press a button, you move on.
• 20:05
  SPEAKER_03: That paradigm shift needs to happen to really hit scale in production, physical AI, right?
• 20:11
  SPEAKER_03: So, no doubts, those are our two key vectors of capability.
• 20:15
  SPEAKER_03: And we have done a really good job with both of them.
• 20:17
  SPEAKER_03: And to your point, I'll not make this a marketing pitch for our company.
• 20:21
  SPEAKER_03: But I am entirely convinced that those are the two critical things to solve.
• 20:25
  SPEAKER_03: If you want to hit scale in physical AI, and companies that do a good job with it, I'm entirely sure are going to be household names in a few years.
• 20:32
  SPEAKER_01: Right.
• 20:32
  SPEAKER_01: So what do you see uh as the well, uh let me ask you a question.
• 20:38
  SPEAKER_01: A lot of times, you know, uh NVIDIA shows up as kind of the default choice uh for you know AI uh hardware.
• 20:46
  SPEAKER_01: Where does that work well and where uh do you need to look at uh edge AI players?
• 20:52
  SPEAKER_03: Yeah, so I'm amazed at what Nvidia has done in the last 10 years.
• 20:56
  SPEAKER_03: I don't know if anybody in the planet is not an amazing company, and if you will, they've kind of led the charge on AI and AI adoption globally.
• 21:05
  SPEAKER_03: But our thesis is that I think they're really fundamentally built around a GPU-centric architecture that was built for graphics, and they've extended that capability and it's fit well in the construct of data centers and in the cloud.
• 21:18
  SPEAKER_03: They're a general purpose architecture, but you're gonna see more and more targeted silicon and software architectures that are divergent for the needs of different markets.
• 21:28
  SPEAKER_03: I don't believe there's a one-size-fits-all where a GPU-centric architecture can scale for all problems globally.
• 21:35
  SPEAKER_03: In an early nascent proof of concept, early production, no doubts.
• 21:40
  SPEAKER_03: I think that's gonna be the case.
• 21:41
  SPEAKER_03: But you're gonna see very divergent architectures to solve different problems.
• 21:45
  SPEAKER_03: Particularly as it comes to the physical AI world, like I told you, power and thermal matter a lot.
• 21:51
  SPEAKER_03: Software scalability, I don't think you could really be a CUDA company and scale meaningfully.
• 21:56
  SPEAKER_03: It would need to be open source.
• 21:58
  SPEAKER_03: You need very different architects.
• 22:00
  SPEAKER_03: Architectures, either by silicon or by software, to be meaningfully scaling.
• 22:04
  SPEAKER_03: This is where I think you're going to see that a one-size-fits-all is not going to work.
• 22:08
  SPEAKER_03: And our approach is open source, our approach is solve for power and power efficiency from day one.
• 22:14
  SPEAKER_03: And we are a radically different architecture than GPUs.
• 22:18
  SPEAKER_03: So that's really, if you will, as AI scales, you're going to see a lot of diversion architectures really come into play.
• 22:25
  SPEAKER_01: Say more about your architecture and how it's different.
• 22:29
  SPEAKER_01: Maybe you can dive in a little bit of similar detail.
• 22:31
  SPEAKER_03: At the highest order bit, um, we had the benefit of a clean slate.
• 22:36
  SPEAKER_03: And we said, if we were to land from Mars, what would we build?
• 22:40
  SPEAKER_03: So that's a huge luxury that some of the large public companies do not have.
• 22:44
  SPEAKER_03: So I can do exactly what I want to look at.
• 22:47
  SPEAKER_03: Number two, ours is a very software-centric architecture.
• 22:50
  SPEAKER_03: We knew that AI is going to move at a rapid pace from a software evolution, but silicon cadence is once every three years.
• 22:57
  SPEAKER_03: So how do you really deliver something that's market relevant while silicon has a very different cadence than AI?
• 23:03
  SPEAKER_03: We decided to really take a very software-centric architecture innovation.
• 23:09
  SPEAKER_03: And our approach speaks very differently.
• 23:11
  SPEAKER_03: Third, 70% of the power and the performance gains are more on data management and memory management, almost less to do with AI.
• 23:21
  SPEAKER_03: We have taken full advantage of that architecturally and built something that's very lightweight on on-chip cost and on-chip memory.
• 23:30
  SPEAKER_03: But we have relied on an architecture that uses external DDR.
• 23:34
  SPEAKER_03: So people can load as complicated a lot model that they want, but we have clever software architectures that work with it so that we meet the customers' needs in cost, power, and performance while really giving them scalability on software.
• 23:47
  SPEAKER_03: So these are the things that we have done that's very different.
• 23:50
  SPEAKER_03: And our architecture is divergent and quite different than anybody else's.
• 23:54
  SPEAKER_01: And uh, what about the compatibility with existing software and approaches and methodologies that people are?
• 24:03
  SPEAKER_03: Absolutely.
• 24:03
  SPEAKER_03: So while our architecture is divergent, we are completely open source compliant.
• 24:09
  SPEAKER_03: So what I mean by that is we can work with any ML framework, whether it's Onyx or on PyTorch or TensorFlow, we take them as they are.
• 24:17
  SPEAKER_03: We're also on Linux, we're also on OpenCL, OpenCV.
• 24:20
  SPEAKER_03: So these are well-known practices of 10, 15, 30 years.
• 24:24
  SPEAKER_03: And so everything we do is open source.
• 24:25
  SPEAKER_03: And I've joked, we are the Ellis Island of ML.
• 24:29
  SPEAKER_03: Give us your poor, give us your tired, give us your butt once they come in there, America.
• 24:35
  SPEAKER_03: So that's the closest analogy I draw in, and that what we do inside is pretty complicated and quite different.
• 24:41
  SPEAKER_03: But our interfaces are all industry standard.
• 24:44
  SPEAKER_03: Friction point and migrating from a known base to us is low.
• 24:48
  SPEAKER_01: And what kind of companies are actively investigating your architecture?
• 24:53
  SPEAKER_01: So these are smaller, innovative entrepreneurial companies, or you also have some of the bigger players.
• 25:00
  SPEAKER_01: Who's who's uh looking at your your approach and your methodology?
• 25:03
  SPEAKER_03: We have both camps, absolutely both camps.
• 25:05
  SPEAKER_03: And and I would say in the seven years we run the company, we have gone from early adoption of the market to fast mowers.
• 25:12
  SPEAKER_03: So we are in that phase right now.
• 25:14
  SPEAKER_03: Any of the big company, I mean now it's becoming more and more clear that they have AI and that have knots on AI.
• 25:21
  SPEAKER_03: There's a market separation and even their commercial success.
• 25:25
  SPEAKER_03: So we are seeing everybody jump in.
• 25:28
  SPEAKER_03: And we have a good, happy medium of both early fast mowers that are nimble and small, but also large companies that are also becoming nimble and really forward.
• 25:37
  SPEAKER_03: So we have both.
• 25:38
  SPEAKER_01: And then what what is your competition uh like?
• 25:42
  SPEAKER_01: In other words, you said there's a lot of these players, and there's larger organizations and then niche players.
• 25:47
  SPEAKER_01: Tell us a little bit about that.
• 25:49
  SPEAKER_03: Yeah, so I would say we primarily end up competing mostly with um our GPU friends.
• 25:55
  SPEAKER_03: We also occasionally see DSP-centric competition, primarily from our friends in San Diego.
• 26:01
  SPEAKER_03: Um, and we compete with them well as well, particularly as it comes to automotive and automotive infotainment.
• 26:07
  SPEAKER_03: There is a lot of smart, amazing startups that have come around.
• 26:11
  SPEAKER_03: And most of them are ML accelerators only.
• 26:14
  SPEAKER_03: They don't install the system, system on a chip approach that I think we take.
• 26:19
  SPEAKER_03: And so that's given us a large differentiation against them.
• 26:21
  SPEAKER_03: And the Achilles seal for the industry is definitely software, and that's an area where we have really done well.
• 26:27
  SPEAKER_03: So if I were to summarize, I would really say our primary competition ends up being our GPU friends.
• 26:32
  SPEAKER_03: Primary competition.
• 26:34
  SPEAKER_03: But we are respectful of everybody's capability.
• 26:36
  SPEAKER_03: It behooves us to be paranoid and not take anything for granted.
• 26:41
  SPEAKER_03: We are a startup and we have to fight like hell, anything, everything.
• 26:45
  SPEAKER_03: How many folks are you?
• 26:46
  SPEAKER_03: Uh, we are 200 plus.
• 26:48
  SPEAKER_03: Okay.
• 26:48
  SPEAKER_03: Yeah.
• 26:49
  SPEAKER_03: Our daily struggle is a David versus Goliath story.
• 26:52
  SPEAKER_01: So when you look ahead, uh, where do you see the biggest opportunities in physical AI?
• 26:57
  SPEAKER_01: Uh and where does the hype, particularly around humanoid robots, get ahead of reality?
• 27:03
  SPEAKER_01: I mean, it again, uh what what you know, we it's hard to calibrate because things change so quickly.
• 27:08
  SPEAKER_01: But where what do you see as sort of the near-term and the midterm uh future?
• 27:14
  SPEAKER_03: So I would say uh uh I I I I joke this in a different form.
• 27:18
  SPEAKER_03: We tend to be totally wrong on expectations on things near term, and we tend to be totally wrong on expectations on the long run.
• 27:26
  SPEAKER_03: Right.
• 27:26
  SPEAKER_01: Yeah, so that's just how things overestimate what you can do short term, and you underestimate what you can do long term.
• 27:32
  SPEAKER_03: More eloquently put than I did.
• 27:34
  SPEAKER_01: Yes, correct.
• 27:36
  SPEAKER_01: And so I would say overestimate what you can do in a day, underestimate what you can do in a year, right?
• 27:41
  SPEAKER_03: Yeah.
• 27:42
  SPEAKER_03: So steady state, I would say the volume drivers for physical AI are no doubt gonna be automotive, robotics, and drones.
• 27:50
  SPEAKER_03: Those would be the three volume drivers if I were to pick.
• 27:53
  SPEAKER_03: And I'm using robotics as a broad category, yeah, as you know, there's so many sub-segments even within robotics.
• 27:58
  SPEAKER_03: But you're gonna see this everywhere in medical, you're gonna see this in ACTEC, you're gonna see it in transportation, you're gonna see it in aerospace and defense, you're gonna see it in small vision systems.
• 28:07
  SPEAKER_03: Physical AI is gonna scale everywhere, but if I were to pick three applications, drones, automotive, and robotics.
• 28:14
  SPEAKER_03: So that's where they are.
• 28:15
  SPEAKER_03: And we're seeing the same three vectors today.
• 28:18
  SPEAKER_03: And I say that I think as in as and when these systems get more and more AI enabled, you're gonna see a lot more growth in that vector.
• 28:25
  SPEAKER_03: As it comes to humanoid robots, my view, and I think maybe I'm a little contrarian in where things are at, or maybe I'm not, is in some ways humanoid robots is a more complex problem than autonomous cars.
• 28:38
  SPEAKER_03: There is tremendous hype on humanoid robots right now, and we are participants in it too.
• 28:43
  SPEAKER_03: So I should not be really poo-pooing that too much.
• 28:47
  SPEAKER_03: You're enjoying the benefit of that too.
• 28:49
  SPEAKER_03: But the true production capacity for general purpose humanoid robots and that form factor, where the safety considerations, the scalability considerations, the cost considerations, the power considerations are all factored in.
• 29:02
  SPEAKER_03: It will absolutely happen, but in my view, at a much longer timeline than we're thinking through today.
• 29:08
  SPEAKER_03: What do you see as a timeline?
• 29:10
  SPEAKER_03: I would say 10, 15 years.
• 29:12
  SPEAKER_03: Yeah.
• 29:12
  SPEAKER_03: And so I'm definitely a contrarian in that.
• 29:15
  SPEAKER_03: And particularly for somebody that's living in the middle of it, I think I hurt my colleagues and friends a little more with my protracted timeline.
• 29:23
  SPEAKER_03: But I would say cost alone, right?
• 29:25
  SPEAKER_03: And so, and it all comes down to what's the cost the market's willing to bear.
• 29:29
  SPEAKER_03: If you want things at$10,000 to$15,000 for a humanoid channel purpose robot, somebody to be in your home, fold your laundry, dishes for you, and be a sentry, whatever you want it to be at, or an AI companion or AI guardian, if you will.
• 29:46
  SPEAKER_03: When the compute cost itself is$3,500 to$7,000, yeah.
• 29:51
  SPEAKER_03: It's hard for me to do the math, right?
• 29:53
  SPEAKER_03: So my rule of thumb is if your system cost is X, your compute cost needs to be one-tenth of that.
• 29:59
  SPEAKER_03: Meaningfully while interesting.
• 30:01
  SPEAKER_03: And and the systems that are being deployed today are$150 to 300 watts.
• 30:06
  SPEAKER_03: It's not tenable in a real humanoid form factor, right?
• 30:10
  SPEAKER_03: So we disagree with many of the peers that are rushing into it.
• 30:14
  SPEAKER_01: Pilots are fine.
• 30:15
  SPEAKER_01: You're saying the compute it requires that level of power?
• 30:18
  SPEAKER_01: Absolutely, right?
• 30:19
  SPEAKER_03: And so compute alone.
• 30:20
  SPEAKER_03: What does it need to get down to?
• 30:22
  SPEAKER_03: Less than 50 watts.
• 30:23
  SPEAKER_03: Oh.
• 30:24
  SPEAKER_03: And so so proof of concepts and I mean, I think there's a rush into I'll get my functionality and worry about cost and power later.
• 30:33
  SPEAKER_03: But this is like saying, let me get a Ferrari and I'll figure out how to make it a fiat.
• 30:38
  SPEAKER_03: Some things are not that practical.
• 30:40
  SPEAKER_03: And so you almost need a very different outlook in saying if my cost structure is 15,000, how do I back solve for it meeting my functionality with the cost and power of compute that I need?
• 30:51
  SPEAKER_03: So I worry many are rushing into it, saying, hey, I'll figure out the cost equation later.
• 30:57
  SPEAKER_03: This is not a cloud where you could just throw things there and worry about things later, right?
• 31:03
  SPEAKER_03: So you need to give it a lot of front, upfront thinking.
• 31:05
  SPEAKER_03: I would submit to you that at this point, China, if you were at CES, has done a far better job in pragmatically saying if my cost structure is X,$10,000,$15,000, how do I get the most compute, most power efficient, and how do I bridge the gap?
• 31:21
  SPEAKER_03: Clearly they're not there today.
• 31:26
  SPEAKER_03: But they're iterating around a basis that's a far better one than the one that I think I see around particularly less than MSP.
• 31:34
  SPEAKER_03: So our thesis is we are built for cost and power as a company.
• 31:38
  SPEAKER_03: And so no doubt I'm biased in my outlook, but I've never seen systems that want to be$15,000 built on a cost structure of compute alone at$3,500 or$4,000.
• 31:48
  SPEAKER_03: The economics just are not there.
• 31:50
  SPEAKER_03: And the cost of humanoid robots are not just compute alone.
• 31:54
  SPEAKER_03: I look at the actuators, I look at the overall mechanical components that we have.
• 31:58
  SPEAKER_02: Sure.
• 31:58
  SPEAKER_03: And I would say the median of where the humanoid industry is on cost is 70 to 80,000 today.
• 32:04
  SPEAKER_03: Everybody wants to be 10 to 20,000.
• 32:06
  SPEAKER_03: Yeah.
• 32:07
  SPEAKER_03: And but the how we get their story diters.
• 32:09
  SPEAKER_03: And this is where I think time is going to be a large, large consideration.
• 32:13
  SPEAKER_03: And we've been waiting for autonomous cars for 15 years.
• 32:16
  SPEAKER_02: Yeah, of course.
• 32:17
  SPEAKER_03: And and even mundane, simple tasks like what we're taking for granted on grasping and manipulation at your robotic arm at a human-like capacity is a very complicated problem.
• 32:28
  SPEAKER_03: It's complicated, if not more complicated than autonomous cars.
• 32:32
  SPEAKER_03: I I joke around this with everybody that's done this for 15, 20 years.
• 32:36
  SPEAKER_03: For folks getting into it new, there's a huge exuberance.
• 32:39
  SPEAKER_03: And for folks that have done it for a lot, there's a lot of reticence in jumping into it.
• 32:46
  SPEAKER_03: Somewhere there's a happy medium, and it's gonna happen between those two poll opposite views.
• 32:51
  SPEAKER_03: It's gonna happen, but at a much longer time lane.
• 32:54
  SPEAKER_03: Yeah.
• 32:54
  SPEAKER_03: I submit to you on a totally different compute basis than what people are doing as GPM entry outlooks today.
• 33:00
  SPEAKER_01: Tell me what your roadmap is.
• 33:01
  SPEAKER_01: So, what do you see your organization in the next three to five years?
• 33:06
  SPEAKER_03: Sure.
• 33:06
  SPEAKER_03: Yeah.
• 33:06
  SPEAKER_03: So I think by now our tech is well through.
• 33:09
  SPEAKER_03: No doubts we have an evolution and a roadmap for improving our tech on software and hardware.
• 33:14
  SPEAKER_03: So that's definitely a case.
• 33:16
  SPEAKER_03: A lot happening on robotics and automotive, and we really want to double down on that.
• 33:20
  SPEAKER_03: If I were to pick a macro element of where we are as a company, scaling.
• 33:23
  SPEAKER_03: I mean, that that's that's really our priority and focus.
• 33:26
  SPEAKER_03: Scaling on our customers, on our commercial success, scaling on our partner ecosystem, scaling on our go-to-market, scaling on our software teams and our support teams.
• 33:35
  SPEAKER_03: So if I were to pick one broad theme of what I wake up to every day saying, my God, we need to do better, we need to do more, scaling.
• 33:42
  SPEAKER_01: Well, that's great.
• 33:42
  SPEAKER_01: Well, uh, what an interesting area and and full of change and innovation and fast moving IP.
• 33:49
  SPEAKER_01: And it really is kind of a fasting, as you say, it's an interesting time to be involved and in and alive in this environment and seeing what AI is doing to it to our world.
• 34:00
  SPEAKER_01: So this has been a great conversation.
• 34:02
  SPEAKER_01: Thank you for sharing your insights and you know what it takes to build these systems and and and what we're in for in terms of real world settings.
• 34:11
  SPEAKER_01: And to our listeners, uh, thank you guys for joining us for another episode of the Early AI Podcast.
• 34:16
  SPEAKER_01: So stay with us.
• 34:17
  SPEAKER_01: We're going to continue to explore how AI is moving from experimentation to impact throughout the uh society and and organizations.
• 34:26
  SPEAKER_01: And again, thank you, Krishna.
• 34:27
  SPEAKER_01: I really appreciate your time today.
• 34:29
  SPEAKER_03: Thank you so much, said thoroughly enjoyed the conversation.
• 34:32
  SPEAKER_03: Excellent.
• 34:32
  SPEAKER_01: Well, we'll let you go and we'll see everyone uh next time on the next uh early AI podcast.
• 34:38
  SPEAKER_01: Bye now.
• 34:38
  unknown: Bye.
• 34:40
  SPEAKER_00: Thank you for joining us on another deep dive into AI innovation.
• 34:44
  SPEAKER_00: Tune in next time when we introduce another industry expert and discuss how to maximize AI in your world.
• 34:50
  SPEAKER_00: The Early AI Podcast is sponsored by Early Information Science Invector.
• 34:56
  SPEAKER_00: To learn more, visit early.com.
• 34:58
  SPEAKER_00: That's earlier.com.
• 35:01
  SPEAKER_00: Thank you for tuning in.