Latest / Automated with Brian Heater / Martial Hebert on Why Self-Driving Cars Took So Long and What Everyone Got Wrong About AI

Transcript

Martial Hebert on Why Self-Driving Cars Took So Long and What Everyone Got Wrong About AI · Automated with Brian Heater

• 0:00
  SPEAKER_00: Sometimes people are surprised that it took so long for self-driving car to be deployed.
• 0:04
  SPEAKER_00: The basic technology was ready quite some time ago.
• 0:07
  SPEAKER_00: But now to go from that from something that can truly be validated, that's a completely different game.
• 0:13
  SPEAKER_00: I've been at the Robotics Institute for two years.
• 0:16
  SPEAKER_00: I came here as a postdoc on a project called the Autonomous Land Vehicle Project.
• 0:21
  SPEAKER_00: There used to be a program called the DAPA Image Understanding Program.
• 0:25
  SPEAKER_00: And that program has been instrumental in placing the country as the lead of that field of research, which is essential for everything that has to do with physical AI now.
• 0:36
  SPEAKER_02: I'd love to talk about the arc that's led us to where we are now.
• 0:40
  SPEAKER_00: When we talk about the arc of development, we miss sometimes the angle of talent, expertise, culture.
• 0:50
  SPEAKER_00: Over decades, we have built this culture, this body of expertise, this environment where talent emerges around those topics of autonomy and robotics.
• 1:00
  SPEAKER_02: If I had asked you 10 years ago where we would be at as far as actually having, you know, self-driving cars on the road, you're smiling self-driving cars on the road, do you think that we would be a little further along than we are right now?
• 1:16
  SPEAKER_00: Well, the real answer is a very boring answer, is it depends.
• 1:20
  SPEAKER_00: And here is why we're gonna be a little bit of a little bit of a little bit of a little bit of a little bit of a little bit of welcome back to Unlimited.
• 1:34
  SPEAKER_02: I'm Brian Heater, the managing editor at the Association for Advancing Automation.
• 1:39
  SPEAKER_02: I'm excited to share a great conversation with you this week.
• 1:42
  SPEAKER_02: Um, Marshall Eber is the Dean of CMU's School of Computer Science.
• 1:48
  SPEAKER_02: We recorded this conversation back in March to discuss the opening of the school's massive 150,000 square foot robotics innovation center.
• 1:58
  SPEAKER_02: It's a really great chat with a lot of insight into the state of robotics and physical AI research.
• 2:04
  SPEAKER_02: So thanks to Marshall and thanks to CMU, thanks to you.
• 2:08
  SPEAKER_02: And as always, if you are enjoying the program, please like, subscribe, check out our newsletter, all of that.
• 2:14
  SPEAKER_02: So we're at automated.
• 2:17
  SPEAKER_02: And with all of that, please enjoy this conversation with Marshall Eber.
• 2:22
  SPEAKER_02: You know, we talk a lot about what's coming up next in automation on the show, but if you really want to see the future motion, you've got to be there in person.
• 2:30
  SPEAKER_02: Automate 2026 is where the world's leading innovators, builders, and dreamers come together to show you what's possible.
• 2:37
  SPEAKER_02: Robots, AI, machine vision, motion control, you name it, all automation under one roof.
• 2:43
  SPEAKER_02: And as part of Automate this year, the Humanoid Robot Forum brings together leaders, engineers, and researchers for a two-day deep dive into the real world development, deployment, and commercialization of humanoid robotics.
• 2:56
  SPEAKER_02: Register for free at automateshow.com to join us in Chicago June 22nd through the 25th.
• 3:03
  SPEAKER_02: We will see you there.
• 3:05
  SPEAKER_02: I'm really curious about some of the logistics behind uh creating a 75,000 gallon water tank.
• 3:16
  SPEAKER_02: I mean, that must have been far and away one of the more difficult processes, I assume, of opening this robotics innovation center.
• 3:25
  SPEAKER_00: Yes.
• 3:25
  SPEAKER_00: So thank you for asking about this.
• 3:27
  SPEAKER_00: And thank you for being careful uh calling it a water tank, by the way.
• 3:31
  SPEAKER_00: Most people call it a pool, uh, which involves about a thousand pages of regulation.
• 3:36
  SPEAKER_00: So please let's call it a water tank and not a pool, okay?
• 3:40
  SPEAKER_01: Is there a difference beyond just like government regulation?
• 3:43
  SPEAKER_01: No, just government.
• 3:44
  SPEAKER_01: Okay.
• 3:44
  SPEAKER_00: Just government regulations, yeah.
• 3:46
  SPEAKER_00: So so more seriously, uh I don't think that was a particularly uh logistically uh logistical uh you know uh nightmare, really.
• 3:56
  SPEAKER_00: I think the people who were doing this are very professional and they used to do those kind of things.
• 4:00
  SPEAKER_00: The more important point about it is that we uh had a smaller water tank uh here in one of the buildings on campus, in the second uh basement of that building.
• 4:12
  SPEAKER_00: Uh but this was a much, much smaller water tank.
• 4:15
  SPEAKER_00: Now we have a much larger one.
• 4:17
  SPEAKER_00: You indicated some of the dimensions, and what that means concretely is that we're now able to conduct research that uh and conduct projects that we could not conduct before.
• 4:29
  SPEAKER_00: And this is basically the story of this entire facility, and this is the story of what we're trying to do in robotics is to continue expanding.
• 4:37
  SPEAKER_00: And expanding means being able to experiment on larger scale uh and and experiment in places uh in in in ways that we could not before.
• 4:47
  SPEAKER_02: Yeah, this is interesting, right?
• 4:48
  SPEAKER_02: And and I feel like you probably have as good a perspective on this as anybody, as far as the this is something that we talk about quite a bit on the show, the the data gap right now, physical AI.
• 5:01
  SPEAKER_02: You're somebody who has been involved with uh autonomous vehicles for about four four decades now.
• 5:07
  SPEAKER_02: To a certain extent, those have been in the real world, those have been out there collecting real world data, but this is the biggest issue right now.
• 5:14
  SPEAKER_02: How do we get robots out there, you know, actually collecting this information?
• 5:19
  SPEAKER_02: This seems like a step, right?
• 5:20
  SPEAKER_02: I mean, this isn't quite um it's it's it's a larger tank.
• 5:24
  SPEAKER_02: It's a way to get them sort of out there to collect data within a large body of water.
• 5:30
  SPEAKER_00: Yeah, it's not just uh collecting data, it's also uh simply experimenting.
• 5:35
  SPEAKER_00: Experimenting on larger scale, experimenting with more realistic, uh realistic environments.
• 5:42
  SPEAKER_00: So there's basically um when it comes to physical AI, and in fact, the the fundamental difference between what we call physical AI and let's call it regular AI, the Chat GPT, LLMs and so forth.
• 5:54
  SPEAKER_00: Uh a critical difference is of course the physical aspects.
• 5:57
  SPEAKER_00: What does that mean?
• 5:58
  SPEAKER_00: That means two things.
• 5:59
  SPEAKER_00: One, it's very hard to collect data, right?
• 6:02
  SPEAKER_00: Uh and framing this in the context of underworld of robotics, but this is even harder uh when you want to collect data uh in uh settings that involve interaction with people, for example, co-workers or or home robots that interact with people, right?
• 6:18
  SPEAKER_00: So there's this data collection aspect.
• 6:21
  SPEAKER_00: But there's another aspect, which is the one I was uh answering uh to your question, is the uh experimentation.
• 6:27
  SPEAKER_00: Right?
• 6:28
  SPEAKER_00: You need to experiment with physical things, you need to experiment in real settings, and that you can do as much simulation as you want.
• 6:35
  SPEAKER_00: At some point, you need the real experimentation uh to be able to validate those things.
• 6:41
  SPEAKER_00: Uh in fact, we have a long history of doing this.
• 6:44
  SPEAKER_00: We have another facility off campus called the uh National Robotics Engineering Center, which is in the north of the city where we do large-scale systems integration and and in-the-field uh experimentation.
• 6:57
  SPEAKER_00: So, to your point about the the time that we've been doing this, this is something that we've been doing for for a long time, just looking at how to validate an experiment with those robotic systems.
• 7:09
  SPEAKER_00: Uh the uh difference now with the uh the new wave of AI, if you will, uh AI models, is this idea of uh the importance of the data, the importance of collecting data, uh not just physically, but in simulation, uh and and uh to come together between those AI models and those those physical uh those physical systems.
• 7:31
  SPEAKER_02: So I I really want to go way, way back.
• 7:34
  SPEAKER_02: So I want to go back to to NavLab.
• 7:37
  SPEAKER_02: I want to go back to really sort of the uh what I I think is the beginning for you at least of uh autonomous vehicles.
• 7:45
  SPEAKER_02: You know, what what was the state of the art of this technology when you got involved?
• 7:52
  SPEAKER_00: Right.
• 7:53
  SPEAKER_00: So uh since now you you challenging me to uh talk about my own history, we'll be in trouble, by the way.
• 8:00
  SPEAKER_00: That might take a long time.
• 8:01
  SPEAKER_00: But uh Yeah, well we got we got we got about 45 minutes, so you know what's yeah, so so uh I arrived here as a postdoc.
• 8:08
  SPEAKER_00: Uh and in fact I've been at the Robotics Institute uh 42 years and uh two weeks today, actually, but who's counting?
• 8:16
  SPEAKER_00: Uh and I I came here as a postdoc on a project called the Autonomous Land Vehicle Project, which was the first project actually that was funded by DARPA, so the first federally funded project on what is now autonomy or self-driving.
• 8:33
  SPEAKER_00: Um and the way that started actually is not with something that looks like a car or a truck or a vehicle, really.
• 8:40
  SPEAKER_00: It was a six-wheeled vehicle called Terrogator that was trying to navigate the sidewalks of campus autonomously.
• 8:48
  SPEAKER_00: And then uh there was a development uh by a large team, and we're just uh a postdoc and a starting scientist at the time, of Nav Lab, which was a Chevy van that was equipped for those uh early days of uh safe driving.
• 9:03
  SPEAKER_00: There was a camera, there was uh, I think what is the very first uh LIDAR that was used for uh autonomous uh driving.
• 9:11
  SPEAKER_00: There were two copies of it, one here at CMU and another one at in Denver at what is now Lockett Martin, used to be uh Martin Maeta.
• 9:19
  SPEAKER_02: Aaron Powell Yeah, that it's interesting to hear, you know, uh obviously DARPA has played uh a huge role in the development of robotics generally um um here in the States.
• 9:32
  SPEAKER_02: And and and it's interesting to hear, as you said, that this specific project wasn't you know, I I tend to think of DARPA as um, you know, a lot of the Boston Dynamics work, right?
• 9:44
  SPEAKER_02: Uh um you know, a lot of the the pack robots or or or the humanoids, but that they were working on something that sounds a little more humble at the time, it seems.
• 9:55
  SPEAKER_00: It's interesting you use that word humble.
• 9:58
  SPEAKER_00: Uh I I would have said exactly the opposite.
• 10:00
  SPEAKER_00: They were working on something amazingly ambitious.
• 10:03
  SPEAKER_02: Sure.
• 10:04
  SPEAKER_02: I I I I suppose I mean as far as, you know, this is the defense department, right?
• 10:07
  SPEAKER_02: So it's it's it just an interesting place to start um this sort of defense research uh on on the sidewalks of campus.
• 10:15
  SPEAKER_00: Yeah, on the surface it would look that way.
• 10:18
  SPEAKER_00: Except uh if you generalize the the mission of DAPA and the mission of federal funding in general, uh further than a narrow view of defense, but uh uh uh a broader view of national security and a broader view of of leadership in uh uh areas of technology and science.
• 10:40
  SPEAKER_00: Right?
• 10:41
  SPEAKER_00: If you view this in a much in this much broader uh aperture, then it makes sense.
• 10:47
  SPEAKER_00: As a matter of fact, the uh that project, the ALV project, uh was part of a much broader project called strategic computing.
• 10:55
  SPEAKER_00: And in fact, the real motivation uh originally for that project, ALV, was not just giving autonomy and having things that drive themselves.
• 11:04
  SPEAKER_00: Uh a deeper motivation was to find uh task, uh application that are so hard that they would motivate a revolution in the field of computing.
• 11:18
  SPEAKER_00: That's why I meant in that sense that it was uh a very ambitious view of the world.
• 11:23
  SPEAKER_00: This was not narrowly concentrating on a particular defense application and so forth.
• 11:28
  SPEAKER_00: It was saying, what do we need to look at to really transform the technology and incidentally make sure that the US leads in that transformation?
• 11:39
  SPEAKER_02: Aaron Powell Yeah, that that's interesting.
• 11:41
  SPEAKER_02: So that that's really sort of the the to a set extent, that's the space race argument, right?
• 11:45
  SPEAKER_02: That's we'll go to space and then all of this uh these other breakthroughs will come.
• 11:48
  SPEAKER_00: Aaron Powell Exactly.
• 11:49
  SPEAKER_00: And in fact, uh sometimes we forget that DARPA was created originally to address the the space race.
• 11:55
  SPEAKER_00: Yeah.
• 11:56
  SPEAKER_00: Saturn rockets and all that.
• 11:58
  SPEAKER_02: Aaron Ross Powell which is which is, you know, and and for for I'm sure that most of the people who are watching this are familiar with the DARPA challenges, but that's the idea behind a lot of these, right?
• 12:06
  SPEAKER_02: Is is we put out a specific DARPA challenge, and then all of this interesting innovation kind of uh uh comes from uh the work that goes into those challenges.
• 12:16
  SPEAKER_00: Absolutely.
• 12:16
  SPEAKER_00: And I'll give you another example that is probably not as well known, at least uh uh by the broad audience, uh that is come to my field.
• 12:25
  SPEAKER_00: My field is computer vision.
• 12:26
  SPEAKER_00: That's that's where my uh research is.
• 12:29
  SPEAKER_00: And uh there used to be a program uh called the DARPA Image Understanding Program.
• 12:35
  SPEAKER_00: This was one of the longest running DARPA programs, it was like 15 years or something like that.
• 12:41
  SPEAKER_00: Um and it involved basically all of the research outfits in the country, mostly universities, a few companies, but mostly universities, uh doing fundamental research in computer vision.
• 12:55
  SPEAKER_00: And that program just has been instrumental in placing the country at the as at as the lead of that of that field of research, which is essential, by the way, for everything that has to do with anything that has to do with physical AI now.
• 13:12
  SPEAKER_00: Okay.
• 13:12
  SPEAKER_00: Um and uh the the the spirit of that program again was not looking at some narrow, you know, application defense and all that.
• 13:22
  SPEAKER_00: It was taking a very broad view and and fostering the exchange of idea, the creativity to discover new things and to to invent uh new things.
• 13:33
  SPEAKER_00: And um I think it's important to keep that in mind, you know, when we talk about federal funding and that kind of thing, um sometimes there's that conversation about, well, do we need that?
• 13:45
  SPEAKER_00: Industries doing uh, you know, a lot of things.
• 13:48
  SPEAKER_00: This is where it has its place.
• 13:50
  SPEAKER_00: Is this idea of having a very large uh uh sandbox where uh people can really create and innovate, and not innovate in the sense of creating new products, innovate in the sense of having completely new ideas and completely new direction of research that would not be possible otherwise.
• 14:08
  SPEAKER_00: So this DAPA image understanding uh program is one example.
• 14:11
  SPEAKER_00: The uh strategic computing autonomous land vehicle is another another example of this, and there are there are many like this in when we look at federal funding.
• 14:20
  SPEAKER_02: It's interesting.
• 14:21
  SPEAKER_02: You know, is is there an extent to which then you know federal funding or or or research of that nature is perhaps more um more pliable or more open to change than uh than VC or startup funding, which is very relatively narrow, at least very very focused on um uh you know a specific goal and bringing oftentimes a specific technology to market.
• 14:48
  SPEAKER_00: Yeah, it's it's it's different and it's complementary.
• 14:51
  SPEAKER_00: I I think part of the uh issue sometimes in the conversation is that they're seen as um either in competition or as uh you know completely different universe, right?
• 15:05
  SPEAKER_00: That don't need each other or something.
• 15:07
  SPEAKER_00: But in fact, they are truly complementary.
• 15:12
  SPEAKER_00: You mentioned this aspect of being uh concentrating on some uh narrower uh objective, uh, but another difference is the long term versus the short term.
• 15:24
  SPEAKER_00: Okay.
• 15:25
  SPEAKER_00: Um and uh if you look at uh all of the uh uh you know major discoveries, uh if you go back to the research, the arc and the history of research that that created it, this is a very long-term process.
• 15:45
  SPEAKER_00: And so this is the kind of a of a difference between the two uh uh approaches, but again, a difference in a good way that is actually complementary.
• 15:54
  SPEAKER_02: I'd love to talk specifically about the arc that's led us to where we are as far as uh autonomous driving and how Pittsburgh, you know, CMU, of course, being the major player here, became such an epicenter for um autonomous driving.
• 16:12
  SPEAKER_02: Uh you know, it sounds like uh um uh the autonomous lab vehicle program and nav lab were a big part of that.
• 16:20
  SPEAKER_02: So how do you how do you really get from point A to point B there?
• 16:23
  SPEAKER_00: Yeah, uh by the way, the uh ALV program was certainly a big part of it, but there were other parts.
• 16:28
  SPEAKER_00: There's uh the in the whole robotics institute, for example, uh uh there's a group called the uh Field Robotics Center, for example, that was instrumental in really elevating that that idea of autonomy uh uh to a much uh to a much higher level.
• 16:45
  SPEAKER_00: Uh I think part of the answer to your question is something that we miss sometimes when we talk about investment in research and uh how that arc of development works.
• 16:59
  SPEAKER_00: Uh we tend to think about technology only, right?
• 17:03
  SPEAKER_00: Uh, you know, this particular technique, this this other technique is being deployed, and et cetera, et cetera.
• 17:10
  SPEAKER_00: But we miss sometimes the angle of uh talent, expertise, culture.
• 17:17
  SPEAKER_00: Part of the reason why we have so many um individuals that are in a technical uh uh or management leadership position in the robotics in general and self-driving industry in particular that come from CMU that can be traced back to CMU.
• 17:36
  SPEAKER_00: A big part of that reason is that over all those decades that you describe, we have built this culture, this body of expertise, this environment where talent emerges uh around those topics of autonomy and and robotics.
• 17:53
  SPEAKER_00: It's a much more subtle effect than just pointing to a particular bit of technology.
• 18:00
  SPEAKER_00: As a matter of fact, you were asking about the uh uh early days of the ALV.
• 18:04
  SPEAKER_00: Clearly, some of the algorithms that we used uh at the time, for the most part, uh are not in use anywhere anymore, of course.
• 18:13
  SPEAKER_00: So you could say, well, then what's the connection?
• 18:16
  SPEAKER_00: Well, that's the connection is that by working on those things we learn and we tried other things and we developed other things, and we have this community of people that that gets developed, this community of expertise that gets developed, right?
• 18:32
  SPEAKER_00: And that eventually, once things are ready for uh commercialization, deployment, and all this, we have this this community, we have this expertise, and that's that's our connection to the industry, not just the technology.
• 18:47
  SPEAKER_02: So, in this context, what what does what does culture mean and uh you know what what do you do to uh promote that?
• 18:55
  SPEAKER_00: So there are two things that uh we do here uh that I think are pretty unique.
• 19:02
  SPEAKER_00: I mean they're not unique as ideas when I say them, but the degree to which we do this uh is I think uh a little different than than other places.
• 19:12
  SPEAKER_00: Uh the first one is to take a broad, a very broad view of the of the field.
• 19:17
  SPEAKER_00: And by the field, I don't mean just you know self-driving.
• 19:20
  SPEAKER_00: I mean I mean all of robotics, in fact, all of computer science and everything that has to do with computing in general.
• 19:27
  SPEAKER_00: A broad view that goes from AI algorithms, etc., all the way to physical systems and designs of physical system through the uh human aspects, modeling a human behavior, understanding uh human interaction, etc.
• 19:45
  SPEAKER_00: So that integrates all of those aspects.
• 19:48
  SPEAKER_00: And this is something that we've done in the uh School of Computer Science here uh since the beginning, uh from the beginning from our founders, from their definition of the uh of the field.
• 19:59
  SPEAKER_00: Um and I think that's something that's very important.
• 20:01
  SPEAKER_00: The mistake that uh we need to always avoid is to take a very narrow view of the field that tends to to use a technical machine learning term to overfit on a particular technique, a particular shiny object.
• 20:18
  SPEAKER_00: Okay.
• 20:19
  SPEAKER_00: Um and uh the reality is that when you look at having safe, trustworthy uh system that actually make an impact, it takes all of those things that I talked about, right?
• 20:31
  SPEAKER_00: All that breath of things.
• 20:32
  SPEAKER_00: The second aspect is and and that is more on the cultural aspect, uh, is to uh uh understand the connection between the fundamental research and the impact and the systems and the integration.
• 20:49
  SPEAKER_00: And to not put artificial barrier between the two.
• 20:56
  SPEAKER_00: You may have heard sometimes uh you know, in discussion, uh people say, well, this is this is not really um you know fundamental research or academic research, it's very applied or something.
• 21:08
  SPEAKER_00: Or vice versa, hearing uh you know someone say, Well, this is um this is nice, but this is really uh theoretical stuff that's not going to have an impact.
• 21:19
  SPEAKER_00: You hear sometimes those those things.
• 21:21
  SPEAKER_00: Those are the the very thing to absolutely avoid.
• 21:25
  SPEAKER_00: Right?
• 21:26
  SPEAKER_00: Those are the very things that can completely cure uh uh new ideas.
• 21:30
  SPEAKER_00: Uh and so uh on the cultural side, this is uh this is what what we try to do, is to try to have an environment where we have this this direct connection and this direct um even deeper respect for all of the aspects of research and development, from the most fundamental theoretical uh aspects all the way to the most uh um physical.
• 22:00
  SPEAKER_00: Systems integration aspects.
• 22:02
  SPEAKER_00: And that's essential to our culture.
• 22:04
  SPEAKER_00: And I think that's part of what has served us very well in all these years in making progress.
• 22:11
  SPEAKER_02: Yeah, from the outside, obviously as we're looking at people who are working on these bleeding-edge technologies, we tend to think of them as being very forward-thinking.
• 22:26
  SPEAKER_02: But I think to a certain extent, what you're getting at is that even researchers working on the edge of technology can still be too hung up on certain orthodoxies and certain ways of doing things.
• 22:53
  SPEAKER_02: Yeah, I was thinking about this recently from the standpoint of, I think neural networks are probably a perfect example of this, right?
• 23:01
  SPEAKER_02: I mean, as far as the degree to which they changed the game for artificial intelligence, you know, particularly over the last few years.
• 23:09
  SPEAKER_02: And they completely changed research for a lot of people.
• 23:15
  SPEAKER_02: And I suspect because of that, a lot of people have since had to adapt their work quite a bit.
• 23:22
  SPEAKER_00: Yes, and and and the same applies to them, to the to those techniques, right?
• 23:28
  SPEAKER_00: It does not mean that those techniques are the end of the story.
• 23:32
  SPEAKER_00: And we need to be very careful to not act like at the end of the of the story.
• 23:37
  SPEAKER_00: So it's a constant uh uh evolution uh of the ideas.
• 23:43
  SPEAKER_00: I don't even want to say of the technology or anything like that, but of the ideas, of the of the thinking.
• 23:48
  SPEAKER_00: And we need to have that frame of mind and we need to prepare people.
• 23:52
  SPEAKER_00: That goes, for example, to the way we train students, to have that frame of mind of constant changes and uh to use a very simple expression, to frankly keep an open mind.
• 24:04
  SPEAKER_00: That sounds like a silly thing, but in a scientific or technical uh field, it's sometimes actually uh very hard to keep an open mind.
• 24:13
  SPEAKER_00: It's much easier to center on a particular direction, particular technique.
• 24:17
  SPEAKER_00: Trevor Burrus, Jr.
• 24:17
  SPEAKER_02: What's really interesting to me is you know entering this field, um, you know, I I would say even uh 10, 15, 20 years ago versus today, uh never mind uh 30, 40 years ago, um, that it seems that so much of what you would have been working on would have felt uh theoretical at the time, that it would have felt you know very academic.
• 24:41
  SPEAKER_02: No?
• 24:41
  SPEAKER_00: No, you're you're saying that.
• 24:43
  SPEAKER_00: Yeah, no, no, not not not in our uh not in our case.
• 24:47
  SPEAKER_00: I think I think the the the better uh characterization uh is that we had that spectrum from the theoretical to the uh I like a better term, practical.
• 25:05
  SPEAKER_00: Well, I don't like practical, but you know, more uh possible uh possible or integrated uh side.
• 25:13
  SPEAKER_00: The difference over the years is that what could be achieved on that side, I don't want to say right or left because I don't know how it appears on the screen, but the the difference on that side of the spectrum, right?
• 25:27
  SPEAKER_00: Uh uh the more applied side, uh um is that uh we could not achieve nearly as much as what can achieve can it can be achieved now.
• 25:37
  SPEAKER_00: Right?
• 25:38
  SPEAKER_00: Now one can go much further, much faster in terms of the impact.
• 25:45
  SPEAKER_00: Right?
• 25:46
  SPEAKER_00: So that that's the difference.
• 25:47
  SPEAKER_00: But that was always there, right?
• 25:50
  SPEAKER_00: That that spectrum.
• 25:51
  SPEAKER_02: Yeah, it it's it's been very fascinating to to me speaking with people who have been working as uh again, especially on autonomous vehicles over the past uh 10 to 15 years with all of the money and research that's gone into them.
• 26:07
  SPEAKER_02: You know, obviously we've seen a bit of a boom and bust cycle as far as um funding goes.
• 26:13
  SPEAKER_02: And I I know that this isn't obviously specific to um self-driving, but that there are a lot of things that happen faster than anybody anticipates, and there are a lot of things that that took that have taken longer.
• 26:28
  SPEAKER_02: Um, you know, if I if I had asked you 10 years ago where we would be at as far as actually having you know self-driving cars on the road, you're smiling, self-driving cars on the road.
• 26:41
  SPEAKER_02: Um do you think that we would be a little further along than we are right now?
• 26:47
  SPEAKER_00: Uh so first of all, I'm smiling because at the time that was the number one question that would be asked, of course, right?
• 26:54
  SPEAKER_00: Uh which I consistently refuse to answer, by the way.
• 26:57
  SPEAKER_00: Okay, because I was very skilled.
• 26:59
  SPEAKER_00: I was very skilled at that.
• 27:01
  SPEAKER_00: Um reasons for for not answering the question uh is that, well, the real answer is a very boring answer, is it depends.
• 27:11
  SPEAKER_00: And here is why it's an important it depends.
• 27:14
  SPEAKER_00: The exact condition under which the technology is used matter.
• 27:21
  SPEAKER_00: And they matter not just as to whether it works or not, or whether it's commercially viable or not, it really matters at a fundamental level.
• 27:34
  SPEAKER_00: Are you doing driving in a place that has been mapped extensively, like Remo does in San Francisco and so forth?
• 27:42
  SPEAKER_00: Or are you now trying to drive in a place that you have never seen uh before?
• 27:48
  SPEAKER_00: Right?
• 27:48
  SPEAKER_00: Uh are you um uh driving in conditions that have never rules of the roads or conditions that have never been seen before?
• 28:02
  SPEAKER_00: Uh what is the density of pedestrians, people on foot, etc., around you.
• 28:10
  SPEAKER_00: All of those questions are fundamental questions as to, you know, what is what can be done.
• 28:16
  SPEAKER_00: And in fact, in some cases, they they may require radically different solutions in terms of sensors, in terms of the way the system is trained and all that.
• 28:26
  SPEAKER_00: That's why I consistently, to everybody Changren, by the way, I consistently uh uh with you, at least very careful in answering those questions.
• 28:37
  SPEAKER_00: Because general statements, broad statements like this, from from a from a uh technical standpoint, don't really make sense.
• 28:47
  SPEAKER_00: They don't really apply.
• 28:49
  SPEAKER_00: The second uh thing uh on uh self-driving, but any physical AI, especially that has potentially any uh um interaction with people, and especially catastrophic possible interaction.
• 29:05
  SPEAKER_00: The amount of um effort, and by effort I don't mean just money, I mean uh ideas, uh processes and all that, to do testing and validation and to actually get to a level where the system can be validated to a point where it can be deployed, is uh consistently underestimated.
• 29:29
  SPEAKER_00: Right?
• 29:30
  SPEAKER_00: And I think that's um you know, sometimes people are surprised that it took so long for I think that's where your question was going, that it took so long for self-driving car to be deployed.
• 29:40
  SPEAKER_00: Uh, you know, if you talk to people at those those companies, and we have a lot of CMU connections, uh the the the you know, the the basic technology you're right, w was ready quite some time ago.
• 29:52
  SPEAKER_00: I mean that's true, the b the basic.
• 29:54
  SPEAKER_00: But now to go from that from something that can truly be validated to a level that you can have, you know, the the general public public around, right, can take that risk.
• 30:05
  SPEAKER_00: That's a completely different game.
• 30:07
  SPEAKER_02: Aaron Powell Yeah, i i i it's it's interesting too, you know, something that I've been talking to quite a bit with physical AI companies.
• 30:15
  SPEAKER_02: Um I think field AI is a great example of this.
• 30:18
  SPEAKER_02: And there's somebody, you know, that they're opening up uh space in your new building.
• 30:22
  SPEAKER_02: Um is this idea, getting back to this idea of data collection of the robotics data gap, is this idea of the of the flywheel right now, right?
• 30:32
  SPEAKER_02: I mean, um and and you talked about uh mapping out a city, and certainly we have more data from cars than we do far and away than we do physical real world data from robots.
• 30:46
  SPEAKER_02: So the question is if we need a lot of hard real world data, how much of this is going to come ultimately from simulation?
• 30:55
  SPEAKER_02: How much of it is going to come from real-world data?
• 30:58
  SPEAKER_02: And if it is real world data, how do we get to a point where we can deploy enough robots to start to get that flywheel moving?
• 31:07
  SPEAKER_00: You just gave a wonderful summary, except for one item that you did not mention that I will, a wonderful summary of all the key challenges in the research on uh robotics data fabrication, right?
• 31:21
  SPEAKER_00: Uh and um exactly to your point, the the work is basically figuring out uh how to strike the right balance between the physically acquired data, which is which is limited, uh the simulation, uh perhaps other sources.
• 31:38
  SPEAKER_00: So uh there are other uh uh companies or research group and all that who work with uh videos, right?
• 31:46
  SPEAKER_00: You you have you have um just like you have an enormous corpus of text from uh uh you know people, right, on the internet.
• 31:54
  SPEAKER_00: There's an enormous corpus of videos of people doing things, right?
• 32:00
  SPEAKER_00: You could use that as input to training as well.
• 32:03
  SPEAKER_00: Uh uh accurate physical simulation is another another source, and by the way, it's not just simulation of the uh actions and the interaction, but it's also simulation of entire scenarios, right?
• 32:17
  SPEAKER_00: And being able to do that at scale.
• 32:19
  SPEAKER_00: So all of those things are on the table.
• 32:22
  SPEAKER_00: And uh coming back again to not giving a specific answer, which I'm apparently very good at, uh uh uh obstacle avoidance, as they say.
• 32:33
  SPEAKER_00: Yeah, obstacle avoidance, yeah.
• 32:35
  SPEAKER_00: Uh uh but but it's actually still uh the the the the the real answer is that exactly what approach and what combination of approaches makes sense really depends in a technical way.
• 32:50
  SPEAKER_00: Not not not in ways of investment and things like this, but in a technical way on the environment, on the task, and all that.
• 32:57
  SPEAKER_00: So uh field AI, as the name indicates, looks at kind of field robotics applications, things that are what we call the 3Ds, dirt dirty and dangerous applications, right?
• 33:07
  SPEAKER_00: Which are things where typically you don't have people, for example.
• 33:10
  SPEAKER_00: Uh they have certain characteristics.
• 33:12
  SPEAKER_00: There's a universe of scenarios that they're looking at.
• 33:16
  SPEAKER_00: That informs you know what it means to create data in that scenario.
• 33:21
  SPEAKER_00: There are other entities that look more at um a home robotics situation.
• 33:27
  SPEAKER_00: Well, now the quality of the interaction with a person becomes critical.
• 33:31
  SPEAKER_00: And the safety aspects are very different.
• 33:35
  SPEAKER_00: So you might want to think of a completely different way of thinking about the data, right?
• 33:40
  SPEAKER_00: So that's what actually makes it exciting, right?
• 33:43
  SPEAKER_00: It's a very rich uh area.
• 33:45
  SPEAKER_00: It's not as um, you know, straightforward as, oh, we're going to find w uh, you know, the the the ultimate way of of uh uh fabricating data for for robotics.
• 33:57
  SPEAKER_00: It's actually a very rich uh field of its own.
• 34:00
  SPEAKER_00: Aaron Powell You're a longtime Yenser now, right?
• 34:02
  SPEAKER_00: Is that Yes, that's uh 42 years or so.
• 34:06
  SPEAKER_02: 42 years.
• 34:07
  SPEAKER_02: Um it obviously it's changed quite a bit in in that time, and and I have to imagine that um CMU is a has been a really large part of that.
• 34:17
  SPEAKER_02: Um, you know, something that we were uh I I was I spent uh the last uh nine or so years of my career at TechCrunch and the last several visits I made out to Pittsburgh, we're coming out there and speaking with startups.
• 34:31
  SPEAKER_02: Um, you know, prior to the pandemic, there was a big question of brain drain, right?
• 34:36
  SPEAKER_02: Of of how do we maintain um all of all of uh these really smart people, how do we keep them in this industry and you know, versus going to San Francisco or or New York?
• 34:49
  SPEAKER_02: Um I I have to imagine that to a certain extent, the pandemic probably changed that a little bit, you know, allowed people to work remotely to a certain extent.
• 34:58
  SPEAKER_02: Um how has that changed in in recent years and and to what degree is uh a large project like this robotics center uh a bid to really sort of keep people local and and keep people innovating in the Pittsburgh area?
• 35:14
  SPEAKER_00: Yeah, so it's it's part of um uh this effort to to uh expand the ecosystem here in here in Pittsburgh, right?
• 35:24
  SPEAKER_00: This goes back to that uh what I said earlier about this uh uh harder to quantify effect of the environment and the culture, right?
• 35:35
  SPEAKER_00: Uh so that's basically the uh expanding that that environment.
• 35:39
  SPEAKER_00: And hopefully by doing this, creating more talent and retaining uh retaining more talent.
• 35:45
  SPEAKER_02: Yeah, one of the things um uh that the that you were speaking about around the opening of the center that I thought was very interesting was this idea of cross-pollination, you know, of getting a lot of, you know, obviously people in in in physical AI and and and robotics, but um in sort of uh different corners of that field working within close proximity to each other and giving them sort of opportunities to uh work with one another.
• 36:12
  SPEAKER_00: Yeah.
• 36:12
  SPEAKER_00: Uh yeah, the co-location and the uh it's it it all boils down to this idea of exchanging ideas, right?
• 36:20
  SPEAKER_00: And being able to work together on ideas.
• 36:23
  SPEAKER_00: It's not just um being able to work uh on systems or being able to be in the same office writing code, right?
• 36:32
  SPEAKER_00: Uh it's the idea of of being able to think together and create new uh new ideas.
• 36:39
  SPEAKER_00: And yes, the the intent is for something like the RIC uh to uh uh enhance our ability to work like this with uh with companies, with startups, etc.
• 36:50
  SPEAKER_02: So we uh we we talked a bit about the uh the 75,000 gallon water tank.
• 36:57
  SPEAKER_02: What are some of the other features in this new building?
• 37:01
  SPEAKER_00: Uh a very large drone cage.
• 37:03
  SPEAKER_00: It's far larger than anything we had access to.
• 37:07
  SPEAKER_00: That drone cage is connected inside to uh a very large uh motion capture facility.
• 37:13
  SPEAKER_00: So we can have motion capture both inside and outside.
• 37:17
  SPEAKER_00: We have uh a large outdoor testing area that can be uh reconfigured as well as uh uh indoor testing area.
• 37:25
  SPEAKER_00: Uh interestingly, uh we have uh the uh we, meaning the building has a significant uh wet lab space.
• 37:35
  SPEAKER_00: For example, a uh materials research component to robotics.
• 37:41
  SPEAKER_00: Uh and uh the this this uh building is of course uh half the uh School of Computer Science and jointly with the uh College of Engineering.
• 37:50
  SPEAKER_00: And one of the work that the College of Engineering does in particular, together with our uh colleagues in in robotics, uh uh has to do with uh soft robotics, it has to do with designing new materials, uh both for uh tactile sensing, for artificial skin, artificial muscles, for uh robots, and that kind of thing.
• 38:10
  SPEAKER_00: Uh so there's all kinds of activities around the physical form of robots, looking at radically different ways of thinking about what physical devices look like uh that require those wet lab uh kind of thing that you would not naturally think as being part of robotics or physical AI.
• 38:30
  SPEAKER_02: Yeah, I think you know it's a f I a few times uh ago when I was out there, I I'd spoken with some of the drone team, and and if I remember correctly, um uh swarming behavior was a big part of what was being studied out at CMU.
• 38:43
  SPEAKER_00: Aaron Powell Yeah, that's one aspect.
• 38:45
  SPEAKER_00: So swarming behavior, uh resilience of system.
• 38:50
  SPEAKER_00: Um so again, we go from uh things that are very uh uh autonomy uh uh emphasized, you know, uh autonomous swarming, et cetera, uh all the way to things that are more um systems-oriented.
• 39:06
  SPEAKER_00: So for example, uh in computer science department, uh we look at this issue of how to distribute the computation between a device at the edge, could be a drone, could be a phone, could be you know another device, and a central uh or intermediate facilities.
• 39:25
  SPEAKER_01: How do you're talking about sort of a shared brain across devices to the center?
• 39:30
  SPEAKER_00: How do you distribute the AI processing and the data uh based on the computational resources, based on the network, you know, whether you have good co a good connection, not good connect, or bad connection, et cetera?
• 39:44
  SPEAKER_00: How do you do that optimally and seamlessly in a way that maximizes performance overall?
• 39:50
  SPEAKER_00: You know, that's another that's that's an example.
• 39:53
  SPEAKER_00: Coming back to my uh repeated point about connecting fundamental research to to kind of what is traditionally called applied.
• 40:01
  SPEAKER_00: The way I describe it's a very um uh uh you know practical uh thing, right?
• 40:07
  SPEAKER_00: You you want your device to work as as optimally as it can uh under all conditions, right?
• 40:14
  SPEAKER_00: That's very practical.
• 40:15
  SPEAKER_00: But underneath that, there are very hard fundamental research questions.
• 40:21
  SPEAKER_00: So that's the kind of thing that I love and that I think uh is where progress is, is this this uh this uh combination.
• 40:29
  SPEAKER_00: Um and then we have other, coming back to your question about the drone group, um other um uh uh areas of research that have to do with uh uh specific scenarios, like one of our big projects, the one that actually started, the first one that that started at the at the Robotics Innovation Center is called the Triage Project, uh which is basically a first responder project.
• 40:55
  SPEAKER_00: It does the name Triage.
• 40:57
  SPEAKER_00: And the idea is in a disaster situation, uh earthquake, etc., something what is uh a situation where there's a lot of casualty, to have uh robots drawn coming, being able to assess from a distance the different the uh state of the different victims and be able to make decisions as to what intervention needs to be taken.
• 41:18
  SPEAKER_00: Ground robots that can um uh uh uh apply those interventions all the way to uh evacuation uh autonomously.
• 41:28
  SPEAKER_00: Uh so that's that's a very an example of a very ambitious project that again motivates uh considerable fundamental research.
• 41:36
  SPEAKER_00: For example, you need to develop sensors that can sense at a distance as many medically relevant sites as possible.
• 41:45
  SPEAKER_00: That's one example.
• 41:46
  SPEAKER_00: Uh and that involves uh fundamental research in sensing and in computer vision, etc.
• 41:52
  SPEAKER_02: Yeah, this is and and this this I think speaks to you know what what has been um a large part of your your own work and your own focus as well.
• 42:00
  SPEAKER_02: I I know that you had done uh quite a bit of work into uh what we used to call elder care, what we would call age tech now.
• 42:11
  SPEAKER_02: And it sounds like um to an extent those are motivated in a similar way by uh serving the needs of people first versus having the technology itself come first and then kind of you know backwards engineering what function this could serve for people.
• 42:27
  SPEAKER_00: Yeah.
• 42:27
  SPEAKER_00: And so you're referring to the uh something called the uh Quality of Life Technology Center, which was uh National Science Foundation Center.
• 42:36
  SPEAKER_00: Uh it's interesting to look, as you may have noticed, I like to uh think uh in terms of history and historical terms, right?
• 42:44
  SPEAKER_00: And and um it's interesting to look at at this uh center and and the history uh uh around it.
• 42:52
  SPEAKER_00: Uh this center was several years, uh number of years ago, I don't remember the exact years, it might have been now at least ten years ago or something.
• 43:01
  SPEAKER_00: Um and the ideas behind were uh quite forward-looking in terms of having system helping uh the elderly.
• 43:08
  SPEAKER_00: The important thing here was helping, not replacing uh some capabilities, but actually uh helping.
• 43:17
  SPEAKER_00: And the idea was to uh uh maintain the people in their uh uh everyday life environment as long as possible.
• 43:27
  SPEAKER_00: Right?
• 43:28
  SPEAKER_00: Uh so that was uh uh a very important uh project.
• 43:32
  SPEAKER_00: But the interesting observation there is that if I compare what we were able to do at the time and what could be done now, the progress is is uh considerable.
• 43:43
  SPEAKER_00: And when it comes in particular to rob robots and physical AI, we were at the time very limited by the fact that we needed to use uh fairly complex, uh expensive uh and because They're big and complex, they're not necessarily that safe.
• 44:04
  SPEAKER_00: A robot, right?
• 44:06
  SPEAKER_00: The evolution now with physical AI is that there is at least the possibility, the potential of having devices that are much uh simpler, much uh safer as physical in the physical form uh that can be used in in the home in uh in proximity to people.
• 44:26
  SPEAKER_00: Uh so the evolution is is very interesting in in that uh it's now going to enable uh those those things.
• 44:35
  SPEAKER_02: So moving from the sort of the historical view to to the future, uh, you know, I I'm guessing uh again, obviously the center has been a uh uh a a long time in in the works and and I I imagine everyone's kind of been lobbying for for you know their for their projects to get in there.
• 44:53
  SPEAKER_02: Uh but to an extent you do really have to, you know, you mentioned the the material lab.
• 44:59
  SPEAKER_02: There's the drones, there's this this large um not a pool, a tank.
• 45:05
  SPEAKER_02: Uh but you you know it's it's it's a really it's a it's a it's a big center, but you do have the future-proof, right?
• 45:12
  SPEAKER_02: You do have to create the center that is going to be of use for hopefully you know decades and and generations to come.
• 45:20
  SPEAKER_02: So how how how do you do that?
• 45:22
  SPEAKER_02: How how do you look forward into what uh uh research around physical AI and robotics is going to evolve into and make sure that this center continues to serve its function in the future?
• 45:34
  SPEAKER_00: Yeah, so um I guess there are some things that that will always remain when we talk about physical AI.
• 45:40
  SPEAKER_00: Coming back to earlier in the conversation, when I said when we talk about physical AI, we need to have well, it's kind of an obvious statement, frankly, but we need to have physical testing, right?
• 45:51
  SPEAKER_00: Uh so we need that space, and we need that space.
• 45:54
  SPEAKER_00: Uh to your point about the future, we need that space to be designed in a way that it can evolve and that it can be reconfigured.
• 46:04
  SPEAKER_00: So the testing and development space that we have are um if you look at the building and if you if you look at the space, it's they are very open.
• 46:15
  SPEAKER_00: All right.
• 46:15
  SPEAKER_00: We have not, and this was a little bit controversial by the way, uh, we have not uh prescribed a particular conf uh very uh fixed configuration of those spaces.
• 46:28
  SPEAKER_00: Uh you know, with some kind of uh preconceived idea that uh this is where we're going to test X, Y, and Z, and this is how uh the kind of research that we're going to do.
• 46:40
  SPEAKER_00: We kept those things very open and by construction reconfigurable and and changeable uh over time.
• 46:48
  SPEAKER_00: Right?
• 46:49
  SPEAKER_00: Uh exactly because to your point, we cannot um uh predict uh what uh what the future configuration of the projects is going to be and what kind of work is going to take place.
• 47:01
  SPEAKER_02: Well, I think we're actually at about time right now, but uh it was an absolute pleasure.
• 47:05
  SPEAKER_02: I thank you so much for taking the time.
• 47:07
  SPEAKER_02: Thank you for the opportunity.
• 47:09
  SPEAKER_02: Thank you.
• 47:10
  SPEAKER_02: Uh thank you so much to Marshall and CMU.
• 47:12
  SPEAKER_02: Uh hope to visit that uh huge facility someday.
• 47:15
  SPEAKER_02: I'm inviting I'm inviting myself and the rest of Ada Beta to go uh visit you.
• 47:20
  SPEAKER_02: So hopefully we'll be doing that in the very near future.
• 47:22
  SPEAKER_02: Uh thank you so much for joining in this um thanks uh to you as always for tuning in.
• 47:28
  SPEAKER_02: If you like the show, please like and subscribe.
• 47:30
  SPEAKER_02: Please tell our friends.
• 47:32
  SPEAKER_02: Uh we also have a newsletter with original features that drops every Thursday morning.
• 47:37
  SPEAKER_02: You can find all of that and more over at unlimited.nfm.
• 47:41
  SPEAKER_02: And with all of that, we will catch you just about this time next week with another episode of Unlimated.