Latest / Automated with Brian Heater / Sergey Levine on Why Real-World Data Will Define Physical AI

Transcript

Sergey Levine on Why Real-World Data Will Define Physical AI · Automated with Brian Heater

• 0:00
  SPEAKER_03: Robotics at this stage is at kind of an inflection point where developing more capable and more sophisticated robotic foundation models requires more than just a laboratory science approach.
• 0:11
  SPEAKER_03: It also requires a more holistic kind of industrial undertaking.
• 0:14
  SPEAKER_03: One of the things that I think about a lot is how can we make sure that as robotics researchers we're tackling the right problems in anticipation of the dynamic that we'll see at this larger scale?
• 0:23
  SPEAKER_03: And to me, the kind of work that we're doing at physical intelligence in part serves that purpose.
• 0:28
  SPEAKER_03: What we would want to see ultimately is robots deployed in many different settings, settings that are collecting a lot of data, but it's also very important to do it correctly.
• 0:37
  SPEAKER_00: Is there a way to kind of start aggregating this to really kind of take on that huge data gap that exists right now?
• 0:43
  SPEAKER_03: Actually, getting real-world data and then deploying robots is a lot easier.
• 0:48
  SPEAKER_03: In the research community, people often ballcap.
• 0:50
  SPEAKER_03: There's a lot of work that tries to avoid the need for real-world data.
• 0:54
  SPEAKER_03: I don't think that there's anything wrong with using other data sources, but I think that if we do it on top of models that are trained on lots of real-world data, we'll get a lot further.
• 1:01
  SPEAKER_00: When we reflect back on this moment, how do you think we're going to define this moment in robotics and physical AI in hindsight?
• 1:08
  SPEAKER_01: Do you want like an optimistic answer?
• 1:10
  SPEAKER_00: I want the pessimistic and then the optimistic answer.
• 1:13
  SPEAKER_02: Okay.
• 1:14
  SPEAKER_02: So the optimistic answer is that Animated Podcast.
• 1:32
  SPEAKER_00: My name is Brian Heater.
• 1:33
  SPEAKER_00: I am the managing editor at the Association for Advancing Animation.
• 1:37
  SPEAKER_00: We've got a very special guest for you this week, and Sarah Gay Levine joins us from physical intelligence to discuss the star's approach to training centered around real-world data collection.
• 1:48
  SPEAKER_00: I have actually not gone back and listened to this episode since writing our newsletter feature, but I'm going to go out on a limb and suggest that you not take a shot every time that one of us mentions the data flywheel.
• 2:00
  SPEAKER_00: Thank you so much to Sterugate for joining us.
• 2:02
  SPEAKER_00: Thanks to you as always for listening to the show.
• 2:05
  SPEAKER_00: If you've been enjoying it, please like and subscribe.
• 2:08
  SPEAKER_00: And with that, please enjoy this conversation with Saragunny.
• 2:12
  SPEAKER_00: We talk a little bit about what's coming up next in automation on the show.
• 2:16
  SPEAKER_00: But if you really want to see the future in motion, you've got to be there in person.
• 2:21
  SPEAKER_00: Automate 2026 is where the world's leading innovators, builders, and dreamers come together to show what's possible.
• 2:29
  SPEAKER_00: Robots, AI, machine vision, motion control, you name it, all automation under one roof.
• 2:36
  SPEAKER_00: Register for free at automateshow.com to join us in Chicago June 22nd through the 25th.
• 2:43
  SPEAKER_00: We'll see you there.
• 2:45
  SPEAKER_00: So you're one of, I was trying to do the math on this.
• 2:49
  SPEAKER_00: There are, what, six co-founders of physical AI?
• 2:53
  SPEAKER_00: I I have I haven't, I don't, I feel like I haven't found quite like a definitive resource of the number anywhere.
• 2:59
  SPEAKER_03: There's seven of us.
• 3:00
  SPEAKER_00: There's seven of you, that which is uh obviously, obviously quite a bit.
• 3:04
  SPEAKER_00: Um was was it was it kind of piecemeal?
• 3:07
  SPEAKER_00: Did all of you all of you come together?
• 3:09
  SPEAKER_00: Did this happen over an extended period of time?
• 3:12
  SPEAKER_03: A lot of us have actually worked together for for quite a while before that.
• 3:16
  SPEAKER_03: So on the uh AI and machine learning side of things, many of us worked together previously at um at Alphabet at Google DeepMind, where we developed a lot of the foundations for the kind of robotic models that we're working on now.
• 3:29
  SPEAKER_03: The the two other people that joined after that were uh Anan Esmile, who is the uh the head of hardware here, and uh Kegroom, uh who uh was previously an investor.
• 3:38
  SPEAKER_00: Aaron Powell Did you come over or did that group come over straight from Google Deep Mind?
• 3:41
  SPEAKER_03: Aaron Powell Basically, yes.
• 3:42
  SPEAKER_00: Aaron Powell In the work that you were working on, I mean, obviously they're still doing some uh very interesting things over there.
• 3:47
  SPEAKER_00: What was it something that you felt like you needed to kind of break off and do independently?
• 3:52
  SPEAKER_03: Aaron Powell Robotics at this stage is at kind of an inflection point where developing more capable and more sophisticated robotic foundation models requires more than just um kind of a laboratory science approach.
• 4:06
  SPEAKER_03: It also requires a more holistic kind of um industrial undertaking.
• 4:10
  SPEAKER_03: So uh as an analogy, in the world of LLMs, right, there were fundamental technologies that needed to be developed, transformers, that sort of thing.
• 4:17
  SPEAKER_03: But there was also a lot of what goes into making it work has to do with data collection, data curation, a lot of kind of systems and uh holistic effort that goes beyond just the um the core technical research.
• 4:30
  SPEAKER_03: And in robotics, it's even more like that.
• 4:32
  SPEAKER_03: We needed to develop um new ideas about how data should be collected so that it reflects the demands of uh uh of real-world work.
• 4:42
  SPEAKER_03: So I that that takes more than just kind of like a elaboratory kind of effort.
• 4:46
  SPEAKER_00: Aaron Powell Yeah.
• 4:47
  SPEAKER_00: I uh I've obviously there are a number of different approaches.
• 4:50
  SPEAKER_00: There have been a number of different approaches to to data collection.
• 4:53
  SPEAKER_00: Um is there something, you know, was there something novel or or or something that sort of set your approach apart from what was already existing at the time?
• 5:03
  SPEAKER_03: Aaron Powell I wouldn't say there's any one thing.
• 5:05
  SPEAKER_03: I think it's more about uh addressing the data problem, not just from the standpoint of uh kind of like a research style kind of training set, test set kind of paradigm, but more from the standpoint of like, you know, what kind of experience does a robot really need to be able to perform a wide range of real-world tasks?
• 5:25
  SPEAKER_03: And some of that means uh being really smart about setting up some tasks and smart about like choosing a sufficient diversity of environments, but some of it also means collecting experience that's very realistic, where you actually put the robot into a real world environment where there's some work that needs to be done.
• 5:40
  SPEAKER_03: And we actually do that work, collect the corresponding data, and then get something that's like very representative of real world's choice.
• 5:46
  SPEAKER_00: Yeah, it's interesting.
• 5:47
  SPEAKER_00: We actually just happened to get off the a call with um another San Francisco company, Weave, who like actually they have some robots out and out in the real world right now.
• 5:56
  SPEAKER_00: So they they do have a I don't know if you would call it a are are are taking a flywheel approach right now.
• 6:02
  SPEAKER_00: Or um uh to what extent are you actually deploying systems out in real-world environments at the moment?
• 6:10
  SPEAKER_03: What I would say is this I think that so far we have experiments with deployment.
• 6:14
  SPEAKER_03: So we had, for example, our robot um assembling boxes at the Dandelion Chocolate Factory.
• 6:19
  SPEAKER_03: Uh that's very convenient because dandelion is across the street from our offices, so uh we could set up a robot there and it you know, it spends all day kind of building these boxes that they use to pack up chocolates.
• 6:29
  SPEAKER_03: We have a coffee service at the office.
• 6:31
  SPEAKER_03: Uh so right outside where I'm standing right now, there's a station set up where somebody can go on the company like Slack system, type in like I want uh a latte, and the robot will actually go and make it.
• 6:42
  SPEAKER_03: Um these these are not really like high horsepower kind of commercial efforts.
• 6:47
  SPEAKER_03: They're really experiments to see what happens when uh the robot has to solve real-world tasks.
• 6:52
  SPEAKER_03: And they're also experiments on how data from these kinds of deployments can actually be used to further improve the system.
• 6:58
  SPEAKER_00: Yeah, so obviously like a very different approach than you know we're seeing with a lot of the humanoid companies right now, um, which uh uh at least as far as you know having a big partnership with like a like an automotive company or something like that.
• 7:10
  SPEAKER_00: And to a certain extent, I don't know if I don't know if lower stakes is the right way of putting it, but um certainly the the pressures are different than if you're dealing directly with like a Mercedes, for example.
• 7:21
  SPEAKER_03: Aaron Powell I I think that there's merits to uh to many different kinds of uh uh deployments like that.
• 7:27
  SPEAKER_03: One thing that I think is really interesting about some of the tasks that we've uh uh we have studied is that uh they're they're actually they involve doing work that people want done, like people want a coffee made, but they also involve a lot of edge cases and a lot of kind of uh difficult situations that provide the system with opportunities to improve.
• 7:46
  SPEAKER_03: And we have a few different ways to drive that improvement, including from autonomous experience as well as from human interventions.
• 7:51
  SPEAKER_03: But I think the kind of the ideal set of uh uh jobs for a robotic foundation model that drives improvement are jobs that kind of that you can do, that you can do decently well, but that also offer many opportunities to fix up mistakes, get better, experience kind of some occasional failures that were hard to predict in advance, and then resolve those failures either through reasoning or through adaptation or through human intervention.
• 8:13
  SPEAKER_03: Because then the robot is is learning more.
• 8:14
  SPEAKER_03: It's not just like doing a thing repeatedly, but it's actually acquiring new experiences.
• 8:18
  SPEAKER_00: Yeah, I I guess like in in the specific in instance of a robot that is in your office, you know, making coffee right now.
• 8:25
  SPEAKER_00: Um obviously like the scale isn't really there, right?
• 8:29
  SPEAKER_00: I mean, it's you know, it's it's just making a few coffees at a time.
• 8:32
  SPEAKER_00: Like how how much value can actually be derived from that specific deployment?
• 8:37
  SPEAKER_03: Aaron Powell It's a prototype for a larger system.
• 8:39
  SPEAKER_03: So I think that uh what what you what we would want to see ultimately is uh robots deployed in many different settings, settings that are collecting a lot of data, but uh it's also very important to uh to do it correctly.
• 8:52
  SPEAKER_03: So we need to make sure that we have um the the right technology so that uh all of that experience is actually making the model better.
• 8:59
  SPEAKER_03: We also have to make sure we understand which kinds of situations are the most conducive to collecting useful experience.
• 9:06
  SPEAKER_03: So there may be some very repetitive tasks where there's very limited value, at least for the model, in doing that task for more than a few hours.
• 9:12
  SPEAKER_03: There may be other tasks where there's sort of like an end- endless uh variability.
• 9:16
  SPEAKER_03: And we want to be very smart about uh collecting those kinds of experiences.
• 9:21
  SPEAKER_00: That's interesting.
• 9:21
  SPEAKER_00: I mean what what what constitutes a a useful experience?
• 9:25
  SPEAKER_03: Aaron Powell So I one kind of bit of intuition about this is maybe uh inspired by uh something in um uh in child psychology.
• 9:34
  SPEAKER_03: So there's this notion called the zone of proximal development.
• 9:37
  SPEAKER_03: The zone of proximal development is when you are uh doing something that you're not that completely clueless about, where you have some sense for how to get started, but that provides just enough challenge that there's room for improvement.
• 9:48
  SPEAKER_03: So a a g a good uh uh you know early childhood education system puts children in that zone of proximal development.
• 9:54
  SPEAKER_03: And what we want to do is we want to put our robots in that zone of proximal development.
• 9:57
  SPEAKER_03: Now, I think that's kind of easier said than done because obviously the analogy very quickly breaks down, and then we have to think very hard about uh what that actually means for the model, which might defy our intuition about people.
• 10:07
  SPEAKER_03: Uh so maybe there's some basic motor skills that need to be practiced, maybe there's some higher level uh skills that need to be practiced.
• 10:13
  SPEAKER_03: There's actually a lot of surprises there, but it also depends on how you're supervising the model, right?
• 10:17
  SPEAKER_03: So sometimes we supervise the model with verbal corrections.
• 10:21
  SPEAKER_03: So somebody actually tells it, like, hey, uh in this environment, you should uh pick up the plate and put the plate in the drying rack instead of putting it in the dishwasher.
• 10:28
  SPEAKER_03: That's not action supervision, that's actually verbal supervision, which the model can integrate.
• 10:32
  SPEAKER_03: In other settings, it's a more low-level physical failure.
• 10:34
  SPEAKER_03: So then someone needs to uh actually take over control and illustrate the correction.
• 10:39
  SPEAKER_03: So all of these things, my point is that they all kind of play play together and you have to get all that confluence of factors right.
• 10:44
  SPEAKER_00: Aaron Powell Yeah, I I had heard you describe this in in another interview, and I thought you put it in a really um I mean very plain uh language, but um uh uh an interesting way that I hadn't heard it quite described as before.
• 10:57
  SPEAKER_00: But uh one of the the differences between uh data collection you know with um cars and robotics is that you know cars are obviously like they exist in the real world.
• 11:09
  SPEAKER_00: So they're out there collecting data already.
• 11:11
  SPEAKER_00: And um you you kind of have to go get creative with robots when it comes to collecting data.
• 11:17
  SPEAKER_00: And are there are there ways to scale that?
• 11:20
  SPEAKER_00: I mean, you know, and and and and is does that involve right now taking some of these kind of like baby steps, you know, a weave, for example, or uh doing a few of these pilots with uh you know a chocolate factory box company, um, or they're gonna have to be other creative approaches uh to creating like synthetic data, for example.
• 11:40
  SPEAKER_03: I think it is this is a place where I think there is room for creativity.
• 11:44
  SPEAKER_03: I think it's also one of those things where as the system becomes more and more capable, a lot of the stuff becomes easier to do.
• 11:50
  SPEAKER_03: So uh when we started the company uh two years ago, the biggest challenge for us was getting basic motor skills that work with any degree of reliability.
• 11:59
  SPEAKER_03: Now we're at a point where we can't actually get very good uh basic motor skills for lots of tasks, and the challenge is kind of shifting upwards, where now it's maybe more important to make sure that the robot makes the right decisions in the context of a very long horizon task.
• 12:12
  SPEAKER_03: Uh for example, we we were evaluating a kitchen cleaning uh policy a few days ago, and there the problem is that, okay, the robot is supposed to like clean all wash all the plates, put them in the drying rack, and it drops one of the plates on the floor.
• 12:24
  SPEAKER_03: Now, it drops one on the floor after having washed it, so then it picks it up and like puts it away.
• 12:28
  SPEAKER_03: Well, what it should do is wash it again because it's on the floor, so there's kind of a common sense element.
• 12:33
  SPEAKER_03: The good thing though about this is that as the bottleneck shifts further upward, in some sense we actually actually have more tools that we can bring to bear because those problems also look a lot more like the problems faced with uh by you know LM agents and uh vision language models and so on, kind of other areas of AI.
• 12:47
  SPEAKER_00: Aaron Powell Yeah, I mean how how like ultimately like how finalized of a product, and I guess how generalized of a product do you feel like you're going to have to have to really uh deploy this at scale?
• 13:01
  SPEAKER_03: I think that really depends a lot on uh what kind of uh uh application uh you're thinking about.
• 13:08
  SPEAKER_03: Our take on this is that the right kind of robotic intelligence layer can be helpful across a broad swath of applications.
• 13:16
  SPEAKER_03: Like it's it's very it like really captures the imagination for people to think about, oh, well what would it be like to have a robot in my home that does like household chores?
• 13:23
  SPEAKER_03: And then that's some great applications.
• 13:24
  SPEAKER_03: But there are also lots of other applications, and each of them have uh a different uh uh standard, a different slope, a different set of challenges.
• 13:32
  SPEAKER_03: Some demand much more speed, throughput, robustness, others demand much more of this kind of common sense.
• 13:40
  SPEAKER_03: And we want to improve the model along all those dimensions, but as we do that, certain opportunities will open up to for deployment for these kind of data flywheels, others will open up later, and as long as we're sort of progressing and each of these things kind of bootstraps the next one, then I think this stuff is in good shape.
• 13:55
  SPEAKER_00: Aaron Powell Yeah, I feel it feels like to a certain extent, maybe that the the the company has kind of been built up around um obviously you know being able to have a uh uh a long runway, uh, but you know, in insofar as you you are actually sort of making progress towards a broader goal of generalization, that you're on the right path and that you're not really feeling in the meantime any specific pressure to commercialize a product.
• 14:24
  SPEAKER_03: Well, what I would say is this that um like AI technology is in a really good place right now uh in terms of uh the kind of the the internet-based AI.
• 14:33
  SPEAKER_03: Like you can you can use uh coding agents, you can use LLMs to like help answer questions, like research your next vacation and stuff like that.
• 14:40
  SPEAKER_03: But when it comes to stuff in the physical world, there are a lot of major open challenges.
• 14:45
  SPEAKER_03: And I think that a balance that needs to be struck by anybody that wants to uh really make a fundamental breakthrough in like a real-world applicability of robots is to balance kind of taking advantage of the opportunities, but at the same time actually making a serious investment into solving the really hard problems in the long term.
• 15:04
  SPEAKER_03: And there are really, really hard problems.
• 15:05
  SPEAKER_03: So uh physical uh AI can benefit a lot from LLMs, computer vision, et cetera, but it also requires us to solve fundamentally new challenges that those areas do not have to contend with.
• 15:17
  SPEAKER_03: And it's very important to set up the kind of frameworks, the kind of uh evaluation structures, the kind of data sets that would allow rapidly iterating on those really hard problems.
• 15:27
  SPEAKER_03: And these are not the kind of things that are going to be solved like in a month or in or in two months.
• 15:31
  SPEAKER_03: It's gonna take a very serious scientific effort, which will be helped a great deal by the advances in in all these other areas.
• 15:37
  SPEAKER_00: Yeah.
• 15:38
  SPEAKER_00: Is that though?
• 15:39
  SPEAKER_00: I mean, you know, when when when you know, and I'm sort of coming at this from I was I was at TechCrunch prior to this.
• 15:44
  SPEAKER_00: I'm coming at this from somebody who, you know, has like taken like who to who talks to a lot of VCs and who has talked to a lot of startups and you know who at a certain point there there there is that pressure to kind of to deliver something.
• 15:57
  SPEAKER_00: Um do you but do you do you find that for the most part that uh there is a sort of understanding that this is something that is going to take you know a a long time, but that ultimately it's going to be worth it?
• 16:09
  SPEAKER_03: Yeah.
• 16:10
  SPEAKER_03: Yes.
• 16:10
  SPEAKER_03: I I think the the the short answer is yes.
• 16:12
  SPEAKER_03: Uh and I I think it's also very important to kind of keep in mind the upside here, which is that you know, it's sort of one thing to build an AI-enabled version of an existing robotic systems like like, for example, uh, you know, uh warehouse automation where robots need to pick up objects and things like that.
• 16:31
  SPEAKER_03: Like, you know, there are some solutions that exist, and those solutions can be made incrementally better by leveraging the latest and greatest advances in like computer vision.
• 16:39
  SPEAKER_03: But if we can develop truly general robotic foundation models, the ones where you don't you don't tell it like fold a t-shirt, you tell it like, hey, uh I like to come home at 6 p.m.
• 16:48
  SPEAKER_03: Please make sure dinner is ready, uh, do the laundry on Tuesdays, and uh clean the house on on Thursdays or something.
• 16:54
  SPEAKER_03: Like that's the prompt.
• 16:55
  SPEAKER_03: And then this thing can work for like months and months doing this, and as it does it, further improve, learn about your habits, uh, learn about what goes right and what goes wrong when it performs certain tasks.
• 17:04
  SPEAKER_03: Like that has, I think, a much more uh profound transformative potential than uh kind of incrementally improving the individual areas that are already kind of working.
• 17:15
  SPEAKER_03: And the transformative potential is not from producing, you know, robot butlers.
• 17:18
  SPEAKER_03: The transformative potential is from enabling uh kind of ubiquitous physical intelligence where anybody could experiment with new hardware platforms, new application domains, load in this model, and have it provide uh a respectable first attempt at their application task and then get better from there.
• 17:34
  SPEAKER_03: I think that has enormous potential, far beyond current automation applications.
• 17:38
  SPEAKER_00: So you're talking about um uh like essentially building a platform onto which anyone can uh just develop any kind of uh physical AI application?
• 17:49
  SPEAKER_03: Yeah.
• 17:49
  SPEAKER_03: And and I think I think that's more important than any single application, because uh the truth is that right now, I think that we have you know, in our imagination, everybody has a particular kind of idea of like what they would like a robot to do.
• 18:03
  SPEAKER_03: But I think it it's very tempting to just sort of like think of robots as like a version of a person, but like made of metal.
• 18:09
  SPEAKER_00: I mean, especially because like the ones that we're building like are a version of a person that are made of metal right now.
• 18:13
  SPEAKER_00: Trevor Burrus, Jr.
• 18:14
  SPEAKER_03: Yeah, and that's what we see in science fiction and so on.
• 18:16
  SPEAKER_03: And that's cool.
• 18:16
  SPEAKER_03: Like there's nothing inherently wrong with that, but it's but I think that physical AI has potential for a lot more than that, right?
• 18:24
  SPEAKER_03: Like you could have robots that are designed for specific tasks, robots that are more general, robots that are anywhere in between.
• 18:29
  SPEAKER_03: It's it's like uh you know personal computers, right?
• 18:31
  SPEAKER_03: Or like once there was uh uh general-purpose operating system software, computers can be big, they can be small, they can be in your refrigerator, they can be in your car, they can be on your desk, uh, and they take the form that is most suited for the particular problem that they're addressing.
• 18:46
  SPEAKER_03: So if physical actuation is that ubiquitous, I think it could actually be uh the there there could be much more interesting applications than kind of the most immediate ones that come to mind.
• 18:55
  SPEAKER_00: Aaron Powell Yeah.
• 18:55
  SPEAKER_00: So I mean so so to what extent uh when you're working on any given application, are you thinking about the actual like physical embodiment of the robot?
• 19:04
  SPEAKER_03: One of the things that kind of surprised me is that um we haven't had to do anything particularly special to handle a variety of different robotic platforms.
• 19:14
  SPEAKER_03: Uh like from a kind of a science and engineering standpoint, there's something really interesting about like all the different ways you could tell the model what kind of robot it's controlling and like represent the the model in some clever way so that it sort of maps onto the robot morphology.
• 19:27
  SPEAKER_03: But when we got started, we actually found that we didn't really need to do any of that.
• 19:30
  SPEAKER_03: Uh our model is actually actually very naive when it comes to robot embodiment.
• 19:34
  SPEAKER_03: It takes the images from the cameras, it outputs a fixed-length list of numbers whose length is the maximum size needed for any robot we've ever controlled.
• 19:43
  SPEAKER_03: It's like 38, I think.
• 19:45
  SPEAKER_03: Uh so if the robot requires less than 38 uh degrees of freedom, then it just pads it with zeros and that's it.
• 19:50
  SPEAKER_03: And the rest is all in the model.
• 19:52
  SPEAKER_03: So the model absorbs data from many different robots and basically figures out from looking at the image through the camera what kind of robot it's dealing with and how it should control it.
• 20:00
  SPEAKER_00: I'm really sort of curious as as this team was coming together, it's it's been very interesting to see.
• 20:05
  SPEAKER_00: Obviously, there are these moments that, you know, you mentioned this inflection point, and there there are these moments that we've seen, certainly from the outside, where like there's a chat GPT moment, right, where it's just clear that, like, oh, this is something different.
• 20:18
  SPEAKER_00: Like these systems are accomplishing something that like certainly like we didn't think that they would be able to accomplish in a in a certain amount of time.
• 20:24
  SPEAKER_00: And there's a lot of at universities that there's a lot of uh parallel research happening.
• 20:31
  SPEAKER_00: Obviously, you know, you're collaborating a lot.
• 20:33
  SPEAKER_00: I know you went to Stanford and um you know that you've been at you you've been at Berkeley.
• 20:38
  SPEAKER_00: To what extent when you're really forming and building a team initially are you looking for those people who are doing that kind of like-minded or or parallel research that you've been performing over the years?
• 20:52
  SPEAKER_03: Um yeah, interesting question.
• 20:56
  SPEAKER_03: So there there are a few things I could say about this.
• 20:58
  SPEAKER_03: Uh one thing that's really important in robotics, especially, is that uh these are very integrative disciplines.
• 21:04
  SPEAKER_03: So uh it's very important for any effort like this to succeed to have folks with a a variety of different skill sets, even more so than in like foundation models.
• 21:14
  SPEAKER_03: Like for foundation models, you need folks that are good at systems, that are good with the ML research, that are good at math.
• 21:19
  SPEAKER_03: But for robots, it's also important to have uh a good hardware team.
• 21:22
  SPEAKER_03: It's important to have folks that really know how to build the robot stack, the software stack, people that are very good at thinking about data and tasks.
• 21:29
  SPEAKER_03: So that there's a there's a very wide range of different skill sets.
• 21:31
  SPEAKER_03: I think if uh if I just kind of looked for people just like me, then I probably wouldn't get very far.
• 21:37
  SPEAKER_03: But at the same time, I think it is very important to make sure that any team like this is very aligned on a mission.
• 21:42
  SPEAKER_03: I think one of the things that we have done, I I think quite successfully, is gathered a group of folks that very deeply care about uh the the physical AI problem, the the problem of building general-purpose AI systems for physical uh robots.
• 21:59
  SPEAKER_03: And uh I think it's actually very special to be in an environment where people like are very focused on that mission.
• 22:05
  SPEAKER_00: Yeah.
• 22:05
  SPEAKER_00: But you know, you all you all obviously you you you saw something, you know, at the same time.
• 22:10
  SPEAKER_00: This is it's certainly like this isn't the first time a group of people have gotten together and tried this.
• 22:15
  SPEAKER_00: You all sensed that there was something coming, that that something had been unlocked, that this was the time to really try this in earnest.
• 22:21
  SPEAKER_03: One story I could tell you about here is um uh about a project that we did um about a year before we started the company.
• 22:29
  SPEAKER_03: This was uh it was actually led by um my co-founder, Kwan Vong.
• 22:33
  SPEAKER_03: It was a very academic project, but I think to me at least it was like very eye-opening about the potential for this stuff.
• 22:38
  SPEAKER_03: Uh so we um we thought, okay, can we experiment with trading models on data from many different robots and do so in a way that isn't just kind of like a an internal experiment of our own, but something that kind of impacts the larger research community.
• 22:55
  SPEAKER_03: So what we did is we contacted a number of different robotics research labs, uh, 33 other labs, and we asked them to like essentially donate their data to us.
• 23:04
  SPEAKER_03: So back then, every robotics lab kind of worked in isolation.
• 23:06
  SPEAKER_03: They would get their platform, their particular task, collect some data for it, experiment with different models, and then iterate on this.
• 23:12
  SPEAKER_03: So we said, like, okay, well, since you're not doing anything else with this data anyway, like why don't you share with us and we'll try out some experiments, and in exchange, we'll like send you back the model and you can play with it if you want.
• 23:21
  SPEAKER_03: And I don't I don't think anybody that we contacted really like thought very deeply about this because they're like, okay, like, you know, it's just a little experiment.
• 23:27
  SPEAKER_03: Let's try it out.
• 23:28
  SPEAKER_03: We we trained the model on all of this data, and then we sent it back to a few of the labs that agreed to run some experiments for us, and we just told them, like, hey, take this model and compare it to whatever you're developing on your own system.
• 23:39
  SPEAKER_03: Everybody had something a little different, and it was like suited to their particular tasks.
• 23:43
  SPEAKER_03: Like uh one lab was doing some cable routing, another one was like uh taking out the trash, another one was like putting objects into drawers and things like that, whatever research tasks they had.
• 23:52
  SPEAKER_03: And the model trained across all the different data sets from all these different robots ended up being on average 50% better in terms of success rate than whatever each individual lab was developing.
• 24:03
  SPEAKER_03: And that was really surprising to me, especially uh as a researcher, because I know that if someone has been working on a particular problem with a particular system for a while, like they've really tuned the heck out of that thing.
• 24:13
  SPEAKER_03: So if we just train up a model that we've never run on their robot before and we just send it to them, and immediately they test it and it's better, that shows that something is actually really working.
• 24:22
  SPEAKER_03: Now, these were like academic tasks.
• 24:24
  SPEAKER_03: Like I think if you if you saw videos of these things, you'd think they're pretty dumb.
• 24:28
  SPEAKER_03: But the fact that this was like being evaluated by researchers that have been working on those particular tasks with their own systems and they were finding our model to be better, suggested to me that there was that like spark of potential, the possibility of a generalist system being better than specialists that are specialized to those particular tasks.
• 24:46
  SPEAKER_03: So I think to me that was like a big part of the impetus for doing this thing.
• 24:49
  SPEAKER_00: Aaron Powell That's really interesting.
• 24:51
  SPEAKER_00: Um, you know, and and and again, this is like completely my outsider view.
• 24:55
  SPEAKER_00: But uh I've been doing this for a while, and and I I talk to a lot of researchers.
• 25:00
  SPEAKER_00: Um and again, you know, there there there is collaboration that happens.
• 25:04
  SPEAKER_00: You read a lot of papers and it is and it is cross-schools.
• 25:07
  SPEAKER_00: Um but I don't know, you know, people get busy, they move on to other projects.
• 25:13
  SPEAKER_00: It's it's interesting to hear that for you know whatever reason a lot of this data just kind of sits there.
• 25:19
  SPEAKER_00: Um, is there a lot I don't know.
• 25:24
  SPEAKER_00: It would be I I I I I don't know.
• 25:26
  SPEAKER_00: Uh it'd be it would be interesting to see, you know, is is there a way to kind of start aggregating this to really kind of take on that that huge data gap that exists right now?
• 25:36
  SPEAKER_03: Aaron Powell So in the academic world a few years ago, I think that that would have been a a very reasonable thing to do.
• 25:44
  SPEAKER_03: I think at this point, the kind of industrial scale data collection efforts that we have and and that you know a few other folks have are so large that I think that they would dwarf anything that could be uh accumulated from just uh the open source uh data sets.
• 26:02
  SPEAKER_03: Yeah.
• 26:03
  SPEAKER_03: This is not something I expected, but it is actually quite nice to see that uh, at least in industry, people are now actually taking large-scale data collection very seriously.
• 26:12
  SPEAKER_03: And one of the things that I've said for a while, and maybe I feel like a bit vindicated about, is that real-world data is not actually that costly.
• 26:20
  SPEAKER_03: Like it's it's an industrial effort.
• 26:21
  SPEAKER_03: Like you kind of have to like you have to set it up, you have to get the robots, people have to actually do it, people have to be trained to do it properly, but it's not somehow like profoundly impossible relative to other kinds of industrial efforts.
• 26:33
  SPEAKER_03: It looks impossible from a very research-centric academic standpoint because like you know, it costs money, it costs resources, and costs work, typically the kind of work that professional researchers don't typically enjoy doing.
• 26:45
  SPEAKER_03: But it's uh like it's not harder than like building a house or building a bridge, right?
• 26:50
  SPEAKER_03: It's uh it it's it's work that can be done.
• 26:52
  SPEAKER_03: And if we are serious about taking physical AI to the next level, it makes sense to put in that work and get it right.
• 26:58
  SPEAKER_00: Aaron Powell When you say you were you weren't expecting that, you you just you just didn't think that you would be able to scale or to collect data at the rate that you've been able to collect it since the company started.
• 27:08
  SPEAKER_03: I I meant more uh for the community as a whole.
• 27:10
  SPEAKER_03: Like what I didn't expect is this level of sort of recognition that real-world data collection is actually like a very reasonable path towards getting very effective models.
• 27:21
  SPEAKER_03: Because it's something that in the research community, people often bulk at, right?
• 27:24
  SPEAKER_03: There's a there's a lot of work that tries to avoid the need for real-world data.
• 27:29
  SPEAKER_03: And a lot of it is actually very good work, very good scientific work.
• 27:32
  SPEAKER_03: But you know, utilizing other data sources, utilizing simulation, utilizing videos, these are good things to do.
• 27:36
  SPEAKER_03: But I think sometimes people start working on that because they kind of take it as a given that getting real-world data would be intractable.
• 27:44
  SPEAKER_03: So I don't think that there's anything wrong with using other data sources, but I think that if we do it on top of models that are trained on lots of real-world data, we'll get a lot further.
• 27:51
  SPEAKER_03: And that actually really makes sense because like as people, for example, we can um watch a cartoon on TV with very abstract graphics or or uh as a like a pilot can use a flight simulator.
• 28:02
  SPEAKER_03: And it doesn't have to look perfectly realistic, and they can still get a lot of uh very useful context out of it, but that's because they're grounding that in their own experience in the real world where they understand roughly how things work already, so they can internalize all these other sources of experience.
• 28:15
  SPEAKER_03: And I think robotic foundation models could do that same thing.
• 28:18
  SPEAKER_00: Aaron Powell I guess it maybe to a certain extent it was unpopular because it's almost like it's almost like a brute force approach to collection, right?
• 28:25
  SPEAKER_00: It's just like let's just do as much of this as possible and collect this in um I mean it's not an easy way to do it, right?
• 28:32
  SPEAKER_00: It is to a certain extent kind of it's not telly, but it is like one-to-one data collection, right?
• 28:38
  SPEAKER_03: I think it's actually easy in the grand scheme of things.
• 28:41
  SPEAKER_03: And uh maybe sometimes something that um uh we as researchers get a little bit uh uh mixed up about is easy for us versus easy in the context of like human civilization.
• 28:54
  SPEAKER_03: Okay.
• 28:55
  SPEAKER_03: And in the in the context of like, you know, the world as a whole, actually getting real-world data and then deploying robots that are going to collect more experience and get better and better is a lot easier than inventing some other technologies just to avoid having to do that.
• 29:11
  SPEAKER_03: And because it's a bootstrap problem, it's easier yet because once things are deployed, once they're out there in the world, imagine there's like a million robots out there.
• 29:19
  SPEAKER_03: If they're all collecting experience, the problem is not how to get more data, the problem is what to do with it when you've got it and how much of it to throw out.
• 29:26
  SPEAKER_03: So, because of that, to me, it actually feels very unrewarding to spend like huge amounts of uh kind of technical effort inventing solutions when it's possible to bootstrap things and then be in this easier mode where things are getting better and better.
• 29:39
  SPEAKER_00: This is fascinating.
• 29:41
  SPEAKER_00: Um, you know, and and this this is an interesting point to be at right now.
• 29:45
  SPEAKER_00: That this this is like the flywheel question, right?
• 29:47
  SPEAKER_00: Um we are at a point where we're starting to slowly, slowly deploy robots into the real world.
• 29:54
  SPEAKER_00: Um, you know, I mentioned weave.
• 29:57
  SPEAKER_00: That's that's you know, he told me I asked him how many robots he he deployed.
• 30:01
  SPEAKER_00: He said, we've deployed more robots than we have employees.
• 30:04
  SPEAKER_00: I asked him how many employees he had.
• 30:06
  SPEAKER_00: He said he had 15, so you know, extrapolate from there.
• 30:09
  SPEAKER_00: Uh you talk about these pilots that a lot of the humanoid companies are doing, and you know, most of them I would say are probably are probably single-digit.
• 30:17
  SPEAKER_00: So we're kind of in, I don't want to say we're in a holding pattern, but we're in this place where it's like, how do we get to a point where we can uh deploy that uh the I guess that like minimal viable product, right?
• 30:29
  SPEAKER_00: That mintable, minimal number of robots required to really jumpstart scalable data collection.
• 30:38
  SPEAKER_03: I think to a large extent, this is a problem uh of model capability.
• 30:42
  SPEAKER_03: And uh and that means that uh you you need both the kind of industrial part of it and the research part.
• 30:49
  SPEAKER_03: I don't think either of them in isolation works.
• 30:51
  SPEAKER_03: And I think people sometimes tend to also reduce this a little too much.
• 30:54
  SPEAKER_03: There's you know, kind of one extreme which is like all you have to do is invent technology, and the other extreme is all you have to do is use the technology you have now and pump more data into it.
• 31:01
  SPEAKER_03: And I think neither of those is correct.
• 31:02
  SPEAKER_03: I think what we need to do, and this is maybe the kind of like the boring but pragmatic answer, is that we need to be able to do that.
• 31:07
  SPEAKER_00: I love boring but pragmatic, you know, when we're talking about robots.
• 31:10
  SPEAKER_00: I know, because I mean honestly, again, and and and this is this is the conversation that I that I that I like to have because I think that there's too many.
• 31:18
  SPEAKER_00: I don't want to say it's easy, but there are too many of these like you know five-minute-long humanoid robot videos that are giving really you know unrealistic expectations as far as what a robot can do currently.
• 31:29
  SPEAKER_03: Yeah, that's right.
• 31:30
  SPEAKER_03: So so the the kind of the the the boring pragmatic answer is that you have to go in stages.
• 31:34
  SPEAKER_03: So you you scale up data, maybe by an order of magnitude, and that opens up new avenues for research.
• 31:40
  SPEAKER_03: And then we figure out, okay, given that this level of scale, what do we need to invent to get to the next level of capability?
• 31:46
  SPEAKER_03: Then you go to the next level of scale and then the next level of capability.
• 31:49
  SPEAKER_03: And it it takes it takes time.
• 31:51
  SPEAKER_03: You have to do this carefully, and if you prematurely try to go to the next level of scale, then you don't know what kind of data to collect or how to set up your experiments.
• 32:00
  SPEAKER_03: You might, for example, spend lots and lots of time doing something that will be obviated with better technology.
• 32:05
  SPEAKER_03: Maybe there's some kinds of capabilities that are better off being learned through autonomous experience, and you might waste time learning them through teleoperation, or vice versa.
• 32:14
  SPEAKER_03: So uh one example, alluding to something I said earlier of something uh uh about how research changes the nature of data that you want is uh some of our experience with with uh uh reasoning models.
• 32:25
  SPEAKER_03: So you can have a model that looks at a problem and uh it's maybe uh some high-level problem like clean the kitchen, and instead of just directly outputting actions, it says, okay, given what I'm seeing, the next step is to pick up a plate and put the plate uh in the dishwasher.
• 32:38
  SPEAKER_03: So it's just like uh uh thinking mode for uh like ChatGPT or something like that.
• 32:42
  SPEAKER_03: And then based on that, it outputs the action.
• 32:44
  SPEAKER_03: But now what you can do is you can go in and you can supervise the thoughts instead of supervising the actions.
• 32:49
  SPEAKER_03: And that and that's easier supervision to get because that can be done by somebody like speaking into a microphone, or it can be done by uh a labeler kind of looking at the video after the fact without the need to teleoperate the robot.
• 32:59
  SPEAKER_03: And initially, obviously the the problem was that you need the physical skills, but when our models got good enough, then they could absorb this additional verbal feedback and get better.
• 33:07
  SPEAKER_03: So now that is an additional avenue for data collection that we know can improve the model after we've done the research to develop this.
• 33:14
  SPEAKER_03: And who knows what kind of things might be opened up in the future as we develop new technologies.
• 33:18
  SPEAKER_03: So these things have to keep a pace to make sure that the data collection matches with the needs of the technology.
• 33:23
  SPEAKER_00: Aaron Powell What do you mean exactly when you say supervise the thoughts?
• 33:26
  SPEAKER_03: Aaron Powell It means that somebody goes in and writes text uh that indicates what the robot should have thought at that moment in order to produce the correct action.
• 33:35
  SPEAKER_00: So you're almost going in and like coding.
• 33:38
  SPEAKER_03: A little bit, yeah, but in English.
• 33:40
  SPEAKER_03: So maybe the robot should have picked up the towel instead of the plate, or it should have picked up the plate over here instead of over there.
• 33:47
  SPEAKER_03: And you can go in after the fact and tell it, like, yep, this is what your thought should have been.
• 33:50
  SPEAKER_00: Aaron Ross Powell And this is really like one of the primary places where LLMs can play a pretty key role, it sounds like yeah, yeah.
• 33:57
  SPEAKER_03: Well, and and that and that's by the way, like one open question right now.
• 34:00
  SPEAKER_03: It's like, okay, so you can do this with human labeling.
• 34:02
  SPEAKER_03: You can probably also automate it to a large extent.
• 34:04
  SPEAKER_03: If you can automate it at training time, you can probably also automate it at a test time.
• 34:07
  SPEAKER_03: So is it a problem of getting more labels, or is it a problem of improving the base model?
• 34:11
  SPEAKER_03: Like that's one of the research questions.
• 34:13
  SPEAKER_00: Yeah.
• 34:13
  SPEAKER_00: So so this is really so so the approach right now, I mean, and and this gets back to really what we were talking about at the beginning of the conversation, is is just really is starting small, uh piece by piece, um uh uh uh addressing addressable problems and then just sort of and and then continuing to build there.
• 34:32
  SPEAKER_00: But I I guess once you start the process of building, then it potentially can you can build pretty quickly.
• 34:38
  SPEAKER_03: Aaron Powell I think that this is a place where it's uh very important to um kind of have well, this is something you said earlier, to have a very sober assessment of capabilities.
• 34:49
  SPEAKER_03: And sometimes those capabilities are lower than people think, sometimes they're higher than people think.
• 34:53
  SPEAKER_03: Uh for example, uh there was this um uh uh uh uh blog post that we wrote back in December where um one of our for former colleagues from Everyday Robots, uh Benji Holson, he kind of posted like a little challenge, like these are the tasks that I think would be hard for robots to do.
• 35:09
  SPEAKER_03: And he posted this in uh in the fall, I think it was like in October or so.
• 35:12
  SPEAKER_03: Uh like these were tasks like making a peanut butter sandwich and like washing a frying pan.
• 35:16
  SPEAKER_03: So he was kind of reflecting on what on what he had experienced at uh at EDR back at Alphabet and the kind of tasks that he thought would be challenging with the robotic technologies that exist in 2025.
• 35:27
  SPEAKER_03: We um looked at those uh tasks and we actually found that with our current model, without ri doing any actual research just by collecting a little bit of specialized data, we could actually solve almost all of those tasks.
• 35:41
  SPEAKER_03: So that was an instance where actually the technology is further along than people thought.
• 35:44
  SPEAKER_03: But of course, there's other things that are less far along, right?
• 35:47
  SPEAKER_03: Uh so we could get our robot to like wash the frying pan and and uh uh and uh make a peanut butter sandwich.
• 35:53
  SPEAKER_03: But that like the the interesting part to me is that we didn't actually do anything special for that.
• 35:56
  SPEAKER_03: Like all the all the work was actually done by the people collecting the data.
• 35:59
  SPEAKER_03: The researchers basically like tested some models and that's it.
• 36:01
  SPEAKER_00: Aaron Powell So and the problem, I mean, the the disconnect there is because again, you know, the this idea of being able to show somebody a video of a robot that can do something really impressive in a three-minute video, but ultimately what is what is the disconnect?
• 36:14
  SPEAKER_00: The disconnect is being able to do that in scale and being able to do that across uh you know a variety of different environments.
• 36:21
  SPEAKER_03: Aaron Powell And particularly dealing with the edge cases that arise, right?
• 36:25
  SPEAKER_03: So the peanut butter sandwich, like, okay, maybe you can make the peanut butter sandwich fine, and and like, you know, uh nine out of ten times is perfectly reasonable, but one out of ten times you'll drop the sandwich on the floor.
• 36:35
  SPEAKER_03: And now it's not enough to like start over.
• 36:37
  SPEAKER_03: You have to like you have to clean it up, you have to maybe get the mop out.
• 36:40
  SPEAKER_03: There's like all these other things, all these other behaviors that stem from the need to remedy long tail uh issues.
• 36:48
  SPEAKER_03: And that's actually very h very hard.
• 36:50
  SPEAKER_03: In fact, it it's it's basically equivalent to the entirety of the AI problem, because pretty much anything can happen in real-world settings.
• 36:56
  SPEAKER_03: So getting that kind of common sense to react intelligently to those situations, getting the robustness, the speed, right, you also want the robot to get faster at these things so that it can be uh useful uh in practice, particularly this is especially important in industrial settings.
• 37:10
  SPEAKER_03: So all those things, they are actually quite hard, especially the common sense.
• 37:13
  SPEAKER_03: Uh so I think there's a lot of research to be done there, uh, and that's a lot of what we're focusing on.
• 37:18
  SPEAKER_00: Yeah.
• 37:19
  SPEAKER_00: Well well, I mean, one of the things that was really fascinating uh to me about your company in particular, um and this really does go back to this idea of having like seven founders is a piece of it too, is that it does feel like I mean it does feel like a research facility.
• 37:34
  SPEAKER_00: I mean, it feels like uh it just it feels like I don't want to say a playground, but it just it feels like a just a place where a lot of uh effectively a a large laboratory where a lot of research is being done.
• 37:46
  SPEAKER_00: I mean, it is not structured like a traditional startup as as we would think of it.
• 37:51
  SPEAKER_00: Is that a fair assessment?
• 37:53
  SPEAKER_03: Aaron Powell That's right.
• 37:54
  SPEAKER_03: Uh and this you know, to a large extent, this stems from our belief that the problem really is hard and that there's there are a lot of opportunities to get started.
• 38:05
  SPEAKER_03: There's a lot of a lot of the innovations in AI make it much easier to tackle this problem now than it was like five years ago.
• 38:12
  SPEAKER_03: But there are still fundamental technical challenges yet to be resolved, which will require creativity.
• 38:17
  SPEAKER_03: They'll require hard work too, and they'll require kind of the right initial conditions, the right data, the right uh models, all that stuff, but they'll require creativity also.
• 38:25
  SPEAKER_03: And creativity, uh, in order to unlock that, you really need an environment that promotes that kind of creative thinking and creative problem solving.
• 38:32
  SPEAKER_03: So that's why the company is structured to a large extent as a lab, because we think we need that to solve these hard problems.
• 38:37
  SPEAKER_03: Trevor Burrus, Jr.
• 38:38
  SPEAKER_00: As a creative writing major myself.
• 38:39
  SPEAKER_00: I mean, how how does a laboratory setting um promote creativity in a way that another setting might not?
• 38:46
  SPEAKER_03: Aaron Powell So one really important thing is to reduce the friction for experimentation.
• 38:52
  SPEAKER_03: So if um and this is by the way a big challenge across a modern AI, not just in robotics, but if someone has an idea, if they're lucky and the idea is really, really small, they can try it quickly.
• 39:03
  SPEAKER_03: But sometimes you'll have an idea that you can only try at a particular scale.
• 39:08
  SPEAKER_03: Because we know that machine learning works differently at a large scale than it does at a small scale.
• 39:12
  SPEAKER_03: So one of the big challenges in b in producing a kind of a functional research environment for modern AI research is to make it as easy as possible to try out these ideas, even when they require doing things at a larger scale.
• 39:25
  SPEAKER_03: So that means, for example, really good infrastructure for evaluations.
• 39:28
  SPEAKER_03: How do we test uh to a statistically significant degree whether our models are getting better or worse?
• 39:33
  SPEAKER_03: Uh good infrastructure for uh messing with different parts of the model, with different algorithms.
• 39:38
  SPEAKER_03: How do we test out a new reinforcement learning algorithm when it might require uh thousands of trials in the real world?
• 39:44
  SPEAKER_03: Sometimes you can use simulation for that, sometimes you can't.
• 39:46
  SPEAKER_03: So getting all those structures right uh is actually really, really important.
• 39:50
  SPEAKER_03: And of course, then the other part of it is uh creating uh an intellectually vibrant atmosphere.
• 39:55
  SPEAKER_03: And that you know, involves making sure that the right people join the company, that uh it's a place where people can talk about cool research ideas over lunch, uh, where people want to, you know, stay late and and hack away at some like creative idea that they came up with that morning.
• 40:07
  SPEAKER_03: Uh so these are kind of like the social aspects of it.
• 40:10
  SPEAKER_00: Yeah, and and and fostering almost uh uh uh democratic isn't the the right word, but you know, getting rid of uh, you know, there are founders and and and people do have titles, but getting rid of, I guess, some of the the bureaucracy, um, realizing that you've people are working at the company because they're very smart people and listening to people, you know, when they do come up with those kind of creative or out-of-the-box ideas.
• 40:34
  SPEAKER_03: Yeah.
• 40:34
  SPEAKER_03: So uh we're not unique in this regard, but we don't have titles.
• 40:37
  SPEAKER_03: Uh we have the only title is there's two titles, member of technical staff and CEO.
• 40:42
  SPEAKER_03: And there's only one CEO.
• 40:43
  SPEAKER_03: We're le we're legally required to have one.
• 40:44
  SPEAKER_03: We actually cannot have one.
• 40:46
  SPEAKER_03: Uh but that we're not unique in that regard, actually.
• 40:48
  SPEAKER_03: I think for exactly this reason, many of the uh Silicon Valley AI startups, especially ones that are very research focused, uh do adopt this kind of structure.
• 40:57
  SPEAKER_03: Uh it's also not a good thing.
• 40:58
  SPEAKER_00: It explains it's been really hard for me to connect with people on LinkedIn, and this this explains why to figure out who's who over there.
• 41:05
  SPEAKER_03: But I think it's also it's also a recognition of a certain truth, which is that um, you know, I mentioned that this kind of work requires a heterogeneity of skills, but it also requires everybody to sort of like keep a significant chunk of the problem in in their mind.
• 41:20
  SPEAKER_03: Because if you don't look at this thing holistically, then it's very hard to identify the most important challenge to tackle.
• 41:25
  SPEAKER_03: So everyone uh needs to kind of be good at their particular thing, but they all but people also need to understand the big picture and where their particular uh kind of superpower can fit into addressing that big picture.
• 41:37
  SPEAKER_03: Uh and again, I don't think that's unique to us.
• 41:40
  SPEAKER_03: I think that for any uh effective AI research lab, getting that right is really important.
• 41:44
  SPEAKER_00: Aaron Powell And then remaining uh, you know, it strikes me that sort of remaining keeping one foot in um academics is also extremely useful.
• 41:54
  SPEAKER_00: I mean, you know, almost like being forced to um uh monitor like the breakthroughs that are happening on that side is is also a superpower that you're bringing to the table.
• 42:04
  SPEAKER_03: Aaron Powell One of the things that I'm uh often a bit concerned about, and to be honest, this was part of the reason why I was excited to start this company, is that in the academic research world, um in robotics, I felt like the community might be entering like a little bit of a local optimum where it's hard to work on the most important problems because they're a little bit out of reach due to p non-science factors like availability of data, having the right kind of uh platforms, all that kind of stuff.
• 42:35
  SPEAKER_03: And again, by analogy, if you think about like let's say natural language processing research, uh okay, well, a lot of the technologies that went into LLMs were developed in research labs and in academia, but getting to the point where everybody could do research on LLMs also required an industrial effort.
• 42:52
  SPEAKER_03: And without that, there were still researchers working on problems, but many of those problems were not the right problems, at least in hindsight.
• 42:58
  SPEAKER_03: I mean, obviously at the time they seemed really good problems, but in hindsight, they were not the right problems because this next level of scale changed the dynamic.
• 43:05
  SPEAKER_03: So one of the things that I think about a lot is how can we make sure that as robotics researchers, we're tackling the right problems in anticipation of the dynamic that we'll see at this larger scale.
• 43:15
  SPEAKER_03: And to me, the kind of work that we're doing at physical intelligence in part serves that purpose.
• 43:20
  SPEAKER_03: In part it serves the purpose of helping make sure that we understand like what's the dynamic for that that next level, not just for ourselves, but also for the research community as a whole.
• 43:29
  SPEAKER_00: Yeah, I I'm I'm I'm not gonna I'm not gonna make you do a prediction because I I hate predictions.
• 43:34
  SPEAKER_00: And I and I used to do this and I realized that it's very easy to keep pushing the goalpost, you know, everything's five to ten years, it's perpetually five to ten years.
• 43:43
  SPEAKER_03: Because that's about as far as we can reasonably see things.
• 43:46
  SPEAKER_00: Yeah, especially when it comes to like like some things like like a like AGI.
• 43:51
  SPEAKER_00: But um here's what here's what I will do.
• 43:54
  SPEAKER_00: You know, I you know, when when we reflect back on this moment, obviously a very exciting time, um both from a hardware and a software standpoint.
• 44:06
  SPEAKER_00: How do you think we're gonna define this moment in robotics and and and physical AI in hindsight?
• 44:13
  SPEAKER_01: Do you want like an optimistic answer?
• 44:16
  SPEAKER_00: I want I want the pessimistic and then the optimistic answer.
• 44:20
  SPEAKER_02: Okay.
• 44:22
  SPEAKER_03: So the optimistic answer is that uh I think it's very possible that uh we might end up looking back on kind of the the early to mid-2020s as a bit of a transitional period in AI where people were still training their AI systems largely on data from the internet.
• 44:41
  SPEAKER_03: And I suspect that if we have physical systems that are deployed very broadly in the real world, not like a few hundred robots, but millions of robots, maybe every household will have a robot, then the way that we train AI systems will change fundamentally because we'll have this source of data that is so reflective of human experience that it'll be much easier to train sophisticated uh AI agents, not just for robots, but for everything, own that kind of data.
• 45:07
  SPEAKER_03: And we might look back and say, like, hey, isn't it kind of funny that people used to train their AI systems on data from the internet, which is that thing that we use to like, you know, to post pictures of cats and and and uh swear at people on bulletin boards, right?
• 45:18
  SPEAKER_03: So uh I think that there is something kind of very exciting there because physical experience has a kind of quality, uh it it allows you to understand causal structures, it allows you to understand intuitive physics, and all these things are actually very fundamental to human thinking.
• 45:35
  SPEAKER_03: Like obviously, you know, our brains exist in the physical world.
• 45:38
  SPEAKER_03: Uh and uh I think because of that, we might get much more powerful AI systems that are in some ways actually much more familiar to us from using primarily that kind of data.
• 45:46
  SPEAKER_03: Now, obviously it's a long road to get there, uh, but I think that would be tremendously exciting, and maybe that's actually uh what we'll think of when we look back on this.
• 45:53
  SPEAKER_00: Great.
• 45:54
  SPEAKER_00: Well, uh Sergei, it's been a pleasure.
• 45:56
  SPEAKER_00: Thank you so much for taking the time.
• 45:58
  SPEAKER_00: Thank you, Brian.
• 45:59
  SPEAKER_00: Thanks so much for tuning in to another episode of Untimated.
• 46:02
  SPEAKER_00: Thanks to Saragay Living United for joining us.
• 46:04
  SPEAKER_00: If you've been enjoying the show, the easiest way to support us is to subscribe, to like, to comment.
• 46:10
  SPEAKER_00: You can find more info on the show, you can funding show in the moats, you can fund in the weekly newsletter over at untimated.nfm.
• 46:17
  SPEAKER_00: And with all of that, um, we will see you next week for another episode of Montimated.