Latest / Automated with Brian Heater / Ranjay Krishna on Why Robots Still Fail in the Real World and the Data Problem Holding Them Back

Transcript

Ranjay Krishna on Why Robots Still Fail in the Real World and the Data Problem Holding Them Back · Automated with Brian Heater

• 0:00
  SPEAKER_01: For us and our focus, it's really been about navigation.
• 0:04
  SPEAKER_01: Can we get these robots to move around in spaces that they've never seen before?
• 0:08
  SPEAKER_01: A couple of years ago in 2023, we scaled up all of our simulation environments.
• 0:15
  SPEAKER_01: And the bet that we were making is the sip to real problem is only a problem because our simulation environments lack the diversity that the real world has.
• 0:42
  SPEAKER_00: Recently back from GTC as we are recording this.
• 0:46
  SPEAKER_00: I talked and wrote a lot about physical AI in the past week.
• 0:50
  SPEAKER_00: So it is fortuitous timing to be running an episode with Ranja Krishna, in addition to serving as an assistant professor at the University of Washington.
• 1:00
  SPEAKER_00: He's been working on the robotic data problem at the Allen Institute for some time now.
• 1:05
  SPEAKER_00: A lot of folks at the NVIDIA event seem to share his opinion that more training might actually be accomplished in simulation than had originally been thought.
• 1:14
  SPEAKER_00: It's a great talk, thanks to Ranjay.
• 1:17
  SPEAKER_00: Also, Automated is coming to Boston.
• 1:20
  SPEAKER_00: We are taking our first ever road trip in a few weeks.
• 1:23
  SPEAKER_00: We're recording a bunch of episodes in person, including some in front of a live audience.
• 1:28
  SPEAKER_00: We're also hosting a happy hour from 4 to 6 p.m.
• 1:32
  SPEAKER_00: at Mass Robotics on April 8th.
• 1:34
  SPEAKER_00: Thank you to Maxon for sponsoring that.
• 1:37
  SPEAKER_00: Thanks to you for tuning in.
• 1:39
  SPEAKER_00: If you would like to support Automated, please like and subscribe.
• 1:42
  SPEAKER_00: Please tell a friend.
• 1:44
  SPEAKER_00: Also check out the newsletter over at automated.fm.
• 1:47
  SPEAKER_00: And with that, please enjoy this conversation with Ranjay Krishna.
• 1:52
  SPEAKER_00: So we uh recently uh spoke with uh Sarah Gay over at physical intelligence, and he said something that I hadn't I hadn't heard before from a physical AI company, which was he felt that a lot of companies are maybe overestimating like how how difficult the process of actually like going out in the real world and collecting data is.
• 2:13
  SPEAKER_00: I mean, is that a fair assessment?
• 2:16
  SPEAKER_01: Oh, absolutely.
• 2:17
  SPEAKER_01: Whenever we've had to go out and collect data ourselves, it's it's not just that you need the hardware and everything set up correctly, but you need to train people to give you data that's actually useful for training robots.
• 2:31
  SPEAKER_01: And the training process isn't easy.
• 2:33
  SPEAKER_01: You have to get people to move in ways that feels unnatural to them.
• 2:37
  SPEAKER_01: They're usually teleoperating these robots.
• 2:40
  SPEAKER_01: They have to learn the form factor of the robot that they're moving.
• 2:44
  SPEAKER_01: It's quite a lot of uh quite a lot of actual setup that actually goes into um the entire process before you even have a usable data collection process.
• 2:55
  SPEAKER_00: One of the really interesting topics that we've been discussing quite a bit on the show recently is the data flywheel and this point that we're at right now where it seems very like chicken and the egg of there aren't enough robots deployed or out in the world, or at least like potentially general purpose robots deployed out in the world to collect the physical data needed to deploy that many robots out in the world.
• 3:21
  SPEAKER_00: So, yeah, how do we get to a point where we can actually like start and and what sort of scale do we need to actually start collecting that kind of data?
• 3:29
  SPEAKER_01: The way I look at it, we do have quite a few robots actually already deployed, uh, but they're deployed in very sort of structured environments, usually in factories where you know exactly what they're going to be doing.
• 3:42
  SPEAKER_01: And so in those scenarios, um the data flywheel gets you data that basically looks the same as each other.
• 3:49
  SPEAKER_01: Um, what we really need is robots working in unstructured environments for this data flywheel to become useful.
• 3:56
  SPEAKER_01: And by unstructured, I mean places and environments where it's not clear what's going to happen in those environments, and your robot has to adjust its behavior.
• 4:07
  SPEAKER_01: So, in your home, for example, maybe you spilled something, maybe uh you fell down.
• 4:12
  SPEAKER_01: You'd want your robot to sort of do something in response to things that you're doing.
• 4:17
  SPEAKER_01: And we don't even have robots good enough that or safe enough even that we can deploy them in these unstructured environments.
• 4:25
  SPEAKER_01: And so this data flywheel, to get it even started, we need these robots to be good enough where we can put them out in some unstructured environments.
• 4:35
  SPEAKER_01: Uh, and we're just so far away from that at the moment.
• 4:38
  SPEAKER_00: One of the things that we've seen really rise to prominence, uh, especially recently, is uh synthetic data.
• 4:45
  SPEAKER_00: Um, you know, NVIDIA has been a big proponent of this, which is actually like training these robots in uh in simulation.
• 4:55
  SPEAKER_00: And obviously you can do that a lot faster and a lot more at scale.
• 5:01
  SPEAKER_00: Um one of the things that always jumped out at me though is are you risking the possibility of suddenly like introducing hallucinations and introducing errors into the process by doing things that way?
• 5:14
  SPEAKER_01: I'm a big proponent of uh simulation.
• 5:18
  SPEAKER_01: So NVIDIA's been doing a lot of great work in building out simulation engines and even environments in these simulation engines where you could train these robots.
• 5:28
  SPEAKER_01: The biggest sort of reason why a lot of roboticists have shied away from simulation, at least recently, is uh something they call the sim to real gap.
• 5:38
  SPEAKER_01: The hypothesis is that you can train a robot in simulation, but then it'll only work in that simulation.
• 5:43
  SPEAKER_01: The moment you take it out and deploy them in the real world, they're suddenly not going to be able to adjust.
• 5:49
  SPEAKER_01: And this adjustment, the reason it's so difficult is because all the simulation engines we build, they're all an approximation of our real world.
• 5:58
  SPEAKER_01: We approximate physics, we approximate forces, we approximate material properties.
• 6:04
  SPEAKER_01: And so when your robots do train in these environments, they uh they sort of learn the sort of idiosyncrasies of that simulator.
• 6:15
  SPEAKER_01: And so, if, for example, your gravity isn't exactly right or physics and collisions aren't exactly correct, it'll learn to model those kinds of errors.
• 6:25
  SPEAKER_01: And when you go to the real world, suddenly those errors can compound over time.
• 6:29
  SPEAKER_01: So you might expect a cup to be firmer than it actually is.
• 6:33
  SPEAKER_01: Maybe you might break a glass because you held it too tight.
• 6:37
  SPEAKER_01: All of these material changes or collision changes or physics changes actually makes it very difficult for these robots to adjust.
• 6:44
  SPEAKER_01: Um, now what's interesting though is um we at AI2 have uh been betting big on simulation for quite a while now.
• 6:54
  SPEAKER_01: Um, since about 2017, we've been building out simulation engines very similar to Nvidia.
• 7:00
  SPEAKER_01: But for the longest time, for us and our focus, it's really been about navigation.
• 7:05
  SPEAKER_01: Can we get these robots to move around in spaces that they've never seen before?
• 7:10
  SPEAKER_01: And a couple of years ago in 2023, we scaled up all of our simulation environments.
• 7:17
  SPEAKER_01: And the bet that we were making is simulation, the SIP to real problem is only a problem because our simulation environments lack the diversity that the real world has.
• 7:28
  SPEAKER_01: And so we built out hundreds and thousands of environments.
• 7:32
  SPEAKER_01: These are environments that look like museums, they look like schools or conference rooms or even people's homes.
• 7:39
  SPEAKER_00: And you're you're talking about edge cases and things that might be difficult to otherwise predict in a standard simulation.
• 7:47
  SPEAKER_01: I'm actually talking about just the diversity of data that's available in simulation.
• 7:51
  SPEAKER_01: Um, when you look at a lot of simulation engines, they usually focus on one type of environment.
• 7:57
  SPEAKER_01: Uh, for example, the kitchen.
• 7:58
  SPEAKER_01: Nvidia's built out a lot of simulation engines, we uh environments recently, but a lot of them are just kitchen environments because they want, at the end of the day, be able to deploy robots in people's homes.
• 8:09
  SPEAKER_01: But kitchens only give you some amount of diversity, right?
• 8:12
  SPEAKER_01: It only has uh objects that you typically find in a kitchen.
• 8:16
  SPEAKER_01: And so your robots, even if they do adjust to a set of kitchen environments, they might not be able to go to the living room and do anything meaningful there.
• 8:24
  SPEAKER_01: Because the objects that you see, the sofa, the TV, they might have never seen those before or interacted with them.
• 8:31
  SPEAKER_01: And so suddenly they would get very confused about how to deal with that change.
• 8:35
  SPEAKER_01: And so that's the bet we were making is that if you can scale up the diversity and generate environments that represent all the places that we might ever want to deploy a robot and train these robot models across all of these environments, then suddenly they might just be able to work in the real world.
• 8:54
  SPEAKER_00: To a certain extent, it sounds like a different approach than what I've been hearing from a lot of companies lately, in that, and this is especially coming from the robotics companies themselves, um, the humanoid companies, of let's get really good at doing a couple of tasks first and then sort of scale out from there.
• 9:14
  SPEAKER_00: But it sounds like what you're suggesting here is, hey, let's just let's go for it.
• 9:19
  SPEAKER_00: You know, let's build as big of a model as we can all at once.
• 9:24
  SPEAKER_01: That's maybe a fair assessment.
• 9:25
  SPEAKER_01: If you look at um what's worked really well uh with AI models today, um, if you look at language, for example, for for the longest time, uh linguists were really focused on um solving very specialized tasks.
• 9:40
  SPEAKER_01: You know, tasks like entailment, is what I'm saying going to imply the next thing I'm about to say?
• 9:46
  SPEAKER_01: You know, those kinds of tasks or sentiment analysis is what I'm saying a good sentiment, you know, good thought or a bad thought.
• 9:52
  SPEAKER_01: You know, a lot of those kinds of problems they were working on.
• 9:55
  SPEAKER_01: At some point around um 2017, 2018, we started basically throwing all of these tasks together and started training language models on all of the data that we had available.
• 10:08
  SPEAKER_01: Um, and we saw a lot of what I call um just uh complementary behavior, where learning to do one task allows you to improve on your performance on something else.
• 10:19
  SPEAKER_01: And so today you can ask these models to, you know, draft an email for you, to help you edit things, to give you ideas, to help you brainstorm, even talk through your problems.
• 10:29
  SPEAKER_01: Um and all of them require a different set of skills.
• 10:32
  SPEAKER_01: But because they've been exposed to everything, uh they learn from all of these different areas to improve on any specific skill.
• 10:40
  SPEAKER_01: Uh, and I'd be willing to bet the same thing is true for robotics as well.
• 10:44
  SPEAKER_01: Uh, that if you train these models across all different kinds of capabilities in as many environments as possible with as many form factors as possible, they will get better.
• 10:55
  SPEAKER_00: In a sense, are people, I don't want to say they're, I mean, you know, I don't want to say they're overestimating the complexity, but uh they're they're they're overestimating how dissimilar training for training physical AI is versus these, you know, kind of like large language models that we've seen over the past several years.
• 11:17
  SPEAKER_01: In one sense, I think there are similarities.
• 11:19
  SPEAKER_01: A lot of the problems that we're trying to solve are similar to the problems we've had with um language understanding or visual understanding, where um if you expose them to a lot of different kinds of data and train them with a general purpose machine learning model, they should learn useful skills.
• 11:39
  SPEAKER_01: So that component, I think, is common.
• 11:42
  SPEAKER_01: Uh the component that's maybe not common is that robotics is fundamentally an active process.
• 11:47
  SPEAKER_01: You train these robots to act in the world, and when they act in the world, the world itself changes.
• 11:52
  SPEAKER_01: And that active process is something that's really hard to model.
• 11:56
  SPEAKER_01: Whereas in language, when you feed a model an input, the input stays the same.
• 12:01
  SPEAKER_01: And so they just respond to that one input.
• 12:03
  SPEAKER_01: And but in robotics, everything changes the moment the robot does anything.
• 12:07
  SPEAKER_01: Or if anyone else in the environment around them does anything.
• 12:11
  SPEAKER_01: Um, and that active problem basically makes it so that you can't just make decisions in isolation.
• 12:16
  SPEAKER_01: You have to make decisions sequentially.
• 12:19
  SPEAKER_01: And that sequential decision-making process means that a lot of errors, if you make a mistake on any step, they can compound over time.
• 12:26
  SPEAKER_01: This is what makes deploying these robots so difficult and why they're so unreliable even today.
• 12:31
  SPEAKER_01: Even if you get to a robot model that's about 90% accurate, that 90% accuracy on one point in time means that over five minutes or over 10 minutes, that accuracy is going to go way down because that error is going to compound.
• 12:46
  SPEAKER_00: And it strikes me too, you know, and uh we'll be talking about uh a lot of the work that you your team has been doing more recently over there.
• 12:54
  SPEAKER_00: Is it MOLMO?
• 12:55
  SPEAKER_00: Is it MOLMO?
• 12:56
  SPEAKER_00: MOMO, yeah, yeah.
• 12:58
  SPEAKER_00: You've primarily been testing it with these two research arms.
• 13:02
  SPEAKER_00: You've been testing with a tabletop arm and a mobile manipulator.
• 13:06
  SPEAKER_00: And again, it seems like those are like vastly different levels of complexity, right?
• 13:10
  SPEAKER_00: A stationary arm on a table versus one that's actually moving around as it's interacting with the world.
• 13:15
  SPEAKER_01: Yeah.
• 13:16
  SPEAKER_01: So maybe I can give you a little bit of a background about what we've been doing.
• 13:20
  SPEAKER_01: Yeah.
• 13:20
  SPEAKER_01: Um, so you know, um, at the Allen Institute, like I was saying, we've been really focused on navigation for a long time.
• 13:28
  SPEAKER_01: And a couple of years ago is when we finally decided that our environments were diverse enough where it now became possible for us to start thinking about um more complex robotics tasks.
• 13:39
  SPEAKER_01: So not just moving around in space, uh in the case of navigation, but more complex tasks like being able to pick things up, move things around, open drawers, close doors, uh, things that require you to simultaneously move your body as well as manipulate the objects around you.
• 13:54
  SPEAKER_01: Uh, even if you take the case of opening a door, that's actually quite a hard problem, something that we do so seamlessly.
• 14:01
  SPEAKER_01: But you have to think about what to grab, first of all.
• 14:04
  SPEAKER_01: You need to know where the handle is, you need to move your arm to that handle, uh, grab it.
• 14:08
  SPEAKER_01: But as you're opening the door, the door is going to collide with your body.
• 14:11
  SPEAKER_01: And so you need to move your entire body as you're sort of opening that door.
• 14:15
  SPEAKER_01: And so that process means that you need full control over the entire set of joints on the robot to be able to do that one simple task.
• 14:23
  SPEAKER_01: And that's not easy at all because a lot of the robots that we have have so many joints.
• 14:28
  SPEAKER_01: We're really hoping that if we were to train these models uh to do tasks that are vastly different from each other, and with robots that are vastly different, like you're saying, with some that are uh single arms, that are static, not moving, and entire robot embodiments that where you can move move the entire robot body.
• 14:47
  SPEAKER_01: Um, training across all of these is basically going to allow us to um generalize these models to hold new use cases.
• 14:57
  SPEAKER_00: Reading through the release, that's something that jumped out at me that that I appreciate here is well well, one, you're keeping it open, but two, you're saying, hey, you know, check our work here.
• 15:08
  SPEAKER_00: We're we're offering, we're we're showing this off to you, and we want you to work on this.
• 15:13
  SPEAKER_00: You know, we want to kind of like source this to as many people as possible to see like effectively what parts of this are working and what parts aren't.
• 15:22
  SPEAKER_01: At the Allen Institute, our goal has always been to build breakthrough open AI.
• 15:27
  SPEAKER_01: Um, and we want to tackle the biggest challenges that are out there.
• 15:31
  SPEAKER_01: Um, and robotics is one of those challenges that's really been at the forefront for us for a really, really long time.
• 15:37
  SPEAKER_01: And we want to build open models because our goal at the end of the day is to mobilize the entire scientific community.
• 15:44
  SPEAKER_01: It's to make possible for other scientists to explore questions that they might not be able to explore uh if if because they don't have the infrastructure to do it, or the data to do it, or the models to do it.
• 15:58
  SPEAKER_01: And so what we try to do is make sure that everything that we build is completely open.
• 16:03
  SPEAKER_01: All of our data sets get released publicly, all of our models get released publicly, all of the code that we use to train these models, the infrastructure we train them on, all of it gets publicly open.
• 16:13
  SPEAKER_01: Um, and this allows a lot of people to actually just build on top of our work.
• 16:18
  SPEAKER_01: And of course, they get to test our hypotheses, they get to test our results, and that's really great as well for the research community, because only through reproduction of scientific results can we gain a certain level of confidence that this is actually something that's meaningful.
• 16:33
  SPEAKER_01: Um, and that's really been the biggest sort of driver of for us, for the researchers in our company.
• 16:40
  SPEAKER_01: Um yeah, go ahead.
• 16:42
  SPEAKER_00: Yeah, no, so this is something I was curious about.
• 16:44
  SPEAKER_00: Um, because you you you currently have your, I guess I would say you have your feet in a lot of different worlds.
• 16:49
  SPEAKER_00: You still are in the academic world.
• 16:51
  SPEAKER_00: Uh, you're still you're teaching, um, you're at the institute.
• 16:55
  SPEAKER_00: AI2 and and the institute are connected.
• 16:58
  SPEAKER_00: I don't know if you would say they're synonymous.
• 17:00
  SPEAKER_00: Is AI2 sort of a more commercial side of that?
• 17:04
  SPEAKER_00: How do the two parties interact with one another?
• 17:08
  SPEAKER_01: Uh, so I I guess I have my feet in two different places.
• 17:11
  SPEAKER_01: One of them is uh I'm a professor at the University of Washington.
• 17:16
  SPEAKER_01: Um there I run a research lab with my PhD students, and um we're exploring, you know, novel ideas uh and trying out different kinds of uh but tackling different kinds of problems across a wide range of AI sort of spaces.
• 17:34
  SPEAKER_01: A lot of it is focused primarily on computer vision and of course now language and robotics as well.
• 17:40
  SPEAKER_01: And the second place where I have I spend a lot of time is the Allen Institute or AI2.
• 17:45
  SPEAKER_01: I would say the two places are distinct.
• 17:47
  SPEAKER_01: The Allen Institute is a nonprofit organization.
• 17:51
  SPEAKER_01: Its primary goal is to produce very exciting open AI research uh that other people can build.
• 17:59
  SPEAKER_01: Uh, our goal is to sort of mobilize, like I was saying, the entire scientific community.
• 18:04
  SPEAKER_01: And the problems we tackle there tend to be a little bit larger in scope than the problems we end up tackling at a university.
• 18:11
  SPEAKER_01: And that's largely because there's a difference in the amount of resources you have allocated uh at a university versus at the institute.
• 18:18
  SPEAKER_01: And so at the institute, we're really trying to take big bets.
• 18:22
  SPEAKER_01: Um, and we take on large projects where we bring in multiple collaborators.
• 18:27
  SPEAKER_01: Uh, we collaborate quite quite a lot with uh tons of other universities as well as researchers from uh private institutes as well, um, who are interested in dedicated some amount, dedicating some amount of their time into doing open-ended uh research with us.
• 18:43
  SPEAKER_00: When you say big bets, you know, to me that I, you know, that implies a payoff.
• 18:47
  SPEAKER_00: I mean, is a payoff just, you know, that this thing is successful, or is there also some sort of fiscal payoff as well?
• 18:53
  SPEAKER_00: You know, is there potentially money to be made by creating these successful, even though they're open source models?
• 19:00
  SPEAKER_01: So for us, all of the researchers at the Ellen Institute, we're really driven by impact.
• 19:06
  SPEAKER_01: What we want to do is produce AI that can be for the social good.
• 19:11
  SPEAKER_01: Uh, we want to build things that actually makes the world better.
• 19:15
  SPEAKER_01: And so for us, it's uh never been about making money from the work that we do, but it's through seeing our work being used by others uh and seeing the field progress as fast as possible.
• 19:29
  SPEAKER_01: Um, if you look at the state of AI research today, um, there used to be this golden era until about five years ago, where everything that all of these companies were doing, they were publishing.
• 19:41
  SPEAKER_01: Um, and that meant that we could all move faster because we were all building on each other's work.
• 19:47
  SPEAKER_01: Uh Google and Meta, these companies were putting out research, they were putting out infrastructure to help us at universities do research that we couldn't do before.
• 19:58
  SPEAKER_01: Uh, we were able to try out ideas uh using the kind of compute and scale uh that places like Google and Meta had.
• 20:05
  SPEAKER_01: But over the last few years, a lot of these institutions have essentially closed off all of their research.
• 20:11
  SPEAKER_01: The amount of uh publications have gone down.
• 20:14
  SPEAKER_01: And for most of the most exciting ideas, the things that are actually working really well, they've stopped publishing a lot of these different uh ideas.
• 20:23
  SPEAKER_01: And what that's meant is that progress in AI, although it might look like it's going fast, we could be going a lot faster today.
• 20:30
  SPEAKER_01: But we're not because a lot of these institutions have closed out.
• 20:34
  SPEAKER_01: So, in a sense, uh we at the Allen Institute are filling that gap that all of these other companies originally had.
• 20:42
  SPEAKER_01: Um, and so we're really excited about being able to produce research and really go after problems that require more resources that maybe universities can't tackle and hopefully through that process make it more affordable, make it more easy for other people to build on top of.
• 21:00
  SPEAKER_00: Yeah, so you you studied with uh Fei Fe Li at Stanford.
• 21:03
  SPEAKER_00: That's right, yeah.
• 21:04
  SPEAKER_00: And you know, obviously one of the primary things she's known for is ImageNet.
• 21:09
  SPEAKER_00: And, you know, in a sense, uh the work that you're doing here is kind of creating a physical three-dimensional version of that.
• 21:16
  SPEAKER_00: And it strikes me there's all of these companies out there that are that are collecting data, that are, you know, making their sort of siloed data sets.
• 21:25
  SPEAKER_00: When you're keeping models open like this, do you feel like there could be some sort of centralized data set of physical real-world data?
• 21:34
  SPEAKER_01: You know, naturally, uh a lot of my thinking and a lot of my research interests have been largely shaped by FayFit um during my PhD years.
• 21:43
  SPEAKER_00: I imagine it would be hard not be to be around her and not like, yeah, have a lot of that rub off on you.
• 21:48
  SPEAKER_01: Of course.
• 21:49
  SPEAKER_01: Yeah.
• 21:49
  SPEAKER_01: And you know, uh, we all saw the impact that ImageNet had on the AI community.
• 21:54
  SPEAKER_01: Um, in a sense, you're you're not too far off.
• 21:58
  SPEAKER_01: I'm constantly thinking.
• 22:00
  SPEAKER_01: About what that ImageNed moment looks like for robotics.
• 22:04
  SPEAKER_01: And so we released uh earlier last month a project called Momo Spaces.
• 22:11
  SPEAKER_01: Um and this is our first attempt towards this ImageNet moment for robotics.
• 22:17
  SPEAKER_01: What Momo Spaces is trying to do is take all of these environments that we've created.
• 22:23
  SPEAKER_01: And these environments are super interesting, actually, because scaling up these environments where you could train robots is really difficult because you want them to be as diverse as possible.
• 22:33
  SPEAKER_01: And so the way we create them is by prompting LLMs to tell us, okay, I want to create a conference room.
• 22:40
  SPEAKER_01: What kind of objects would you find in a conference room?
• 22:43
  SPEAKER_01: It tells us.
• 22:44
  SPEAKER_01: And then we use that to go find 3D assets from our library.
• 22:48
  SPEAKER_01: And these 3D assets are assets of chairs, tables, uh, whiteboards like the one behind me.
• 22:55
  SPEAKER_01: Um, and we we then go back to the model and say, okay, what are the constraints for these objects in these 3D spaces?
• 23:02
  SPEAKER_01: Where would you find a whiteboard?
• 23:04
  SPEAKER_01: Where would you find a chair relative to a table?
• 23:06
  SPEAKER_01: Then we take all of those constraints and we treat it as an optimization problem and create an environment with all of these assets in there.
• 23:13
  SPEAKER_01: And then once we have it, we actually go ahead manually verify whether that environment looks realistic, whether that's an actual place that might exist.
• 23:21
  SPEAKER_01: And we also verify that the material properties of these objects are correct.
• 23:26
  SPEAKER_01: That, you know, the cup is something that has the right weight, uh, the right kind of material properties as well.
• 23:33
  SPEAKER_01: And then it's ready for us to start testing with real robots.
• 23:37
  SPEAKER_01: We've created about 230,000 of these environments and they have millions of objects in them.
• 23:43
  SPEAKER_01: And um, of course, it's not just that they have millions of objects, all of them also have come with places where you can grasp them.
• 23:52
  SPEAKER_01: So we've annotated all of that information as well, meaning that if you wanted to train a robot in this environment, we'll already tell you, hey, these are the places where you can grasp this object to pick it up and do something with it.
• 24:02
  SPEAKER_01: And we're now building out, which we're hoping we'll release soon, uh, is MomoBot.
• 24:08
  SPEAKER_01: And MomoBot is a project that's got multiple different aspects to it.
• 24:13
  SPEAKER_01: One aspect is data generation.
• 24:16
  SPEAKER_01: Because we have these simulated environments, we know everything that's in those environments.
• 24:20
  SPEAKER_01: We know exactly where that object is, we know what shape it has, we know where to grasp it.
• 24:25
  SPEAKER_01: We can generate data very, very easily.
• 24:28
  SPEAKER_01: All we need is uh some GPUs, and then we can automatically scale up uh data generation.
• 24:33
  SPEAKER_01: We've been generating over the last couple of months millions of robot trajectories.
• 24:38
  SPEAKER_01: And to do this in the real world would be, like we were talking about, very, very expensive.
• 24:42
  SPEAKER_01: You'd have to train people to do it, you'd have to buy the robots, you'd have to put these robots in spaces, move them around, and that's a lot of effort.
• 24:50
  SPEAKER_01: And if you wanted to take a bunch of robots and move them to 230 different thousand different locations, that's going to take you years to do.
• 24:58
  SPEAKER_01: Uh, whereas for us, we can do this in a matter of a couple of months.
• 25:03
  SPEAKER_01: Um, and now that we have all of this data, we've been training and coming up with new kinds of architectures to uh to train these kinds of models.
• 25:11
  SPEAKER_01: And we've been seeing some very, very exciting results.
• 25:14
  SPEAKER_01: Uh, something that's been very surprising for us is that we can actually train these models in simulation and only in simulation, and they actually generalize to the real world.
• 25:25
  SPEAKER_01: Um, meaning that without ever seeing anything in the real world, we now have robots that can open doors, that can pick objects up, move them around, that can also move around themselves in real world spaces.
• 25:38
  SPEAKER_01: That's a very exciting finding for us because I think now we have a proof of concept that simulation is a viable strategy for training these robot foundation models.
• 25:48
  SPEAKER_01: And what we really needed was that Image Net moment, that sort of diversity of data and environments from which we can train.
• 25:58
  SPEAKER_00: It's always fascinating to me.
• 25:59
  SPEAKER_00: Uh, you know, I'm I I was a humanities major.
• 26:01
  SPEAKER_00: It's always fascinating to me to hear researchers say that they're like positively surprised at an outcome, that it that it turned out better, you know.
• 26:11
  SPEAKER_00: And I know that like in and around um uh like Transformers, large large language models, there's a lot of conversations around uh black boxes and like not really quite understanding like the inner machinations of them.
• 26:25
  SPEAKER_00: Um when you say you were surprised that it worked out like better than you thought it would, what what do you what do you mean by that?
• 26:34
  SPEAKER_00: And and what do you attribute that to?
• 26:37
  SPEAKER_01: So uh I mostly attribute that to public opinion.
• 26:41
  SPEAKER_01: Uh you mentioned your previous conversation with somebody at Physical Intelligence.
• 26:45
  SPEAKER_01: Physical Intelligence put out a blog uh a few months ago where they were arguing that simulation is not a viable strategy, that the only way to train these robots is by collecting real-world data and scaling up real world robot data.
• 27:00
  SPEAKER_01: And they've been doing a lot of that.
• 27:02
  SPEAKER_01: They've been hiring uh really talented people to come in and collect data across uh what I understand are hundreds of robots.
• 27:11
  SPEAKER_01: Um and that's going well.
• 27:13
  SPEAKER_01: Uh, I'm everyone's very excited by the models that they're putting out, everyone's using those models.
• 27:19
  SPEAKER_01: Um, what we're hoping to show is an alternative strategy.
• 27:23
  SPEAKER_01: Uh, and it's a strategy that not a lot of people are betting on because it hasn't panned out in the past.
• 27:28
  SPEAKER_01: We've had cases where people have tried to build things from simulation, and they've always run across this problem where things just don't work in the real world when you train in simulation.
• 27:38
  SPEAKER_01: Meanwhile, we're showing that not only does it work, but it allows you to have robot models that are that have capabilities that existing models just do not have currently.
• 27:49
  SPEAKER_01: So, for example, um in simulation, we can train a robot uh or collect the data, collect a robot doing uh collect uh training data of a robot doing something.
• 28:01
  SPEAKER_01: And while it's doing it, we can move the camera that it's using to see the world around randomly in its environment.
• 28:09
  SPEAKER_01: Um and that's really hard to do in the real world uh because usually the camera is mounted on top of the robot or on its arms or someplace, and so it's usually fixed relative to the robot.
• 28:18
  SPEAKER_01: But in simulation, there's nothing stopping us from moving that camera around.
• 28:22
  SPEAKER_01: Now, the reason this is exciting is because what we're showing now is not only does this model that we train in simulation work in the real world, if its robot camera is kind of detached a little bit, maybe you didn't attach it to the body correctly, so it's wiggling a little bit or moving around, um, it still works.
• 28:38
  SPEAKER_01: It's still able to generalize because we trained it uh with these cameras moving around.
• 28:44
  SPEAKER_01: And so in the real world, you can actually move the camera around and it'll still work uh and do what it's supposed to.
• 28:50
  SPEAKER_00: That is a good point.
• 28:51
  SPEAKER_00: I was speaking, I can't remember which physical AI company, but yeah, they brought up a similar point, which is that, you know, the minute you put a robot out into the world, much like us, like they start to degrade.
• 29:02
  SPEAKER_00: So it's never going to be exactly the same system that it was when you released it.
• 29:07
  SPEAKER_01: And you can model these degradations in simulation.
• 29:10
  SPEAKER_01: Um, and you know, we're what we're trying to do is account for all of these kinds of faults and build them into how we collect data.
• 29:17
  SPEAKER_01: And because we can scale this up so much faster than in the real world, um, you can build in these capabilities.
• 29:24
  SPEAKER_01: We talked about compounding errors a couple of minutes ago.
• 29:27
  SPEAKER_01: And even there, we can automatically create data where the robot actually fails to do something and then has to correct its behavior.
• 29:35
  SPEAKER_01: And all of that is very easy to scale up as well.
• 29:38
  SPEAKER_01: You just inject some errors into how you generate your data.
• 29:42
  SPEAKER_00: I think you know, the other uh immediate upside that I see of creating really sort of small models and and having robots out in the world doing focused tasks is that there's a more immediate deployment there, right?
• 29:54
  SPEAKER_00: It's and and it sounds like to a certain extent what we're talking about here is is longer term.
• 29:59
  SPEAKER_00: We're talking about these huge models, right?
• 30:01
  SPEAKER_00: You know, are these things, you know, are are is the research that we're talking about right now, are these models that we're talking about right now things that are useful, that actually like can be used and can be deployed on systems in the real world right now?
• 30:16
  SPEAKER_01: So you're right.
• 30:17
  SPEAKER_01: What we're doing at the Allen Institute is very much taking longer-term bets.
• 30:21
  SPEAKER_01: We're trying to take bets uh that might not pay off for a year or two or even five years, uh, and build models that we're hoping is going to drive the next wave of innovation.
• 30:32
  SPEAKER_01: And so, in that sense, a lot of the things that we're betting on and the infrastructure we're building is for research and that eventually that research will mature enough for it to become deployable in the real world.
• 30:44
  SPEAKER_01: We do have smaller versions of these models and we can deploy them in the existing hardware, um, but they don't work uh like you would expect if you from uh an industry grade robotics application, of course.
• 30:57
  SPEAKER_01: Um, and to get there, I think there's still a lot of heavy lifting uh that needs to go into uh building useful robot systems.
• 31:05
  SPEAKER_01: Uh, and we're really tackling the intelligence problem.
• 31:07
  SPEAKER_01: We're tackling the problem of figuring out what to do, how to act, how to move, how to grab things, how to move things around.
• 31:13
  SPEAKER_01: And aside from that, there's all of these other issues around how do you build the right hardware, how do you make sure they're compliant, how do you make them safe for a specific deployment.
• 31:23
  SPEAKER_01: All of those considerations go beyond just um building an intelligent system.
• 31:28
  SPEAKER_01: They're they're very much dependent on uh the application you want to build for and the domains that you want to deploy these robots.
• 31:36
  SPEAKER_01: So we're gonna need to build work together with a lot of domain experts to actually make them deployable.
• 31:42
  SPEAKER_00: I assume that we just have to take for granted that there are always going to be edge cases, right?
• 31:47
  SPEAKER_00: Like you put robots out in the world for long enough and there are just going to be weird things that happen.
• 31:51
  SPEAKER_00: You hear these stories about self-driving cars and just bizarre things that nobody can predict happen.
• 31:57
  SPEAKER_00: Is there a way to prepare for those for a certain extent based on these models that you're currently building?
• 32:06
  SPEAKER_01: There are a ton of challenges that the research community has really been focused on for many years, or at least talking about for decades.
• 32:14
  SPEAKER_01: One of those is this idea of continual learning um, or in context learning, even.
• 32:20
  SPEAKER_01: With language models, this has become quite uh quite, you know, it quite good.
• 32:25
  SPEAKER_01: Today you can tell a language model, hey, here's some examples of uh inputs and outputs, and here's a new input, produce the output.
• 32:34
  SPEAKER_01: Um, so you can show it examples of bad essays and good essays, and then ask it to improve your current essay and make it better.
• 32:43
  SPEAKER_01: Um, that kind of behavior is so far away from for robotics today.
• 32:49
  SPEAKER_01: You know, ideally what you'd want is a robot system where if you bring it home, uh maybe you have a particular object um that looks very different, or you want to hold very differently.
• 33:01
  SPEAKER_01: Uh, maybe it's something that's unique.
• 33:03
  SPEAKER_01: Um, you'd want to be able to teach the robot how to do it.
• 33:06
  SPEAKER_01: Um, and to teach it, you'd probably do the thing that you want yourself and then expect it to learn by watching you.
• 33:13
  SPEAKER_01: Uh, this is something we do very often with each other.
• 33:15
  SPEAKER_01: We tell each other, hey, this is how you clean this mug, or this is how you uh clean this statue.
• 33:21
  SPEAKER_01: Um if we want robots to be able to learn by watching us, that's going to require us to do all kinds of new, build entirely new capabilities.
• 33:30
  SPEAKER_01: They're going to need to map, first of all, their own form factor, their own body, which looks a lot different from uh a person's body.
• 33:38
  SPEAKER_01: Uh their arms are going to move and have very different kinds of joints and flexibility than the robot.
• 33:43
  SPEAKER_01: So that mapping itself is really hard.
• 33:45
  SPEAKER_01: Then aside from that, they also now need to put themselves in the perspective of the person doing the task.
• 33:50
  SPEAKER_01: So when you watch someone do it, uh you're you're really a third-person view of somebody performing an action.
• 33:57
  SPEAKER_01: Uh, what you want, of course, is to map that into a first-person view so that you can do it yourself.
• 34:02
  SPEAKER_01: Um, and that's really difficult.
• 34:04
  SPEAKER_01: And then aside from that, you also want this continual learning where oftentimes we imagine what it would be like to do something.
• 34:12
  SPEAKER_01: So, for example, maybe you, you know, you watched a game of tennis, you could imagine yourself playing tennis, you could imagine how your arms would need to move in order to hit the ball.
• 34:22
  SPEAKER_01: Now, that kind of imaginative process, that kind of simulation is something that these robots currently lack.
• 34:27
  SPEAKER_01: And so this ability to remap uh different bodies to themselves, this ability to change perspectives, this ability to continually learn, all of those components need to be built in place.
• 34:38
  SPEAKER_01: And we haven't even talked about memory and you know, long-term tasks, long horizon tasks, all of these things are still very much open-ended questions.
• 34:46
  SPEAKER_00: The sense that I get, and I've gotten this from a lot of people, and I think you know, you you might be getting at this a little bit too, is that um the solution is gonna going to come in many forms when it comes to training, pre-training, post-training, and that things like teleop, that you know, watching by learning video, that's simulation, and that training in the real world, like these are all going to play a role in making robots better.
• 35:12
  SPEAKER_01: One of the things that we agree on in my team is we're making a big bet on simulation, but by no means do we think simulation alone will be the solution.
• 35:23
  SPEAKER_01: Um, what we're going to want are algorithms that actually do learn in the real world eventually.
• 35:28
  SPEAKER_01: We want these robots to perform tasks, realize they made a mistake, and then not only have error corrective behavior, but learn from that experience.
• 35:36
  SPEAKER_01: Uh, and so we do want systems that do adjust and adapt to the real world.
• 35:41
  SPEAKER_01: And that requires these models to also have reasoning capabilities.
• 35:45
  SPEAKER_01: Um, and so another line of work uh that we've been working on is called Momo Act.
• 35:50
  SPEAKER_01: And it's a series of ACT models that are really focused on this reasoning capability of figuring out what is the 3D space around me and how should I move in that 3D space and how should I adapt um as that 3D space changes over time?
• 36:05
  SPEAKER_00: So we we spoke about this a little bit before, and you know, obviously you said um, you know, that you you have a great deal of respect for what NVIDIA is doing, and you know, you work in collaboration with them.
• 36:16
  SPEAKER_00: Um, and obviously they're playing this this huge role in just pushing robotics forward generally.
• 36:22
  SPEAKER_00: But what is it that I guess what is it that their sort of simulation right now is is lacking when it comes to training robots?
• 36:31
  SPEAKER_01: Yeah, uh NVIDIA is doing a wonderful job with simulation engines.
• 36:35
  SPEAKER_01: Their Isaac simulator is one that we use as well in-house here at the Allen Institute.
• 36:41
  SPEAKER_01: The difference, I think, is the diversity of simulation data.
• 36:46
  SPEAKER_01: Um, they've been working on a set of projects uh called Robocasa, where, like I was saying, they've built kitchen environments.
• 36:53
  SPEAKER_01: So these are single rooms with kitchen items where they train robots.
• 36:59
  SPEAKER_01: The biggest difference is we're building simulation environments that are really large and very diverse.
• 37:07
  SPEAKER_01: We have large houses with uh five to ten rooms, we have conference centers with multiple rooms, uh, we have libraries, we're building hospitals, and all of them have a wide diversity of objects.
• 37:21
  SPEAKER_01: Uh, and that diversity is what we're betting on.
• 37:24
  SPEAKER_00: Um scale, really.
• 37:27
  SPEAKER_01: It's scale.
• 37:27
  SPEAKER_01: It's scale and diversity, yeah.
• 37:30
  SPEAKER_00: Yeah.
• 37:30
  SPEAKER_00: It it's interesting.
• 37:32
  SPEAKER_00: You know, you were using uh linguistics as a kind of a metaphor before, and it that jumped out at me because, you know, I was reading, I was reading your looking at your homepage, and one of the things you said is my research bootstraps machine learning using frameworks from behavioral and social sciences.
• 37:50
  SPEAKER_00: So again, uh, you know, I'm a humanities guy.
• 37:53
  SPEAKER_00: Um and and I'm uh I'm curious what it means to sort of look at these um, you know, at these, at these other fields and how they ultimately like do inform the work that you're currently doing.
• 38:08
  SPEAKER_01: There's many different aspects to this.
• 38:10
  SPEAKER_01: A lot of my work does draw on ideas and frameworks from um a ton of different cognitive scientists and behavioral scientists.
• 38:19
  SPEAKER_01: I can give you a couple of examples.
• 38:22
  SPEAKER_01: You know, we were looking at Maurice Merlot-Ponti.
• 38:24
  SPEAKER_01: He's a philosopher from early 1900s to mid-1900s, and he was one of those few people that really talked about the importance of embodiment.
• 38:33
  SPEAKER_01: Um, how his main argument was that um intelligence isn't something that is only in your mind.
• 38:40
  SPEAKER_01: It's very much something that is part of your body, that your body does a lot of the thinking for you so that your mind doesn't have to.
• 38:49
  SPEAKER_01: Um, for example, he talks about how if someone throws a ball at you, um, you're not predicting the trajectory of that ball uh and where it's going to go, and then decide to move your arm to that location.
• 39:00
  SPEAKER_01: Your body's already done all of that for you.
• 39:03
  SPEAKER_01: It knows exactly where to move its, move your arms to be able to catch that ball.
• 39:07
  SPEAKER_01: Um, if you touch something hot, you're not thinking, oh, I'm I'm touching something hot.
• 39:12
  SPEAKER_01: I need to move my hand away.
• 39:13
  SPEAKER_01: Your body's already reacting and changing that.
• 39:16
  SPEAKER_01: Similarly, he talks about how even the environments that we look at are very much defined not just by symbols that we associate with it, but by what your body could do with it.
• 39:28
  SPEAKER_01: So, for example, uh, you don't look at a chair and say that's a chair, you think about it as something you can sit on.
• 39:33
  SPEAKER_01: You don't look at a staircase and say that's a staircase and these are individual steps.
• 39:38
  SPEAKER_01: You look at it as something you can step on to climb onto things.
• 39:41
  SPEAKER_01: Um, and so all of these different ways that your body reacts to things, he calls it motor intentionality.
• 39:47
  SPEAKER_01: And that's a lot of how we're also thinking about building models as well.
• 39:52
  SPEAKER_01: Today, if you look at how most robotics models are built, they're built with this language component, uh this LLM, this large language model component in there.
• 40:01
  SPEAKER_01: And I would be willing to bet that all of that is completely wrong.
• 40:04
  SPEAKER_01: Uh, I don't think language should be the medium through which we think about embodiment or through which we build robotics models, because our body doesn't use symbols and language to be able to act in the world.
• 40:17
  SPEAKER_01: Uh instead, we go directly from sensors to uh reactions.
• 40:22
  SPEAKER_01: Um, and so this mapping from computer vision to robotics, that's the right mapping without that language intermediary in the middle.
• 40:30
  SPEAKER_01: So as things move forward, I think this dependence on language models is one of those things that should go away.
• 40:36
  SPEAKER_01: Uh and so in my in my lab at uh the University of Washington, we are exploring these kinds uh of models, uh, models that I call visual grounding models.
• 40:45
  SPEAKER_01: So models that directly go from sensors and grounds what it sees in some representation so that it can act on it.
• 40:53
  SPEAKER_01: And it's very much uh a representation that's defined by actions you can perform on those things and not by the symbols that you associate with them.
• 41:01
  SPEAKER_00: Maybe this is even more of a hardware question then, but uh, you know, I wonder if that means uh a more decentralized processing for the robots themselves, you know, in actuators and pieces themselves that kind of that do react to their environment, but also react as kind of a whole in tandem.
• 41:23
  SPEAKER_01: Yeah, I could I could totally see that as well.
• 41:25
  SPEAKER_01: Um, you know, sensors are expensive today when we're building these robot hardware.
• 41:30
  SPEAKER_01: Uh you've got, of course, uh maybe a few force torque sensors, um, and of course a few cameras, but that's maybe it.
• 41:37
  SPEAKER_01: Ideally, yes, we would have robot embodiments that have a lot more sensors to them that can react to a lot of different kinds of stimuli.
• 41:45
  SPEAKER_01: Uh, you know, we feel um air across our body that helps us adjust to the spaces that we're in.
• 41:52
  SPEAKER_01: Uh when someone taps us on our shoulder, we know exactly where the person is.
• 41:56
  SPEAKER_01: All of these things allow us to adapt very quickly to our environment.
• 41:59
  SPEAKER_01: Our body facilitates so much of our thinking.
• 42:02
  SPEAKER_01: And so I totally agree.
• 42:03
  SPEAKER_01: Uh, it's also a hardware problem.
• 42:06
  SPEAKER_01: I would love to see, you know, that we have really good hardware coming out now that are able to do a lot of exciting things.
• 42:11
  SPEAKER_01: I'm sure we're gonna have a sense of revolution as well at some point.
• 42:15
  SPEAKER_01: Well, Professor Krishna, thank you so much for taking the time today.
• 42:18
  SPEAKER_01: Thank you.
• 42:19
  SPEAKER_01: Thank you, Brad.
• 42:21
  SPEAKER_00: Thank you, Daranjay, and thanks to the folks at the Allen Institute for setting that up.
• 42:26
  SPEAKER_00: Uh, thanks to you as always for tuning in.
• 42:28
  SPEAKER_00: Please like and subscribe on your preferred podcasting platform and maybe some of your non-preferred ones as well.
• 42:36
  SPEAKER_00: Uh, go check out the newsletter over at automated.fm.
• 42:40
  SPEAKER_00: All right.
• 42:41
  SPEAKER_00: And uh with that, we will see you this time next week for another episode of Automated.