Latest / Going Direct Conversations / #43 Chris Paxton & Michael Cho: Robopapers And Future Of Robot Learning

Transcript

#43 Chris Paxton & Michael Cho: Robopapers And Future Of Robot Learning · Going Direct Conversations

• 0:00
  SPEAKER_03: When you see a video of a robot doing something,
• 0:01
  you should always assume that it can do literally what it can
• 0:04
  see and nothing else.
• 0:05
  Today I'm sitting down with Chris Paxton and Michael Cho to
• 0:07
  talk all things robotics.
• 0:09
  The two of them are co-hosts of a weekly robotics podcast
• 0:12
  called RoboPapers.
• 0:13
  SPEAKER_00: I see two huge bottlenecks in robotics.
• 0:15
  SPEAKER_00: So one is obviously lack of robotics data.
• 0:18
  SPEAKER_00: The other huge bottleneck I see is real-world evaluation.
• 0:21
  They interview robotics researchers to discuss a recently
• 0:25
  published research paper each week.
• 0:27
  SPEAKER_02: I think these robots are funny before they are useful.
• 0:29
  SPEAKER_02: Now,
• 0:29
  SPEAKER_02: the two of them are really at the forefront of robotics research,
• 0:32
  SPEAKER_02: which is the reason why I wanted to have the conversation with them.
• 0:35
  SPEAKER_02: Chris currently works at Agility Robotics and is a roboticist,
• 0:38
  and Michael has been in and around the robotics space pretty
• 0:41
  much his entire
• 0:42
  SPEAKER_02: career.
• 0:43
  SPEAKER_03: So we train the robot how to do something in a simulation,
• 0:46
  SPEAKER_02: and then it allows us to work in the real world.
• 0:48
  SPEAKER_02: I hope you enjoyed the conversation, and to remember to go direct.
• 0:58
  SPEAKER_02: All right, we are live.
• 0:59
  SPEAKER_02: Chris, Michael, how are we doing today?
• 1:03
  SPEAKER_00: Yeah.
• 1:03
  SPEAKER_00: Great.
• 1:03
  SPEAKER_00: How are you?
• 1:04
  SPEAKER_00: It's morning here in Hanzhou in China.
• 1:07
  Hopefully,
• 1:08
  I don't get kicked out by the firewall,
• 1:09
  but otherwise,
• 1:11
  yeah,
• 1:11
  it's great to be here.
• 1:13
  SPEAKER_02: Great.
• 1:13
  SPEAKER_02: So, yeah, Mike's currently in China.
• 1:15
  Chris is in Pittsburgh,
• 1:16
  and I'm in San Francisco,
• 1:17
  so we're a little spread out.
• 1:19
  SPEAKER_02: So, hey, listen, guys, it was...
• 1:22
  I was excited to reach out to you all because for most folks
• 1:26
  that don't know,
• 1:27
  you guys started a podcast not too long ago called RoboPapers
• 1:31
  where you're doing
• 1:34
  weekly interviews with folks that are putting out a variety of
• 1:37
  technical papers
• 1:37
  SPEAKER_02: related to research around robotics and kind of moving the field forward.
• 1:41
  And for everyone that's listening or watching,
• 1:43
  obviously you all know that
• 1:46
  SPEAKER_02: If we're ever going to decentralize the real economy,
• 1:47
  SPEAKER_02: the heart of that new system is decentralized production,
• 1:49
  SPEAKER_02: of which the heart of that is robotics and automation.
• 1:53
  So I'm excited to,
• 1:54
  as you guys like to say,
• 1:55
  geek out with you a little bit today.
• 1:57
  SPEAKER_02: And so thanks again.
• 1:59
  SPEAKER_02: So I want to start here.
• 2:00
  SPEAKER_02: So where did the two of you guys meet?
• 2:07
  SPEAKER_03: That's a good question.
• 2:12
  SPEAKER_03: That's a better answer.
• 2:13
  SPEAKER_03: Yeah, that's right.
• 2:15
  SPEAKER_03: I think Sarah must be.
• 2:17
  SPEAKER_00: So yeah, I'm the odd one because I'm actually not a researcher.
• 2:19
  SPEAKER_00: Chris is a really good researcher.
• 2:21
  SPEAKER_00: I'm not even a researcher.
• 2:23
  But I've been really fascinated by this kind of like AI
• 2:28
  research for like a decade now.
• 2:30
  SPEAKER_00: Literally,
• 2:30
  SPEAKER_00: my hobby is for the last 12 years,
• 2:33
  SPEAKER_00: let's say,
• 2:34
  SPEAKER_00: or maybe even longer,
• 2:35
  SPEAKER_00: I don't know,
• 2:35
  SPEAKER_00: something like 10 years,
• 2:36
  SPEAKER_00: sorry,
• 2:36
  SPEAKER_00: something like 10 years,
• 2:37
  SPEAKER_00: is to read papers before I sleep.
• 2:39
  SPEAKER_00: I've been doing that for something like 10 years.
• 2:42
  SPEAKER_00: What happened was,
• 2:43
  SPEAKER_00: I think back in 2016,
• 2:45
  I was in the Bay Area and I crashed this course at Stanford
• 2:50
  called CS231N.
• 2:52
  SPEAKER_00: which then was taught by this one PhD student called Andre Carpathic.
• 2:56
  SPEAKER_00: So I was there in the front row.
• 2:59
  I have no idea what's going on,
• 3:00
  but then I thought that he was just a phenomenal teacher.
• 3:03
  SPEAKER_00: And since then I've been reading like computer vision, mostly CV papers.
• 3:08
  in the initial years and of course like in more recent years
• 3:11
  like robotics paper
• 3:11
  but that's that's how and then of course bump into Chris and
• 3:14
  one of those robotic
• 3:15
  conference and I think Chris is just one of those rare
• 3:18
  researchers who us I don't
• 3:20
  know how Chris find the time but he finds the time to share
• 3:23
  right and which I think
• 3:24
  SPEAKER_00: is very rare and I thought
• 3:27
  SPEAKER_00: Yeah, it's been a real privilege for me to do this with him.
• 3:30
  SPEAKER_00: Anyway, I should stop yapping.
• 3:32
  SPEAKER_00: Yeah, it's been a lot of fun.
• 3:34
  SPEAKER_03: I'm pretty sure I was thinking about it where we met.
• 3:36
  SPEAKER_03: We met at ICRA in Yokohama, right?
• 3:38
  SPEAKER_03: So last year in Japan.
• 3:39
  SPEAKER_00: Is that right, in person?
• 3:41
  SPEAKER_03: Yeah, I think that's right.
• 3:42
  SPEAKER_03: I think so, yeah.
• 3:43
  SPEAKER_03: Yeah, so a year and a half, right?
• 3:44
  SPEAKER_03: That was...
• 3:46
  And ironically,
• 3:46
  also,
• 3:47
  I learned deep learning from Andrew Karpathy's lectures,
• 3:49
  too.
• 3:50
  SPEAKER_03: So I think that's true of basically everyone of my generation in AI.
• 3:54
  SPEAKER_03: So it's, anyway.
• 3:57
  SPEAKER_02: Cool.
• 3:58
  SPEAKER_02: And so, Chris, I'll start with you.
• 4:00
  SPEAKER_02: So kind of your journey in the robotics space,
• 4:04
  SPEAKER_02: you were at NVIDIA and Meta for a little while and now at Agility.
• 4:08
  SPEAKER_02: Is that right, Agility?
• 4:09
  SPEAKER_01: Yeah.
• 4:11
  So just kind of walk through kind of your kind of experience in
• 4:14
  the embodied AI
• 4:15
  SPEAKER_02: space,
• 4:16
  SPEAKER_02: getting kind of up to date.
• 4:18
  So I actually got my start in my PhD at Johns Hopkins
• 4:22
  University in Maryland.
• 4:26
  SPEAKER_03: Initially, we were doing surgical industrial robotics.
• 4:29
  We did a bunch of work on how to teach robots to do different
• 4:34
  tasks and try to make
• 4:35
  SPEAKER_03: them easier to use and all this.
• 4:36
  SPEAKER_03: And this was all before deep learning really worked.
• 4:39
  So it's all stuff that it's ironic looking back at it now,
• 4:42
  because a lot of it doesn't like
• 4:43
  It ended up being quite different,
• 4:45
  I think,
• 4:46
  than the things that actually we use these days.
• 4:48
  SPEAKER_03: But then after that,
• 4:49
  SPEAKER_03: I went to...
• 4:50
  SPEAKER_03: Actually,
• 4:50
  SPEAKER_03: during that,
• 4:51
  SPEAKER_03: I missed one,
• 4:52
  SPEAKER_03: which is that I did a co-op at Zoox,
• 4:54
  SPEAKER_03: so I went to Zoox for like a year and a half.
• 4:56
  SPEAKER_03: So Zoox is a self-driving car company.
• 4:59
  SPEAKER_03: So that was... But then I did post-doc.
• 5:04
  SPEAKER_03: And was there for a while, moved to Pittsburgh.
• 5:06
  SPEAKER_03: And then I moved to Pittsburgh, then I moved to Meta.
• 5:10
  SPEAKER_03: And since then,
• 5:11
  SPEAKER_03: I've been at startups after Meta,
• 5:13
  SPEAKER_03: which is,
• 5:14
  SPEAKER_03: yeah,
• 5:15
  SPEAKER_03: so I've done,
• 5:16
  SPEAKER_03: I've worked in a lot of different parts of robotics over the years,
• 5:20
  SPEAKER_03: actually.
• 5:20
  SPEAKER_03: I mean, yeah, many different robotics fields and industries.
• 5:24
  SPEAKER_02: I don't know.
• 5:25
  Is there kind of an underlying theme with kind of your research
• 5:29
  or what you work on
• 5:29
  SPEAKER_02: that you enjoy doing across the different robotic systems?
• 5:32
  SPEAKER_02: Or is it kind of pretty wide ranging?
• 5:35
  SPEAKER_03: I think it's been wide-ranging,
• 5:36
  SPEAKER_03: but my research area,
• 5:38
  the area that I've always been excited about is long horizon
• 5:41
  reasoning and planning
• 5:42
  SPEAKER_03: for robots.
• 5:43
  SPEAKER_03: How do we get robots to do multi-step tasks?
• 5:46
  SPEAKER_03: How do we get them to reason and be able to interact with the world?
• 5:51
  SPEAKER_03: One of the things I was most excited about last year was working on
• 5:56
  the work on Meta and the work that I did last year was around
• 5:59
  home robots and
• 6:01
  reasoning about how can they reason about and represent their
• 6:03
  environments to be
• 6:04
  SPEAKER_03: able to make long horizon decisions.
• 6:06
  SPEAKER_03: What that means in practice is me being able to tell my robot,
• 6:09
  SPEAKER_03: hey,
• 6:09
  can you go find me my drink and have it go and search the house
• 6:12
  and look for it and
• 6:13
  SPEAKER_03: pick it up and bring it to you,
• 6:14
  SPEAKER_03: stuff like that.
• 6:16
  Because obviously this is actually still fairly challenging for
• 6:20
  robotics,
• 6:21
  that kind of stuff.
• 6:23
  And I think home robotics is something that I've always been
• 6:25
  very excited about,
• 6:26
  SPEAKER_03: passionate about,
• 6:27
  SPEAKER_03: which is not quick.
• 6:29
  SPEAKER_02: And Mike, so Mike, over to you.
• 6:31
  So obviously,
• 6:32
  outside of RoboPapers,
• 6:36
  what's the day job or what do you do every day?
• 6:40
  SPEAKER_00: Yeah.
• 6:41
  SPEAKER_00: So,
• 6:42
  SPEAKER_00: yeah,
• 6:42
  SPEAKER_00: I'm the co-founder of this project called BitRobot,
• 6:44
  SPEAKER_00: where we're trying to use crypto incentives to incentivize,
• 6:49
  SPEAKER_00: well,
• 6:49
  SPEAKER_00: gather resources to push forward like embodied AI research.
• 6:53
  SPEAKER_00: I don't know whether I lost you at the moment I say crypto,
• 6:56
  SPEAKER_00: it just made me like 10 times less serious.
• 6:58
  SPEAKER_00: But I genuinely believe you.
• 7:02
  SPEAKER_02: Not at all, but go ahead.
• 7:04
  SPEAKER_00: Yeah.
• 7:05
  So how I started in this space was I literally was just with my
• 7:10
  younger brother,
• 7:11
  SPEAKER_00: Sam,
• 7:11
  SPEAKER_00: who's my co-founder.
• 7:12
  SPEAKER_00: So during the pandemic, we were just like...
• 7:16
  SPEAKER_00: On the weekend, we'll... So he's in China, I'm in Singapore at the time.
• 7:21
  SPEAKER_00: And during the pandemic, I want to say it.
• 7:23
  SPEAKER_00: So it's the end of the world.
• 7:24
  We don't know what we should do,
• 7:26
  but we want to have a reason to keep in contact.
• 7:28
  SPEAKER_00: So every weekend, we'll come together, build our own sidewall robot.
• 7:32
  It honestly just started out as a weekend hacking project to
• 7:36
  keep in touch with my brother.
• 7:39
  And then we decided to start a company because our observation
• 7:41
  was that a lot of
• 7:42
  sidewalk robots that were built by these bigger startups and
• 7:46
  the Amazons of the
• 7:47
  SPEAKER_00: world,
• 7:48
  SPEAKER_00: their robots are pretty expensive,
• 7:50
  SPEAKER_00: like $10,000 or more.
• 7:51
  SPEAKER_00: Because at the time, we were assembling our own.
• 7:53
  SPEAKER_00: I have a pretty good handle on the build of materials.
• 7:57
  SPEAKER_00: And so we thought that if you take the business worldview,
• 8:02
  SPEAKER_00: maybe we already have the recipe,
• 8:06
  SPEAKER_00: all this offline RL invitation learning stuff,
• 8:08
  SPEAKER_00: which I think four years ago is starting to show some signs of life.
• 8:12
  And if you look at these robots as real-world robotic data
• 8:16
  collection devices,
• 8:18
  SPEAKER_00: like $10,000 robot will just not scale, like won't get you the data.
• 8:21
  SPEAKER_00: So we thought,
• 8:22
  SPEAKER_00: okay,
• 8:22
  SPEAKER_00: very naively,
• 8:23
  SPEAKER_00: to be honest,
• 8:24
  SPEAKER_00: at that time,
• 8:24
  SPEAKER_00: me and my brother have never done robotics as a real,
• 8:27
  SPEAKER_00: we are not roboticists to begin with.
• 8:28
  We thought that maybe someone should build a toy that's like
• 8:30
  $500,
• 8:30
  that's like a science.
• 8:33
  sidewall robot toy basically and make we thought it can be
• 8:37
  built at that price by
• 8:39
  just making a dumb robot so yank out the big gpu very cheap
• 8:44
  cameras basically and
• 8:46
  just have it 100 like human control so no ai so just a dumb
• 8:51
  robot this is during
• 8:53
  SPEAKER_00: the pandemic right so the other of the fact that we made was that
• 8:57
  SPEAKER_00: There was this huge investment that was brought forward,
• 8:59
  SPEAKER_00: let's say,
• 8:59
  SPEAKER_00: a decade so that we can have video calls like this.
• 9:02
  SPEAKER_00: Right.
• 9:02
  SPEAKER_00: So because before that,
• 9:03
  SPEAKER_00: the Internet wasn't really geared towards like real time communication,
• 9:06
  SPEAKER_00: but because of that.
• 9:07
  SPEAKER_00: So we thought,
• 9:08
  SPEAKER_00: OK,
• 9:08
  maybe it's possible for someone in New York to control a robot
• 9:12
  toy somewhere else.
• 9:12
  SPEAKER_00: And that could be a really interesting gamification.
• 9:14
  SPEAKER_00: So honestly, it all started off like that.
• 9:17
  Everything along the way,
• 9:19
  you know,
• 9:20
  it's a stumble into it after the other.
• 9:24
  I'll say why I got a lot more into this robotic research is
• 9:28
  from this robotic game.
• 9:29
  So think of it almost like a Pokemon Go where you have human
• 9:32
  gamers that drive
• 9:33
  SPEAKER_00: around,
• 9:33
  SPEAKER_00: like pick up virtual items on a map.
• 9:37
  SPEAKER_00: We started gathering data from very early on,
• 9:39
  and it turns out that data was actually really useful for some
• 9:43
  of the folks that
• 9:44
  SPEAKER_00: are doing urban navigation robotic research.
• 9:47
  And that's how we started working with some of these really top
• 9:51
  researchers.
• 9:53
  SPEAKER_00: Like,
• 9:53
  SPEAKER_00: for example,
• 9:53
  we co-organized for a couple of times now this competition
• 9:57
  called Earth Rover
• 9:58
  SPEAKER_00: Challenge,
• 9:59
  SPEAKER_00: where we pit the top human gamers in our game.
• 10:03
  SPEAKER_00: versus AI models that are built by different labs, right?
• 10:06
  So,
• 10:07
  UT Austin,
• 10:08
  Seoul National University,
• 10:09
  National University of Singapore.
• 10:11
  And that one is actually co-organized together with many folks
• 10:13
  from DeepMind.
• 10:15
  SPEAKER_00: And so,
• 10:16
  SPEAKER_00: yeah,
• 10:16
  SPEAKER_00: I mean,
• 10:17
  SPEAKER_00: I thought that there's something there because I was such a small,
• 10:19
  SPEAKER_00: tiny little project.
• 10:23
  SPEAKER_00: to,
• 10:23
  to get the attention of this top researcher to spend time with
• 10:25
  me to organize
• 10:26
  SPEAKER_00: competition like this.
• 10:27
  SPEAKER_00: I thought there is something there.
• 10:29
  SPEAKER_00: And so, yeah.
• 10:29
  SPEAKER_00: So anyway, yeah.
• 10:32
  SPEAKER_02: So essentially just like to summarize then,
• 10:36
  SPEAKER_02: so that kind of the idea was game game,
• 10:38
  SPEAKER_02: the gamification of basically tele-operation of,
• 10:42
  SPEAKER_02: of robots to collect data to help in the training process.
• 10:45
  SPEAKER_02: Is that kind of the best way to describe it?
• 10:48
  SPEAKER_00: Yeah, I, yeah, pretty much it.
• 10:50
  SPEAKER_00: Yeah.
• 10:50
  SPEAKER_00: I would say that's not the only thing that we do now,
• 10:52
  SPEAKER_00: but that's definitely,
• 10:53
  SPEAKER_00: we're still,
• 10:54
  SPEAKER_00: it's,
• 10:54
  SPEAKER_00: it's a big thesis of mine.
• 10:55
  SPEAKER_00: I,
• 10:55
  SPEAKER_00: I,
• 10:56
  I kind of feel that we're still a long way from having robots
• 10:59
  that actually work in
• 11:00
  SPEAKER_00: homes,
• 11:00
  SPEAKER_00: which is what,
• 11:01
  SPEAKER_00: you know,
• 11:01
  SPEAKER_00: Chris's dream.
• 11:02
  SPEAKER_00: It's my dream too,
• 11:02
  SPEAKER_00: but I think,
• 11:04
  SPEAKER_00: you know,
• 11:06
  SPEAKER_00: there's too many bottlenecks,
• 11:08
  SPEAKER_00: but,
• 11:08
  SPEAKER_00: uh,
• 11:08
  SPEAKER_00: I think these robots are funny before they are useful.
• 11:11
  So I think like the current product market fit is actually
• 11:14
  doing entertainment with robots.
• 11:16
  SPEAKER_00: Whereas we've seen,
• 11:17
  I actually also have a separate project that do this like
• 11:19
  humanoid fighting thing.
• 11:22
  SPEAKER_00: Yeah.
• 11:22
  SPEAKER_00: So those couple of things that you may have seen in San Francisco,
• 11:25
  SPEAKER_00: like,
• 11:25
  SPEAKER_00: uh,
• 11:25
  SPEAKER_00: you know,
• 11:26
  SPEAKER_00: I was kind of behind it for a number of these events.
• 11:29
  SPEAKER_00: So yeah.
• 11:31
  SPEAKER_00: Um,
• 11:33
  SPEAKER_00: Yeah.
• 11:33
  SPEAKER_02: So let's talk about those bottlenecks, right?
• 11:36
  So obviously,
• 11:38
  it's common knowledge that obviously there's a training gap.
• 11:43
  SPEAKER_02: And obviously,
• 11:44
  SPEAKER_02: it's like you need the most diverse,
• 11:47
  large data sets and to deal with edge cases and with larger
• 11:52
  context windows.
• 11:53
  SPEAKER_02: We can go on and on and on.
• 11:55
  SPEAKER_02: Can we drill,
• 11:56
  I want to get each of your perspectives on really where the
• 11:59
  main gap lies and the
• 12:00
  SPEAKER_02: main bottleneck lies because it's so hard to determine between what,
• 12:04
  especially folks that are building robots put out there from a
• 12:06
  marketing standpoint
• 12:07
  SPEAKER_02: and how far away we are to actually make them useful.
• 12:09
  SPEAKER_02: So Chris, maybe I'll go over to you first.
• 12:11
  If you were to boil down if there's one or two things that
• 12:15
  really stand out to you of...
• 12:19
  SPEAKER_02: the bottleneck around the current bottleneck around training gaps and,
• 12:22
  SPEAKER_02: you know,
• 12:23
  SPEAKER_02: where are we really at?
• 12:24
  SPEAKER_02: What is the current state of the industry?
• 12:26
  SPEAKER_03: I mean,
• 12:27
  I think that the hardest things to do are to get robots out
• 12:30
  there into the real
• 12:31
  SPEAKER_03: world and make them do stuff,
• 12:33
  SPEAKER_03: which,
• 12:33
  SPEAKER_03: I mean,
• 12:34
  SPEAKER_03: it's kind of circular,
• 12:35
  SPEAKER_03: right?
• 12:35
  SPEAKER_03: But,
• 12:35
  SPEAKER_03: like,
• 12:36
  getting robots and deploying them and making them work reliably
• 12:40
  and robustly and
• 12:42
  SPEAKER_03: survive.
• 12:42
  It's like,
• 12:43
  can you make the robot actually work for a long period of time?
• 12:46
  SPEAKER_03: It's like, I mean...
• 12:48
  SPEAKER_03: We need data, but that's secondary to getting robots out there.
• 12:54
  SPEAKER_03: I think a lot of the time, making them work.
• 12:57
  SPEAKER_03: Once they work, the data comes in naturally.
• 12:59
  SPEAKER_03: They're always producing it.
• 13:05
  SPEAKER_02: Michael, what's your perspective?
• 13:09
  SPEAKER_00: Yeah, I think broadly speaking, I see two huge bottlenecks in robotics.
• 13:14
  SPEAKER_00: One is obviously lack of robotics data.
• 13:18
  SPEAKER_00: I think that,
• 13:19
  SPEAKER_00: so if you look at the internet,
• 13:20
  like I think that the whole internet is just the output of
• 13:23
  humans' conscious
• 13:24
  SPEAKER_00: thoughts,
• 13:24
  SPEAKER_00: right?
• 13:25
  SPEAKER_00: So we're able to record those things down.
• 13:27
  SPEAKER_00: That's why those are great training models for LLM, VLMs and whatnot.
• 13:31
  SPEAKER_00: But the way humans like,
• 13:32
  SPEAKER_00: you know,
• 13:33
  SPEAKER_00: move in a physical world,
• 13:34
  SPEAKER_00: like the way I move my water bottle,
• 13:35
  SPEAKER_00: I can only show you.
• 13:37
  SPEAKER_00: but I can't record it at,
• 13:38
  SPEAKER_00: let's say,
• 13:38
  SPEAKER_00: 30 hertz,
• 13:39
  SPEAKER_00: you know,
• 13:40
  SPEAKER_00: the exact angle,
• 13:41
  SPEAKER_00: the velocity and all that.
• 13:42
  So these action labels,
• 13:44
  because we can't record it,
• 13:46
  so that's why the data is totally missing.
• 13:49
  SPEAKER_00: And except,
• 13:50
  SPEAKER_00: I guess,
• 13:50
  SPEAKER_00: maybe Tesla,
• 13:51
  they put a human in a robot on wheels and then they call it
• 13:54
  what the human driver
• 13:54
  SPEAKER_00: is doing.
• 13:54
  SPEAKER_00: So those action labels.
• 13:58
  So I think the robotic data is very much needed and there are
• 14:01
  three types of
• 14:02
  SPEAKER_00: robotics data,
• 14:03
  like tablet operation and video data and also like synthetic or
• 14:07
  simulation data.
• 14:09
  SPEAKER_00: So that's one big bottleneck, just data and three types of data.
• 14:12
  SPEAKER_00: The other huge bottleneck I see is real-world evaluation.
• 14:15
  SPEAKER_00: And I think this is something that is very unique to physical AI.
• 14:22
  SPEAKER_00: In the digital AI run,
• 14:23
  SPEAKER_00: like if you,
• 14:24
  SPEAKER_00: let's say,
• 14:25
  SPEAKER_00: if DeepSig or OpenAI,
• 14:26
  SPEAKER_00: if they drop another reasoning model,
• 14:27
  SPEAKER_00: typically,
• 14:28
  SPEAKER_00: you know,
• 14:28
  SPEAKER_00: obviously they have a bunch of benchmarks.
• 14:30
  SPEAKER_00: And if you look at all those benchmarks,
• 14:32
  SPEAKER_00: right,
• 14:32
  SPEAKER_00: like for example,
• 14:32
  SPEAKER_00: they will take,
• 14:33
  SPEAKER_00: let the AI do,
• 14:34
  SPEAKER_00: let's say,
• 14:34
  SPEAKER_00: a math Olympian test.
• 14:37
  SPEAKER_00: That test can be done like just like that, right?
• 14:39
  SPEAKER_00: So even though there's no perfect benchmarks,
• 14:42
  SPEAKER_00: getting the benchmarks is never a bottleneck in the iteration cycle.
• 14:48
  SPEAKER_00: Whereas the physical AI,
• 14:49
  SPEAKER_00: like you can test all you want in simulation,
• 14:51
  SPEAKER_00: but ultimately,
• 14:52
  SPEAKER_00: like literally rubber hits the road,
• 14:54
  SPEAKER_00: right?
• 14:54
  SPEAKER_00: You know, you need to see how they fail.
• 14:56
  SPEAKER_00: And because you can't speed up time,
• 14:59
  SPEAKER_00: Like,
• 15:00
  SPEAKER_00: how do you know,
• 15:01
  SPEAKER_00: for example,
• 15:01
  whether you build a grid model unless you measure how long does
• 15:04
  it take before it
• 15:05
  SPEAKER_00: fails?
• 15:05
  SPEAKER_00: Like, this is the only way to do it.
• 15:07
  SPEAKER_00: And so I think real world eval would be a huge bottleneck.
• 15:11
  SPEAKER_00: Literally,
• 15:11
  SPEAKER_00: if you look at robotics,
• 15:12
  SPEAKER_00: I mean,
• 15:13
  SPEAKER_00: Chris and I talk about this all the time,
• 15:15
  SPEAKER_00: there's no benchmarks.
• 15:16
  SPEAKER_00: You know,
• 15:17
  there's no benchmarking because it is just nearly impossible to
• 15:21
  have benchmarks in
• 15:22
  SPEAKER_00: robotics.
• 15:23
  SPEAKER_00: Whereas everybody's robots difference.
• 15:25
  SPEAKER_03: Everybody's house is different.
• 15:27
  SPEAKER_00: Yeah, exactly.
• 15:28
  SPEAKER_00: In that paper, they have this success rate.
• 15:30
  SPEAKER_00: You just have to take it that,
• 15:33
  SPEAKER_00: okay,
• 15:33
  SPEAKER_00: they did their work and it is what they say,
• 15:35
  SPEAKER_00: but no one can verify it.
• 15:38
  SPEAKER_03: Sure.
• 15:38
  SPEAKER_03: It's getting better, but yeah, definitely not.
• 15:40
  SPEAKER_03: Oh, that's so hard.
• 15:41
  SPEAKER_03: Yeah.
• 15:42
  SPEAKER_02: All right.
• 15:44
  SPEAKER_02: Well, it's an interesting question, right?
• 15:45
  SPEAKER_02: I guess this one's for you,
• 15:46
  SPEAKER_02: Chris,
• 15:47
  because obviously you're working on or agility is working on
• 15:50
  the humanoid form
• 15:51
  SPEAKER_02: factor.
• 15:52
  SPEAKER_02: And it's kind of like when the world of robotics is like, just show me.
• 15:57
  It's very different than some abstract or benchmarking process
• 16:05
  to benchmark LLMs.
• 16:07
  SPEAKER_03: Is that okay?
• 16:09
  SPEAKER_03: Yeah.
• 16:09
  The thing is that you can't just get a good feel for how a
• 16:12
  robot works from a video
• 16:13
  SPEAKER_03: because you can always get a good video.
• 16:15
  SPEAKER_03: Honestly,
• 16:16
  the other thing is that I think people have seen lots of really
• 16:18
  good robot videos
• 16:19
  SPEAKER_03: for a long time because they've seen Star Force or whatever.
• 16:22
  SPEAKER_03: You've seen robots that can do things that they can't even do now.
• 16:26
  I think you have to be able to show robots in the real world
• 16:28
  and see it for
• 16:31
  SPEAKER_03: yourself to be able to see what it can do.
• 16:33
  SPEAKER_03: I think a lot of videos can be misleading.
• 16:39
  I wanted to drill into the different types of training,
• 16:43
  methods for training.
• 16:44
  SPEAKER_02: I wanted to get your perspective on it.
• 16:48
  SPEAKER_02: Because that's also another aspect that's kind of a little bit unclear.
• 16:53
  SPEAKER_02: What is the most effective?
• 16:54
  SPEAKER_02: Is it a mix of all of these?
• 16:55
  SPEAKER_02: Let's start with just teleoperation.
• 16:58
  SPEAKER_02: Michael,
• 16:58
  what's your perspective on what does teleoperation truly allow
• 17:02
  you to do to solve
• 17:04
  SPEAKER_02: some of the training gaps that currently exist?
• 17:08
  SPEAKER_00: Yeah,
• 17:08
  I would say teleoperation is the gold standard amongst the
• 17:11
  three types of data I
• 17:12
  SPEAKER_00: mentioned just now.
• 17:14
  SPEAKER_00: But obviously, it's extremely costly to collect.
• 17:19
  I say it's a gold standard because you are doing in the real
• 17:22
  physical world with a
• 17:23
  SPEAKER_00: real physical robot.
• 17:25
  SPEAKER_00: So you don't suffer from, let's say, embodiment gap.
• 17:29
  And you collect exactly,
• 17:31
  typically,
• 17:31
  I mean,
• 17:31
  this is humans controlling the robot,
• 17:33
  right?
• 17:33
  SPEAKER_00: So you also,
• 17:35
  SPEAKER_00: in a way,
• 17:36
  SPEAKER_00: collect human intention,
• 17:37
  SPEAKER_00: like what would a human do,
• 17:40
  SPEAKER_00: like in this instance,
• 17:41
  SPEAKER_00: right?
• 17:41
  So there's some social compliance stuff that also naturally get
• 17:45
  collected.
• 17:46
  SPEAKER_00: So I think it's great for all those reasons.
• 17:48
  SPEAKER_00: But of course,
• 17:49
  SPEAKER_00: these robots break all the time,
• 17:51
  SPEAKER_00: you know,
• 17:51
  SPEAKER_00: and you can't respawn them,
• 17:53
  SPEAKER_00: you know,
• 17:53
  SPEAKER_00: like in a simulation.
• 17:54
  SPEAKER_00: So that's a huge thing.
• 17:56
  SPEAKER_00: Yeah.
• 17:56
  SPEAKER_00: So I,
• 17:56
  SPEAKER_00: I think my thing is that if you can gamify it,
• 17:58
  SPEAKER_00: then,
• 17:59
  SPEAKER_00: then hopefully you get more people to constantly do it,
• 18:02
  but you still cannot get away from the fact that these things
• 18:04
  will break all the
• 18:05
  SPEAKER_00: time and makes it very,
• 18:07
  SPEAKER_00: very hard to scale.
• 18:08
  SPEAKER_02: Yeah.
• 18:09
  SPEAKER_02: Yeah.
• 18:10
  SPEAKER_02: What's your thought, Chris, on the teleoperation side?
• 18:13
  SPEAKER_03: I mean,
• 18:13
  SPEAKER_03: I think that,
• 18:13
  SPEAKER_03: yeah,
• 18:13
  SPEAKER_03: like you're saying,
• 18:14
  SPEAKER_03: that's the gold standard,
• 18:16
  SPEAKER_03: getting real robot data.
• 18:17
  SPEAKER_03: And I think the gold standard is actually probably,
• 18:20
  SPEAKER_03: the best data is probably the data the robot collects itself.
• 18:23
  SPEAKER_03: It's probably like reinforcement learning kind of data.
• 18:25
  SPEAKER_03: But that's pretty rare still.
• 18:26
  SPEAKER_03: But the robots are...
• 18:33
  SPEAKER_03: Teleoperation data is great as long as you can actually get it, right?
• 18:36
  SPEAKER_03: But while scaling, it's hard.
• 18:39
  And so that was actually the next question was kind of
• 18:44
  reinforcement learning.
• 18:45
  SPEAKER_02: So Chris,
• 18:46
  can you kind of explain that for folks that might not be so
• 18:50
  understanding of what
• 18:51
  SPEAKER_02: that actually means in the term of robotics space?
• 18:53
  SPEAKER_03: Yeah, sure.
• 18:54
  Yeah,
• 18:55
  so I apparently even wrote a book chapter on this for what it's
• 18:57
  worth.
• 19:01
  SPEAKER_03: So reinforcement learning is... What's the best way to describe this?
• 19:06
  SPEAKER_03: I really should have a better answer for you off the top of my head.
• 19:09
  But it's basically the idea that the robot needs to be able to
• 19:13
  teach itself from
• 19:14
  SPEAKER_03: its own experiences.
• 19:15
  SPEAKER_03: So the way that you or I learn to perform a task is not by watching.
• 19:20
  SPEAKER_03: I don't watch... If I wanted to learn how to...
• 19:23
  SPEAKER_03: hit a tennis ball.
• 19:24
  SPEAKER_03: I think that's like to serve a tennis ball or something like this.
• 19:29
  SPEAKER_03: Like I wouldn't do that by just watching a bunch of YouTube videos.
• 19:31
  SPEAKER_03: I would do that by tossing a tennis ball and trying to hit it, right?
• 19:34
  And then do practicing for days or weeks or months or years
• 19:38
  depending on how good I
• 19:38
  SPEAKER_03: wanted to get,
• 19:39
  SPEAKER_03: right?
• 19:40
  SPEAKER_03: And that's what reinforcement learning is in general.
• 19:42
  It is saying that the robot is going to just like sit there and
• 19:45
  take in information
• 19:46
  SPEAKER_03: about the world and it's going to
• 19:49
  SPEAKER_03: And it's going to make a decision and try something,
• 19:52
  SPEAKER_03: and if it fails,
• 19:53
  SPEAKER_03: it's going to take that and think about it and do better next time,
• 19:55
  SPEAKER_03: hopefully.
• 19:56
  SPEAKER_03: Or if it succeeds, it's going to be like, okay, that was good.
• 19:59
  SPEAKER_03: Let's try again a little bit.
• 20:02
  SPEAKER_03: Let's try to do more like that.
• 20:04
  And the reason that I'm saying it's the best is because it's
• 20:09
  the same reason that
• 20:10
  SPEAKER_03: it's best for you or me.
• 20:12
  SPEAKER_03: If you want to learn to do something,
• 20:13
  SPEAKER_03: You'll have your own embodiment and your own version,
• 20:16
  SPEAKER_03: your own thoughts about how to do that,
• 20:18
  SPEAKER_03: and you're never going to really get that from watching someone.
• 20:21
  SPEAKER_03: But you'll do pretty good from watching someone.
• 20:23
  SPEAKER_03: You'll learn a lot from watching someone,
• 20:25
  SPEAKER_03: but you still at some point always need to do it yourself.
• 20:27
  SPEAKER_02: Right?
• 20:28
  SPEAKER_02: And so that's a good segue into kind of another aspect,
• 20:33
  SPEAKER_02: which is kind of one I really wanted to better understand,
• 20:35
  SPEAKER_02: which is learning from kind of human video.
• 20:37
  SPEAKER_02: So the concept of can you collect,
• 20:39
  SPEAKER_02: you know,
• 20:39
  SPEAKER_02: you collect data,
• 20:40
  SPEAKER_02: you label it,
• 20:41
  SPEAKER_02: you provide contacts on and on and on.
• 20:43
  SPEAKER_02: You have your own data schema.
• 20:45
  SPEAKER_02: So Mike,
• 20:46
  SPEAKER_02: what's kind of your perspective on,
• 20:48
  because of just what Chris said around like you actually have
• 20:50
  to do the thing to
• 20:51
  SPEAKER_02: learn the thing.
• 20:52
  SPEAKER_02: At some point, at some point.
• 20:55
  SPEAKER_03: You can get pretty far without it, right?
• 20:58
  Let's dig into that because this is like a very abstract thing
• 21:00
  for folks that
• 21:01
  SPEAKER_02: aren't in the robotic space.
• 21:02
  SPEAKER_02: So how useful is kind of human video for training robots, Mike?
• 21:06
  SPEAKER_02: I mean, do you think, is that a critical piece right now?
• 21:09
  SPEAKER_02: And will it be a critical piece going forward?
• 21:10
  SPEAKER_02: Or is it going to be the robot physically doing
• 21:15
  SPEAKER_02: like physically watching someone and not video.
• 21:19
  SPEAKER_00: Yeah, I think it's a critical piece, but you, it can't be the only piece.
• 21:23
  SPEAKER_00: I think, like I said just now, there are three types of data.
• 21:26
  SPEAKER_00: You don't need all three types is my view.
• 21:28
  SPEAKER_00: You need to co-train on all three types.
• 21:30
  SPEAKER_00: Video being one of them.
• 21:31
  SPEAKER_00: Video is great because it's so easy to collect video.
• 21:34
  SPEAKER_00: We already have a lot of it on YouTube, let's say.
• 21:37
  You can also quite easily just collect ecosystem video,
• 21:40
  that sort of stuff.
• 21:42
  SPEAKER_00: The issue,
• 21:43
  SPEAKER_00: well,
• 21:43
  SPEAKER_00: I mean,
• 21:44
  SPEAKER_00: the great thing about video is because it's so easy to collect,
• 21:46
  SPEAKER_00: you get a lot of diversity of environments and tasks.
• 21:52
  SPEAKER_00: And you want the diversity, right, to so-called
• 21:57
  hopefully allow the model to see more things and then
• 21:59
  generalize better ultimately.
• 22:01
  SPEAKER_00: But I also think that there's a pretty low threshold to what,
• 22:05
  SPEAKER_00: like the performance of a model that you only train from video.
• 22:08
  SPEAKER_00: Because these videos doesn't come with action labels.
• 22:11
  SPEAKER_00: Unlike, let's say, teleoperation, it's hard to think about it.
• 22:13
  SPEAKER_00: So you always have to use some techniques to infer actions.
• 22:17
  SPEAKER_00: But those actions are always, you know, like approximation.
• 22:22
  SPEAKER_00: Not only that, because every embodiment is different.
• 22:25
  SPEAKER_00: Imagine you have an ecosystem video of some humans doing something.
• 22:30
  SPEAKER_00: The thing is,
• 22:30
  SPEAKER_00: all of us have hands,
• 22:31
  SPEAKER_00: but all of our hands look similar,
• 22:33
  SPEAKER_00: but they're actually also distinctly different.
• 22:36
  SPEAKER_00: And I tend to think that in robotics, all the minor detail matters a lot.
• 22:40
  SPEAKER_00: Millimeters, you're talking about 50 milliseconds latency.
• 22:45
  could be the difference between you falling down and standing
• 22:48
  up,
• 22:48
  you know?
• 22:49
  SPEAKER_00: And so when you actually apply in the real physical,
• 22:51
  SPEAKER_00: because the real physical world,
• 22:53
  SPEAKER_00: like physics is just brutal,
• 22:54
  SPEAKER_00: like gravity is brutal.
• 22:55
  SPEAKER_00: Like if you just miss by that little bit,
• 22:58
  SPEAKER_00: it's the difference between you're able to do it and not doing it.
• 23:01
  SPEAKER_00: And so I think video is great,
• 23:02
  SPEAKER_00: again,
• 23:03
  SPEAKER_00: to give you that whole very broad concept,
• 23:05
  SPEAKER_00: okay,
• 23:05
  SPEAKER_00: what is playing tennis,
• 23:06
  SPEAKER_00: what is playing basketball,
• 23:07
  SPEAKER_00: things like that,
• 23:08
  SPEAKER_00: right?
• 23:08
  SPEAKER_00: But if you actually want to play basketball,
• 23:11
  SPEAKER_00: I think you can't just watch basketball.
• 23:14
  SPEAKER_00: Yeah, something, yeah.
• 23:17
  SPEAKER_02: And so Chris,
• 23:18
  SPEAKER_02: when you think about what Mike just said,
• 23:20
  which is obviously capturing video in the diversity of
• 23:24
  environments is actually a
• 23:25
  SPEAKER_02: very easy thing to do,
• 23:26
  SPEAKER_02: obviously,
• 23:26
  SPEAKER_02: with all the devices that we have today.
• 23:28
  SPEAKER_02: Like as a roboticist, what needs,
• 23:32
  SPEAKER_02: If you were to start collecting a bunch of this data,
• 23:34
  what would need to be built on top of it for actually to be
• 23:37
  useful for someone like
• 23:38
  SPEAKER_02: you to actually train the robots that you're working on?
• 23:42
  So like Michael was saying,
• 23:44
  one of the things that you need to do is kind of
• 23:47
  the robot needs to be able to understand the actions that are
• 23:50
  being performed,
• 23:51
  right?
• 23:51
  SPEAKER_03: So what that means is a lot of these methods that you'll see,
• 23:55
  SPEAKER_03: and we've talked about a couple,
• 23:57
  SPEAKER_03: we've had a bunch of these on our podcast,
• 23:59
  SPEAKER_03: right?
• 23:59
  SPEAKER_03: There's a few ways that you can do it.
• 24:01
  SPEAKER_03: One of them is like,
• 24:03
  actually like predicting like this is where my hand this is
• 24:05
  where the hand is in
• 24:06
  the video and like but obviously that's super limiting because
• 24:09
  most videos that we
• 24:11
  take like on youtube don't have like clear videos clear
• 24:14
  pictures of your hands and
• 24:15
  so on so like ideally things get less structured and you can
• 24:19
  kind of throw more
• 24:20
  data into the mixture but then you have to figure out how to
• 24:22
  actually train on all
• 24:23
  kinds of heterogeneous data and that is different missing field
• 24:28
  it's a complicated
• 24:28
  SPEAKER_03: and open research area in my opinion honestly
• 24:31
  SPEAKER_03: A lot of the things that you see companies doing,
• 24:34
  SPEAKER_03: like Tesla,
• 24:35
  SPEAKER_03: for example,
• 24:35
  is just like they actually seem to have just like a setup where
• 24:39
  you can very
• 24:40
  SPEAKER_03: clearly see what the robot is doing,
• 24:42
  SPEAKER_03: right?
• 24:42
  SPEAKER_03: Which is not, sorry, not what the robot's doing, what the human's doing.
• 24:47
  SPEAKER_03: Both, right?
• 24:48
  SPEAKER_03: Because it needs to know what it's doing to be able to learn off of it.
• 24:51
  SPEAKER_03: But that's obviously not,
• 24:52
  SPEAKER_03: that wouldn't be the end goal if you wanted to,
• 24:54
  SPEAKER_03: if you really want to train off all of YouTube or something.
• 24:57
  SPEAKER_03: We don't have that yet.
• 24:58
  SPEAKER_03: That's not something that we're,
• 24:59
  SPEAKER_03: that currently people are doing.
• 25:02
  SPEAKER_03: They're useful.
• 25:07
  So it's like a different kind of video,
• 25:08
  I think,
• 25:08
  than you're usually thinking of.
• 25:09
  SPEAKER_03: And this is a real limitation.
• 25:11
  SPEAKER_02: So when you say a different type of video,
• 25:13
  SPEAKER_02: are you talking about like,
• 25:14
  SPEAKER_02: are you talking about,
• 25:15
  SPEAKER_02: let's use an example,
• 25:17
  SPEAKER_02: right?
• 25:17
  SPEAKER_02: So let's use the example of like bartending, for instance, right?
• 25:21
  So are you talking about like a dedicated video to like slicing
• 25:25
  a lime or preparing
• 25:29
  SPEAKER_02: a cocktail and like having very intentional data to like...
• 25:34
  SPEAKER_03: Usually, yeah, that's right.
• 25:36
  SPEAKER_03: But also,
• 25:37
  SPEAKER_03: usually,
• 25:37
  SPEAKER_03: depending on how you're doing it a lot of the time,
• 25:39
  that means that you need to be able to estimate where the hands
• 25:42
  are in the video,
• 25:42
  SPEAKER_03: which means that it's not just
• 25:44
  SPEAKER_03: a video of a bartender doing a lime.
• 25:45
  It's like a video from the bartender's point of view of their
• 25:48
  hands slicing a lime or whatever.
• 25:51
  SPEAKER_03: So that means that because to be able to know what the robot should do,
• 25:54
  SPEAKER_03: I need to know where this finger is a lot of the time.
• 25:58
  Otherwise,
• 25:58
  because that's what the robot's going to have to do itself
• 26:00
  later.
• 26:01
  So a lot of learning from human video stuff ends up getting
• 26:03
  very complicated like this.
• 26:05
  SPEAKER_03: Now, of course, that's not all of it.
• 26:06
  There's quite a lot of different ways that people have learned
• 26:08
  from human video.
• 26:10
  SPEAKER_03: But yeah.
• 26:12
  SPEAKER_00: Yeah,
• 26:12
  I should also just add actually video alone is not going to get
• 26:17
  us there also to
• 26:18
  human performance because like you can imagine if you take an
• 26:21
  egocentric video,
• 26:22
  SPEAKER_00: there's a lot of occlusion.
• 26:23
  SPEAKER_00: Let's say when I pick up a box,
• 26:24
  you cannot see the rest of my big fingers because the box is
• 26:27
  covering the fingers,
• 26:27
  SPEAKER_00: but I'm actually touching the box,
• 26:28
  SPEAKER_00: right?
• 26:29
  SPEAKER_00: So we humans take it for granted.
• 26:31
  SPEAKER_00: We have very, very good touch sensors all over our body, right?
• 26:34
  SPEAKER_00: Especially the fingers, dips and all that.
• 26:37
  SPEAKER_00: So we actually are organizing this robotic origami competition.
• 26:43
  SPEAKER_00: We announced it a couple of weeks ago.
• 26:45
  And we actually are co-organizing that competition together
• 26:48
  with some folks from
• 26:49
  SPEAKER_00: the Japan Origami Association.
• 26:51
  SPEAKER_00: So these are like the real human experts doing origami.
• 26:54
  So we actually invited like a grandmaster of the origami
• 26:58
  association to this robotic conference.
• 27:00
  SPEAKER_00: And then next to that,
• 27:01
  the old man doing origami is is a robot doing origami and the
• 27:04
  robot absolutely like
• 27:05
  it's very impressive but nothing compared to the old man but
• 27:08
  what i want to
• 27:08
  highlight though is i was like standing in front of this like
• 27:11
  grandmaster for like
• 27:12
  10 minutes just observing what he's doing half the time he's
• 27:16
  talking to people
• 27:17
  SPEAKER_00: around him he's not even looking it's all touch
• 27:22
  SPEAKER_00: Yeah.
• 27:23
  SPEAKER_00: Yeah.
• 27:24
  SPEAKER_00: Like, you know, making his paper crane on that.
• 27:25
  SPEAKER_00: So,
• 27:26
  SPEAKER_00: so I think,
• 27:27
  SPEAKER_00: so,
• 27:27
  SPEAKER_00: so there's also a very big hardware thing that we,
• 27:30
  SPEAKER_00: we just don't have good tough senses,
• 27:31
  SPEAKER_00: right?
• 27:32
  SPEAKER_00: Anywhere near as good as ours.
• 27:35
  SPEAKER_00: Sure.
• 27:35
  In terms of like,
• 27:36
  you know,
• 27:36
  like fatality at the cost that it's,
• 27:39
  that makes sense.
• 27:40
  SPEAKER_00: So that's also another one.
• 27:42
  And I think you're understanding a little bit how good the
• 27:44
  robot part was.
• 27:46
  SPEAKER_03: The robot guy,
• 27:48
  SPEAKER_03: I walked by this booth,
• 27:49
  SPEAKER_03: obviously,
• 27:49
  SPEAKER_03: at this conference with lots of top researchers.
• 27:52
  And a lot of them were saying,
• 27:53
  this is one of the coolest robot manipulation things I've seen.
• 27:56
  SPEAKER_03: And he's just getting blown away by the old man next door.
• 27:59
  SPEAKER_03: Right.
• 27:59
  SPEAKER_03: So like, that's just like, I want to just be clear.
• 28:02
  SPEAKER_03: It was actually very good.
• 28:03
  SPEAKER_03: The robot part was very good.
• 28:05
  SPEAKER_03: It was just, yeah.
• 28:08
  SPEAKER_02: Yeah.
• 28:08
  SPEAKER_02: Yeah.
• 28:09
  SPEAKER_02: We take it for granted how like, how unbelievable our hands are, right?
• 28:12
  Like just the dexterity and like what you can manipulate and
• 28:16
  how you can manipulate.
• 28:17
  SPEAKER_02: It's pretty, it's pretty remarkable.
• 28:19
  All right,
• 28:20
  let's go to kind of one last one,
• 28:21
  which is sim to reel or reel to sim.
• 28:25
  SPEAKER_02: So Chris,
• 28:26
  SPEAKER_02: maybe kind of break that down for us,
• 28:29
  SPEAKER_02: what that means and kind of your perspective on the level of use,
• 28:36
  SPEAKER_02: like usability in training and training.
• 28:40
  SPEAKER_03: Yeah,
• 28:40
  SPEAKER_03: so this is actually something that I worked on for a very long time,
• 28:42
  SPEAKER_03: like six plus years,
• 28:44
  SPEAKER_03: I think.
• 28:45
  SPEAKER_03: But so robotics.
• 28:48
  So yeah,
• 28:49
  SynthReal means that we train the robot how to do something in
• 28:52
  a simulation.
• 28:54
  SPEAKER_03: And then it also has to work in the real world, right?
• 28:59
  So I take the robot,
• 29:00
  and maybe I create a digital twin of the task that I want it to
• 29:03
  perform.
• 29:05
  If it's an industrial task or an industrial kidding task,
• 29:07
  maybe I would just
• 29:09
  spawn all the objects that I need to place on a table and have
• 29:12
  a copy of the robot
• 29:14
  and have it sit and have and then maybe I create a whole bunch
• 29:16
  like a million of
• 29:17
  these robots in simulation I have them all practice picking and
• 29:20
  placing the object
• 29:21
  and into into the target and whatever that right and I use that
• 29:26
  data like I
• 29:27
  SPEAKER_03: simulate cameras or whatever sensors that I may have access to and then I
• 29:32
  SPEAKER_03: train a policy, train a neural network on that simulated data.
• 29:37
  SPEAKER_03: And the sim-to-real part, that's the sim part.
• 29:39
  The sim-to-real part is then I have to set that up in such a
• 29:42
  way that I can then
• 29:43
  SPEAKER_03: take that policy and use it in the real world.
• 29:45
  SPEAKER_03: So this is the way that a lot of things work right now,
• 29:49
  particularly when it comes to all the dancing robot videos that
• 29:51
  you see are
• 29:52
  SPEAKER_03: sim-to-real.
• 29:55
  SPEAKER_03: all of them are trained this way.
• 29:57
  SPEAKER_03: But there's a lot of other stuff like central manipulation is another,
• 30:00
  SPEAKER_03: is an active research area,
• 30:02
  SPEAKER_03: so.
• 30:04
  SPEAKER_02: Let's go over to RoboPapers.
• 30:06
  SPEAKER_02: Thank you for that.
• 30:07
  SPEAKER_02: So actually,
• 30:08
  SPEAKER_02: before we do that,
• 30:09
  SPEAKER_02: I wanna,
• 30:10
  SPEAKER_02: I guess if we were to step back,
• 30:12
  SPEAKER_02: when you think about all these different training methods,
• 30:14
  and obviously several folks working on a variety of both open
• 30:18
  source and their own
• 30:19
  SPEAKER_02: closed source robot foundational models and multimodal models and world
• 30:22
  SPEAKER_02: foundational models,
• 30:23
  SPEAKER_02: whatever you wanna call it,
• 30:26
  SPEAKER_02: I want to get each of your perspectives on,
• 30:28
  SPEAKER_02: on,
• 30:28
  SPEAKER_02: on,
• 30:29
  SPEAKER_02: on how do we,
• 30:30
  SPEAKER_02: how do we ultimately solve this training gap?
• 30:33
  Like what is the thing that we can do today to move the field
• 30:36
  forward in the most
• 30:37
  SPEAKER_02: effective way,
• 30:38
  SPEAKER_02: I guess,
• 30:38
  SPEAKER_02: uh,
• 30:39
  SPEAKER_02: around the training gap.
• 30:40
  SPEAKER_02: If there's one thing that you pick, you pick, we could do.
• 30:45
  SPEAKER_02: Gosh,
• 30:46
  SPEAKER_00: uh,
• 30:47
  SPEAKER_00: I think you need,
• 30:48
  SPEAKER_00: yeah,
• 30:48
  SPEAKER_00: it's very hard to pick one thing because like,
• 30:50
  SPEAKER_00: it's a,
• 30:51
  SPEAKER_00: it's very unclear which one is actually going to scale.
• 30:53
  SPEAKER_00: Like all, all these, uh,
• 30:55
  SPEAKER_03: I mean, if it's like wave your magic wand, infinite money.
• 30:58
  SPEAKER_03: Sorry.
• 30:58
  SPEAKER_03: Go ahead.
• 31:00
  SPEAKER_03: I have an answer.
• 31:01
  SPEAKER_00: I have talked to so many different researchers now.
• 31:04
  SPEAKER_00: Different researchers have different,
• 31:05
  SPEAKER_00: like,
• 31:06
  SPEAKER_00: kind of,
• 31:06
  SPEAKER_00: let's say,
• 31:07
  SPEAKER_00: I don't know,
• 31:08
  SPEAKER_00: like,
• 31:08
  SPEAKER_00: inclination of biases.
• 31:11
  SPEAKER_00: Some people think, you know, TeddyWarp is really the way.
• 31:15
  SPEAKER_00: Some people are really betting on, like, role model.
• 31:18
  SPEAKER_00: I mean, I should say something about simulation, right?
• 31:20
  SPEAKER_00: So to me,
• 31:22
  SPEAKER_00: Simulation data.
• 31:24
  SPEAKER_00: Simulation have kind of two types now.
• 31:26
  SPEAKER_00: You have the traditional simulation engine,
• 31:29
  SPEAKER_00: and then you have the wall model,
• 31:30
  SPEAKER_00: which is the new stuff.
• 31:32
  SPEAKER_00: Because the wall model is, in a way, a simulator as well.
• 31:34
  SPEAKER_00: It just behaves very, very differently and trains very differently.
• 31:39
  SPEAKER_00: I think simulation has got to...
• 31:41
  SPEAKER_00: Okay,
• 31:41
  SPEAKER_00: if I were to bet,
• 31:42
  SPEAKER_00: I feel that simulation alone can get us 80%,
• 31:44
  SPEAKER_00: 90% there.
• 31:44
  SPEAKER_00: It's my guess.
• 31:44
  SPEAKER_00: Okay.
• 31:47
  SPEAKER_00: But because you need 99.9999% so it's not enough therefore.
• 31:51
  SPEAKER_00: But it can probably move us all near to 90 something percent.
• 31:56
  SPEAKER_00: The issue with simulation,
• 31:57
  SPEAKER_00: I don't know whether Chris you agree with me,
• 31:59
  SPEAKER_00: at least with the classical simulation engine,
• 32:01
  SPEAKER_00: is that I believe contrary to popular belief,
• 32:04
  SPEAKER_00: we're not bottlenecked by compute actually,
• 32:07
  SPEAKER_00: I think.
• 32:08
  SPEAKER_00: I think that in a way, the
• 32:12
  SPEAKER_00: its capabilities is also its curse, right?
• 32:14
  Because with a very good simulation engine,
• 32:17
  you can literally simulate anything you want.
• 32:19
  And that's a big problem because the search space or the
• 32:23
  exploration space is nearly infinite.
• 32:26
  SPEAKER_02: It's too wide, yeah.
• 32:28
  SPEAKER_00: Yeah,
• 32:28
  SPEAKER_00: you can imagine,
• 32:30
  SPEAKER_00: you can use a simulation engine to,
• 32:31
  SPEAKER_00: let's say,
• 32:31
  SPEAKER_00: simulate a robot frying egg,
• 32:33
  SPEAKER_00: let's say.
• 32:35
  You can imagine in your head maybe an infinite way to set up
• 32:37
  that scene,
• 32:38
  that task.
• 32:39
  SPEAKER_00: Different lighting, different robots, different eggs and whatnot.
• 32:42
  And so it will take a crack researchers like Chris to
• 32:45
  drastically shrink that
• 32:47
  search space into something that is tractable with the compute
• 32:50
  budget that he has.
• 32:52
  SPEAKER_00: But as researchers do that,
• 32:54
  SPEAKER_00: well,
• 32:54
  SPEAKER_00: so,
• 32:55
  SPEAKER_00: you know,
• 32:55
  they are missing a lot of actually potential edge cases that
• 32:58
  may happen in the real
• 32:58
  SPEAKER_00: world that they could have simulated,
• 33:00
  SPEAKER_00: but they just don't know.
• 33:01
  SPEAKER_00: I mean, they would have messed up.
• 33:03
  SPEAKER_00: And so I think that's a big part of it.
• 33:05
  SPEAKER_00: So if I look at it that way,
• 33:06
  SPEAKER_00: actually the bottleneck to a simulation,
• 33:09
  SPEAKER_00: a big bottleneck is having more people like Chris,
• 33:11
  SPEAKER_00: honestly,
• 33:13
  SPEAKER_00: because that process is very curated.
• 33:15
  SPEAKER_00: Like it's very handcrafted.
• 33:16
  SPEAKER_00: Like you need,
• 33:17
  SPEAKER_00: like Chris will have to come up with very like a man,
• 33:21
  SPEAKER_00: like handcrafted,
• 33:22
  SPEAKER_00: like the rewards functions for the reinforcement learning,
• 33:26
  SPEAKER_00: things like that.
• 33:27
  SPEAKER_00: It's all very handcrafted.
• 33:28
  SPEAKER_00: So, which means actually the human in the loop,
• 33:31
  SPEAKER_00: and unique researchers to do it.
• 33:32
  SPEAKER_00: Like, you know, it's a very, very specialized skill set.
• 33:37
  SPEAKER_00: That's a big bottleneck.
• 33:39
  SPEAKER_00: World model,
• 33:40
  SPEAKER_00: on the other hand,
• 33:40
  SPEAKER_00: all this video generation,
• 33:41
  SPEAKER_00: I think it's also very,
• 33:42
  SPEAKER_00: very interesting.
• 33:44
  SPEAKER_00: I think they potentially can move the needle a lot.
• 33:46
  SPEAKER_00: But you can imagine to build a world model,
• 33:47
  SPEAKER_00: you also need a lot of probably like video data and tele-op data too.
• 33:52
  So you still need a lot of data,
• 33:54
  a huge amount of data to build a decent world model,
• 33:58
  right?
• 34:00
  SPEAKER_00: Yeah, so it's hard to pick one.
• 34:03
  SPEAKER_00: I feel that tele-op data, video data, simulation,
• 34:08
  SPEAKER_00: all are needed.
• 34:08
  SPEAKER_00: Like we, we need all.
• 34:10
  SPEAKER_00: And there's a lot of papers that we've come across,
• 34:12
  SPEAKER_00: you know,
• 34:12
  SPEAKER_00: that we've interviewed that do co-training of various sorts,
• 34:16
  SPEAKER_00: right?
• 34:17
  SPEAKER_00: They'll,
• 34:17
  SPEAKER_00: they'll,
• 34:18
  SPEAKER_00: they'll,
• 34:20
  SPEAKER_00: they'll,
• 34:20
  SPEAKER_00: they'll scale it using some video or like,
• 34:22
  SPEAKER_00: you know,
• 34:22
  SPEAKER_00: do the real to seem the real thing.
• 34:24
  SPEAKER_02: Yeah.
• 34:25
  SPEAKER_02: So, yep.
• 34:26
  SPEAKER_02: Sure.
• 34:26
  SPEAKER_02: What's your thought, Chris?
• 34:28
  SPEAKER_02: If there was kind of what,
• 34:28
  SPEAKER_02: if you have one thing that kind of stands out in your mind,
• 34:30
  SPEAKER_02: if someone who's actually in there doing,
• 34:32
  SPEAKER_02: you know,
• 34:32
  SPEAKER_02: working on it every day,
• 34:34
  SPEAKER_03: I mean,
• 34:35
  I think that if you were going to wave a magic wand and had
• 34:38
  infinite money,
• 34:38
  SPEAKER_03: then I think you'd just have lots and lots of robots doing.
• 34:43
  SPEAKER_03: I think that they worked.
• 34:45
  SPEAKER_03: But practically speaking, I think Michael's correct.
• 34:50
  SPEAKER_03: It seems like you need a little bit of everything,
• 34:52
  SPEAKER_03: because they're all good at slightly different things.
• 34:54
  SPEAKER_03: And they all have different pluses and minuses.
• 34:56
  SPEAKER_03: Sure.
• 34:58
  SPEAKER_03: Yeah.
• 34:59
  SPEAKER_03: So the real solution is going to be a mix.
• 35:02
  So,
• 35:02
  Chris,
• 35:02
  I actually saw a recent tweet that you put out that caught my
• 35:07
  eye.
• 35:07
  SPEAKER_02: And I don't have it in front of me, so I might butcher it a little bit.
• 35:12
  SPEAKER_02: But basically,
• 35:13
  the premise was the reality of it is if we want low-cost
• 35:18
  generalized robots,
• 35:19
  SPEAKER_02: we need a bunch of robots in the field learning and actually doing stuff.
• 35:22
  That's,
• 35:23
  SPEAKER_03: yeah,
• 35:23
  SPEAKER_03: that's increasingly kind of what I,
• 35:25
  SPEAKER_03: my opinion on a lot of this is just like,
• 35:27
  SPEAKER_03: it'll be solved with that.
• 35:29
  Like we have,
• 35:29
  and that's one of the things is that like the robots are so
• 35:32
  much better.
• 35:32
  Like,
• 35:32
  so I,
• 35:33
  I've,
• 35:34
  I've been doing this since taken one form or another since like
• 35:37
  2014.
• 35:37
  SPEAKER_03: Right.
• 35:38
  SPEAKER_03: So like I've been doing robotics for a long time.
• 35:41
  SPEAKER_03: And when,
• 35:41
  SPEAKER_03: when I started,
• 35:42
  we paid like $40,000 for a UR5 industrial arm that failed like
• 35:47
  every few days.
• 35:48
  SPEAKER_03: And like we had a, we had another, like this really, really,
• 35:52
  SPEAKER_03: not so great robot arm that like,
• 35:57
  They had hand-tuned PID controllers where one of the grad
• 36:00
  students stayed up and
• 36:01
  slammed into the wall and broke it into a million pieces and
• 36:03
  that kind of stuff.
• 36:04
  SPEAKER_03: And so this was the reality.
• 36:07
  SPEAKER_03: And that shuts down the lab for months, many months.
• 36:11
  SPEAKER_03: So that was the reality for a long time.
• 36:12
  SPEAKER_03: And now you can buy a $20,000 unitree robot from Walmart.
• 36:15
  SPEAKER_03: Or you can 3D print one of those SO200 or
• 36:25
  SPEAKER_03: What's it?
• 36:25
  SPEAKER_03: Vector's robot.
• 36:26
  SPEAKER_03: I forget that.
• 36:27
  SPEAKER_03: The robot.
• 36:28
  SPEAKER_03: The various little robot things.
• 36:30
  SPEAKER_03: You can 3D print.
• 36:32
  SPEAKER_03: You can build these things.
• 36:35
  We're so much closer to that world with infinite robots
• 36:38
  everywhere,
• 36:39
  I think,
• 36:39
  than we used to be.
• 36:40
  SPEAKER_03: So that's...
• 36:42
  SPEAKER_02: Yeah, it's cool.
• 36:43
  SPEAKER_02: It's cool seeing it all come together.
• 36:44
  SPEAKER_02: And as someone who,
• 36:46
  SPEAKER_02: I guess,
• 36:46
  without that kind of background in robotics and the more folks
• 36:51
  that I speak to,
• 36:51
  SPEAKER_02: especially on the research side,
• 36:52
  SPEAKER_02: and someone always thinking about like market,
• 36:56
  market application and how to get adoption and useful
• 36:58
  application of these
• 37:00
  SPEAKER_02: different robotic systems.
• 37:01
  SPEAKER_02: Yeah.
• 37:02
  SPEAKER_02: It's very clear that it's such an exciting time.
• 37:04
  SPEAKER_02: It's very clear that there's a lot of movement and it's moving fast,
• 37:08
  SPEAKER_02: but yet I'm challenged because most of the robotics companies that I see,
• 37:12
  I think the path to any type of adoptability or market
• 37:17
  adoptability is very
• 37:18
  SPEAKER_02: challenging.
• 37:20
  SPEAKER_02: still at this case.
• 37:20
  And so I don't know if you guys think from like a market
• 37:22
  viability standpoint that
• 37:25
  the vast majority is still not viable or do you actually see
• 37:28
  kind of a window here
• 37:31
  SPEAKER_02: where it's going to start becoming really viable with any type of scale?
• 37:36
  Well,
• 37:36
  I think it's absolutely viable if you look at it as an
• 37:38
  entertainment thing,
• 37:40
  right?
• 37:40
  SPEAKER_00: So Unitree is putting up... I love it.
• 37:43
  SPEAKER_02: He's pulling in the entertainment plug.
• 37:45
  SPEAKER_02: Give it to us.
• 37:46
  SPEAKER_00: No, no, for real, for real.
• 37:47
  SPEAKER_00: Like Jordan, I absolutely believe in it.
• 37:49
  SPEAKER_00: Like,
• 37:49
  SPEAKER_00: you know,
• 37:50
  SPEAKER_00: if you look at,
• 37:51
  SPEAKER_00: Unitree is going to start shipping the R1,
• 37:53
  SPEAKER_00: which is a humanoid plastic key.
• 37:57
  SPEAKER_00: Yeah, sorry.
• 37:58
  SPEAKER_00: I mean, Chris, you saw it too, right?
• 38:03
  SPEAKER_00: In Korea.
• 38:03
  SPEAKER_00: So $6,000, $7,000, something like that.
• 38:05
  SPEAKER_00: So at that price point,
• 38:06
  SPEAKER_00: I imagine every bar,
• 38:09
  SPEAKER_00: every disco in the country is going to have a few of these.
• 38:12
  SPEAKER_00: Yeah.
• 38:13
  SPEAKER_00: so that patrons can, you know, every bar is going to have their own.
• 38:16
  It's like having a karaoke system,
• 38:17
  maybe cheaper than a karaoke system,
• 38:19
  let's say.
• 38:20
  SPEAKER_00: You know,
• 38:21
  SPEAKER_00: like,
• 38:21
  SPEAKER_00: you know,
• 38:21
  SPEAKER_00: all these bars in Texas,
• 38:22
  SPEAKER_00: they have all this,
• 38:23
  SPEAKER_00: like,
• 38:23
  SPEAKER_00: you know,
• 38:24
  SPEAKER_00: the pool thing,
• 38:24
  SPEAKER_00: the mechanical pool.
• 38:25
  SPEAKER_00: It's like that, you know, so people are going to have that.
• 38:29
  And so,
• 38:30
  and so that's,
• 38:32
  honestly,
• 38:32
  this is how I think we'll get to the first 100,000 huminoid.
• 38:35
  SPEAKER_00: It's just bars, entertainment venue, buying a bunch of these, and then...
• 38:43
  SPEAKER_00: And then that will get some of this supply chain,
• 38:46
  SPEAKER_00: you know,
• 38:46
  SPEAKER_00: more honed up,
• 38:47
  SPEAKER_00: you know,
• 38:47
  SPEAKER_00: you're talking about the actuators,
• 38:48
  SPEAKER_00: the batteries and all that.
• 38:50
  SPEAKER_00: And once you get to that run,
• 38:51
  SPEAKER_00: you know,
• 38:52
  SPEAKER_00: you get a lot more instances of how these things may fail,
• 38:55
  SPEAKER_00: right?
• 38:56
  SPEAKER_00: You know, a lot of these actuators may fail, especially on the shoulder.
• 39:00
  SPEAKER_00: So you just can't really know all this until you apply in the real world.
• 39:05
  And I think honestly,
• 39:06
  like the entertainment,
• 39:07
  everything is the first use case.
• 39:11
  SPEAKER_00: 100,000 units at least.
• 39:13
  Not the application that most people are thinking about,
• 39:15
  but I do think that's realistic.
• 39:18
  SPEAKER_02: Well,
• 39:18
  SPEAKER_02: you know,
• 39:18
  SPEAKER_02: I think what I like about the perspective,
• 39:20
  I actually have thought a lot about this because I just started
• 39:22
  seeing some of
• 39:23
  these robots showing up at conferences and events and trade
• 39:25
  shows and not like
• 39:27
  SPEAKER_02: robotics trade shows,
• 39:28
  SPEAKER_02: just like generalized,
• 39:29
  SPEAKER_02: right?
• 39:29
  SPEAKER_02: And you have like the Pepsi branded humanoid and whatever it might be.
• 39:35
  I think that that kind of perspective is nice because it's kind
• 39:37
  of like you take
• 39:37
  kind of the seriousness out of it and it's kind of like playful
• 39:40
  and it's playful
• 39:41
  SPEAKER_02: and fun versus like trying to be so overly,
• 39:44
  SPEAKER_02: I don't know,
• 39:45
  SPEAKER_02: just like intense about like...
• 39:47
  SPEAKER_02: We need to solve general labor and we need to do this and this.
• 39:50
  SPEAKER_02: It's just like, you know.
• 39:53
  SPEAKER_00: Well, it's good for fundraising.
• 39:54
  SPEAKER_00: I mean, because that's it.
• 39:55
  SPEAKER_00: Multiple trillions of dollars of impact.
• 39:58
  SPEAKER_00: Of course, of course.
• 39:59
  SPEAKER_00: And humanity should solve it.
• 40:00
  SPEAKER_00: I mean, we need good researchers like Chris to keep going at it.
• 40:05
  SPEAKER_00: They shouldn't be distracted by human affecting.
• 40:08
  SPEAKER_00: But in the meantime,
• 40:09
  SPEAKER_00: I mean,
• 40:10
  SPEAKER_00: that's one way to get a lot of robots out there,
• 40:13
  SPEAKER_00: especially humans.
• 40:15
  SPEAKER_00: It's to just say this is a fun thing.
• 40:18
  SPEAKER_00: It doesn't matter if it breaks, it's just for fun.
• 40:20
  SPEAKER_00: And so,
• 40:21
  SPEAKER_00: and that could be,
• 40:22
  SPEAKER_00: I mean,
• 40:22
  SPEAKER_00: that's how actually,
• 40:23
  SPEAKER_00: honestly,
• 40:24
  SPEAKER_00: I believe in it because I started off again with a sidewalk robot.
• 40:27
  SPEAKER_00: I didn't pitch it as something that delivers food for you.
• 40:30
  SPEAKER_00: I just say this is Pokemon Go in the real world.
• 40:33
  SPEAKER_00: Go out with it.
• 40:34
  SPEAKER_00: It's a couple hundred dollars.
• 40:35
  SPEAKER_00: Even if it breaks, no one care.
• 40:37
  SPEAKER_00: It's just a toy.
• 40:38
  SPEAKER_00: And it turns out, guess what?
• 40:39
  The data collected are now used by all the top researchers that
• 40:42
  do this stuff,
• 40:43
  right?
• 40:43
  SPEAKER_00: And so you can't do it in all robotic cases,
• 40:47
  SPEAKER_00: but I think where possible,
• 40:48
  SPEAKER_00: you should try to find an application that is fun for robotics.
• 40:52
  Maybe some of the things,
• 40:53
  the data that can be used could be,
• 40:54
  I don't know.
• 40:55
  SPEAKER_00: Or let's say even the humanoid, let's say they can't be boxing 24-7.
• 41:00
  SPEAKER_00: When they're not boxing,
• 41:00
  maybe some researchers can use those to do real-world events
• 41:04
  for the serious
• 41:04
  SPEAKER_00: research stuff.
• 41:06
  SPEAKER_00: Because the researchers are all desperate for more robots like Chris say.
• 41:09
  So if there's a business model like entertainment that can fund
• 41:13
  and justify the
• 41:15
  SPEAKER_00: manufacturing costs and deployment costs,
• 41:18
  SPEAKER_00: but they are not going to be used for fun things 24-7,
• 41:21
  SPEAKER_00: then maybe we can tap those for research purposes.
• 41:25
  sure data collection let's let's move over to for a few minutes
• 41:28
  to robo papers so
• 41:32
  as i mentioned at the beginning of the pod you're you're you
• 41:35
  you're going to very
• 41:36
  kind of deeply technical conversations with with robot
• 41:39
  researchers and so chris
• 41:40
  just kind of walk us through kind of the genesis of why you
• 41:43
  guys wanted to do it
• 41:44
  SPEAKER_02: and what's your goal of of um of doing the the weekly pod
• 41:50
  SPEAKER_03: So this is actually Michael's idea, for what it's worth.
• 41:52
  SPEAKER_03: So he was the genesis of it all.
• 41:54
  SPEAKER_03: So I think he's the person to ask.
• 41:58
  SPEAKER_03: But I think the idea is.
• 41:59
  SPEAKER_03: Yeah, go ahead.
• 42:01
  SPEAKER_00: Well,
• 42:01
  SPEAKER_00: I mean,
• 42:01
  there's a very selfish reason because I would love to talk to
• 42:03
  all these researchers
• 42:04
  SPEAKER_00: and ask them questions that I don't understand.
• 42:07
  SPEAKER_00: I still don't understand.
• 42:09
  SPEAKER_00: The more I get into the podcast, the more I realize I don't understand.
• 42:12
  SPEAKER_00: But I think it's a good thing, right?
• 42:13
  That's the only way to learn,
• 42:14
  like get really uncomfortable,
• 42:15
  but then just ask them.
• 42:17
  SPEAKER_00: And I think it's just a great privilege for,
• 42:19
  SPEAKER_00: you know,
• 42:19
  SPEAKER_00: both of us to be able to talk to these researchers,
• 42:21
  SPEAKER_00: ask them all these questions.
• 42:23
  SPEAKER_00: And also that there's nothing like it, like, you know, I think, uh,
• 42:28
  SPEAKER_00: There's really nothing like it even now.
• 42:29
  SPEAKER_00: It's a very small audience because it's very technical.
• 42:36
  SPEAKER_00: But those people who care about this technicality, they will listen.
• 42:42
  SPEAKER_00: I think it's a decent couple.
• 42:44
  SPEAKER_00: I don't know, Chris, how you think about it.
• 42:45
  SPEAKER_00: The way I think about it is Chris obviously is a real expert.
• 42:48
  SPEAKER_00: I'm like the guy who is trying to,
• 42:50
  SPEAKER_00: who knows enough to ask,
• 42:51
  SPEAKER_00: but then I'm like,
• 42:52
  I feel that I can be that bridge to maybe bring it down to
• 42:55
  earth a little bit,
• 42:56
  SPEAKER_00: you know,
• 42:57
  SPEAKER_00: and talk like a,
• 43:00
  SPEAKER_00: because I'm really not a researcher.
• 43:03
  So I think that's a decent combo,
• 43:05
  but it's still quite technical to be honest.
• 43:07
  SPEAKER_00: And we want to keep it that way.
• 43:09
  SPEAKER_00: Because if you want to order like,
• 43:11
  SPEAKER_00: you know,
• 43:11
  SPEAKER_00: the big industry application,
• 43:13
  there's so many like VC podcast that I think that the market
• 43:16
  has plenty of that.
• 43:17
  SPEAKER_00: But for something for this particular niche,
• 43:18
  SPEAKER_00: I think there's literally no podcast like that,
• 43:20
  SPEAKER_00: like long conversation with the technical expert doing very,
• 43:23
  SPEAKER_00: very specific research and just get them
• 43:26
  SPEAKER_00: you know, to talk about their work.
• 43:27
  SPEAKER_00: And usually they're very willing to share and tell you very frankly,
• 43:31
  SPEAKER_00: what is the capabilities and more importantly,
• 43:33
  SPEAKER_00: what are the limitations?
• 43:35
  And sometimes you can't really just get that by reading the
• 43:37
  paper is you get a lot
• 43:39
  SPEAKER_00: more to try them talking.
• 43:40
  SPEAKER_00: Yeah.
• 43:41
  SPEAKER_02: Yeah, absolutely.
• 43:43
  SPEAKER_02: Is there an area of research
• 43:47
  SPEAKER_02: This is Chris for you.
• 43:48
  Is there an area of research that the more folks that you're
• 43:51
  speaking to at
• 43:52
  RoboPapers that's getting you really excited about maybe some
• 43:55
  kind of breakthroughs
• 43:56
  coming through around robotic training or anything else related
• 43:59
  to the field?
• 44:02
  I think the thing that has been most exciting lately has been
• 44:05
  reinforcement learning.
• 44:06
  SPEAKER_03: There's a lot of interesting stuff going on.
• 44:09
  A lot of people are looking into how to do reasoning models for
• 44:11
  robotics and how to
• 44:13
  SPEAKER_03: do long horizon reasoning for robots.
• 44:15
  SPEAKER_03: Like I said, that's stuff that I've always found very fascinating.
• 44:20
  SPEAKER_03: I don't know.
• 44:21
  SPEAKER_03: There's so much exciting stuff happening, though.
• 44:22
  I think there's lots of separate little things which are really
• 44:25
  cool going on.
• 44:27
  SPEAKER_00: Chris,
• 44:27
  I kind of feel that maybe you have changed your mind a little
• 44:33
  bit about like
• 44:34
  SPEAKER_00: eco-centric video as we do more podcasts around research around that.
• 44:39
  SPEAKER_00: I don't know why Chris has his doubts.
• 44:44
  SPEAKER_03: Yeah, definitely.
• 44:46
  SPEAKER_03: I definitely have my doubts.
• 44:47
  SPEAKER_03: I still have plenty of doubts, but that's, yeah.
• 44:51
  SPEAKER_02: I'm coming around about it.
• 44:52
  SPEAKER_02: I'm definitely coming around about it.
• 44:54
  SPEAKER_02: Why do you say that?
• 44:55
  SPEAKER_02: First of all, you say ego or ecocentric?
• 44:58
  SPEAKER_00: Ego.
• 44:58
  SPEAKER_00: Ego.
• 44:59
  SPEAKER_00: Ego-centric videos.
• 45:00
  SPEAKER_02: Oh, ego.
• 45:01
  SPEAKER_02: Ego, like first person.
• 45:02
  SPEAKER_02: So tell me why you said that, because that's interesting to me.
• 45:05
  SPEAKER_02: What's changed?
• 45:06
  SPEAKER_00: Well, because I know like Chris is pretty bearish about this stuff.
• 45:10
  SPEAKER_00: But I mean,
• 45:10
  SPEAKER_00: we still try to keep an open mind,
• 45:13
  SPEAKER_00: you know,
• 45:13
  SPEAKER_00: so we've talked to a bunch of researchers that have papers on it.
• 45:16
  SPEAKER_00: And I kind of feel that Chris is, I mean, he has his doubts, you know.
• 45:22
  SPEAKER_00: I think that's the thing with experts, right?
• 45:25
  Because of his years of experience,
• 45:26
  he must have biases,
• 45:27
  right,
• 45:28
  naturally.
• 45:29
  Whereas I'm like a blank piece of paper,
• 45:31
  so I was like,
• 45:32
  oh,
• 45:32
  okay,
• 45:32
  I have no idea.
• 45:34
  SPEAKER_00: So as an observer, I'm just looking.
• 45:37
  SPEAKER_00: And so even for Chris,
• 45:38
  SPEAKER_00: I can see,
• 45:39
  SPEAKER_00: okay,
• 45:40
  SPEAKER_00: I mean,
• 45:40
  SPEAKER_00: I should speak on behalf of you,
• 45:41
  SPEAKER_00: Chris,
• 45:41
  SPEAKER_00: but I think it's,
• 45:44
  SPEAKER_00: yeah,
• 45:44
  SPEAKER_00: it forced us to open up,
• 45:46
  SPEAKER_00: yeah,
• 45:46
  SPEAKER_00: a little bit,
• 45:47
  SPEAKER_00: maybe previous biases.
• 45:48
  SPEAKER_00: Yeah.
• 45:49
  SPEAKER_03: Definitely.
• 45:50
  SPEAKER_03: I think it's cool to see people,
• 45:51
  SPEAKER_03: like,
• 45:54
  SPEAKER_03: there are so many different opinions that people have,
• 45:56
  SPEAKER_03: lots of cool research.
• 45:58
  SPEAKER_03: Yeah, I feel like I learn a ton from this.
• 46:00
  SPEAKER_03: I definitely feel like I'm,
• 46:01
  SPEAKER_03: I mean,
• 46:02
  SPEAKER_03: the fact that I've been doing this for so long makes me,
• 46:04
  SPEAKER_03: like,
• 46:04
  SPEAKER_03: by default,
• 46:05
  SPEAKER_03: bearish about most things.
• 46:07
  SPEAKER_03: Like, once you end up going into some stuff, like, I mean, mind changes.
• 46:12
  SPEAKER_03: Yeah.
• 46:12
  SPEAKER_03: It's changed a lot, honestly, over the last year.
• 46:16
  SPEAKER_02: So talk to me about the egocentric data for you,
• 46:21
  SPEAKER_02: either historically or why you're kind of more bearish on it.
• 46:24
  SPEAKER_02: Just kind of explain why from a research standpoint.
• 46:28
  SPEAKER_03: Historically,
• 46:28
  SPEAKER_03: I actually talked about it earlier,
• 46:30
  SPEAKER_03: is that it requires more engineering than it looks to make it work well.
• 46:35
  SPEAKER_03: I think that's historically been the case,
• 46:38
  SPEAKER_03: but these days,
• 46:39
  SPEAKER_03: like Michael was saying,
• 46:40
  SPEAKER_03: there's some stuff,
• 46:41
  SPEAKER_03: the things that seem to change it are that you can now,
• 46:45
  SPEAKER_03: with
• 46:49
  SPEAKER_03: modern model architectures and training.
• 46:51
  SPEAKER_03: It seems to be that you can do a little bit more co-training,
• 46:54
  SPEAKER_03: do a little bit less hand engineering to pull the right frames out.
• 46:59
  SPEAKER_03: With a lot of this, yeah.
• 47:02
  SPEAKER_03: The problem with most robotic stuff,
• 47:05
  SPEAKER_03: like you were mentioning earlier about the business side,
• 47:08
  SPEAKER_03: and whether or not it was ready,
• 47:10
  the problem with most robotic stuff is that it generally takes
• 47:13
  a couple PhDs to
• 47:14
  SPEAKER_03: make it work.
• 47:16
  So the more engineering you have to do,
• 47:18
  the less I buy it as something that will...
• 47:22
  SPEAKER_03: scale, I guess.
• 47:24
  SPEAKER_03: And that's been the source of my skepticism a lot of the time.
• 47:32
  SPEAKER_03: It seems like wishful thinking that this will work.
• 47:34
  But then you start to see,
• 47:36
  oh,
• 47:36
  yeah,
• 47:36
  this is actually kind of working for people.
• 47:38
  SPEAKER_03: All right, maybe my mind can be changed by that.
• 47:42
  SPEAKER_03: Yeah.
• 47:44
  SPEAKER_02: Cool.
• 47:45
  SPEAKER_02: Yeah.
• 47:47
  SPEAKER_02: Michael, I have to ask you.
• 47:51
  You mentioned incentive structure and to collect data and
• 47:57
  crypto is a way to use
• 47:59
  SPEAKER_02: incentive structures.
• 48:01
  SPEAKER_02: So that's super interesting to me for a variety of reasons.
• 48:04
  SPEAKER_02: But break down a little bit more about what you're doing there.
• 48:09
  SPEAKER_00: Yeah,
• 48:09
  SPEAKER_00: so to be honest,
• 48:12
  SPEAKER_00: I was a crypto skeptic three,
• 48:14
  SPEAKER_00: four years ago until I discovered this project called Helium.
• 48:19
  These days,
• 48:20
  they call these decentralized physical infrastructure networks.
• 48:24
  SPEAKER_00: So,
• 48:24
  SPEAKER_00: I mean,
• 48:25
  SPEAKER_00: the way maybe I explain it is,
• 48:29
  SPEAKER_00: I mean,
• 48:29
  SPEAKER_00: if you look at,
• 48:29
  SPEAKER_00: let's say,
• 48:29
  SPEAKER_00: for example,
• 48:30
  SPEAKER_00: everyone knows Bitcoin.
• 48:31
  SPEAKER_00: Bitcoin is a project that has never raised any money.
• 48:35
  And yet it has amassed a huge amount of compute and every year
• 48:38
  it's spent,
• 48:40
  I read somewhere like 0.5% of global electricity bill is used
• 48:43
  to mine Bitcoin.
• 48:44
  SPEAKER_00: So huge amount of resources gathered without raising money as a project.
• 48:49
  SPEAKER_00: So there is something very unique about crypto that is very different
• 48:55
  as a way to do capital formation or basically gather resources
• 48:59
  that is different
• 49:00
  from the limited liability company kind of what we're used to
• 49:06
  for a few hundred
• 49:06
  SPEAKER_00: years,
• 49:06
  SPEAKER_00: right?
• 49:09
  SPEAKER_00: And yeah,
• 49:10
  so I do think it maps quite well for some of these things we
• 49:14
  need to advance and
• 49:15
  SPEAKER_00: body AI.
• 49:17
  A lot of this data,
• 49:18
  I think ultimately should be collected what they call in the
• 49:21
  wild.
• 49:22
  SPEAKER_00: I think in theory,
• 49:22
  you can spin up some warehouses to collect teleoperation data
• 49:26
  full and close,
• 49:26
  SPEAKER_00: but
• 49:28
  I tend to think that those models that are trained just using
• 49:32
  like robots in the
• 49:33
  SPEAKER_00: warehouse will overfit to those control environment.
• 49:37
  SPEAKER_00: Ultimately, these AR models are just compression of the underlying data.
• 49:41
  SPEAKER_00: So actually,
• 49:44
  SPEAKER_00: I have a sense that for research to,
• 49:47
  SPEAKER_00: I mean,
• 49:47
  SPEAKER_00: if you want to build very,
• 49:48
  SPEAKER_00: very generalizable model,
• 49:50
  SPEAKER_00: you want the underlying data to be collected in the world out there.
• 49:55
  SPEAKER_00: And so it seems to me, it's something very hard to do centralized.
• 49:58
  SPEAKER_00: And crypto is a great way to just, it's like a bit permissionless, right?
• 50:02
  SPEAKER_00: You get people to just show up,
• 50:03
  SPEAKER_00: hey,
• 50:04
  SPEAKER_00: help me collect this data in your home,
• 50:06
  SPEAKER_00: and then we give you a token in exchange,
• 50:07
  SPEAKER_00: right?
• 50:08
  It's almost like,
• 50:09
  kind of like what,
• 50:10
  I mean,
• 50:10
  fundamental Bitcoin is doing,
• 50:11
  right?
• 50:11
  SPEAKER_00: It's like, you help me do this, I give you Bitcoin, something like that.
• 50:16
  SPEAKER_00: So,
• 50:17
  SPEAKER_00: and I feel that if structured the right way,
• 50:19
  SPEAKER_00: you can get a lot of resources at scale globally.
• 50:22
  Whether,
• 50:24
  when I say resources,
• 50:25
  it could be the teleoperator,
• 50:27
  the man hour,
• 50:28
  right?
• 50:28
  SPEAKER_00: Someone needs to control the robot,
• 50:30
  SPEAKER_00: someone needs to deploy the robots,
• 50:32
  SPEAKER_00: someone needs to pay for the robots,
• 50:35
  SPEAKER_00: right?
• 50:36
  SPEAKER_00: And so, and even real estate, right?
• 50:38
  SPEAKER_00: I mean, these robots need to be in somewhere, right?
• 50:41
  SPEAKER_00: So paying for the real estate as well.
• 50:43
  SPEAKER_00: So all these, I feel that can be incentivized with crypto if done well.
• 50:47
  SPEAKER_00: So anyway,
• 50:48
  SPEAKER_00: I don't want to shoot my project too much,
• 50:50
  SPEAKER_00: but effectively,
• 50:51
  SPEAKER_00: that's my belief for the last few years.
• 50:53
  SPEAKER_00: So really want to really get, yeah, take a good crack at using it.
• 50:58
  SPEAKER_00: Yeah, I do believe it.
• 51:00
  SPEAKER_00: I mean, you know, if it's done well, I think it can be,
• 51:03
  SPEAKER_00: It definitely won't be the scale of all these large AI labs,
• 51:08
  but I think it still can be maybe a few billion of US dollar
• 51:13
  kind of resources that
• 51:14
  SPEAKER_00: can be gathered this way.
• 51:15
  SPEAKER_00: I think crypto is way more mature now.
• 51:18
  SPEAKER_00: There's a lot of capital within the crypto market.
• 51:21
  And I think if we can activate those capital and then pull it
• 51:26
  and then turn it
• 51:26
  SPEAKER_00: around and give to people like Crystal,
• 51:27
  SPEAKER_00: then again,
• 51:29
  SPEAKER_00: accelerate their work,
• 51:30
  SPEAKER_00: I think that would be a great thing to try.
• 51:33
  SPEAKER_02: Cool.
• 51:33
  SPEAKER_02: All right, you guys ready for a little rapid fire for a rip-up?
• 51:38
  SPEAKER_00: Yeah, let's do it.
• 51:39
  SPEAKER_02: Sure.
• 51:39
  SPEAKER_02: All right.
• 51:43
  SPEAKER_02: I'll start with you, Chris, and then Mike, I'll go over to you.
• 51:47
  SPEAKER_02: What's one myth about robotics you wish you would die?
• 51:52
  SPEAKER_03: I don't know.
• 51:52
  SPEAKER_03: I don't have a good answer for that one.
• 51:55
  SPEAKER_03: What are myths about robotics right now?
• 52:00
  SPEAKER_03: I think a lot of people overestimate what things can do from videos.
• 52:05
  SPEAKER_03: Let's say that.
• 52:06
  I think that generally the rule of thumb when you see a video
• 52:10
  of a robot doing
• 52:10
  something is that you should always assume that it can do
• 52:12
  literally what it can see
• 52:13
  SPEAKER_03: and nothing else.
• 52:14
  And I think that a lot of people tend to generalize a little
• 52:18
  bit more than that.
• 52:19
  SPEAKER_03: There.
• 52:20
  SPEAKER_03: That would be my
• 52:23
  SPEAKER_00: Yeah,
• 52:23
  SPEAKER_00: for me,
• 52:24
  SPEAKER_00: I think it's even like,
• 52:27
  I wonder whether we're in a local minimum where the whole world
• 52:31
  is just trying to
• 52:31
  gather a lot of data and then throw in a lot of compute to
• 52:34
  solve and for the AI.
• 52:37
  Chris and I talk about it often,
• 52:38
  like I call it the mosquito intelligence.
• 52:41
  Like,
• 52:41
  you know,
• 52:41
  I can barely like catch,
• 52:43
  like,
• 52:43
  you know,
• 52:43
  kill a damn mosquito.
• 52:44
  The thing is so tiny,
• 52:46
  probably running at 0.1 watt,
• 52:48
  you know,
• 52:48
  must be have...
• 52:49
  Must have a very,
• 52:50
  very tiny,
• 52:50
  like whatever embodied AI model that's in this brain.
• 52:55
  SPEAKER_00: But it's got excellent physical intelligence.
• 52:57
  SPEAKER_00: So there's something missing there that nature has figured out that we,
• 53:01
  SPEAKER_00: we,
• 53:01
  SPEAKER_00: I feel that we're just maybe using,
• 53:04
  SPEAKER_00: don't know how many percentage of global GDP to,
• 53:06
  SPEAKER_00: to gather this data and,
• 53:08
  SPEAKER_00: and compute to solve something.
• 53:09
  SPEAKER_00: Yeah.
• 53:10
  SPEAKER_00: I don't know.
• 53:12
  SPEAKER_00: So I don't know whether it's Amoeba,
• 53:16
  SPEAKER_00: but you know,
• 53:16
  SPEAKER_00: I,
• 53:17
  SPEAKER_00: I just thought that there's something there.
• 53:19
  SPEAKER_02: Yeah.
• 53:21
  SPEAKER_02: What's,
• 53:21
  SPEAKER_02: uh,
• 53:21
  what's the most overhyped narrative and you can choose even AI
• 53:26
  or robotics today,
• 53:27
  SPEAKER_02: Chris.
• 53:32
  SPEAKER_02: You're into stuff every day.
• 53:33
  SPEAKER_02: You gotta have a guess on this one.
• 53:34
  SPEAKER_02: Yeah, yeah, yeah.
• 53:35
  SPEAKER_02: I don't know.
• 53:35
  SPEAKER_03: I don't know.
• 53:36
  SPEAKER_03: I don't know.
• 53:41
  SPEAKER_03: I don't tend to think.
• 53:41
  SPEAKER_03: I mean, I'm just very optimistic right now.
• 53:43
  SPEAKER_03: I don't know.
• 53:44
  SPEAKER_03: I don't think there's a lot that's super overhyped.
• 53:46
  SPEAKER_03: I still think that there are...
• 53:50
  You could argue that individual companies are overhyped or
• 53:53
  something like that,
• 53:54
  SPEAKER_03: which I don't necessarily know if I want to go start something.
• 53:58
  But I think that generally...
• 54:01
  No,
• 54:02
  I think that it's hard for me to think of a direction.
• 54:04
  SPEAKER_03: I think all of it's worth investigating at least.
• 54:09
  SPEAKER_03: I don't know.
• 54:09
  SPEAKER_03: Michael, do you have a better answer than me?
• 54:13
  Yeah,
• 54:14
  maybe the overhyped is maybe some of the timelines that's been
• 54:17
  thrown around.
• 54:18
  SPEAKER_03: Yeah.
• 54:19
  But I suspect it's over hype in the near term,
• 54:23
  but under hype in the long term.
• 54:25
  SPEAKER_00: You know, the typical.
• 54:26
  SPEAKER_03: I think that's fair.
• 54:27
  Like,
• 54:27
  I think everything,
• 54:28
  I think all of these are equally good directions.
• 54:31
  I think that people are,
• 54:32
  things will generally take longer than you expect.
• 54:34
  SPEAKER_03: Like, that's always the rule.
• 54:36
  SPEAKER_00: Yeah.
• 54:36
  SPEAKER_00: Especially robotics.
• 54:37
  SPEAKER_00: Yeah.
• 54:38
  SPEAKER_00: Especially in robotics.
• 54:39
  SPEAKER_00: Physics is brutal, man.
• 54:41
  SPEAKER_00: No mercy.
• 54:41
  SPEAKER_00: Right.
• 54:43
  SPEAKER_02: Yeah.
• 54:43
  SPEAKER_02: Yeah.
• 54:45
  SPEAKER_02: Chris, if you weren't building robots, what would you be doing?
• 54:51
  SPEAKER_03: I don't know.
• 54:52
  SPEAKER_03: Right now.
• 54:53
  SPEAKER_03: Like, well, yeah.
• 54:56
  So part of the reason that I started doing this,
• 55:00
  like,
• 55:00
  I do a lot of,
• 55:01
  like,
• 55:01
  writing my blog.
• 55:02
  SPEAKER_03: I like writing about stuff.
• 55:03
  So that's,
• 55:04
  like,
• 55:04
  I have a blog in addition to all this that I keep up most
• 55:08
  weeks.
• 55:09
  SPEAKER_03: Not all.
• 55:09
  SPEAKER_03: Not all.
• 55:10
  SPEAKER_03: Yeah.
• 55:10
  SPEAKER_03: So that's part of the genesis of a lot of this, actually.
• 55:14
  SPEAKER_03: How about you, Michael?
• 55:18
  SPEAKER_00: Oh, gosh.
• 55:18
  SPEAKER_00: No robotics.
• 55:19
  SPEAKER_00: It would be very boring, isn't it?
• 55:21
  SPEAKER_00: I don't know.
• 55:22
  SPEAKER_00: I do some music on the side.
• 55:25
  SPEAKER_00: I street sing sometimes.
• 55:27
  SPEAKER_00: Maybe I street sing more, but I think robotics is more fun.
• 55:30
  SPEAKER_00: I do robotics, I guess.
• 55:36
  SPEAKER_02: Who else should I talk to going direct about robotics invited AI?
• 55:39
  SPEAKER_02: Any name come to mind?
• 55:41
  Yeah,
• 55:41
  you should maybe talk to some of these founders from the
• 55:45
  Chinese manufacturer.
• 55:46
  SPEAKER_00: I think they will offer a very interesting perspective.
• 55:49
  Yeah.
• 55:51
  SPEAKER_00: Yeah.
• 55:52
  SPEAKER_00: Most of them only speak Chinese though.
• 55:53
  SPEAKER_00: So that's the problem.
• 55:55
  SPEAKER_00: If you can find someone.
• 55:56
  SPEAKER_00: Yeah.
• 55:56
  SPEAKER_00: I think they would, yeah.
• 55:59
  SPEAKER_00: They will offer a very different perspective from,
• 56:01
  SPEAKER_00: yeah,
• 56:02
  SPEAKER_00: what we typically read about in the West,
• 56:05
  SPEAKER_00: I think.
• 56:08
  SPEAKER_02: Chris?
• 56:11
  SPEAKER_03: You already talked to Bernt and to Physical Intelligence,
• 56:14
  SPEAKER_03: who are two of the top ones that I would have recommended,
• 56:16
  SPEAKER_03: right?
• 56:18
  SPEAKER_03: I think they're doing really cool stuff.
• 56:21
  SPEAKER_03: Let's see.
• 56:26
  SPEAKER_01: Wow, you're getting on the top.
• 56:27
  SPEAKER_01: I know, man.
• 56:28
  SPEAKER_03: I'm a huge fan of what... I know, I'm really bad at that.
• 56:32
  SPEAKER_03: It's late at night for me.
• 56:33
  SPEAKER_03: No, it's not that late.
• 56:34
  SPEAKER_03: It's not even that late.
• 56:35
  SPEAKER_03: I think...
• 56:39
  SPEAKER_03: One that I've liked lately has been Dyna.
• 56:42
  SPEAKER_03: Dyna Robotics startup, they do just making the robots work all the time.
• 56:46
  SPEAKER_03: There's a couple others that are doing home robots,
• 56:49
  SPEAKER_03: home robotics,
• 56:50
  SPEAKER_03: like Weave,
• 56:51
  SPEAKER_03: that I think are really cool.
• 56:53
  I think that they just have a different perspective on this,
• 56:57
  not going with the humanoid.
• 56:59
  Whether that's right or not,
• 57:00
  it's interesting to see the difference in design choices.
• 57:03
  SPEAKER_03: There's a lot of like,
• 57:05
  SPEAKER_03: I love the open source stuff coming out of Hug Me Face,
• 57:08
  SPEAKER_03: like the community they've built up.
• 57:10
  SPEAKER_03: I don't know, like it's hard to come up with a single answer.
• 57:12
  SPEAKER_03: I think there's a lot of cool stuff happening.
• 57:15
  SPEAKER_02: Yeah.
• 57:16
  SPEAKER_02: Michael, I might take you up on that offer.
• 57:18
  I think it would be a very interesting conversation from one of
• 57:21
  the Chinese
• 57:21
  SPEAKER_02: manufacturers,
• 57:23
  SPEAKER_02: like totally.
• 57:23
  SPEAKER_00: Yeah.
• 57:25
  SPEAKER_00: How can folks reach you, Chris?
• 57:33
  SPEAKER_03: Twitter usually X, so it's less sliced, yeah.
• 57:37
  SPEAKER_00: Yeah.
• 57:38
  SPEAKER_00: Same.
• 57:38
  SPEAKER_00: Same.
• 57:38
  SPEAKER_00: And Michael.
• 57:39
  SPEAKER_00: Yeah.
• 57:40
  SPEAKER_00: Same.
• 57:41
  SPEAKER_00: Same.
• 57:42
  SPEAKER_00: Same.
• 57:43
  SPEAKER_00: Yeah.
• 57:44
  SPEAKER_00: Just follow us on global papers.
• 57:45
  SPEAKER_00: I actually, that's a, yeah, there you go.
• 57:47
  SPEAKER_00: That's right.
• 57:47
  SPEAKER_00: Follow me.
• 57:48
  SPEAKER_00: The shout out.
• 57:50
  SPEAKER_03: We tagged in every post anyway.
• 57:55
  SPEAKER_02: We'll put up, we'll put obviously link links in the show notes.
• 57:57
  SPEAKER_02: Well, listen guys, I appreciate it.
• 57:59
  SPEAKER_02: Chris, I know that is getting a little late for you.
• 58:01
  SPEAKER_02: So like I said, I, thank you.
• 58:05
  SPEAKER_02: I,
• 58:06
  SPEAKER_02: I really dig what your folks are doing on the RoboPaper side.
• 58:11
  And I think I wanted to give the audience your perspective on
• 58:16
  it because you're
• 58:16
  SPEAKER_02: speaking with a lot of researchers.
• 58:18
  SPEAKER_02: And yeah, and so we'll include that all in the show notes.
• 58:21
  SPEAKER_02: And yeah, I look forward to keeping in touch.
• 58:23
  SPEAKER_02: And yeah, we'll go from there.
• 58:26
  SPEAKER_02: Okay.
• 58:27
  SPEAKER_02: Awesome.
• 58:27
  SPEAKER_02: Thanks.
• 58:28
  SPEAKER_02: Thanks for having us.
• 58:28
  SPEAKER_02: A lot of fun.
• 58:29
  SPEAKER_00: Bye.
• 58:29
  SPEAKER_00: Have a good one.
• 58:30
  SPEAKER_00: Cheers.
• 58:30
  SPEAKER_00: Bye.