Latest / What's Your Problem? / Using AI to Build Better Robots

Transcript

Using AI to Build Better Robots · What's Your Problem?

0:15
Speaker 1: Pushkin for a long time. Now, we've had a lot
0:21
Speaker 1: of technological innovation in virtual things in bits, you know,
0:26
Speaker 1: the Internet, digital images, large language models, etc. We have
0:30
Speaker 1: had noticeably less innovation in actual things and things made
0:36
Speaker 1: of atoms, things that would hurt if you dropped them
0:39
Speaker 1: on your foot. Now that seems to be changing. People
0:43
Speaker 1: are using innovations in bits, improvements in computing and communications
0:47
Speaker 1: and AI to drive innovation in actual things, everything from
0:51
Speaker 1: batteries to garbage cans to airplanes. Next up robots. I'm
1:03
Speaker 1: Jacob Boldstein and this is What's Your Problem, the show
1:05
Speaker 1: where I talk to people who are trying to make
1:08
Speaker 1: technological progress. My guest today is Peter Chi. He's the
1:12
Speaker 1: co founder and CEO of Covariant. Peter's work at Covariant
1:17
Speaker 1: was partly inspired by the work of Fafe Lead, who
1:20
Speaker 1: coincidentally is the AI researcher I interviewed just last week
1:24
Speaker 1: on the show. Peter's problem is this, how do you
1:28
Speaker 1: take the AI breakthroughs of the past decade or so
1:32
Speaker 1: and make them work in robots.
1:39
Speaker 2: So to really tell the story of robotics, like we
1:44
Speaker 2: have to tell the story of robotics even without AI
1:48
Speaker 2: like robotics for a very long time. It's a field
1:51
Speaker 2: that you would actually find in mechanical engineering departments of universities.
1:56
Speaker 2: Like it's largely a hardware problem. It's a control problem,
1:59
Speaker 2: like how can you design the moti well, how can
2:02
Speaker 2: you design the gearbox? Well?
2:03
Speaker 1: Yeah, right?
2:04
Speaker 2: Can you design like the control algorithm so that you
2:07
Speaker 2: can get the robot to a exact xyz location in
2:11
Speaker 2: the three D physical world like without oscillating around and
2:15
Speaker 2: you can.
2:15
Speaker 1: Making the thing move? How do you build the parts
2:17
Speaker 1: that make the thing move the way we.
2:19
Speaker 2: Wanted to move exactly? Like it's all about like telling
2:23
Speaker 2: this piece of machinery that we call robot to do
2:26
Speaker 2: the thing that's exactly what we tell it to do,
2:29
Speaker 2: which turned out to be like obviously a fairly difficult
2:32
Speaker 2: engineering problems, and that's why people have woke on it
2:34
Speaker 2: for many decades. But it has gotten really good.
2:38
Speaker 1: And so this is like this is like the classic
2:40
Speaker 1: kind of image you see from a car assembly line
2:44
Speaker 1: of like a robot arm you know whatever, welding a
2:48
Speaker 1: part onto the body of a cart again and again
2:52
Speaker 1: again all day long.
2:53
Speaker 2: Yeah, exactly right.
2:54
Speaker 1: So they're good at robots are clearly good at welding
2:57
Speaker 1: the same part onto the same car a million times?
3:00
Speaker 1: What are the limits of that approach? What were the
3:02
Speaker 1: problems people were bumping up against.
3:05
Speaker 2: Yeah, so the problem is that in order to use
3:08
Speaker 2: that kind of robotics, it has a really big limitations
3:13
Speaker 2: on your environment. Right. You basically need to be able
3:15
Speaker 2: to reduce your task to be solvable by repeated motion. Right,
3:22
Speaker 2: And so if you look at like, how like these
3:24
Speaker 2: kind of assembly lines that use classical robots, they always
3:28
Speaker 2: feed the material into exactly the same place. So no
3:32
Speaker 2: matter how how the way that they came in from
3:34
Speaker 2: their suppliers and whatnot, you always need to load them
3:37
Speaker 2: up in exactly the same way because like there's really
3:39
Speaker 2: no adaptivity at all that these robots have because they're
3:43
Speaker 2: just executing the same thing again, and it.
3:45
Speaker 1: Just it has to be they're very precise, but their
3:47
Speaker 1: whole environment has to be super homogeneous, the same every
3:51
Speaker 1: time exactly.
3:53
Speaker 2: Like, So, like that's the problem one, but that's very difficult, Like,
3:56
Speaker 2: not everything can be reduced that way. The second problem
3:59
Speaker 2: with it is even in the case that you can
4:02
Speaker 2: reduce the problem to that kind of pure mechanical repeated motion,
4:07
Speaker 2: it's still very expensive because you still need to program
4:10
Speaker 2: a robot to do that one specific task, and if
4:14
Speaker 2: you change your task slightly, you need to reprogram everything,
4:17
Speaker 2: typically from scratch, and that means like robot are not
4:21
Speaker 2: just extremely rigid, which limits like the range of capabilities
4:26
Speaker 2: that they can rich and do. And it's also very
4:30
Speaker 2: expensive even on the very fixed rigid capability will.
4:34
Speaker 1: Right, And so you need something that's the same every time,
4:37
Speaker 1: and you need to be doing a lot of it
4:39
Speaker 1: because otherwise the economy of scale just doesn't work out.
4:42
Speaker 1: It's too expensive to try and get the robot to
4:45
Speaker 1: do something else.
4:46
Speaker 2: Exactly.
4:47
Speaker 1: So I know that you as a as a student,
4:50
Speaker 1: as an undergrad and a grad student, if I have
4:53
Speaker 1: it right, you worked in the lab of this professor
4:56
Speaker 1: at Berkeley who for a long time had been trying
4:59
Speaker 1: to teach robots to fold towels. Yes, which is an
5:03
Speaker 1: amazing problem because it's one of those ones that seems
5:06
Speaker 1: so simple, right, it seems like way easier than riveting
5:09
Speaker 1: parts onto a car or whatever, but turned out to
5:12
Speaker 1: be in fact much harder for robots, right. And I
5:16
Speaker 1: feel like that's telling like why was that so hard
5:19
Speaker 1: for robots?
5:20
Speaker 2: And what do we learn from that when you do
5:22
Speaker 2: welding on a car body, like as we have discussed,
5:25
Speaker 2: like you can reduce the problem to just simple mechanical
5:28
Speaker 2: repeated motion. But because like these a piece of fabric
5:33
Speaker 2: is flexible, is deformable, like it can come in many
5:37
Speaker 2: many different kinds of shapes. It has many different possibilities, right, and.
5:40
Speaker 1: It's much more complex than a than a car body.
5:43
Speaker 1: Weirdly not into exactly, but when you think about it's like,
5:46
Speaker 1: oh it could have a little fast possibilities.
5:50
Speaker 2: It has a lot more possibility and how it can
5:52
Speaker 2: present itself to you. And because exactly of that, like
5:56
Speaker 2: recall like the first limitation of traditional robotics, which is
6:00
Speaker 2: it can only work with problems that can be reduced
6:03
Speaker 2: to repeated motion like and towel folding and folding apparel
6:09
Speaker 2: items is exactly one of those things that cannot be
6:12
Speaker 2: reduced because.
6:13
Speaker 1: It's just a little bit different every time it tells
6:16
Speaker 1: a little bit of a different shape. It might be
6:18
Speaker 1: sitting on the table and it'll folded over some.
6:20
Speaker 2: Weird exactly, like if it's folded onto itself, like how
6:24
Speaker 2: much is it folded onto yourself? Like how much wrinkle
6:26
Speaker 2: does it have? Like all of those make a big
6:28
Speaker 2: difference in terms of what the robots should do with it,
6:32
Speaker 2: and so that like it's a really good example of
6:35
Speaker 2: something that's traditional robotics cannot solve and you really need
6:40
Speaker 2: AI to solve it.
6:42
Speaker 1: And when your co founder started working on the problem,
6:45
Speaker 1: it was sort of before the kind of modern era
6:49
Speaker 1: of AI that we're living in now, right, And I
6:53
Speaker 1: read that there was this big moment in kind of
6:56
Speaker 1: the origin of the company was when this database essentially
7:03
Speaker 1: of labeled images called image net was released, Which was
7:08
Speaker 1: interesting to me because I just talk to Faye Lee
7:10
Speaker 1: for this.
7:10
Speaker 2: So we have is actually one of the advisor investors.
7:16
Speaker 1: Just the coincidence. For the record, she didn't put me
7:18
Speaker 1: in touch with you, but tell me about sort of
7:21
Speaker 1: why why image net was meaningful in the in the
7:26
Speaker 1: birth of your company.
7:28
Speaker 2: Yeah, there was actually a bigger lesson here than just robotics.
7:33
Speaker 2: The bigger lesson is that artificial intelligence is actually becoming
7:38
Speaker 2: simpler and simpler, Like when you look at the field
7:42
Speaker 2: of artificial intelligence fifteen twenty years ago, like it used
7:46
Speaker 2: to be many different sub fields. Like the people that
7:48
Speaker 2: work on robotics have a completely different tool sets than
7:51
Speaker 2: the people that work on computer vision, and the people
7:54
Speaker 2: that are working on computer vision had a different two
7:56
Speaker 2: sets from people that are working on natural language processing.
7:59
Speaker 1: Well, and they feel different, Right, Like, teaching computers to
8:02
Speaker 1: understand an image, you know, to sort of deal with
8:07
Speaker 1: an image of the world and understand what it means,
8:09
Speaker 1: feels quite different than teaching a computer to have a conversation.
8:14
Speaker 1: It doesn't. It's not obvious that you would use the
8:16
Speaker 1: same tools to do exactly.
8:17
Speaker 2: Those and that definitely was the consensus, Like why.
8:21
Speaker 1: Would it be the same?
8:22
Speaker 2: Yeah, what would it be the same? Like it feels
8:23
Speaker 2: very different, Like it feels like you need different kind
8:25
Speaker 2: of data, and I would just like say it. Essentially,
8:29
Speaker 2: the field of AI is becoming more and more unified,
8:32
Speaker 2: Like the methodology, the model that you would use is
8:35
Speaker 2: actually becoming similar and similar and sometimes it's even the
8:39
Speaker 2: same across these very different fields of robotics, computer vision, language.
8:43
Speaker 1: So it's basically, you build a neural network and then
8:46
Speaker 1: you just train it on a bunch of images or
8:49
Speaker 1: train it on a bunch of documents, and what you
8:52
Speaker 1: train it on is what determines sort of what it's
8:55
Speaker 1: good for.
8:55
Speaker 3: Is it like that that that basically is it like
8:59
Speaker 3: it's it's like it instead of each sub feel coming
9:05
Speaker 3: up with different and think about it as like hand
9:09
Speaker 3: programmed intelligence, right, like let's try to break down what's
9:13
Speaker 3: a sentence means, like break into different parts and when
9:16
Speaker 3: you approach a computer vision problem, or let's try to
9:19
Speaker 3: come up with different features and some features represent an edge,
9:22
Speaker 3: some features represent the background.
9:24
Speaker 2: Instead of like trying to manually program the kind of
9:28
Speaker 2: quite human intelligence into AI, you're basically taking a step
9:32
Speaker 2: back and say, I'm just going to create a very
9:34
Speaker 2: flexible learning mechanism, which is an artificial new net, and
9:39
Speaker 2: then we're just going to feed it a lot of data.
9:41
Speaker 2: And if you have different types of problems that you're solving,
9:45
Speaker 2: you're just feeding like this new net different types of data,
9:49
Speaker 2: but they're still like really the same kind of mechanism.
9:51
Speaker 2: Like then that is a drastic departure from how artificial
9:55
Speaker 2: intelligence used to be done.
9:57
Speaker 1: And so in that world, then the differentiation is just
10:00
Speaker 1: in the data set that you are feeding the totally model.
10:03
Speaker 2: Totally and in fact, like I mean, this is like
10:07
Speaker 2: jumping way forward in time. We were still talking about
10:10
Speaker 2: image net. But if you look at really the most
10:13
Speaker 2: popular technologies today, these large language models. When you use
10:17
Speaker 2: different companies large language models, it's really what you're saying,
10:22
Speaker 2: like as if I'm using GPT chat, gipt four versus
10:26
Speaker 2: using Google's bought backed by Germini versus Anthpics, Claude or
10:33
Speaker 2: coheres command model.
10:34
Speaker 1: You're just naming all the different big language large language models. Now.
10:37
Speaker 2: Yeah, and like when you think about these different big
10:39
Speaker 2: language models, you're just really referring to the different data
10:43
Speaker 2: sets that's see behind.
10:44
Speaker 1: That they were trained on. So this is interesting kind
10:47
Speaker 1: of abstract big picture talk. I want to kind of
10:50
Speaker 1: map this now onto the story of Covariate, the story
10:54
Speaker 1: of your company. Right, so we're going back in time. Now,
10:58
Speaker 1: tell me about the moment when you decided to start
11:01
Speaker 1: the company. What's going on?
11:02
Speaker 2: Yeah, so the moment that we decide to start a
11:04
Speaker 2: company is you'll refer to this image net mode like
11:09
Speaker 2: this like large data set that actually first time like
11:14
Speaker 2: really taught people that you can train a network to
11:18
Speaker 2: solve one specific task really work.
11:21
Speaker 1: So this is twenty twelve. There's when a neural net
11:28
Speaker 1: trains on image net and does a really good job,
11:32
Speaker 1: way better than anybody has ever done. Any Then any
11:35
Speaker 1: model has ever done of identifying objects.
11:38
Speaker 2: In exactly right, exactly. That was really significant because that
11:42
Speaker 2: means if you can collect a lot of data for
11:45
Speaker 2: a single task, and if you can get a group
11:48
Speaker 2: of PhDs to work on a model for that single task,
11:52
Speaker 2: you basically have AI. Like you can solve that task
11:55
Speaker 2: really well. I mean like you might need to iterate
11:58
Speaker 2: on your data, you might need to iterate on your algorithm,
12:01
Speaker 2: but ultimately you can solve that task really well.
12:03
Speaker 1: You're basically saying, if you can gather the data, you
12:07
Speaker 1: can gather a shitload of data of whatever kind, then
12:11
Speaker 1: you can get AI around.
12:14
Speaker 2: For that for that kind, right, And which is like
12:16
Speaker 2: why you saw artificial intelligence like really started working like
12:20
Speaker 2: after twenty twelve, Like like Google, Facebook, all of these
12:24
Speaker 2: companies like have a lot of AI based applications, but
12:27
Speaker 2: they are largely not democratized because like in order to
12:31
Speaker 2: get any single AI working, you still need a lot
12:33
Speaker 2: of data and you still need a team of PhDs
12:36
Speaker 2: to work on it. So it was a huge breakthrough
12:40
Speaker 2: in AI, but it was not sufficient for really widespread
12:43
Speaker 2: usage just because the barrier to create one AI is
12:46
Speaker 2: so high. Okay, and then comes the second inflection point
12:51
Speaker 2: in the history of AI, which is really the start
12:54
Speaker 2: of foundation models. So like I'm talking about the most
12:58
Speaker 2: initial version of GPT, right, Like I'm talking about like
13:02
Speaker 2: these large language models that are trained on multiple tasks
13:06
Speaker 2: so that they're incredibly generalizable, Like you can ask to
13:09
Speaker 2: do something new and it can do it really well.
13:12
Speaker 2: And also it performed better at single task than specialized model.
13:16
Speaker 1: And so just to be clear, like until a few
13:19
Speaker 1: years ago, people thought reasonably that if you want to
13:23
Speaker 1: build AI to whatever translate language right, you would work
13:28
Speaker 1: really hard on that. You would try and build an
13:31
Speaker 1: AI specifically designed to be really good at translating text
13:36
Speaker 1: from one language to another. But this really surprising result,
13:42
Speaker 1: this really surprising thing that emerged from just work people
13:45
Speaker 1: were doing, was, in fact, that's not the best way
13:48
Speaker 1: to get AI to translate language. It's just throw everything
13:52
Speaker 1: you can, all the words on the internet at an
13:55
Speaker 1: AI model and just say, figure out everything about language,
13:58
Speaker 1: figure out how to answer questions about history, and figure
14:02
Speaker 1: out how to translate, and figure out how to give
14:05
Speaker 1: me a recipe for you know, pasta. And it turns
14:08
Speaker 1: out that LAD technique gets you better results at each
14:12
Speaker 1: specific thing than trying to build specialized model exactly.
14:15
Speaker 2: And that is really the magic of foundation models. And
14:18
Speaker 2: that's the thing that we're not obvious to people outside
14:22
Speaker 2: of open Ai for a very long time. And because
14:26
Speaker 2: we came from open Ai, a lot of the founding
14:28
Speaker 2: team at Coverin came from open Ai. We saw that
14:30
Speaker 2: inside earlier, and that inside allow us to start Covariant
14:36
Speaker 2: to build foundation models for robotics way before other people
14:40
Speaker 2: even believed in the.
14:42
Speaker 1: Approach, even people in the field, even people in the field.
14:45
Speaker 1: So so, yeah, so you were when did you go
14:47
Speaker 1: to open Ai? You were you went to work at
14:49
Speaker 1: open Ai.
14:50
Speaker 2: We went to open Ai when it was about ten
14:54
Speaker 2: ish people like like sometime in twenty sixteen.
14:57
Speaker 1: Okay, And and when do you sort of personally have
15:03
Speaker 1: this realization and you're not the only one to have it,
15:06
Speaker 1: but when do you see that the power of foundation models?
15:12
Speaker 2: There are two things on it, Like, so the first
15:15
Speaker 2: thing is that early on at open Ai, we believed
15:20
Speaker 2: in the idea of scaling, like really scaling up the
15:24
Speaker 2: model and scaling up the data sets. And you actually
15:27
Speaker 2: see models getting like getting increasingly smarter as you actually
15:31
Speaker 2: scale them up. So one is that, and then the
15:34
Speaker 2: other one is I would say we we had conviction
15:39
Speaker 2: in foundation model for robotics probably earlier than foundation model
15:44
Speaker 2: for language. And this is like the one key thing
15:46
Speaker 2: is that if you think about building a large language
15:51
Speaker 2: model that tried to compress the whole internet of knowledge,
15:55
Speaker 2: you still need to compress many things that are not
15:58
Speaker 2: quite related to each other. Right, Like maybe you're browsing
16:01
Speaker 2: on Wikipedia and you have to recite the composition of
16:05
Speaker 2: materials of soil on the moon, and you also need
16:09
Speaker 2: to learn how to play chess. What there is there's
16:12
Speaker 2: really nothing in common with these two things. Yeah, these
16:15
Speaker 2: two parts of the knowledge, but you are asking one
16:17
Speaker 2: AI model to learn all of these. And the place
16:21
Speaker 2: that make a lot of sense to us is that
16:24
Speaker 2: there's only one physical world. Like even when you have
16:27
Speaker 2: many different robots that need to do different things in
16:31
Speaker 2: different factories, different warehouses, they are still interacting in the
16:34
Speaker 2: same physical world. And so building a foundation model for
16:38
Speaker 2: robotics like has this amazing property of grounding that like,
16:42
Speaker 2: no matter what kind of tasks that you're asking this
16:45
Speaker 2: foundation model to learn, well, it's just learning with the
16:48
Speaker 2: same sets of physics, Like the same surrounding.
16:50
Speaker 1: Is the literal ground with the models.
16:52
Speaker 2: The literal ground exactly, and.
16:54
Speaker 1: The models have to understand just how the physical world works,
16:57
Speaker 1: and that you drop a thing, it will fall and etc.
17:02
Speaker 2: And it's if it's something that is like deformable, you
17:05
Speaker 2: push it, it would move a little bit. If something
17:07
Speaker 2: is rigid like you would sly, if something is rollable,
17:10
Speaker 2: it would roll away.
17:11
Speaker 1: Like.
17:12
Speaker 2: These are the type of things that like, no matter
17:14
Speaker 2: where you are on Earth and what type of robots
17:17
Speaker 2: like body you're using, is the same, right, and so
17:19
Speaker 2: like if you can be one single foundation model that
17:22
Speaker 2: can learn from all of these different data, it would
17:25
Speaker 2: be incredibly powerful.
17:26
Speaker 1: So just to just to state it clearly, So you're
17:29
Speaker 1: at open AI, you're seeing the power of foundation models,
17:33
Speaker 1: you decide to leave and start the company that is
17:36
Speaker 1: now Covariant, Like what are you setting out to do
17:39
Speaker 1: when you when you start the company?
17:42
Speaker 2: Yeah, So when we started Covariant, we had this really
17:46
Speaker 2: strong conviction that like, there should be a future that
17:51
Speaker 2: has a lot of autonomous robots doing all the things
17:55
Speaker 2: that are repetitive, injury prone, dangerous, like and so that
18:00
Speaker 2: can really revolutionize the physical world, make it a lot
18:04
Speaker 2: more abundant and to really enable that future of autonomous robots,
18:09
Speaker 2: you need really smart AI, and like because of the
18:12
Speaker 2: inside that we just talk about, like we believe that
18:15
Speaker 2: AI had to be a foundation model. We believe like
18:17
Speaker 2: you should have single model that learn from all these
18:20
Speaker 2: two different robots together and become smarter together.
18:23
Speaker 1: So I mean, so the basic idea is, the dream
18:28
Speaker 1: is to build one AI foundation model for robots. Basically, yeah,
18:35
Speaker 1: in the same way that you can ask chat GBT
18:37
Speaker 1: anything in language and it can answer you in language
18:40
Speaker 1: about any different thing. You have a model where you
18:42
Speaker 1: just sort of make it be the brain of any
18:45
Speaker 1: robot and that robot can sort of see the world
18:48
Speaker 1: and move and pick things up and behave in the
18:51
Speaker 1: world exactly.
18:53
Speaker 2: And there's one key problem, Like the key problem is
18:56
Speaker 2: that unlike foundation model for language, where you can scrape
19:01
Speaker 2: the whole Internet of text as your pre training data,
19:05
Speaker 2: there's nothing that is equivalent in the case of robotic
19:09
Speaker 2: I mean, there are some images online, there are some
19:11
Speaker 2: YouTube videos online, but by and large, like they don't
19:14
Speaker 2: really give you the same type of data that are
19:20
Speaker 2: in the form of robots interacting with the world. And
19:22
Speaker 2: the big problem is because they're just not that many robots,
19:25
Speaker 2: they're just doing interesting things in the world. And so
19:28
Speaker 2: a big chunk of what we set out to build
19:31
Speaker 2: as a company is recognizing that we need to build
19:34
Speaker 2: fundation model for robotics. And in order to be a
19:37
Speaker 2: foundation model for robotics, you need to have large data sets.
19:41
Speaker 2: And in order to create large data sets, you have
19:43
Speaker 2: to have robots that are actually creating value for customers
19:47
Speaker 2: in production at scale, because if you're only collecting data
19:51
Speaker 2: in your own lab, there's only so much data. And
19:54
Speaker 2: so I would say the last six years of Covariance
19:59
Speaker 2: is largely focused on really building autonomous robot systems that
20:05
Speaker 2: work really well for customers, and they're doing interesting things
20:08
Speaker 2: to a level of autonomy and reliability that have not
20:12
Speaker 2: been hit before.
20:16
Speaker 4: In other words, Peter and his colleagues at Covariant built
20:19
Speaker 4: robot arms that businesses are paying to use out in
20:22
Speaker 4: the world. But to some extent, those robot arms are
20:26
Speaker 4: just a means to an end because they're not just
20:29
Speaker 4: doing warehouse work. They're collecting more and more.
20:32
Speaker 1: Data that Peter and his colleagues are feeding back into
20:35
Speaker 1: their AI model to try and make it get better
20:38
Speaker 1: and better. In a minute, what those robot arms are
20:43
Speaker 1: actually doing out in the world and what they're learning.
20:56
Speaker 1: So let's talk about what the robots you have built
21:02
Speaker 1: are doing out in the world. Besides generating the data
21:05
Speaker 1: that you will use to train the next generation of robots,
21:08
Speaker 1: what are they actually doing in the world right now today.
21:11
Speaker 2: Yeah, So when we started the company, we soveyed the
21:14
Speaker 2: landscape pretty carefully and then we selected warehouse and logics
21:19
Speaker 2: as the primary sector that we focus on today. When
21:22
Speaker 2: you are stopping online shops, like when you click a
21:26
Speaker 2: button and something shows up next day, there's tremendous amount
21:30
Speaker 2: of complexities behind that back end logistics of getting things
21:34
Speaker 2: to you. And typically it's estimated that each item is
21:38
Speaker 2: touched fifteen to twenty times between when you click a
21:41
Speaker 2: butttom to buy and when it shows up.
21:44
Speaker 1: In contrast, that fifteen to twenty touches of getting a
21:48
Speaker 1: thing from the warehouse to your door is your opportunity.
21:52
Speaker 2: Exactly right, And like combining with that, people don't want
21:56
Speaker 2: to drive in the middle of the night two hours
21:59
Speaker 2: into a suburb to work in a warehouse, Like it's
22:03
Speaker 2: just a kind of job that has extremely high turnover rate,
22:06
Speaker 2: Like not the kind of job that people stay there
22:08
Speaker 2: for a very long time, and so there's a tremendous
22:11
Speaker 2: amount of desire for more robotics and more automations in
22:15
Speaker 2: those environment to do picking up objects, sorting them into
22:20
Speaker 2: the right compartment of boxes, and then packing it nicely
22:24
Speaker 2: and then shipping it out to you as a customers.
22:28
Speaker 1: Tell me a little bit more specifically, I mean, what's
22:30
Speaker 1: one thing one of your robots is doing today in
22:35
Speaker 1: a warehouse somewhere on the earth.
22:37
Speaker 2: Yeah, so we would get like pretty detail and giky here,
22:41
Speaker 2: and so we'd actually tell you a little bit of
22:43
Speaker 2: like the needy, greitty details of like warehouse. Like, so
22:49
Speaker 2: let me describe what the robot is doing. Like the
22:50
Speaker 2: robot is doing, is I have a toad full of
22:55
Speaker 2: items that come up to the robots, and then I
22:57
Speaker 2: would need to grab one thing at a time, like we're.
23:00
Speaker 1: Is like a tote bag with a bunch of different stuff.
23:02
Speaker 2: And a bunch of different stuff in it, and like
23:05
Speaker 2: because like these stuffs are all layout in a chaotic
23:09
Speaker 2: way and like they're overlapping with each other. If you're
23:12
Speaker 2: not careful, you might drag out multiple items at the
23:14
Speaker 2: same time. And these items all have different shapes like
23:17
Speaker 2: they might be transparent, they might be reflective, they might
23:19
Speaker 2: be hard to see.
23:20
Speaker 1: This is hard, like to go back to our old
23:23
Speaker 1: Like riveting parts on a car is easy, Folding a
23:27
Speaker 1: towel is hard. This is hard because it's heterogeneous. Things
23:30
Speaker 1: look different, They come differently every time. This is hard
23:33
Speaker 1: for robots. Hard for a sort of classical robot to do.
23:37
Speaker 2: Impossible for a classical impossible.
23:39
Speaker 1: Not just hard, impossible. Can your robots do it?
23:44
Speaker 2: Yeah? They can do it? How extremely well?
23:47
Speaker 1: How did you solve How did you solve it?
23:49
Speaker 2: How does it work in the end of the day?
23:50
Speaker 2: Like it the way that it operates us very similar
23:53
Speaker 2: to how like humans vision system work, Like we have
23:57
Speaker 2: two eyes and then like by two eyes looking at
23:59
Speaker 2: something like we like we can figure out what's the
24:03
Speaker 2: depth of a certain items, like because our two eyes
24:06
Speaker 2: can triangulate a single point the three D world and
24:10
Speaker 2: it's the same kind of mechanisms and so like you
24:12
Speaker 2: can just use multiple regular cameras, just like the one
24:15
Speaker 2: that you have on your iPhone, and by having multiple
24:18
Speaker 2: ones of those like you give the new net the
24:22
Speaker 2: ability to triangulate what's happening.
24:24
Speaker 1: Just the way our two eyes allow us to see
24:27
Speaker 1: depth essentially exactly right, And are there other things like
24:31
Speaker 1: like weight, I mean the arm is going to be
24:33
Speaker 1: picking things up like weight or whether you know, presumably
24:36
Speaker 1: there's like could be a shirt in a plastic bag,
24:39
Speaker 1: could be a box, or something's are rigid, some things
24:41
Speaker 1: are deformable.
24:42
Speaker 2: Yeah, So what we have found is that if you
24:46
Speaker 2: just have a visual understanding of the world that is
24:50
Speaker 2: as robust as human, you go a really long way. Right,
24:54
Speaker 2: Like so when I when I pick up a cup, right, like,
24:58
Speaker 2: I'm not doing a lot of calculations on my how
25:02
Speaker 2: is my fingers faced? Like exactly, it gets translated to
25:05
Speaker 2: the cup and making sure it holds right.
25:07
Speaker 1: It's part of the miracle of being a person though, Right,
25:09
Speaker 1: it's like a really hard problem to pick.
25:11
Speaker 2: It's a very hard problem. But then like your brains
25:13
Speaker 2: subconsciously solve it for you, like your your system one
25:16
Speaker 2: thinking somewhat solved that.
25:18
Speaker 1: For thinking, you don't have to think about yeahm.
25:20
Speaker 2: Hm, exactly, And and you can like imagine like when
25:24
Speaker 2: you do this, like in fact, like even if my
25:26
Speaker 2: fingers are numb, I can still do this perfectly like
25:29
Speaker 2: just because like so it acquires this intuitive understanding of
25:35
Speaker 2: interaction with physical will so well that you can do it.
25:38
Speaker 1: So basically vision, vision gets you most of the way there.
25:41
Speaker 2: I would say, vision and then the ability to intuit
25:45
Speaker 2: physics from your visual input that you'll get.
25:48
Speaker 1: That told me that second. What is wild though, Like
25:51
Speaker 1: I mean into it, I mean into it in the
25:54
Speaker 1: in the context of it, I mean sort of make inferences.
25:57
Speaker 1: I mean intuit is yeah, okay, yeah.
26:00
Speaker 2: But like by by into it, I mean like it's
26:03
Speaker 2: not doing some kind of detail physical calculation.
26:07
Speaker 1: It's not doing math. It's not doing math.
26:09
Speaker 2: Yeah, it's doing a kind of high level pattern matching
26:11
Speaker 2: of well, like based on how these things looks, this
26:15
Speaker 2: is likely going to be a successful way to approach
26:18
Speaker 2: the item and interact with it.
26:21
Speaker 1: What what's next?
26:23
Speaker 2: So what is next immediately is very exciting? Right. We
26:28
Speaker 2: are now getting to a place that by we, I
26:31
Speaker 2: mean we as in AI community has gotten to a
26:36
Speaker 2: place that we have enough computation power and algorithmic and
26:43
Speaker 2: modeling understanding that can allow us to extract a lot
26:48
Speaker 2: out of data.
26:49
Speaker 1: Right, So from any given amount of data, you can get.
26:54
Speaker 2: More, you can get more out of it. Right.
26:57
Speaker 1: It's exciting for you because data is such a constraint
27:00
Speaker 1: on what you're trying to do exactly.
27:02
Speaker 2: And then like we're building up this large robotic data
27:05
Speaker 2: set by tapping into a lot of these events, is
27:11
Speaker 2: it gives us the ability to even get more out
27:13
Speaker 2: of the data sets that we're building and allow us
27:16
Speaker 2: to build smarter than better robotics foundation models that perform
27:21
Speaker 2: better at the current tasks that they're supposed to do
27:23
Speaker 2: and also power more robots.
27:26
Speaker 1: When people talk about concerns around AI, they often kind
27:29
Speaker 1: of have jokingly use the phrase killer robots, which is
27:32
Speaker 1: usually like a metaphor or something. But in your instance,
27:35
Speaker 1: because you are building robots, and because you are building
27:38
Speaker 1: by design a model that is supposed to be used
27:41
Speaker 1: for lots of different purposes, I can in fact very
27:44
Speaker 1: easily imagine killer robot applications of your work. Like that
27:47
Speaker 1: seems like a very plausible thing someone could do with it, Like,
27:51
Speaker 1: is that something you think about worry about?
27:54
Speaker 2: I would say very Fortunately, in the very near to
27:59
Speaker 2: medium term use cases, we are very safe because like
28:03
Speaker 2: all of these industrial robots are very much confined by
28:06
Speaker 2: the stations that they're designed in. Curse like industrial robots
28:11
Speaker 2: heavy machineries that are subject to regulations and are very
28:14
Speaker 2: carefully They are like careful design guidelines and compliance requirements
28:20
Speaker 2: for them. They are already by design safe.
28:23
Speaker 1: You're saying your model is built for robots basically for
28:28
Speaker 1: robot arms. I mean, is that essentially what the model
28:31
Speaker 1: you're building is really a foundation model for robot arms
28:34
Speaker 1: that are built to just be in one place, pick
28:36
Speaker 1: things up, put them down, that sort of thing. It's
28:39
Speaker 1: not a foundation model that you could have map onto
28:42
Speaker 1: a car or something, or even to a robot that
28:44
Speaker 1: walks around. It wouldn't work for that.
28:46
Speaker 2: That would not be the near term use cases. Like
28:49
Speaker 2: so like because the near term use cases are more
28:52
Speaker 2: in this safe by construction setting, it allows us to
28:57
Speaker 2: not more about that problem and in fact, like have
29:00
Speaker 2: basically no way to misuse the technology in the way
29:04
Speaker 2: we need to. But I do agree with you, like
29:06
Speaker 2: as we actually unleash this model to a set of
29:09
Speaker 2: use cases, like when these robots can actually interact with
29:12
Speaker 2: the world in a lot more freeform way those other cases,
29:15
Speaker 2: that the safety considerations become a lot more important and
29:20
Speaker 2: there's definitely a lot more work that need to be
29:22
Speaker 2: done for that to be reliable.
29:29
Speaker 1: We'll be back in a minute with the lightning round.
29:33
Speaker 1: M there is a lightning round that we're gonna do
29:44
Speaker 1: now for the end of the interview, what household chore
29:48
Speaker 1: do you wish that a robot could do?
29:52
Speaker 2: Cleaning up kitchen? Yeah, but I don't like the cleanup
29:57
Speaker 2: of it.
29:59
Speaker 1: That seems like a really hard one, like putting stuff away.
30:03
Speaker 1: Basically I'm wiping the counter maybe less hard, but like
30:06
Speaker 1: putting stuff away seems like a really hard job for
30:08
Speaker 1: a robot.
30:09
Speaker 2: It is. And these are the type of jobs that
30:12
Speaker 2: start to get to how we are limitation, Like, these
30:15
Speaker 2: are the type of jobs that start to get to
30:18
Speaker 2: like you probably do want a humanoid robot, like you
30:20
Speaker 2: probably do want something that kind of moves and conform
30:26
Speaker 2: to the human standard of interacting with the world.
30:29
Speaker 1: Because the kitchen is optimized, right, or you would have
30:31
Speaker 1: to redesign the kitchen for a robot, and but then
30:34
Speaker 1: that would suck because then you couldn't get your plates
30:37
Speaker 1: because they'd be in some random spot or whatever. Okay,
30:40
Speaker 1: so you left open ai in twenty seventeen. In the
30:44
Speaker 1: past year, open ai became like this household word, GPT
30:48
Speaker 1: became a household word. Were you surprised as a you know,
30:53
Speaker 1: old school former open AI guy, were you surprised by
30:57
Speaker 1: how how wild the world went for GPT or by
31:00
Speaker 1: how good it was how soon?
31:03
Speaker 2: I was definitely surprised by the speed of it. I
31:07
Speaker 2: was surprised by the speed of bull of the technologists
31:10
Speaker 2: development and the speed of adoption. But I was not
31:14
Speaker 2: surprised by the fact that it could be dispig and
31:18
Speaker 2: it could be bigger.
31:19
Speaker 1: You know, when you were talking about sort of warehouse
31:22
Speaker 1: and getting data from you know, picking and packing basically,
31:27
Speaker 1: I thought, of course, as anyone would, of Amazon, and
31:31
Speaker 1: I've read that they're working on some kind of robot
31:34
Speaker 1: arm I feel like they would just have so much
31:37
Speaker 1: data that they could gather if they wanted, just because
31:40
Speaker 1: they're so big, they have so many warehouse I mean
31:41
Speaker 1: the same way that say Google just gets tons of
31:44
Speaker 1: data every day with every Google search and the way
31:46
Speaker 1: people like I feel like that would be very hard
31:48
Speaker 1: to compete with, But.
31:50
Speaker 2: We also don't need to compete with them. They are
31:52
Speaker 2: also a very large role that Amazon is not serving,
31:56
Speaker 2: and there are a lot of customers that don't have
31:59
Speaker 2: the same degree of engineering team data access as Amazon, and.
32:05
Speaker 1: You could be the shop by of you could be
32:07
Speaker 1: the Shopify of warehouse robots.
32:09
Speaker 2: There are all of these people that still need help,
32:12
Speaker 2: and we very gladly help them.
32:14
Speaker 1: What was the first robot you personally built.
32:19
Speaker 2: I think it's probably one of the first pick and
32:21
Speaker 2: place robot e Covariant.
32:23
Speaker 1: You didn't build them when you were kid there.
32:25
Speaker 2: I didn't build them when I was kid.
32:27
Speaker 1: What made you get into robots?
32:31
Speaker 2: I would say I'm an AI person first and robot
32:36
Speaker 2: person set, and a big part of the interest in
32:41
Speaker 2: robotics is probably driven by my interests in AI. Like
32:44
Speaker 2: it just like we have just not make as much
32:49
Speaker 2: progress for AI in the physical world. That's AI in
32:52
Speaker 2: the digital world, and to a large degree, like I
32:56
Speaker 2: think we have to make progress there because like ultimately
32:59
Speaker 2: we live in the physical world. Like you're creating all
33:01
Speaker 2: these intelligence and amazing things in the digital world. That
33:04
Speaker 2: is all great, but where's AI in the physical world?
33:07
Speaker 2: Like this remarkably little progress there despite how much AI
33:12
Speaker 2: has moved forward. And so to some degree, like it's
33:14
Speaker 2: a it's driven by a conviction that like AI has
33:20
Speaker 2: to progress forward and AI will have a large impact
33:24
Speaker 2: in the physical world.
33:25
Speaker 1: What do you understand about robots that most people don't.
33:28
Speaker 2: I think the most interesting thing about robots that I
33:34
Speaker 2: would say, like we understand a covariant that maybe outside
33:37
Speaker 2: of the company don't is making one robot work is
33:47
Speaker 2: obviously hard and fun, but making a lot of robots
33:50
Speaker 2: work as scale for a lot of customers take a
33:53
Speaker 2: lot of operational discipline. Like it's about like doing many
33:59
Speaker 2: many things right, like before robots go into a facility,
34:03
Speaker 2: like how should they prep the site so the robots
34:06
Speaker 2: actually work. To ship robots at scale, it's a competencies
34:13
Speaker 2: that requires a lot of operational excellence, and that is
34:17
Speaker 2: something that I would say, like when most people think
34:20
Speaker 2: about robots like big thing about is like this sexy,
34:22
Speaker 2: interesting technology, they don't think about having to nail a
34:26
Speaker 2: thousand steps, a thousand small steps well in order to
34:30
Speaker 2: have robots actually have an impact in the world at scale.
34:33
Speaker 1: That's the leap from the academic lab to being a
34:36
Speaker 1: real company selling real products in the world. Great, anything
34:41
Speaker 1: else do you want to talk about?
34:42
Speaker 2: No? I enjoyed this conversation.
34:44
Speaker 1: Yeah, likewise, thank you. Theater Chen is the co founder
34:55
Speaker 1: and CEO of Covariate. Today's show was produced by Edith
35:00
Speaker 1: Russolo and Gabriel Hunter Chang. It was edited by Karen
35:04
Speaker 1: Chakerji and engineered by Sarah Bugueer. You can email us
35:08
Speaker 1: at at Pushkin dot f M. I'm Jacob Goldstein, and
35:12
Speaker 1: we'll be back next week with another episode of What's
35:14
Speaker 1: Your Problem