Latest / Automated with Brian Heater / Jagdeep Singh: Teaching Robots From Internet Video at Rhoda AI

Transcript

Jagdeep Singh: Teaching Robots From Internet Video at Rhoda AI · Automated with Brian Heater

• 0:00
  SPEAKER_02: Despite all the billions that have been raised, despite all the cool videos that have been published, I actually can't think of a single example of a generalist autonomous robot that's doing useful work in the field today.
• 0:12
  SPEAKER_02: Coming from outside the space, as I ramped up on understanding what the issues were, why robotics hadn't seen the same kind of progress that other AI models had seen, it became clear that the central issue was the lack of data.
• 0:25
  SPEAKER_02: The dominant paradigm in robotics is the VLA, the vision language action model.
• 0:29
  SPEAKER_02: Obviously, we have a different approach.
• 0:30
  SPEAKER_02: Our belief is that while VLAs are very helpful, they too have a ceiling.
• 0:36
  SPEAKER_00: You're working with a lot of these existing customers, in a lot of cases with off-the-shelf hardware.
• 0:42
  SPEAKER_00: So why ultimately is the company developing its own hardware?
• 0:46
  SPEAKER_02: We played with a bunch of third-party hardware and found these issues with payload capacity, reliability, safety, and dexterity.
• 0:52
  SPEAKER_02: So those are the four key things that are missing from today's humanoids.
• 0:55
  SPEAKER_00: And you get the sense that the partners that you're working with right now are eventually going to move towards a more general purpose hardware?
• 1:02
  SPEAKER_02: Oh, absolutely.
• 1:03
  SPEAKER_02: The tasks that are left that are not automated are tasks that really can't be well automated through traditional means.
• 1:09
  SPEAKER_02: And those tasks really require the generality of what the humanoids are.
• 1:12
  SPEAKER_02: The more immediate question, I think, for the industry is do you have an AI model that can just deal with the messiness of the real world?
• 1:49
  SPEAKER_00: Thrilled this week to take another deep dive into one of the bleeding edge pioneers of physical AI.
• 1:56
  SPEAKER_00: Jagdeep Singh is the CEO and founder of Rode AI, a startup that has been showcasing some extremely impressive and promising progress in robotic learning.
• 2:07
  SPEAKER_00: We go really deep on this one.
• 2:09
  SPEAKER_00: Please enjoy.
• 2:10
  SPEAKER_00: Thanks as always for listening.
• 2:11
  SPEAKER_00: If you've been enjoying the show, it would be a huge help if you would like and subscribe, tell a couple of friends, and while you're at it, why not like and check out and subscribe to the newsletter over at automated.fm.
• 2:25
  SPEAKER_00: Now please enjoy this conversation with Jack Deep Singh of Rhoda AI.
• 2:30
  SPEAKER_00: So the biggest thing I'm always curious about in the life of the serial founder is when it's clear to make the jump from this current company that you built and are running into this entirely new venture.
• 2:48
  SPEAKER_02: Yeah, I don't think there's a formula for that.
• 2:50
  SPEAKER_02: I think you know there's a lot of factors that go into it.
• 2:53
  SPEAKER_02: In my particular case, the last two companies I've run, I ran for on the order of 10 or more years.
• 3:00
  SPEAKER_02: Both of those we took public.
• 3:02
  SPEAKER_02: And you know, at some point running a public company gets a bit fatiguing, and you know, it's time for me to step down and let uh you know uh uh the the next level of leadership take over.
• 3:13
  SPEAKER_02: Uh but I I'm always keeping track of new technologies, and so there was always uh you know um a sort of uh uh an interest in what's uh in in in in new disruptions that are around the corner.
• 3:28
  SPEAKER_02: Uh and so if if uh if the time that uh it's right for me to step down from a company coincides with a new disruption that's uh around the corner, then everything comes together and it's kind of a perfect storm.
• 3:42
  SPEAKER_00: Yeah, I I one of the things that that I suspect a lot of people are wondering who who follow the industry is what is it precisely that's happened in the last, I guess, you know 24 months, you know, two to three years that has caused all of these physical AI companies, including Rhoda, to like suddenly start coming out of the woodwork right around the same time.
• 4:09
  SPEAKER_02: Well, I think obviously the the biggest change is that um, you know, uh the the underlying AI technology has really advanced to the point where you can do more useful things now, right?
• 4:19
  SPEAKER_02: So robotics used to be uh an area where uh you know you had uh a modular approach.
• 4:27
  SPEAKER_02: There was a perception module and a planning module and a control module and so on.
• 4:32
  SPEAKER_02: Um and and uh with the emergence of end-to-end AI models uh that learned purely based on data, and and even things like CNNs, which at one point were helpful, really introduce a um predefined bias into the into how the problem is solved.
• 4:48
  SPEAKER_02: And and a more end-to-end approach just feeds in data into the network, the network learns what it needs to learn and then produces the output.
• 4:56
  SPEAKER_02: Uh and those approaches allow um for a higher ceiling on what the models can achieve, right?
• 5:03
  SPEAKER_02: So if you the the classic uh, I guess um you know uh curve that you look at here is you know if you are data constrained or compute constrained, then having uh human biases, uh you know, inductive biases inserted into the model itself can be helpful because you can do more with data or less compute.
• 5:23
  SPEAKER_02: But those inductive biases also introduce ceilings on how good the models can get.
• 5:28
  SPEAKER_02: Uh, and if you go to a more end-to-end approach, you do need more compute and more data to get useful results, but the model might be able to learn things and go beyond what those inductive biases can can uh can provide.
• 5:40
  SPEAKER_02: Uh and so what's happened in robotics is that uh with with the uh uh you know uh increased amount of compute available, and of course the the sophistication of models, not only things like attention, which is obviously been critical, but but a lot of progress in the area of diffusion, um, you know, you've been able to uh process data end-to-end.
• 6:00
  SPEAKER_02: And that's enabled uh a way to do robotics that uh that doesn't require the traditional modular approach.
• 6:07
  SPEAKER_02: And I think that's been the key.
• 6:08
  SPEAKER_02: So if you look at, you know, obviously the prevailing paradigm, the dominant paradigm in robotics is the VLA, the vision language action model, right?
• 6:15
  SPEAKER_02: And what they do, of course, is they start with a pre-trained language model or vision language model, uh, which gives them a good prior on language, uh, and then they post-train it using some robot teleoperation data.
• 6:27
  SPEAKER_02: Uh and that teleoperation data uh is um you know uh that that training happens end-to-end.
• 6:34
  SPEAKER_02: So the models can can learn to do pretty useful things.
• 6:37
  SPEAKER_02: Um and and that's that's you know been a big part of the progress of the last few years.
• 6:41
  SPEAKER_02: Obviously, we have a different approach, our belief is that while VLAs are very helpful and they've shown results that you know uh perhaps go beyond what traditional robotics can do, uh, that they too have a ceiling.
• 6:54
  SPEAKER_02: And we think that ceiling comes from the fact that that there's just not enough teleop data that you can collect, and and more importantly, that you can't get enough diversity in that data set to really learn a good uh general prior on how the world moves.
• 7:06
  SPEAKER_02: And so, of course, our approach uh as you uh as you know is is is is a little bit different.
• 7:11
  SPEAKER_02: We we looked for an internet scale data set that we could tap into uh that had the scale and the diversity of the internet from which we could learn physics.
• 7:20
  SPEAKER_02: And and of course, uh that data set is internet video.
• 7:23
  SPEAKER_02: Uh and when we started, that was a bit contrarian, but quite contrarian.
• 7:27
  SPEAKER_02: Uh it's increasingly becoming more consensus uh these days.
• 7:32
  SPEAKER_02: Um people call them ward models, uh, even though it's not a technically uh accurate term.
• 7:37
  SPEAKER_02: Uh but um but yeah, so so so uh the the the short answer to your question is that uh the emergence of um you know uh the I should say the maturity of techniques such as um you know uh uh uh attention and and and diffusion uh coupled with uh the with uh larger data sets have really uh been the key to to drive this recent uh uh this recent uh activity.
• 8:03
  SPEAKER_00: Yeah, and you uh you have a technical background.
• 8:06
  SPEAKER_00: Um you know you you studied in school, you've you've been in the industry for a really long time, but you don't specifically have a robotics background.
• 8:14
  SPEAKER_00: So what are those conversations that you were having?
• 8:17
  SPEAKER_00: Who are you talking to, and like what specifically did you see that gave you confidence that this was the next big thing for you?
• 8:25
  SPEAKER_02: Yeah, so sometimes being from outside the space can actually be an advantage.
• 8:31
  SPEAKER_02: Um and I say that in the sense that uh sometimes if you're uh deep into a space, you can be part of the groupthink that everybody else is a part of.
• 8:41
  SPEAKER_02: Uh so coming from outside the space, uh as I ramped up on understanding what the issues were, why robotics uh hadn't seen the same kind of uh progress that other AI models had seen, from language models to image generation to video generation models, um, it became clear that the central issue was the lack of data.
• 9:02
  SPEAKER_02: Uh and um so as I dug into what the reasons were that there wasn't enough data, uh it then became clear that uh that the dominant approach was one where data was being collected using teleoperation.
• 9:19
  SPEAKER_02: And and uh that seemed to be uh you know a clearly limiting way of doing things, but um most of the leading uh voices in the field felt like teleoperation was the best and the richest form of data.
• 9:32
  SPEAKER_02: Uh, and and anything else would be a uh, you know, would just not be good enough.
• 9:36
  SPEAKER_02: Uh and and um as I started talking to people that I knew, for example, Gordon Retstein, one of our co-founders here, was in the faculty at Stanford, with whom I started a company in the past called Raxium that we sold to Google.
• 9:49
  SPEAKER_02: Gordon runs the computational imaging lab at Stanford where they do a lot of work on generative models and computer vision.
• 9:55
  SPEAKER_02: So that's a bit of a different perspective from traditional robot learning type folks.
• 10:00
  SPEAKER_02: Uh, but in discussing ideas with him, you know, the question came up well, why don't people use internet video to learn?
• 10:08
  SPEAKER_02: Uh and um uh you know uh we couldn't think of a good reason why people couldn't couldn't use internet video.
• 10:14
  SPEAKER_02: Uh and so he pointed me to one of his top PhD students ever, uh a guy named Eric Chan, who was an amazing, uh amazing guy in terms of both you know just being incredibly high IQ, but also such a deep thinker and um you know just high highly emotionally intelligent.
• 10:29
  SPEAKER_02: We started discussing this and and um you know we we uh became clear to us that yeah, it should be possible to use inner video to learn physics.
• 10:38
  SPEAKER_02: Uh and and um because there's just such a massive quantity of of these videos, massive diversity of these videos.
• 10:44
  SPEAKER_02: Uh and importantly, uh the vast majority of these videos cover real physics, how the world actually works, not they're not animations, they're not simulations.
• 10:52
  SPEAKER_02: So, you know, we we decided to try this approach.
• 10:55
  SPEAKER_02: Uh, and again, we were outsiders in the field to some extent, uh, not not outsiders in terms of um, you know, AI models and computer vision and generative AI, but outsiders relative to robot learning.
• 11:06
  SPEAKER_02: Uh and um and uh you know uh sometimes you need that uh kind of uh an outside uh perspective to try something different.
• 11:14
  SPEAKER_02: In this case, uh it worked, right?
• 11:16
  SPEAKER_02: The models uh were trained on video and they need only between 10 and 20 hours of robot data instead of tens of thousands of hours of robot data.
• 11:25
  SPEAKER_02: So uh, you know, uh a lot of times that that fresh perspective can be exactly what's needed to try something different.
• 11:33
  SPEAKER_00: Yeah, uh group think is is uh an interesting term in this context.
• 11:37
  SPEAKER_00: And I'm wondering, um one of the things that I've been trying to sort of parse out myself is, you know, obviously again, we're seeing all of these physical AI companies and we're seeing uh I'm seeing like you know, five, six news stories a week about all these breakthroughs that everybody's doing.
• 11:54
  SPEAKER_00: And it ends up being really hard to sort of parse out, you know, what the actual uh the real breakthroughs are, um, you know, and what what actually is going to sort of translate into real world progress.
• 12:05
  SPEAKER_00: So I'm curious, you know, in terms of when you are doing this work internally, um how do you do that vetting and and what do you decide what to actually share with the world?
• 12:18
  SPEAKER_02: So um let me answer this question in two parts.
• 12:21
  SPEAKER_02: Uh part one, relative to what I look for, the way I approach projects is I may have mentioned to you before, but we I I look for four things that I think every uh great company uh has in common.
• 12:34
  SPEAKER_02: One is uh, of course, there needs to be a big unsolved problem, right?
• 12:38
  SPEAKER_02: In this field of robotics.
• 12:39
  SPEAKER_02: It's you know, it's you could argue as big as problems get, maybe the biggest market ever, right?
• 12:45
  SPEAKER_02: Uh the market for work, right?
• 12:47
  SPEAKER_02: Uh second, I look for disruptive technology, you know, uh something that is different from what everybody else is doing.
• 12:54
  SPEAKER_02: Because if it's just the same as everybody else, then it's a commodity, and commodities have a difficult time making enough margins to re-invest in RD.
• 13:02
  SPEAKER_02: Uh so disrupt something, something that's um uh ideally something that's contrarian, not just different, but but but really um you know, uh completely the opposite of what everybody else is trying uh is really important.
• 13:14
  SPEAKER_02: Uh third, you need to have an exceptional team.
• 13:15
  SPEAKER_02: This is sort of obvious, but the team has to be you know world class and you know you have to be able to hold your own against the best CTOs from the best companies in the world.
• 13:22
  SPEAKER_02: And four, uh it's really important is you have to have early customer validation, right?
• 13:26
  SPEAKER_02: Are the customers you're you're you're targeting, uh are they excited before you ever really build a product?
• 13:33
  SPEAKER_02: Um and and uh uh in this case, all four items were in place.
• 13:37
  SPEAKER_02: But you have to you have to realize that when you start a company, those four items, those four attributes are really just hypotheses.
• 13:44
  SPEAKER_02: You may have a hypothesis about how big a market is, or hypotheses around uh different ways to solve the problem, etc.
• 13:50
  SPEAKER_02: Uh, and your job as a startup founder is to validate the hypotheses in much the same way that in a scientific lab, you you come up with hypotheses for what might be the truth, but you have to validate the hypotheses through experiment.
• 14:02
  SPEAKER_02: Uh and that's what we did here, right?
• 14:04
  SPEAKER_02: So we we went out and talked to customers.
• 14:06
  SPEAKER_02: Uh, we did some early work on the uh the approach with the generation.
• 14:10
  SPEAKER_02: Um, you know, we spoke to investors about the fundability and so on.
• 14:13
  SPEAKER_02: Uh, and only after we got real validation uh did we decide to increase the scale of the investment that we're making, right?
• 14:19
  SPEAKER_02: So that's the way that I look at things.
• 14:20
  SPEAKER_02: Now, from your perspective, as someone who observes the industry and has done that for a long time very well, you're right.
• 14:29
  SPEAKER_02: It is hard to distinguish uh when there are all these claims being made, you know, which of these claims is is gonna work.
• 14:36
  SPEAKER_02: And I think the easiest way to do it is just to um look for market traction.
• 14:40
  SPEAKER_02: Which of these approaches is actually winning in the marketplace?
• 14:43
  SPEAKER_02: So the real test, the real test of who wins in this space is not gonna be, you know, uh who can put out a more uh you know uh uh more breathless press release or show videos of their robots doing things in a laboratory setting, but who is actually getting deployments, right?
• 15:01
  SPEAKER_02: Who actually is getting traction with real customers, uh, which models are good enough to really deploy in the real world?
• 15:07
  SPEAKER_02: Because that, in my view, is the central challenge with robotics today, with intelligent robotics, is that the the models, VLAs, are good enough to do a lot of cool demos in the lab, because in the lab setting, the environment can be set up to match the distribution of the training data very closely.
• 15:24
  SPEAKER_02: And in that case, the model can work.
• 15:26
  SPEAKER_02: In the real world, you don't have that luxury.
• 15:28
  SPEAKER_02: The real world will present to you what it what it is, which is messy and diverse and highly variable.
• 15:33
  SPEAKER_02: And if the model can't deal with that variability, it's gonna fail.
• 15:37
  SPEAKER_02: Uh, and so um the real test of which of these approaches is gonna be successful is which of these approaches has a robust enough model to be able to generalize into the real world.
• 15:47
  SPEAKER_02: Now, obviously, based on what we've seen, we believe that the DVA, the direct video action model that we're that we're uh pioneering, uh has that ability to generalize and be more robust uh to where it can work in the world.
• 16:00
  SPEAKER_02: Uh but uh you know uh until we start reporting on that traction, it will be hard for people to figure out which what the winning approach is.
• 16:07
  SPEAKER_02: I think a year from now, it'll be probably a lot more clear uh which have won and which have not won, just based on the ones that have real traction in the world.
• 16:16
  SPEAKER_02: So um, yeah, it's I I I agree.
• 16:18
  SPEAKER_02: It's a difficult, uh you have a difficult job trying to distinguish winners from non-winners, uh, but but to some extent that data that you need isn't all in yet.
• 16:28
  SPEAKER_02: Uh as a founder myself, you know, I I can uh look at these hypotheses and and and and the results of the hypothesis validation uh to have an opinion.
• 16:38
  SPEAKER_02: Uh, but you know, uh it's hard for somebody that isn't uh, you know uh uh well even somebody that is deeply technical, uh, I think would have a hard time sometimes distinguishing you know uh winners from non-winners.
• 16:53
  SPEAKER_00: Yeah, and I think an important distinction here too is that there is that kind of first wave of customers that are initially really interested in coming on board because you know, obviously you have this background, it's a novel technology, they want to get involved, and then there are the ones who stay on, and then there's the question of actual like ROI and at what point that actually kicks in, right?
• 17:16
  SPEAKER_00: So they're they're the early pilots, and then there's the actual like real world in in usage.
• 17:22
  SPEAKER_00: I mean, where are you at?
• 17:24
  SPEAKER_00: Where is your technology at on that timeline?
• 17:28
  SPEAKER_02: Yeah, there's no question that the economics are a big part of uh of what will cause the customers to to stick with the given approach over time.
• 17:35
  SPEAKER_02: Uh but but but what I'm talking about is is a bit even precedes that economic uh question.
• 17:41
  SPEAKER_02: It's simply around can the model even work?
• 17:43
  SPEAKER_02: Right?
• 17:43
  SPEAKER_02: Before you can look at the economics, the solution has to has to actually work.
• 17:48
  SPEAKER_02: Um, you know, uh and and and and the problem here I think is that the conventional approach, uh, I don't think is going to be robust enough to even work in the face of all the diversity that the real world has to offer.
• 18:01
  SPEAKER_02: So uh just getting the first few deployments where you're actually running in a production setting uh you know day in and day out with you know with with uh with again all of the variability that the real world has to uh to throw at you, um, that will be the first real test of which models have have potential versus the ones that don't.
• 18:20
  SPEAKER_02: And then you're right.
• 18:21
  SPEAKER_02: Um once once you have if you have a number of models that that work, then the question becomes, okay, where you know, which ones can deliver compelling enough economics to where it's a win for the customer while while the company uh offering the product can you know still makes enough money to be able to reinvest in RD.
• 18:37
  SPEAKER_02: Uh and so obviously we we've got our own internal financial models uh that that that um that give us confidence that we've got a you know a sustainable you know business model.
• 18:46
  SPEAKER_02: But the more immediate question, I think, for the industry is is do you have an AI model that can just deal with the uh the messiness of the real world?
• 18:56
  SPEAKER_00: Yeah, and I think there has to be an expectation too with these customers that you know uh regardless of the model, like there are always gonna be there's always gonna be long tail, there's always gonna be edge cases, and a big part of the question is what's built into this model to be able to sort of correct itself on the fly.
• 19:15
  SPEAKER_02: Totally.
• 19:16
  SPEAKER_02: Well, this is um you're tapping into one of the things that we think is the biggest one of the biggest advantages of the approach that we're taking.
• 19:22
  SPEAKER_02: So the fact that we've pre-trained on all these hundreds of millions of videos is of course what allows us to post-train on a very small amount of data, right?
• 19:30
  SPEAKER_02: Between 10 and 20 hours of data to start.
• 19:32
  SPEAKER_02: Um but that that small amount of data, the advantage that gives us then is that is that uh when we encounter those corner cases uh in deployment, uh, we can uh we can record those scenarios uh and then use them to further uh fine-tune the model uh and make it even more robust.
• 19:50
  SPEAKER_02: And and those small amounts of data are sufficient uh for the model to really learn from because again, it has this very strong prior upon which it's building.
• 19:58
  SPEAKER_02: Uh whereas if you needed, you know, if you don't have that prior and you need tens of thousands of hours of data, then it becomes harder to learn from just one or two uh long tail data points.
• 20:10
  SPEAKER_02: Right.
• 20:11
  SPEAKER_02: So I think uh the reason why we're excited about what we're doing is we think the models uh when we ship them will be good enough to actually work autonomously uh you know uh for the the the uh the majority, the vast majority of the time.
• 20:24
  SPEAKER_02: And then if we run into a uh an edge case that requires human intervention, um, that actually is a really good thing because the model will learn from those edge cases and become more robust, which then allows for more deployments and more uh uh data collection and better models.
• 20:41
  SPEAKER_02: It kicks off that whole classic data flywheel that that um you know uh everybody obviously looks for.
• 20:48
  SPEAKER_02: Uh but but you you know you have to have the right, you have to earn the right to collect that edge case data by having a model that's good enough to deploy uh to begin with, right?
• 21:00
  SPEAKER_00: Yeah, one of the things I wanted to clarify, so so you said models plural, and I'm wondering, you know, are are are you working towards one big model that works across use cases, or will there be individualized models for different jobs and different fields?
• 21:16
  SPEAKER_00: Oh yeah.
• 21:17
  SPEAKER_02: No, I was trying to be modest when I said models.
• 21:19
  SPEAKER_02: I was trying to talk about other other companies building models that I see.
• 21:23
  SPEAKER_02: Uh but in our case, we have one model.
• 21:25
  SPEAKER_02: It's uh pre-trained on video, post-trained on customer use cases.
• 21:28
  SPEAKER_02: We do require that our customers allow us to train our model on data we collect from uh operating in their facilities, uh, because that's how we learn.
• 21:38
  SPEAKER_02: That's the biggest single thing we get from early deployments is data on those corner cases.
• 21:42
  SPEAKER_02: Um uh so if a customer says, you know, we're not gonna let you uh post train on this data, then we say, well, uh in that case, you know, we're not gonna be able to work with you because that's that's really the value that we get.
• 21:53
  SPEAKER_02: But the benefit of that is that is that then the model um just uh you know, uh the learning compounds across customers.
• 22:00
  SPEAKER_02: Because every time you get more data, that data is now, it makes this model stronger across the board.
• 22:06
  SPEAKER_02: And that's really the key thing.
• 22:08
  SPEAKER_02: So it's a single model, and we're going to use the latest checkpoint that we get from a given customer to then post-train additional data points from new customers.
• 22:22
  SPEAKER_00: Have you gotten a significant number of customers or potential customers giving you pushback on that idea of collecting data to train the models?
• 22:33
  SPEAKER_02: We actually haven't.
• 22:40
  SPEAKER_02: We just need an irrevocable perpetual license to be able to use the data to train our model.
• 22:45
  SPEAKER_02: So we're not trying to like sell this data to other parties.
• 22:49
  SPEAKER_02: We're not trying to prevent them from using this data for their own purposes if they find another supplier that can leverage the data.
• 22:55
  SPEAKER_02: Great.
• 22:56
  SPEAKER_02: They can do that.
• 22:57
  SPEAKER_02: But what they can't do is preclude us from using that data to make our model better.
• 23:02
  SPEAKER_02: And it's not that it's not that bizarre a concept.
• 23:07
  SPEAKER_02: If you think about it, when you hire a new employee, right, the employee has a certain base level of training that they come in with, but they may need to learn your specific processes, right?
• 23:16
  SPEAKER_02: And if you if you train them on your processes and then they leave and go to some other company, you can't then like, you know, unlearn what they've learned, right?
• 23:25
  SPEAKER_02: You can't make them forget if they've learned to use a hammer, you know, because your process required a hammer and they go somewhere else.
• 23:34
  SPEAKER_02: Well, they still know how to use a hammer, right?
• 23:36
  SPEAKER_02: So uh that you know what they can't do is take your proprietary process with them.
• 23:41
  SPEAKER_02: And that we're not going to do that.
• 23:43
  SPEAKER_02: All we're doing is is the basic skills that we've learned are now a part of the model.
• 23:49
  SPEAKER_00: Something that you said to me the last time we spoke that uh really stayed with me, and I've run it by a number of other physical AI companies since then.
• 23:59
  SPEAKER_00: Um I'll I'll actually uh uh I'll quote it back to you.
• 24:03
  SPEAKER_00: I know I know people love it when I do this to them, but uh you said um uh it turns out we found out the opposite models that worked best were the ones that were less curated, uh when there's a bunch of other random stuff in there uh that again might not involve human beings.
• 24:18
  SPEAKER_00: So so uh effectively um you know what we were talking about at the time was that you found out that the data used to trade models was often better when it wasn't like the video wasn't that specific task.
• 24:32
  SPEAKER_00: It was other tasks that built that that informed the physics.
• 24:35
  SPEAKER_00: And I've run that by a number of companies since then, and and they they all seem to agree with that.
• 24:41
  SPEAKER_00: I mean, what is the you know, and and you said in hindsight you it kind of dawned on you that maybe that should have been obvious.
• 24:48
  SPEAKER_00: I mean, what was the breakthrough there?
• 24:51
  SPEAKER_02: You know, um when we tell people that we want to um you know build models to automate um tasks like factory use cases or logistics and warehouses, a lot of times they'll say, Oh, you know, there's a data set we know of that you know, where it's just about people who are doing that task or that type of task.
• 25:11
  SPEAKER_02: And they're there there's companies, for example, collecting egocentric data where they they have workers wear you know uh GoPro type cameras on their heads and they're they're delivered getting data doing the tasks.
• 25:22
  SPEAKER_02: And they say that that sounds like it's uh it's it's the most important data set for you.
• 25:26
  SPEAKER_02: Uh and and um you know the the so so it it's it's easy to think that that you know uh if you're trying to get the robot to do a certain task, the data on that particular task is uh is all you need.
• 25:41
  SPEAKER_02: Uh and and so it was a bit counterintuitive for us uh to realize that, well, um, you know, this more curated data set of human beings doing tasks uh didn't perform as well as as broader data sets.
• 25:55
  SPEAKER_02: And I think the reason why it makes sense that that would be the case, of course, is that is that the job of the pre-training, right, is not to learn a specific task.
• 26:04
  SPEAKER_02: It's to understand the dynamics of how that domain works.
• 26:08
  SPEAKER_02: In the case of robots, the pre-training should allow them to understand uh how motion in the world works, how the laws of physics uh work, basically.
• 26:18
  SPEAKER_02: The specific task training is going to come in the post-training.
• 26:21
  SPEAKER_02: So the pre-training almost doesn't need to worry about specific tasks, it just needs to make sure the model understands things like weight and momentum and basic aspects of how things move, how much force you have to apply to make something, you know, like make a plastic bottle warp and so on and so forth.
• 26:36
  SPEAKER_02: Or even things like which direction do you twist uh a bottle cap to open it versus to close it.
• 26:42
  SPEAKER_02: Um these are all things that are not task-specific, they're just general sort of uh, you know, uh uh general levels of understanding about the uh the the way the world works.
• 26:52
  SPEAKER_02: Uh and then the task-specific learning comes from that post-training.
• 26:56
  SPEAKER_02: Uh the 10 to 20 hours of task-specific post-training is then enough uh to be able to train it.
• 27:00
  SPEAKER_02: So we don't look for um, you know, uh uh we don't particularly try to go out and look for uh video data sets that match the task we're trying to we're trying to perform.
• 27:10
  SPEAKER_02: We we just try to build a uh a diverse enough data set uh to where uh we we can um you know uh uh learn how the word works.
• 27:18
  SPEAKER_02: We do we do obviously do some level of curation in the sense that we don't want any particular class of video to dominate the pre-training.
• 27:25
  SPEAKER_02: So for example, if there's a whole bunch of baseball videos in the um in the pre-training, you know, some of those are useful, but if you have like you know 80% baseball videos, uh you're not you're not gonna learn other aspects of how the world works.
• 27:40
  SPEAKER_02: So you have to make sure that there's some level of balance.
• 27:42
  SPEAKER_02: Uh but what you can't do is really is really completely cut out uh entire classes of uh video.
• 27:49
  SPEAKER_00: Yeah, so uh uh effectively the uh the video is extremely valuable when it comes to pre-training the model, when it comes to giving the uh the systems an idea of how the world works, but ultimately you do need to get it to the point where you have sufficient real-world robotic deployment in order to kick start the physical flywheel.
• 28:13
  SPEAKER_00: Is that right?
• 28:14
  SPEAKER_02: Well, so uh I I would break it down into three different levels of training.
• 28:18
  SPEAKER_02: Uh the first level is, of course, uh the pre-training.
• 28:20
  SPEAKER_02: And that this is a vast, vast majority of training that we do is on this internet video, right?
• 28:25
  SPEAKER_02: Uh again, hundreds of millions of videos or clips that we trained on.
• 28:29
  SPEAKER_02: Um, the next phase is the post-training.
• 28:32
  SPEAKER_02: That's already extremely data efficient because we only need between 10 and 20 hours of data on a given task, right?
• 28:37
  SPEAKER_02: So we could, you've seen the videos on our website, maybe you know, uh there's a task there, an automotive task where we're decanting ball bearings.
• 28:44
  SPEAKER_02: Uh and then that task, you know, we've learned that task with on the order of 10 to 20 hours of video.
• 28:49
  SPEAKER_02: Another task where we're folding these um plastic containers called Conticos at another company.
• 28:55
  SPEAKER_02: Uh, and again, that task was learned with just a few, like 10 to 20 hours worth of worth of data.
• 29:00
  SPEAKER_02: That's the second level.
• 29:01
  SPEAKER_02: The third level is the uh is the the in-production edge and corner case data that you get where the model is failing on a particular edge case and requires human intervention to correct.
• 29:15
  SPEAKER_02: Uh and that's an even smaller data set than the 10 to 20 hours of uh of post-training.
• 29:19
  SPEAKER_02: Uh, but again, very powerful because that's uh that reflects a data point that's never been seen before, uh and the model really learns from this data point.
• 29:26
  SPEAKER_02: So those are the three levels of the hierarchy, and they drop pretty dramatically.
• 29:30
  SPEAKER_02: So the first one is massive, the second one is already really small, 10 to 20 hours, and the third one is even smaller.
• 29:36
  SPEAKER_02: It might just be a data point or two uh that you can learn from Yeah.
• 29:42
  SPEAKER_00: Well, one of the things that you pointed out to me earlier, and it was in reference to those boxes, were was that um that that you would use those boxes in the laboratory and that you were able to get them to run really efficiently.
• 29:54
  SPEAKER_00: And then once you actually tried to use them with real-world boxes in the lab, like initially the model struggled, right?
• 30:02
  SPEAKER_00: Because you know, these are boxes that have been like battered and bruised and been through the system.
• 30:06
  SPEAKER_00: And there's a lot of like physics and things that you hadn't encounted for initially.
• 30:10
  SPEAKER_02: No, exactly.
• 30:11
  SPEAKER_02: So so we we almost had to relearn the task uh because you know, we we we got there uh thinking we'd mastered this task, and we found that, well, there was, you know, you know, the the latches may be broken, which requires a different strategy for how you how you you know you fold the the sides.
• 30:29
  SPEAKER_02: The ball transfer tables on which we rotate these containers were rusted.
• 30:33
  SPEAKER_02: So the amount of frictional forces were very different than what we're used to.
• 30:37
  SPEAKER_02: Uh so we had to relearn the task.
• 30:38
  SPEAKER_02: And the point I was making there was that we were able to relearn the task in just literally on the order of three hours or so of data in the factory.
• 30:47
  SPEAKER_02: Uh so again, this incredibly high level of data efficiency is one of the main benefits of um of uh uh this kind of video peak training.
• 30:55
  SPEAKER_02: In fact, if you look at the paper, the technical report on our website, uh, the title is that causal video models are data efficient learners, because that's one of the key takeaways for us is we can learn with very little robot data.
• 31:06
  SPEAKER_02: Uh and that's uh that that's one of the things that really appeals to customers.
• 31:11
  SPEAKER_02: You know, you know, you're a robotics guy, you know that with conventional robotics, it can take weeks, if not longer, to program robots to do you know, to do uh tasks.
• 31:19
  SPEAKER_02: Hard code to code them.
• 31:21
  SPEAKER_02: Um and and actually with modern intelligent robots, if you have to collect thousands of hours of robot data, then it also might take weeks.
• 31:30
  SPEAKER_02: Uh so the this ability to be able to perform a new task with just 10 to 20 hours or even a single digit number of hours is a really big advantage uh for uh ease of deployment.
• 31:42
  SPEAKER_02: Customers really want to just get these robots up and running.
• 31:46
  SPEAKER_02: Uh and and you know, we live in a instant gratification culture, right?
• 31:50
  SPEAKER_02: Uh you know uh having to wait weeks or months uh you know just uh it's hard to do because usually when you want something, you've already you've already decided that you need it now.
• 32:01
  SPEAKER_02: Uh and so this is one of the key uh reasons why customers are excited, actually.
• 32:06
  SPEAKER_00: So one of the things uh that I was really curious about is you know, you're working with a lot of these existing customers and a lot of cases with off-the-shelf hardware.
• 32:18
  SPEAKER_00: Um so why ultimately is the company developing its own hardware?
• 32:25
  SPEAKER_02: Yeah.
• 32:25
  SPEAKER_02: So the hardware we're working with right now is good enough to accomplish uh a lot of the use cases that we are working with, uh, but it's not good enough to really address the full scope of what we want to address.
• 32:38
  SPEAKER_02: In particular, uh there are four key requirements that we want from our hardware.
• 32:43
  SPEAKER_02: We want it to be able to have uh a sufficiently high payload capacity.
• 32:48
  SPEAKER_02: Uh so the OSHA standard for uh humans uh in the US is on the order of 23 kilograms of weight uh that can be lifted um uh safely.
• 32:58
  SPEAKER_02: Uh Europeans uh uh and in Asia have very similar uh kind of standards.
• 33:03
  SPEAKER_02: Uh we want the robot to be able to lift on the order of twice that.
• 33:07
  SPEAKER_02: So close to the body, it's 23 kilograms, is what the OSHA standard is.
• 33:11
  SPEAKER_02: Far from the body, it's on the order of uh eight or nine kilograms.
• 33:15
  SPEAKER_02: We want to double that.
• 33:16
  SPEAKER_02: So like 45 kilograms close to the body, 18 or 19 far from the body.
• 33:20
  SPEAKER_02: Um we couldn't find humanoid hardware out there that has that kind of capacity.
• 33:26
  SPEAKER_02: Uh most humanoid hardware out there today, um, there and there are probably over a hundred humanoid companies around the world.
• 33:32
  SPEAKER_02: But most of them, to be candid, are are are toys.
• 33:35
  SPEAKER_02: Uh so they they can lift a small amount of weight uh but um uh not a lot.
• 33:40
  SPEAKER_02: And even that weight that they can supposedly lift, uh they can't do it day in, day out uh reliably.
• 33:47
  SPEAKER_02: That's the second requirement.
• 33:48
  SPEAKER_02: We want to be able to be reliable and work in an industrial environment continuously for three years without burning out the motors.
• 33:56
  SPEAKER_02: As I think I may have mentioned uh last time we spoke, we we we started out by buying a few humanoids uh and we we burned out one motor within a day and the second motor within a week.
• 34:07
  SPEAKER_02: Uh so it became clear to us that reliability was an unsolved problem.
• 34:10
  SPEAKER_02: The third requirement is safety.
• 34:12
  SPEAKER_02: Uh we're not aware of any humanoids that have CE certification to be able to operate uh around humans in the in the in the Europe, for example.
• 34:20
  SPEAKER_02: Um and that's a critical requirement for us.
• 34:22
  SPEAKER_02: We really want to have every subsystem of our robot be CE certified or safety certifiable.
• 34:27
  SPEAKER_02: That requires a lot of uh actually it's a lot of work, a lot more work than I realized when we started doing this.
• 34:32
  SPEAKER_02: And the final thing is dexterity, right?
• 34:34
  SPEAKER_02: There's a lot of people that are not talking about robot hands, but uh you don't really have any hands that can do real work today outside of just simple power grasps, which is kind of open and close the whole fist.
• 34:44
  SPEAKER_02: Uh and and the reason for that goes beyond just the mechanical actuation of the hand, uh, includes things like how do you teleoperate the hand?
• 34:53
  SPEAKER_02: Without teleoperation, you can't collect data.
• 34:55
  SPEAKER_02: Without data, you can't uh train an autonomous model to use the hand.
• 34:58
  SPEAKER_02: So those are the four key things that are missing from today's humanoids.
• 35:02
  SPEAKER_02: Uh and uh the only um uh uh way we saw to address those requirements was to put it on hardware.
• 35:08
  SPEAKER_00: So when we're talking full stack, we're I mean like hardware-wise, like truly full stack, like you are in fact developing the hand as well.
• 35:16
  SPEAKER_02: 100%.
• 35:17
  SPEAKER_02: I mean there's you know, our our development philosophy is quite simple.
• 35:20
  SPEAKER_02: Uh it is that um we don't want to develop anything that we can buy off the shelf.
• 35:26
  SPEAKER_02: So if a solution exists, we want to buy it.
• 35:28
  SPEAKER_02: On the other hand, uh part B of that philosophy is if something um that we need is not available on the market, then we aren't afraid to develop it ourselves.
• 35:38
  SPEAKER_02: And I think by applying that that filter, both of those filters, you end up with um uh by definition, almost uh you know, uh uh a scenario where everything you're working on is truly um advancing the frontier.
• 35:50
  SPEAKER_02: So yeah, we're a full stack company.
• 35:52
  SPEAKER_02: Uh the AI model is obviously um you know one of the key missing pieces, so we'll focus on that.
• 35:56
  SPEAKER_02: Uh but the hardware is absolutely another missing piece.
• 35:59
  SPEAKER_02: Well, one one comment I'll say for both the hardware and the and the AI model, the key requirement is is generality, right?
• 36:06
  SPEAKER_02: So um for the AI model, generality is pretty obvious what it means.
• 36:11
  SPEAKER_02: It means you know you can basically you don't these are not bespoke um you know uh use case uh specific uh models that can do anything basically.
• 36:21
  SPEAKER_02: Um uh for the hardware, what generality means is that the same hardware doesn't require a lot of um uh mechatronic um uh retrofitting around it.
• 36:33
  SPEAKER_00: Modularity, yeah.
• 36:34
  SPEAKER_02: Yeah, you don't need things like conveyance uh mechanisms to bring goods to the robot and so on and so forth.
• 36:39
  SPEAKER_02: The robot can move, uh it can, you know, it has kind of a humanoid form because that form is very general.
• 36:45
  SPEAKER_02: Uh and that generality is really key because if you're doing a bespoke piece of hardware for every use case, um it becomes economically unattractive very quickly.
• 36:55
  SPEAKER_02: Uh so that's kind of the the key is on both the software and the uh and the hardware is to make it um uh general.
• 37:01
  SPEAKER_00: So generality uh when you're looking at manipulators, uh how closely I mean, I guess we're probably speaking in abstractions here because obviously like you haven't unveiled anything, but how closely do you expect to hue to the human hand?
• 37:18
  SPEAKER_02: I think that um you need to have enough degrees of freedom to where you can perform uh tasks that go beyond simple, like you know, uh gripper open-close type tasks or or simple power grasp type tasks for the whole hand.
• 37:31
  SPEAKER_02: But you may not need all five fingers worth of uh uh degrees of freedom.
• 37:36
  SPEAKER_02: Uh in particular, uh it seems like a three-fingered hand can do a lot.
• 37:41
  SPEAKER_02: Uh in fact, most uh uh tasks can be done with a three-fingered hand, uh, and the extra two fingers appear to be mostly necessary for strength.
• 37:49
  SPEAKER_02: Uh so we're looking at those options and and and without pre-announcing our products, uh the key is we want to have enough degrees of freedom to do a job, but if you have too many degrees of freedom, you end up uh becoming less reliable.
• 38:01
  SPEAKER_02: Uh and because reliability is is also one of our four key requirements for the hardware uh in conjunction with with dexterity, we need to strike uh a healthy balance there.
• 38:10
  SPEAKER_00: Yeah, it I mean it strikes me that that so much of what we're talking about um centers around timing, you know, and and and uh approach to market, um, you know, uh ROI, deliverables, things like that.
• 38:26
  SPEAKER_00: Obviously, you're leading with the model, uh, you know, you're you're working with this off-the-shelf hardware.
• 38:32
  SPEAKER_00: Ultimately you're working towards generality, both in terms of the model and the hardware.
• 38:39
  SPEAKER_00: Um, that's gonna take time.
• 38:41
  SPEAKER_00: As you mentioned, there's something in the neighborhood of you know, a hundred or whatever humanoid companies.
• 38:47
  SPEAKER_00: And the reason why most of them are are that we're seeing right now are um toy-like is because those are the ones like that have actually come to market.
• 38:55
  SPEAKER_00: But obviously, there's a lot of companies working towards this industrial humanoid.
• 38:59
  SPEAKER_00: Why does it make sense to just then develop this in-house versus you know, waiting out, developing, you know, working on the model and then waiting for any one of these number of companies doing this work to develop a humanoid that does fit your needs?
• 39:17
  SPEAKER_02: Well, we don't know if anybody is going to develop something that meets our needs.
• 39:21
  SPEAKER_02: So uh the reason I say that is we're not just developing the humanoid system itself, but in order to address the needs uh of the humanoid system, we needed to go to the subsystem level.
• 39:33
  SPEAKER_02: So, for example, we're developing our own actuators because not only couldn't we find a fully built humanoid to get the job done, we also couldn't find components like the actuators that could we could use to assemble a humanoid.
• 39:44
  SPEAKER_02: So there's a high level of vertical integration here.
• 39:47
  SPEAKER_02: It's the same thing with tactile sensing, right?
• 39:50
  SPEAKER_02: If you want the ability to sense force and pressure, both both normal and shear forces on the fingertips, for example, uh there really aren't a lot of good solutions in the market today.
• 39:58
  SPEAKER_02: We've we've done a pretty extensive survey of what's available.
• 40:00
  SPEAKER_02: There's a lot of technologies from you know uh uh you know uh optical, magnetic, and so on and so forth that are resistive, uh, but uh there's nothing that meets the requirements of having the right level of sensitivity, the dynamic range, the economics, uh, the reliability uh to make it all work.
• 40:16
  SPEAKER_02: So we're finding ourselves building uh these component-level technologies to enable the system, which then allows the AI model to run on it.
• 40:24
  SPEAKER_02: So it's very much a full-stack company.
• 40:26
  SPEAKER_02: We're not doing this because we're masochists and just want to take on stuff on our own.
• 40:29
  SPEAKER_00: There might be a little bit of that, but yeah.
• 40:31
  SPEAKER_02: But we're doing it because in the end, you know, we want to solve a real problem that our customers have.
• 40:36
  SPEAKER_02: And we just are in talking to our suppliers, we've been unable to find a roadmap uh from them uh that suggests that we're gonna be able to do that entirely off the shelf.
• 40:45
  SPEAKER_02: Uh now we're not building every component ourselves.
• 40:48
  SPEAKER_02: Where we can find components, we're absolutely leveraging them.
• 40:51
  SPEAKER_00: Yeah, well, it it's interesting.
• 40:53
  SPEAKER_00: Um if I'm remembering correctly, so so the company uh effectively comes out of stealth um in March, and uh uh it it had been really only around for about 18 months prior to that, all the development.
• 41:08
  SPEAKER_00: At what point in that process did it really become clear that okay, we're gonna have to do full stack?
• 41:16
  SPEAKER_02: Oh, no, the full stack thing, we we've kind of um, you know, we've kind of we're prepared to do that from day one.
• 41:22
  SPEAKER_02: Uh we we know we we um we started out by focusing on the AI model and and and trying to buy hardware that we potentially could could um reduce the amount of development we had to do internally.
• 41:34
  SPEAKER_02: Uh and and and that was a it was a good experience.
• 41:37
  SPEAKER_02: We played with a bunch of third-party hardware uh and and and found these issues with, as I said, payload capacity, reliability, safety, uh, and dexterity.
• 41:45
  SPEAKER_02: Uh and realized very quickly that our original vision of of the full stack uh robot company uh in fact is the one we have to pursue.
• 41:53
  SPEAKER_02: Right.
• 41:54
  SPEAKER_02: So so we've been doing this since this day one.
• 41:56
  SPEAKER_02: It's also very helpful if you're a robot company.
• 41:59
  SPEAKER_02: You know, you can't you can't not get your hands dirty with the hardware because there are so many aspects to like how um the control software works that impacts what the AI model does.
• 42:09
  SPEAKER_02: Uh, you know, uh you could argue that in in some ways the the behavior of the AI model is um as much a function of of the control loops at the low level uh you know uh of the hardware as they are of the A model itself.
• 42:22
  SPEAKER_02: Uh so things like you know, like games, right?
• 42:25
  SPEAKER_02: What kind of um game do you provide for for your actuators, for example?
• 42:29
  SPEAKER_02: Um the these are all things that have uh a really big impact.
• 42:32
  SPEAKER_02: I mean, is it stiff?
• 42:33
  SPEAKER_02: Is it springy?
• 42:34
  SPEAKER_02: Uh these are all things that really impact the ability of the A model to run.
• 42:37
  SPEAKER_02: And so owning that full stack really helps us build a better system overall.
• 42:42
  SPEAKER_00: And and you get the sense that these the customers, the partners that you're working with right now um are eventually going to move towards humanoids, towards um a more general purpose hardware?
• 42:53
  SPEAKER_02: Oh, absolutely.
• 42:54
  SPEAKER_02: Um, in fact, um you know the way we look at it, everything that can be automated with conventional sort of classical robotics should be automated that way, and most of it already is automated that way.
• 43:05
  SPEAKER_02: So by definition, the tasks that are left that are not automated are tasks that really can't be uh you know well automated through traditional means.
• 43:13
  SPEAKER_02: Uh and those tasks really require the generality of the humanoid.
• 43:17
  SPEAKER_02: But when I say humanoid, I'm speaking a bit loosely, right?
• 43:20
  SPEAKER_02: We don't define humanoids as arms and hands and legs, for example.
• 43:23
  SPEAKER_02: Our robots don't have legs.
• 43:25
  SPEAKER_02: Uh we we use wheel bases.
• 43:27
  SPEAKER_02: You can more precisely call it a mobile manipulator.
• 43:30
  SPEAKER_02: Uh seven degree of freedom arms with with uh you know bidexterous by exactly dexterous uh hands, uh vision system, obviously, uh and but on a mobile base.
• 43:40
  SPEAKER_02: So we have full mobility, but we don't need to navigate stairs.
• 43:44
  SPEAKER_02: And very importantly, we are uh we are stable balance wise.
• 43:48
  SPEAKER_02: So if if we have to e stop the robot, uh we don't have to worry about the robot collapsing and either uh damaging itself or worse injuring a human being, right?
• 43:56
  SPEAKER_02: So that in fact many customers have told us directly that.
• 44:00
  SPEAKER_02: They're really grateful that we're building uh wheel bases because they know how to use that.
• 44:04
  SPEAKER_02: All these industrial logistical customers already have AMRs and AGVs running around on their floor.
• 44:10
  SPEAKER_02: So they're used to that kind of paradigm.
• 44:11
  SPEAKER_02: And this is simply an AMR or AGV with arms that can do actual manipulation, not just mobility.
• 44:17
  SPEAKER_00: Yeah, I it it it strikes me that you're solving like 50% or more of the safety, potential safety issues right up the bat by not doing dynamic stability.
• 44:26
  SPEAKER_02: Absolutely.
• 44:27
  SPEAKER_02: Absolutely.
• 44:29
  SPEAKER_02: The thing is you cannot deploy a robot in a production environment.
• 44:32
  SPEAKER_02: By production I mean, you know, manufacturing or logistics without the ability to e-stop it.
• 44:38
  SPEAKER_02: E-stop is critical.
• 44:39
  SPEAKER_02: If the robot does something weird, uh you have to be able to say stop or press a button or whatever.
• 44:45
  SPEAKER_02: And if you do that, it has to be safe.
• 44:46
  SPEAKER_02: And you can't guarantee safety if it's on legs.
• 44:50
  SPEAKER_02: It's just not there's no way to do that.
• 44:51
  SPEAKER_02: And that's one reason why these human knowledge have had a hard time getting CE certification.
• 44:55
  SPEAKER_00: Yeah, one of the things I'm curious about as far as uh uh sort of roadmaps and and timing.
• 45:03
  SPEAKER_00: Um it it seems like it not not that you were you weren't hiding the fact that you were working on hardware, but you were very deliberate about launching the company as a physical AI company and you know really sort of announcing or revealing the hardware later.
• 45:19
  SPEAKER_00: Um why why was that your priority as source timeline?
• 45:24
  SPEAKER_02: I mean, I I think that it really just is a function of um of the majority of different levels of our stack, right?
• 45:30
  SPEAKER_02: So the AI model is in is actually, Kennedy, in very good shape.
• 45:33
  SPEAKER_02: Uh and as you've pointed out at the beginning of this uh this session, uh, you know, we are running it on off-the-shelf hardware today, and that hardware is good enough to do real tasks.
• 45:42
  SPEAKER_02: Uh not the full range of what we think the model can can take on, but enough to be useful to our customers.
• 45:48
  SPEAKER_02: And so there was no reason to delay launching our AI model uh until our hardware was ready.
• 45:54
  SPEAKER_02: Uh so we're planning on shipping that AI model on that off-the-shelf hardware um you know very soon.
• 46:00
  SPEAKER_02: Uh, we I don't want to again pre-announce uh the timing and get in trouble with my marketing team, but uh we're gonna be deploying.
• 46:06
  SPEAKER_02: We we think that I mean you might be surprised to hear this, but we think that you know we could be the first autonomous manipulation capable robot that's deployed at scale, right?
• 46:20
  SPEAKER_02: Despite all the hype that is in the space, right?
• 46:22
  SPEAKER_02: Uh, because many of the players in the space are just focused on doing uh demos and demo videos, uh, and then not really tackling what's required to make a model actually industrial strength enough to work in the real world.
• 46:35
  SPEAKER_02: Right?
• 46:35
  SPEAKER_02: You have to be able to have um you know high success rates, you have to be able to deal with things like safety, you have to be able to deal with things like the time it takes to train the model to perform a task, right?
• 46:45
  SPEAKER_02: Reliability and success rates alone are a major.
• 46:48
  SPEAKER_02: I don't know if you followed the physical space, but you know, um many of these videos and demos that we see of robots doing cool things, uh, you know, are are uh oftentimes one take out of many takes that are that are uh and then they simply you know publish the one that that worked.
• 47:05
  SPEAKER_02: Uh it's like the old days of Boston Dynamics where they used to show robots doing backflips, and then occasionally you get the the the blooper wheel where they show you all the clips that didn't work, right?
• 47:13
  SPEAKER_02: Um but that's not what we want to do.
• 47:15
  SPEAKER_02: That's fine for a demo, but for real deployments, the the metric is the opposite.
• 47:19
  SPEAKER_02: In the demo, you care about the one time it worked because you're gonna capture that video and publicize that.
• 47:24
  SPEAKER_02: For for actual deployments, you care about the the one time it doesn't work because that number has to be very, very low.
• 47:30
  SPEAKER_02: You can't you can't really have it not work very often.
• 47:32
  SPEAKER_02: So every single time you do it, it has to basically work.
• 47:35
  SPEAKER_02: The customer in the end is buying a product and they expect the robot to work.
• 47:40
  SPEAKER_02: Now, you can one advantage you have with the kind of model that we're building is you can actually, you know, if the robot is struggling, you can remotely teleoperate the robot and and get it to address uh the issue we're struggling with.
• 47:54
  SPEAKER_02: And by the way, that produces really high quality data uh of corner cases on that long tail, which makes the model better, right?
• 48:01
  SPEAKER_02: Uh but from a customer standpoint, um, you know, you it just needs to work.
• 48:05
  SPEAKER_02: Um and if you're having to intervene too often, then the economics are not going to work, right?
• 48:09
  SPEAKER_02: So um, so yeah, so I think this is this is one of the key differences in what we're doing.
• 48:13
  SPEAKER_02: We really want to build products that we can deploy in the field, not just things that we you know impress other industry insiders with.
• 48:23
  SPEAKER_00: Well, yeah, so the well, there's there's there's two things.
• 48:26
  SPEAKER_00: There, I mean, obviously these are very closely related, but there's two things here, right?
• 48:29
  SPEAKER_00: There's there's the deployment, and then there's at what point you decide to show it off, right?
• 48:38
  SPEAKER_00: And and what you decide to show off.
• 48:39
  SPEAKER_00: And it seems like that's also something you're being very deliberate about.
• 48:43
  SPEAKER_00: You know, you alluded to the fact that like there are these wheel-based robots out there, that they are with customers, you know, that that there is some work being done.
• 48:52
  SPEAKER_00: Um what point do you feel, what point of maturity will the hardware have to be at in order for you to really feel comfortable showing it off?
• 49:04
  SPEAKER_02: No, it's a great point.
• 49:05
  SPEAKER_02: We we we we very deliberately, very consciously um don't want to show uh you know, uh uh hardware or software uh that isn't really ready for prime time.
• 49:15
  SPEAKER_02: Right.
• 49:16
  SPEAKER_02: So what that does is it it uh you know it causes us to be a little um later in uh what we announce, but what we do announce, uh we we want it to be real.
• 49:25
  SPEAKER_02: We don't want to be one of the many, many companies in this space that just um you know are showing demonstrations uh and not deployable products.
• 49:34
  SPEAKER_02: So uh everything we've talked about so far um has generally been things that we you know have a high level of confidence uh in relative to you know actually being deployable.
• 49:45
  SPEAKER_00: Yeah, I guess what um again, abstract question, but like what what does real mean?
• 49:51
  SPEAKER_02: You mean relative to deployability?
• 49:53
  SPEAKER_00: No, I mean you said you want you want it to you want you want to be real.
• 49:57
  SPEAKER_00: So you know real doesn't mean at scale.
• 50:01
  SPEAKER_02: No, no, real, real, real, real just means very simply that that the success rate has to be high enough to where it can be deployed, you know, uh and do actual work.
• 50:09
  SPEAKER_02: It can't just be uh, you know, well, I'm gonna try it this thing a hundred times and and if it works once, I have a video of it working and I can I can send, right?
• 50:18
  SPEAKER_02: Reliability, repeatability.
• 50:20
  SPEAKER_02: Repeatability, success rate.
• 50:21
  SPEAKER_02: Those are critical.
• 50:22
  SPEAKER_02: Success rate.
• 50:23
  SPEAKER_02: Yeah.
• 50:23
  SPEAKER_02: I mean, in the end, the customer is measuring you based on on success rate, right?
• 50:26
  SPEAKER_02: But well, typically most customers have two main KPIs, uh, key performance indicators.
• 50:32
  SPEAKER_02: One is um success rate, right, or reliability, how often does it succeed in the task?
• 50:37
  SPEAKER_02: Um, but two is cycle time.
• 50:39
  SPEAKER_02: How quickly does it get the job done?
• 50:41
  SPEAKER_02: And can it, can it, can I meet my my throughput requirements, whether it's a manufacturing floor or a warehouse uh to get the job done.
• 50:48
  SPEAKER_02: Um so but and and there may be others for different customer different use cases, but those are the two that are typically common to uh most use cases.
• 50:56
  SPEAKER_02: Uh and when I say real, that's what I'm talking about.
• 50:58
  SPEAKER_02: It has to be able to work.
• 50:59
  SPEAKER_02: Like what we don't want to do is show videos that are sped up 5x, uh, because you know that's not practical in the real world uh if you're running at one-fifth the speed of a human.
• 51:08
  SPEAKER_02: Uh we don't want to show videos of just kind of hero results where it worked once you know out of n attempts.
• 51:15
  SPEAKER_02: Uh we really want to be much, we want to build a brand uh candidly, uh, around being a company where if we show a video of something, uh it means that this thing is actually reliable enough to be deployed doing that particular use case that we're demonstrating.
• 51:30
  SPEAKER_00: Yeah, I mean it sounds to me, you know, when you say something like uh first autonomous manipulation capable robot that's deployed at scale.
• 51:37
  SPEAKER_00: So that's two things.
• 51:38
  SPEAKER_00: That's one, you're saying, okay, well, what we've seen so far is going to take longer than probably most people are thinking.
• 51:46
  SPEAKER_00: But then the other thing that's saying to me is, you know, that you're as a company are perhaps closer than than people think, which also in turn says to me that like you know, you might be nearing a point in the not too distant future when you are ready to show off this hardware.
• 52:03
  SPEAKER_02: Absolutely.
• 52:03
  SPEAKER_02: Uh I think uh or or you know, the the model working in real real uh customer environments with off-the-shelf hardware.
• 52:10
  SPEAKER_02: Uh I think I think you know, the my main point here uh is that is you know, for those of us um, you know, uh that that follow this physical AI space, you know, the last year and a half or two years have seen a tremendous level of activity.
• 52:25
  SPEAKER_02: Um lots of companies have raised a lot of capital.
• 52:27
  SPEAKER_02: There have been a ton of demos and videos, and and you know, you if you're an outside observer, you might be forgiven for thinking that, oh, robots are real.
• 52:35
  SPEAKER_02: We already have these humans and they can do it.
• 52:37
  SPEAKER_02: But the reality is, despite all the billions that have been raised, despite all the cool videos that have been published, right?
• 52:44
  SPEAKER_02: I actually can't think of a single example of uh a generalist autonomous robot that's doing useful work in the field today.
• 52:53
  SPEAKER_02: It's kind of a it's kind of a huge gap if you think about it.
• 52:55
  SPEAKER_02: Like the gap between the hype and the reality is is is quite substantial.
• 52:59
  SPEAKER_02: And and um you know, we just we we just did not want to be yet another company with the.
• 53:07
  SPEAKER_00: Well, that uh I mean that that this this is interesting, and this and this speaks to the timing, this speaks to the priority, this speaks to like how quickly you want to get to market, this speaks to questions of of ROI, because you know, when we when we talk about um generalist robot out in the field, uh you know, I would say, okay, well, you know, agility obviously they're probably the closest as far as like actually having robots deployed, but a big part of the reason why that's the case is because they're approaching the market by taking on a few tasks at a time versus arriving at the market as a general purpose robot.
• 53:46
  SPEAKER_00: Now, obviously, like you're pushing back the timeline quite a bit if you want to like hit the market as a general purpose, but is that you know you you you don't want this hardware to deploy at scale without it being able to be at least multicapable?
• 54:03
  SPEAKER_02: Absolutely.
• 54:04
  SPEAKER_02: We we we want it to be you know uh uh generalist, meaning it's not just doing one task.
• 54:10
  SPEAKER_02: I want it to be manipulation capable, right?
• 54:11
  SPEAKER_02: So I think with agility, no question that they have robots in the field.
• 54:15
  SPEAKER_02: Uh I'm not sure how general they are.
• 54:18
  SPEAKER_02: I think they appear to do mostly tote moving type use cases.
• 54:21
  SPEAKER_02: And I'm not sure how uh how manipulation capable they are.
• 54:24
  SPEAKER_02: I mean, I'm not sure if we consider that manipulation.
• 54:26
  SPEAKER_02: I mean, it is in a way, but yeah, but it's it's a fairly limited form of manipulation.
• 54:30
  SPEAKER_02: So uh what I'm referring to is you know, is um, you know, you've seen robots folding t-shirts, making sandwiches, making coffee and so on.
• 54:38
  SPEAKER_02: None of that stuff, or you know, or or or or even handling packages, uh, you know, uh in a what looks like a warehouse facility, but none of those are deployed.
• 54:48
  SPEAKER_02: Uh it's just they're just all demos.
• 54:50
  SPEAKER_02: Uh and if you if you if you believe the hype from the companies producing these videos, you you might think, oh, this is all deployed.
• 54:58
  SPEAKER_02: But uh, but you know, we've uh we've we've now talked to you know uh you know 100 plus customers, and and we just haven't seen any examples of any uh any next-gen autonomous manipulation capable robots that are actually uh you know uh doing work in the field.
• 55:17
  SPEAKER_00: Yeah, and we we can um we can wrap up on this, but something I was curious about when I was sort of you know catching up and and looking at some of the the headlines around the company since the last time we spoke.
• 55:29
  SPEAKER_00: Um, one of them was signing a lease for a lot of real estate and in Sunnyvale, uh 71,000 square feet, which you know, I'm when we're talking about that large of a space, to me, it strikes me like, okay, are we talking about physically I training?
• 55:46
  SPEAKER_00: Are we talking about like real world like deployment training?
• 55:50
  SPEAKER_00: Because that is, you know, obviously I think everybody agrees that that is important, but but Rhoda's approach has always really been video first.
• 56:00
  SPEAKER_02: Yes, 100%.
• 56:01
  SPEAKER_02: No, that's not true.
• 56:02
  SPEAKER_02: I think the big benefit of our approach, as we spoke about last time we talked, is it is it um there's a couple of benefits of the video first approach.
• 56:09
  SPEAKER_02: One is that the model learns a robust prior on how things move, which allows it to be more capable of dealing with unexpected scenarios.
• 56:18
  SPEAKER_02: Um it doesn't just fail, it comes up with a reasonable guess as to what should happen.
• 56:21
  SPEAKER_02: Edge cases, and then that can be dealt with no easier.
• 56:24
  SPEAKER_02: But the second benefit of that is that it reduces the amount of time you need to uh to collect data to train to post-train the model.
• 56:32
  SPEAKER_02: So instead of like hundreds or maybe thousands of hours of data to post-train the model, uh, you can get by with just tens, low tens, in fact, of hours of data collection.
• 56:43
  SPEAKER_02: Um you might have missed this, but but um in the March launch, we actually put a lot of different features into a single blog post.
• 56:49
  SPEAKER_02: But one of the features we put in there was this notion of demo following.
• 56:53
  SPEAKER_02: And that's a really powerful technique.
• 56:54
  SPEAKER_02: In fact, the last couple of weeks I've seen some announcements around this from other third parties in the industry.
• 56:59
  SPEAKER_02: But uh, that was the idea that the robot could be trained to do a task by simply uh following a human demonstration of a task with a single demonstration.
• 57:07
  SPEAKER_02: Uh, that's a very powerful technique that we like to think that we we've uh you know we pioneered, uh, and we're obviously continuing to work on that.
• 57:13
  SPEAKER_02: But the combination of only needing a few tens of hours of teleoperation data uh plus techniques like um that demo following technique is also called in-context learning, uh, really allow uh for um you know really rapid deployment of a wide set of use cases.
• 57:28
  SPEAKER_02: So to come back to your question, uh we we do have a fair amount of space that we that we uh have leased.
• 57:35
  SPEAKER_02: Uh and the reason for all that space is we obviously need space for our uh to house our people.
• 57:41
  SPEAKER_02: Uh we also need space for the hardware lab, right?
• 57:43
  SPEAKER_02: We're a full-style company.
• 57:44
  SPEAKER_02: Hardware requires a lab.
• 57:46
  SPEAKER_02: We got to develop stuff there.
• 57:48
  SPEAKER_02: Uh, and we also do need some space for for robot uh data collection uh because even though we have this pre-trained video model, uh we do require some uh uh use case specific uh data collection, uh which requires robots to be to be teleoperated.
• 58:03
  SPEAKER_02: So uh all those things together uh you know take up that space and and uh uh and we also have some some headroom for growth as well.
• 58:10
  SPEAKER_02: So that's the reason why we have that that much space.
• 58:13
  SPEAKER_00: That's interesting.
• 58:13
  SPEAKER_00: So so so it the majority of the post-training then is um teleop.
• 58:20
  SPEAKER_02: Yeah, uh the the post-training is is uh uh is teleop um uh simply because you know that you can actually do the actual use case and the robot can learn uh regard very quickly.
• 58:32
  SPEAKER_00: Got it.
• 58:32
  SPEAKER_00: Well, uh Jagdeep, always a pleasure.
• 58:35
  SPEAKER_00: Thank you so much.
• 58:35
  SPEAKER_00: Thanks for making time, buddy.
• 58:36
  SPEAKER_00: Take care.
• 58:37
  SPEAKER_00: Thank you.
• 58:39
  SPEAKER_00: Thanks so much to Jagdeep Sting from Roda AI for setting that up.
• 58:42
  SPEAKER_00: Thanks to you as always for sticking around until the end and listening.
• 58:46
  SPEAKER_00: If you're a fan of the program, you can support us by liking and subscribing on the platform of your choosing and subscribing to our newsletter over at automated.fm.
• 58:57
  SPEAKER_00: And with that, we'll see you next week for another episode of Automated.