Latest / Automated with Brian Heater / From Diffusion Policy to Dexterity: Robotics in Review with Erin McColl of Toyota Research Institute

Transcript

From Diffusion Policy to Dexterity: Robotics in Review with Erin McColl of Toyota Research Institute · Automated with Brian Heater

• 0:09
  SPEAKER_00: This holiday season, we taught robots how to pack supply kits for people in need.
• 0:16
  SPEAKER_00: These robots are now fully autonomous, and we taught them without writing a single line of code.
• 0:31
  SPEAKER_01: What's this?
• 0:32
  SPEAKER_01: Two special episodes, two weeks in a row.
• 0:35
  SPEAKER_01: Consider this our end-of-the-year gift for giving automated such a warm welcome over these past few months.
• 0:42
  SPEAKER_01: This week we are highlighting some of the year's top robot videos along with Toyota Research Institute's Aaron McCall.
• 0:48
  SPEAKER_01: We are also discussing some of the broader trends that shaped a wild, wild year for the industry before unveiling the winner of our 2025 Reader's Poll.
• 1:01
  SPEAKER_01: Don't forget to like and subscribe, and we will see you on the other side.
• 1:06
  SPEAKER_01: Well, thank you for joining us.
• 1:07
  SPEAKER_01: It's been uh it's been a few months since September.
• 1:11
  SPEAKER_01: Yeah.
• 1:12
  SPEAKER_01: Since we last saw you.
• 1:13
  SPEAKER_01: What what have you been up to?
• 1:14
  SPEAKER_01: What has TRI been up to?
• 1:16
  SPEAKER_02: It's coming out to the end of the year, so we've been up to a lot of cool things.
• 1:19
  SPEAKER_02: I've actually been traveling quite a bit over in Japan, working with some of our partners, understanding what's happening over in the ecosystem.
• 1:25
  SPEAKER_02: Uh, we're also getting ready for end-of-year celebrations and wrapping up some of our projects.
• 1:32
  SPEAKER_02: So yeah.
• 1:32
  SPEAKER_01: So for you, robot-wise, automation-wise, AI-wise, would have kind of been the, I guess, defining subjects of 2025.
• 1:42
  SPEAKER_02: Yeah, I mean the big one is clearly at TRI, the diffusion policy.
• 1:45
  SPEAKER_02: That was a big outcome for us to be able to show off this ability to do imitation learning where there is no writing of code and you can teach the robots to do these really complex behaviors.
• 1:55
  SPEAKER_02: I think that's completely sort of shifted the landscape and the conversation, and it's sort of brought TRI into play with many of people really interested in how do they make this work for them in the real world.
• 2:05
  SPEAKER_02: Um, so that's been huge.
• 2:06
  SPEAKER_01: Yeah, so what what is the real world application for diffusion policy?
• 2:11
  SPEAKER_02: Yeah, we don't really know yet.
• 2:13
  SPEAKER_02: I mean, it's still early stages, we're still working on the performance and reliability of it, and obviously that dictates some of the applications.
• 2:19
  SPEAKER_02: So, someone with 99.9% uptime, maybe this is not the right thing to start with, but maybe other applications that make sense where failure is acceptable sometimes.
• 2:29
  SPEAKER_02: So I think you're gonna see changes in the early stages of applying diffusion policy to the later stage applications.
• 2:35
  SPEAKER_02: So maybe where consumer products, consumers might be more flexible as opposed to an industry product.
• 2:41
  SPEAKER_02: But we don't know yet because there's other competing priorities also about data.
• 2:45
  SPEAKER_02: So data is a huge part of these things, and so the first applications need to be helping us collect that data to improve that performance.
• 2:52
  SPEAKER_01: We should maybe back up a little bit uh so the people who are looking at me right now don't kill me.
• 2:56
  SPEAKER_01: Well, we should explain like what TRI is because it occupies a very interesting space, right?
• 3:01
  SPEAKER_01: And and and you alluded to it as far as actually sort of like responding to what people are are requesting.
• 3:08
  SPEAKER_01: So in terms of uh research projects, how do you actually sort of interact with the the public?
• 3:16
  SPEAKER_02: Yeah, so there's there's layers of the public, right?
• 3:19
  SPEAKER_02: There is obviously potentially our industry partners, because we have a very close relationship with Toyota, there's our university partnerships, and there's just more broadly trying to build robotic products that really help society.
• 3:31
  SPEAKER_02: So there's many layers of the groups that we interact with.
• 3:34
  SPEAKER_02: Uh, but we do try very hard to do use-inspired research so that we're not just inventing technologies and then presenting them to the world and saying this is what we have, what do you want to do with it?
• 3:45
  SPEAKER_02: But actually, a huge part of my job is going out to these various groups and saying, what are your needs and challenges?
• 3:50
  SPEAKER_02: What do you need solved that doesn't yet have like a solution in the market?
• 3:55
  SPEAKER_02: And maybe you need that cutting-edge technology, but it's just not there yet.
• 3:58
  SPEAKER_02: So for us, that's potentially people in manufacturing, logistics, it's consumer concepts like about robots in the home, helping the elderly.
• 4:07
  SPEAKER_02: It's also with our university partners, what they're hearing and seeing.
• 4:10
  SPEAKER_02: Really just taking that information and using it to help define our research strategy so that then when the capabilities do come to fruition, we've now known that there's a need for those, and it's much easier to potentially move it out into the world.
• 4:24
  SPEAKER_01: So, hypothetically, how would like slicing an apple fit into that conversation?
• 4:29
  SPEAKER_02: Yeah, uh obviously you've seen some of our videos on diffusion policy.
• 4:33
  SPEAKER_02: So, a lot of those tasks were really set around about being in the home because a big motivating pillar for us is aging society and keeping people having purpose and wanting to keep them capable in their lives.
• 4:46
  SPEAKER_02: What that looks like for different types of people can be very different.
• 4:49
  SPEAKER_02: Maybe in the home they really enjoy cooking, but for arthritis reasons, they can no longer cut the apple.
• 4:55
  SPEAKER_02: So we take a lot of inspiration for trying to look at tasks as good challenge tasks for us to think about, but also in these social environments.
• 5:04
  SPEAKER_02: Maybe there is a use case where now there's a collaborative relationship with this robot who's helping you cook and keep that passion alive and keep you in the home.
• 5:12
  SPEAKER_01: Yeah, I asked partially because obviously, you know, the conversation around robotics in the home has come up again for very for very interesting reasons.
• 5:21
  SPEAKER_01: And I suspect that a lot of people suddenly feel like um it's a very realistic possibility that's that's on the horizon.
• 5:30
  SPEAKER_01: Um as we were going through the process of uh, I guess kind of vetting some of these videos, you know, it's very important to make for us to make sure that this is the real stuff that's happening in in the real world, um, you know, that that it's like evidence-based, that that it can be backed up, you know, and obviously um I think you know, an institution like like TRI, along with the the companies that we're focusing on here, along with you know, the um research institutions, the schools, it's evidence-based work.
• 6:02
  SPEAKER_01: Um and the videos themselves, like they're not super overproduced, right?
• 6:08
  SPEAKER_01: I mean, that's that's what we're looking at in the videos that you've brought to show us today.
• 6:13
  SPEAKER_02: Yeah, it's uh it's interesting because we are seeing a lot of these uh demonstrations for things from the home or around the environment.
• 6:22
  SPEAKER_02: And like I said, we are use-inspired, but we still consider this research and that still gives it a long trajectory.
• 6:29
  SPEAKER_02: So I think that sort of framing of there's potentially there might be in the home and it is really exciting, but I think it's still important to note that these aren't products, and that sort of overproduced, shiny product video is really not what we want because we haven't proven this to ourselves yet that this is viable.
• 6:47
  SPEAKER_02: It is us experimenting, and we are taking an idea to ground that experimentation, but until we really know that this is gonna work and it's practical, uh, it's just insights into what we're learning.
• 7:00
  SPEAKER_01: Yeah, I mean, that's always been a fascinating thing uh to me about TRI is that they're not products and that um, you know, Toyota does seem that that might not seem that Toyota has a vested interest in the institute and has continued to support it.
• 7:16
  SPEAKER_01: Um, you know, if we're talking about like really like long tail research and things that are going to take a long time to not only productize but monetize like ultimately what's in it for a large corporation like Toyota.
• 7:31
  SPEAKER_02: Yeah, uh I think there's a lot of things on the way of that you can claim success for to monetization.
• 7:37
  SPEAKER_02: Like that is a goal, and it's obviously the easiest goal for businesses to understand.
• 7:44
  SPEAKER_02: Exactly.
• 7:45
  SPEAKER_02: But there's a lot we do along that way.
• 7:49
  SPEAKER_02: So um, while Toyota funds us to work on these things and come up with these ideas, sometimes the success comes when they walk the halls with us and they see the robots like we never thought about it from that perspective because we're coming from a very industrial point of view and they have all of the safety requirements, and that sometimes narrows their frame of how they can think about a solution.
• 8:09
  SPEAKER_02: And so often the the big wins are actually some of that inspiration that inspires them to think about their roadmaps differently, where they want to go with robotics differently, and they can use that to even work with their other vendors.
• 8:20
  SPEAKER_02: So it may not be our widget or product necessarily that makes it to that commercialization, but we're shaping that whole landscape for them.
• 8:28
  SPEAKER_01: Yeah, yeah, and and um one of the interesting, I think, pieces of background here that I probably most of our audience is aware of, but Japan specifically has been very ahead of the rest of the world when it comes to assistive robotics.
• 8:42
  SPEAKER_01: And a big piece of that is that they have an aging population.
• 8:46
  SPEAKER_01: And and that's that's been a big piece, I think, of what you've been doing doing here as well.
• 8:50
  SPEAKER_01: And that certainly like that plays into the research that you're doing in the home.
• 8:54
  SPEAKER_02: Yeah, definitely.
• 8:55
  SPEAKER_02: It's a huge focus for Japan because when I first came here, it felt like a futuristic problem of like we're preparing for an aging society.
• 9:03
  SPEAKER_02: But the reality is it's already there.
• 9:05
  SPEAKER_02: They're already feeling the impacts on their labor market, on the care they need to provide to the elderly.
• 9:10
  SPEAKER_02: So we feel it when we go there and we see it.
• 9:13
  SPEAKER_02: And one that's very motivating for us because it there's a real need for this, but it obviously shapes a lot of how we think about what we do.
• 9:20
  SPEAKER_01: Yeah, so uh I one of the things I wanted to talk about that's been of like sort of particular interest to, I think me and anyone who's following humanoids when it talks to when we speak about sort of pragmatism and timelines is the problem of uh dexterity and manipulation.
• 9:37
  SPEAKER_01: Um, one of the videos that you have is uh is is is like a hand reaching into a microwave.
• 9:44
  SPEAKER_01: You told me that the success rate is what, 40 to 50 percent?
• 9:48
  SPEAKER_02: It varies, but you know, with these diffusion policies, we we are seeing that as you increase data and variance of data, the success rate does improve.
• 9:57
  SPEAKER_02: So through the theory, we know that this is a trajectory that we think is going to deliver really cool results.
• 10:03
  SPEAKER_02: But right now, with you know limited sets of data, maybe only 200 examples, through this imitation learning, uh, the reliability is not something you'd put in a factory.
• 10:12
  SPEAKER_02: And similarly, in your home, like uh with pets and children around, not all the failure cases are necessarily known either.
• 10:19
  SPEAKER_02: So I think there is still a gap for us to close to really think about putting these with consumers, whether they're businesses or end customers.
• 10:28
  SPEAKER_01: Why is dexterity, why is manipulation such a hard problem to solve?
• 10:34
  SPEAKER_02: It's yeah, many reasons.
• 10:36
  SPEAKER_02: Uh I mean, if you just look at the traditional, like before we had these deep learning methodologies, it is very difficult for an engineer to really think through the exact like path planning, manipulation, uh, and to architect that for so many different domains of objects, environments, interactions.
• 10:55
  SPEAKER_02: The complexity of that space is huge.
• 10:59
  SPEAKER_02: And so the breakthrough of this is that no longer do we have to think about every single example and hard code it, but we can think about training for all of those examples.
• 11:09
  SPEAKER_02: There's still a lot of challenges in that of getting the data, having robot teachers, teleoperation.
• 11:13
  SPEAKER_02: There's a whole suite of issues there.
• 11:16
  SPEAKER_02: But dexterity is something that we as humans rely on so deeply.
• 11:21
  SPEAKER_02: I mean, tools are the thing that sort of made us intelligent and able to build the world around us that we have.
• 11:27
  SPEAKER_02: And that variability of the things that we can work with makes us so generalizable that we just have not reached that with robots yet.
• 11:36
  SPEAKER_01: So, so you know, TRI sees this problem, dexterity.
• 11:40
  SPEAKER_01: Obviously, a lot of people see the problem.
• 11:41
  SPEAKER_01: And is it how do you decide which issues you're focused on and how much time to devote to them?
• 11:48
  SPEAKER_02: Yeah.
• 11:49
  SPEAKER_02: Uh, this is something that we think about a lot going back to that sort of finding the needs.
• 11:53
  SPEAKER_02: There are a lot of problems out there, particularly around the dexterity, um, more problems than we could possibly solve.
• 12:00
  SPEAKER_02: And so there is one degree of finding a problem that also matches kind of the level of technology that we think we could do.
• 12:08
  SPEAKER_02: Like we've definitely seen examples of things where, you know, super flexible items in tight spaces, like our engineers look at that problem, they're like, no way, that's really hard.
• 12:17
  SPEAKER_02: We're not there yet.
• 12:18
  SPEAKER_02: But maybe something that's uh a little bit less dexterous, maybe more fixed objects, we so, okay, maybe, maybe this is the right level.
• 12:26
  SPEAKER_02: It's hard enough that we don't know if we do it, but maybe we leverage their technical expertise to sort of, yeah, I think I think we could try this.
• 12:35
  SPEAKER_02: That's how we identify challenge problems from the technical side.
• 12:39
  SPEAKER_02: But then there's also needs driven of how big is this solution going to be if we do solve it?
• 12:46
  SPEAKER_02: If it's a one-off thing, uh, do you really want to invest all of that money into something that's going to solve a very niche problem?
• 12:54
  SPEAKER_02: Or is this particular dexterous problem something that we think could scale?
• 12:58
  SPEAKER_02: To could really go to, oh, if we could do this type of object, that means we could go across these three industries or three, these three things in home.
• 13:07
  SPEAKER_02: So robotics is just such a capital-intensive endeavor that you want to make sure you're solving things and investing where you think that return is.
• 13:15
  SPEAKER_02: And even though we're a research institute and we don't think about commercializing things, it is a really good analogy for us to think about the scale of problem.
• 13:23
  SPEAKER_01: Yeah, I'm kind of I'm curious, like if I take a look at TRI and the way you you have your lab set up, like how much I can kind of like map that to sort of the broader research that's happening in in the community.
• 13:37
  SPEAKER_01: You know, like as you know, I've been to the offices um a number of times over the years.
• 13:42
  SPEAKER_01: You know, I was here when um there was a kitchen set up, I was here when it was like supermarket rose.
• 13:47
  SPEAKER_01: And I think at this point it's it's an industrial setting, is that right?
• 13:50
  SPEAKER_02: We have a bit of both.
• 13:51
  SPEAKER_02: So we still have our supermarket setting, and that's really it's it's a really great challenge problem.
• 13:56
  SPEAKER_02: It gives us a way to work with tons of different types of objects where the robot is coming in to pick up a bag of rice or a really big gallon container of bleach or fresh produce.
• 14:08
  SPEAKER_02: And so that still exists, and we still have sort of the kitchen type environments, and that's where a lot of that research is of do we have the capabilities to do this?
• 14:18
  SPEAKER_02: And that environment also gives us something that's very measurable.
• 14:20
  SPEAKER_02: We send those same robots out into a real store every quarter, give it a shopping list, and measure success.
• 14:27
  SPEAKER_02: And so that's at the research end of the scale.
• 14:30
  SPEAKER_02: But as those capabilities evolve in that environment, we're then learning how to, okay, what is it about our segmentation of the objects, our grasping strategies, our navigation, that we can take those components and maybe put them into an industrial setting?
• 14:45
  SPEAKER_02: Because we've proved their repeatability, reliability in that challenge setting.
• 14:50
  SPEAKER_02: Can we move that into an industrial setting?
• 14:52
  SPEAKER_02: And this is not necessarily to build a product, but it's to prove can we go to that next thing, that thing that's a little bit more grounded, that has maybe requirements that we could truly gather and start measuring ourselves in a really grounded way.
• 15:04
  SPEAKER_01: Yeah, to it to take it back to one of the things I was talking about at the top of the conversation, and um something I've been referring to, and I'm not gonna uh throw shade on your parent company, but as kind of the like car commercialification of a lot of these robot videos, um, is the setting of what I would say are maybe unrealistic expectations about what robots can do in the short term.
• 15:33
  SPEAKER_01: Um what role can TRI and other researchers play to like help, I guess, reset some of those expectations?
• 15:45
  SPEAKER_02: Yeah.
• 15:45
  SPEAKER_02: Yeah, it's something that we think about a lot here because when you're at the the leading edge, you also see all the failure cases and you get really cool videos, and maybe you get a really great demo and it does something really impressive and you caught it on a camera and you put it out to the world.
• 16:01
  SPEAKER_02: But the reality is how many takes did that take?
• 16:05
  SPEAKER_02: How much of it is teleoperated, how much of that scene is crafted to enable that really flashy video.
• 16:12
  SPEAKER_02: So I think part of our role as a research institute is to be really honest with ourselves and to be honest with our audience as well.
• 16:20
  SPEAKER_02: So when we think about publishing our videos, we want to we want to show off the cool things that we're doing, but we also want it to be grounded in reality.
• 16:28
  SPEAKER_02: If these are all the ways it doesn't work, and these are the things we're still learning.
• 16:32
  SPEAKER_02: And these are the things that it performs really well at, and we're seeing a trend in the positive direction, but there's still more to do.
• 16:39
  SPEAKER_02: So yeah, I think it's quite important sort of our role in that ecosystem, both for just general audiences, but also for Toyota.
• 16:48
  SPEAKER_02: That's part of what they invest in us in, that we are the experts in robotics and AI for them, that they can come to us and ask, how real is this?
• 16:57
  SPEAKER_02: Is this something that we should be investing in?
• 16:59
  SPEAKER_02: Is this something that we should be really pursuing quite hard?
• 17:02
  SPEAKER_02: So yeah, we take it uh yeah very seriously to be able to think about that.
• 17:05
  SPEAKER_01: This is a bit of a big campaign of mine, and not just, but also from a content perspective, but normalize the blue bar reel.
• 17:11
  SPEAKER_02: Yeah.
• 17:13
  SPEAKER_02: I I think it it says a lot of uh, okay, how many times did it fail and what are the failure mechanisms and showing those videos just as much as the ones where it it's impressive because both ends of that spectrum have value.
• 17:29
  SPEAKER_01: What do we what do you feel about um normalizing um, if not torturing, at least annoying robots?
• 17:37
  SPEAKER_02: It's interesting.
• 17:38
  SPEAKER_02: I mean, there's a long history of this sort of torturous behavior to the robots of the push and the pull and distracting them.
• 17:44
  SPEAKER_02: And it does have a reason for it.
• 17:47
  SPEAKER_02: And that idea is people are trying to demonstrate the rely, like the robustness to the robot to respond.
• 17:54
  SPEAKER_02: Um, and that's both in its maybe stability control, but also its ability to handle disturbances to the environment.
• 18:01
  SPEAKER_02: Um I've always thought along history of like, you know, when our robot overlords do eventually watch YouTube and see these things of us kicking robots, how are they gonna take it?
• 18:12
  SPEAKER_01: I probably wrote that story if you tap them in the picture if yeah.
• 18:15
  SPEAKER_02: Exactly.
• 18:16
  SPEAKER_02: Um so there there is a reason, but I think often the general consumer of these videos may not even understand the reason of like, why are we pushing these robots around?
• 18:25
  SPEAKER_02: And but it it starts from a place of it's a way for these scientists to show the control systems of their robots and how performant they really are.
• 18:34
  SPEAKER_02: Uh, do I think it's a good thing?
• 18:36
  SPEAKER_02: Maybe we've matured beyond that, and there's newer, better ways to show that, but it seems to have stuck in a lot of these productions.
• 18:43
  SPEAKER_02: Sure, yeah.
• 18:43
  SPEAKER_01: I mean, uh a cheeky question in this context, I asked specifically because of the work that you were doing with Botswynamics and the hockey stick.
• 18:51
  SPEAKER_02: Yeah.
• 18:52
  SPEAKER_02: I mean, that hockey stick is famous at this point on its own.
• 18:55
  SPEAKER_02: So I think it has to show in most of the videos, and it's for the same reason of how well can these robots function around these disturbances.
• 19:03
  SPEAKER_01: Yeah, can you talk a little bit about what the hockey stick was doing or what the person with the hockey stick was doing?
• 19:09
  SPEAKER_02: Yeah, so in this in this video, we're showing TRI's diffusion policy applied to Boston Dynamics eAtlas platform.
• 19:16
  SPEAKER_02: So this was a partnership for two research groups to come together and show with their control system and their platform and teleoperation, can we take our diffusion policy, our large behavior models, and apply it to this humanoid structure?
• 19:30
  SPEAKER_02: And so in that video, what you're seeing is a long horizon task where the robot is unpacking these objects into a cart or a shelf, and the hockey stick is closing the bin, pulling it in and out.
• 19:42
  SPEAKER_02: Uh, this is really showing that disturbance to the scene, that the robot is able to respond to these disturbances, that it's not a pre-programmed situation where for every single uh disturbance there's an if statement of if the lid is closed.
• 19:57
  SPEAKER_02: Like that's not scalable and doesn't work.
• 19:59
  SPEAKER_02: So by having that interfering actor, which is the hockey stick in this case, you can see that the robot in real time very quickly is responding and understands what the true state needs to be and how it works around these disturbances.
• 20:12
  SPEAKER_01: So talking of, I guess, uh sympathizing with the robot, I think that's something that we've kind of done to ourselves, right?
• 20:19
  SPEAKER_01: Because like we've gone out of our ways, our way to make them more humanoid, or um obviously we have all these biologically inspired robots, which brings us to our first video.
• 20:30
  SPEAKER_01: We have a a non-anthropomorphic robot from uh Apple.
• 20:35
  SPEAKER_01: This is uh one of the research projects that they were working on.
• 20:38
  SPEAKER_01: And and and we both found this interesting, but I I think for for different reasons.
• 20:41
  SPEAKER_01: I found this interesting because you know I've been writing about Apple for a really long time, and there's rumors that they're working on uh on a consumer robot and behind the scenes, and this is um you know a rare instance where Apple actually kind of has to show their hand because that's that's what you do when you're working on research projects.
• 21:01
  SPEAKER_01: Why why did this jump out at you?
• 21:03
  SPEAKER_02: I really liked this one because it was less biologically inspired.
• 21:08
  SPEAKER_02: I think, you know, with the lamp, uh it it has a very playful sort of attitude to it, and you can see there's this uh interaction with this person that it's clearly trying to take a lot of what we're learning about uh robotics and uh robotic user interface, and instead of this humanoid form factor, it's a piece of furniture.
• 21:30
  SPEAKER_02: Uh which for me seems like a pathway to more robotics in homes as opposed to the generalized platform.
• 21:39
  SPEAKER_02: Uh it it seems more approachable to me.
• 21:41
  SPEAKER_02: It's a lamp.
• 21:42
  SPEAKER_02: Um, I would love it if all of my furniture was able to interact with me in fun ways where I could speak to it, come close, go away.
• 21:48
  SPEAKER_02: So I I thought that was quite cool.
• 21:50
  SPEAKER_01: Yeah, it's certainly you'd you'd probably be uh I think much nicer to your furniture just in general.
• 21:56
  SPEAKER_02: I think it's way less imposing in your Home as well.
• 22:00
  SPEAKER_02: Like the idea of a large, heavy humanoid robot walking around kind of creeps me out a little bit in my house.
• 22:06
  SPEAKER_02: But the idea that my lamp like looks at me when I walk in the door and maybe like lights up, like that's a cool concept that seems more friendly.
• 22:14
  SPEAKER_01: Yeah, yeah, not to go too far down that road, but uh you know, obviously like safety is is a really uh big concern when it comes to humanoid robots, and especially when it comes to humanoid robots in a home.
• 22:24
  SPEAKER_01: We were talking a little bit about elder care um and why that's uh gonna be a potentially a big market for for home robots.
• 22:32
  SPEAKER_01: Um safety has been I I I would say um uh you know, speaking I I guess totally subjectively, that um as far as like humanoid companies go, that uh I think agility has been doing a really good job being kind of at the forefront of safety when it comes to humanoids.
• 22:51
  SPEAKER_01: But a lot of progress I think has been made just with all the companies across the board.
• 22:55
  SPEAKER_01: Um so our second video that we're looking at right now is from Agility.
• 23:00
  SPEAKER_01: Um it's a little bit longer.
• 23:01
  SPEAKER_01: We've cut this one down a bit.
• 23:03
  SPEAKER_01: Uh so this one is another torturing the robot video to a certain extent.
• 23:08
  SPEAKER_01: Um, why did this one catch your eye?
• 23:11
  SPEAKER_02: I think what I liked about this one is is to your point, agility is doing a good job at really thinking about this safety case uh to get their robots out in the world.
• 23:19
  SPEAKER_01: They're also torturing, to be fair, they're also torturing their employees.
• 23:22
  SPEAKER_01: That is true.
• 23:22
  SPEAKER_02: They made their people do it as well.
• 23:25
  SPEAKER_02: And actually, that's what I kind of liked about it.
• 23:27
  SPEAKER_02: I think in all these other robot videos, we just see this antagonistic, someone poking it, kicking it.
• 23:32
  SPEAKER_02: But in this video, they made an attempt to really talk through why they are doing this?
• 23:36
  SPEAKER_02: Why is it important?
• 23:37
  SPEAKER_02: And they tried to think about the science of the right experiment to actually test this.
• 23:41
  SPEAKER_02: Yeah.
• 23:41
  SPEAKER_02: And I think that's I think that's a good thing for people to understand maybe why roboticists are torturing their robots.
• 23:48
  SPEAKER_01: Yeah, why why um, you know, after like having watched this video, why is that sort of like uh that magic trick of pulling the um, you know, the the tablecloth out from under the table?
• 23:58
  SPEAKER_01: Why is that a good way to test the robot?
• 24:00
  SPEAKER_02: I think because it's one so fast and there is no way for the robot to know.
• 24:05
  SPEAKER_02: Like they talk about in this video, there is no perception, there is no, they can't see a stick or a human coming.
• 24:10
  SPEAKER_02: It's purely control-based, reactionary, um, literally pulling the rug out from under them.
• 24:16
  SPEAKER_02: And I think the fact that they did it with a human as well really shows that how quick can that response time be to uh gain its stability.
• 24:24
  SPEAKER_02: And there's so much variation of how when you pull it out from under that, uh, how much it could potentially shift.
• 24:31
  SPEAKER_01: Another, I guess, research institute that has been doing some some very fascinating work in recent years is is Disney.
• 24:38
  SPEAKER_01: Um, you know, they they actually, I think they, as far as I know, they showed this robot that we're seeing right here, which looks, I think it's probably like a droid, a Star Wars droid.
• 24:48
  SPEAKER_01: It looks like it, yeah.
• 24:49
  SPEAKER_01: Yeah, they showed this off at uh at GTC last year.
• 24:52
  SPEAKER_01: Um and they're really doing, I mean, again, talk about like anthropomorphizing, talk about like personifying a robot, like they're really projecting some emotions onto this thing.
• 25:05
  SPEAKER_02: Yeah, it's it's it's cute, right?
• 25:07
  SPEAKER_02: Like it definitely gives you those Disney vibes of bringing uh to life a character in a way that I think is new and exciting.
• 25:15
  SPEAKER_02: What I I thought was really cool about this is obviously they're mapping happy, sad, angry, shy.
• 25:21
  SPEAKER_02: But they trained it based on a teleoperator.
• 25:24
  SPEAKER_02: So originally they had someone teleoperating an interaction, someone who was used to having this robot express happy, sad uh in the way they do for their production.
• 25:35
  SPEAKER_02: And then they took that and they trained models to now have it instinctively react and obviously testing it out with a number of people to see how that interaction went.
• 25:43
  SPEAKER_02: And I thought that was kind of cool, the using the human experience of how to operate these robots to have personality and then embedding that experience into the program.
• 25:52
  SPEAKER_01: Yeah, and another part that I think is really interesting because I think what forms like most of the work that they're doing, I because I think this is probably out of Zurich, a lot of stuff they do is is the these are for um for theme parts, right?
• 26:05
  SPEAKER_01: Like the what's it, Galaxy's Edge, is that the SARS theme park?
• 26:08
  SPEAKER_01: So um, you know, these are meant to work fairly autonomously, so you you project like a mood on them so they can have that mood as they operate autonomously in the world.
• 26:18
  SPEAKER_02: Yeah, and you can see just the practicalities of making that scalable.
• 26:21
  SPEAKER_02: Of if every robot needs a teleoperator out on the floor, then it doesn't really work.
• 26:26
  SPEAKER_02: But now if you can let these robots loose in the park and they're responsive to the people, it seems like a really cool project to sort of test out these ways of making these uh these robots independent.
• 26:38
  SPEAKER_01: So we're going from Disney Imagineering to uh former Disney Imagineer, Scala Valley.
• 26:44
  SPEAKER_01: This is uh cartwheel robotics, a company out of uh the Reno-based, a Reno-based company.
• 26:50
  SPEAKER_01: Um, this is them showing off Yogi, uh or at least like an early conception of Yogi, a home robot that they've been working on.
• 26:57
  SPEAKER_01: Um different, you know, and and I obviously this is what's interesting at the beginning of the video.
• 27:02
  SPEAKER_01: We're seeing some of these, uh, we're seeing, you know, the uh apptronic uh robot, we're seeing some of the humilities that we've already seen in the world.
• 27:11
  SPEAKER_02: Yeah, what was a really big takeaway for me on this one is when I think about social robots at a societal level, it's really clear like why there's a need with like loneliest epidemics, aging society.
• 27:23
  SPEAKER_02: Like there's clearly a need for social robots, but it's been hard for me to see so far at the individual level that same need.
• 27:31
  SPEAKER_02: Like how many people are going out and saying I need, you know, tens of thousands dollar robot to be social.
• 27:36
  SPEAKER_02: But I think companies like this are doing really cool work to flip that, of if that robot now is approachable and interactable and exciting, that at that individual level we might start seeing more of those uptake of social robots.
• 27:50
  SPEAKER_02: And I think this guy's a really cool example of that.
• 27:52
  SPEAKER_01: Yeah, we were talking, you and I are talking before about um you know successful Australians in robotics, and and Grace Brown came up as somebody I think who's who's doing a really good work as far as making um you know human accessible robots.
• 28:09
  SPEAKER_02: Yeah, it's really cool to see what they're doing.
• 28:11
  SPEAKER_02: As an Australian, it's it's super exciting to see more and more roboticists uh having these great successes.
• 28:16
  SPEAKER_02: And yeah, with their robot, uh, I I can't wait to see where they go with it.
• 28:20
  SPEAKER_02: So, similarly in that social robot space, um, maybe intuition robotics is another example of this, which our sister company, Twitter Ventures, um, has funded and worked with them.
• 28:29
  SPEAKER_02: I think they have their LEQ, similar older adults, how can we work in that social environment to really bridge that gap and maybe help with some of these big societal challenges?
• 28:38
  SPEAKER_01: So you and I, as I said earlier, we met for the first time in September in Seattle at our big uh humanoid robotics forum.
• 28:48
  SPEAKER_01: Um and I would say I'm allowed to say this because I I helped program the event.
• 28:55
  SPEAKER_01: Not a big deal.
• 28:57
  SPEAKER_01: Uh but one of the one of the more um like fascinating uh sessions that we had was this uh UCSD, the surgery uh robot, robotic surgeon.
• 29:07
  SPEAKER_01: This is um some work that they're redoing out of UC San Diego, and it's really sort of taking the idea of a robot surgeon to potentially like a whole nother level.
• 29:18
  SPEAKER_02: Yeah, yeah, this is really cool research to see because I think what's interesting is they are we're doing quite dexterous tasks.
• 29:24
  SPEAKER_02: Tasks that honestly I didn't expect us seeing from robots for a long time.
• 29:29
  SPEAKER_02: And obviously it's early stage, there's research, but it's a really cool proof of concept to see with these platforms, this dexterity, what tasks can we push the limit of in medicine?
• 29:40
  SPEAKER_02: Um, and it's it's quite, I guess, sort of shocking to think of this idea of like going to the doctor and getting stitches from a robot or I mean talk about like the you know whether you want the human touch or not.
• 29:52
  SPEAKER_01: Yeah, you know, this is this is definitely gonna be a debate.
• 29:55
  SPEAKER_02: I mean, like robotics is not new in medicine.
• 29:57
  SPEAKER_02: Like you look back at the Da Vinci robots and 20 years, 25 years.
• 30:02
  SPEAKER_02: Yeah, and there's just a long history there because medicine is a high precision um task.
• 30:07
  SPEAKER_02: And so having a robot with like millimeter precision and accuracy, it makes a lot of sense.
• 30:12
  SPEAKER_02: Um, those tend to be in surgeries where I guess the patient is usually asleep and it's obviously with a doctor in the loop.
• 30:19
  SPEAKER_02: So I can see the benefit for robotics there.
• 30:22
  SPEAKER_02: Um, but yeah, maybe there's nurses alongside these robots helping out uh with the patient interaction.
• 30:29
  SPEAKER_01: Yeah, one of the things I think that's really interesting about uh work like this, um, and uh Professor Michael Yipp is um was was uh leading this project, um, is that I'd I he would probably disagree with this, but I think maybe to a certain extent this is also almost like an art project or like a social commentary that gets us having that conversation.
• 30:55
  SPEAKER_01: Um we had, you know, we we also had uh Brad Porter at the event to kind of serve as I guess like a counterbalance to some of these humanoid conversations and you know, whether like, I don't know, is this overkill?
• 31:06
  SPEAKER_01: Do we actually like need a humanoid in the operating room?
• 31:09
  SPEAKER_02: That was actually my initial reaction when I watched this video.
• 31:12
  SPEAKER_02: I was a little bit like, is that what we want?
• 31:15
  SPEAKER_02: And I do think there is a place for that of in these research environments.
• 31:19
  SPEAKER_02: I think we can come up with really cool technical solutions, but there is an importance for that social feedback and commentary.
• 31:26
  SPEAKER_02: And as researchers, we should be listening and hearing from many customers, from the potential patients, from the doctors, from the nurses.
• 31:34
  SPEAKER_02: When they see this for the first time, how do they feel about it?
• 31:37
  SPEAKER_02: Is there a place?
• 31:38
  SPEAKER_02: It's actually a technique we use quite often is to show the research and then just see what is that reaction from people.
• 31:44
  SPEAKER_02: So I feel similarly about this video that I think there's a place for that in air of how do people feel about a robot doing their stitches in a surgery room.
• 31:52
  SPEAKER_01: All right, and then uh finally, this brings us to, I guess, uh our winner voted on by uh readers of the newsletter and watchers of the show of the same name.
• 32:02
  SPEAKER_01: Uh, this is work being done out of the RAI Institute uh over in Cambridge.
• 32:07
  SPEAKER_01: Um, I don't know, I I just really like this because they're just like, hey, let's just do like an entirely new weird form factor and see if we can get it to work in the real world.
• 32:16
  SPEAKER_02: Yeah, I feel similarly.
• 32:17
  SPEAKER_02: It was kind of cool.
• 32:18
  SPEAKER_02: I was the same with the lamp.
• 32:19
  SPEAKER_02: Like I love seeing these different form factors that are trying new things and really pushing the limits of various stability and mobility tasks.
• 32:28
  SPEAKER_02: So yeah, I thought this was really, really cool research.
• 32:31
  SPEAKER_01: Yeah, I think I'm I'm trying to remember, I think it was like, you know, in order to say it was success, a success, we have to have it go like to to to pull it off like 12 times in a row, something like that.
• 32:44
  SPEAKER_02: Yeah, I actually that resonated a lot because uh similar, we talked about our grocery challenge of actually setting ourselves tasks that are measurable to know that our research is it doing things that are performant reliable.
• 32:57
  SPEAKER_02: And so it was really yeah, nice to hear they thought about that quite similarly and how they measure.
• 33:02
  SPEAKER_02: Is this doing what we need it to do?
• 33:04
  SPEAKER_01: Yeah, I I wonder, you know, I this is something I and I'd like to get your take on this, because uh, you know, obviously, like, you know, you you and your team are doing some uh sort of similar work in uh similar area.
• 33:17
  SPEAKER_01: Um what's your sense of how much of a project like this is like, hey, let's see this cool robot in the world and how much of this is actually like useful research?
• 33:28
  SPEAKER_02: I think it's a mix of both.
• 33:30
  SPEAKER_02: I mean, there's definitely things that I can't speak exactly to in this project, but they're probably pushing their capabilities of what can we do in new ways and then new capabilities we can develop in this.
• 33:41
  SPEAKER_02: And so they're uncovering sort of breakthroughs.
• 33:44
  SPEAKER_02: Uh, but at the same time, there's obviously uh when you do that, you have an opportunity to put these form factors and these breakthroughs out to the world, and you get a lot of feedback if you're if you're engaging various people.
• 33:55
  SPEAKER_02: And it's quite surprising sometimes, some of the things we put out to the world and the reactions we do get of, oh, I can see a place for that.
• 34:02
  SPEAKER_02: And often in the things that we don't always expect, and then it goes the other way as well of, oh no, we're not actually that interested in this vision model or this type of sensor, and you just don't get the same uptake.
• 34:12
  SPEAKER_02: So I think it's a virtuous cycle, um, and there's a lot to be said about putting the research and the capabilities out there, but then intentionally getting that uh like product market fit feedback.
• 34:23
  SPEAKER_02: It's not a product yet, but it's like solution needs feedback.
• 34:26
  SPEAKER_02: Uh yeah, so I think there's uh a lot they could do with sort of industries that you may not ever expect that may be into these things.
• 34:33
  SPEAKER_01: Yeah, uh we're so we're seeing a lot of simulation in this video.
• 34:36
  SPEAKER_01: And you know, I think a point that they're making in here as well is like sim obviously like simulation is only gonna get you so far.
• 34:42
  SPEAKER_01: And we're seeing a lot of bloopers, which is again, I I approve of very nice.
• 34:45
  SPEAKER_01: Yep.
• 34:46
  SPEAKER_01: Um simulation is only gonna get you so far, you actually like have to build a thing and put it out in the real world to really know how it's gonna perform.
• 34:53
  SPEAKER_02: Yeah, it was a really great point they make, and there's like that interaction between like the ML, the simulation, and the hardware teams.
• 35:00
  SPEAKER_02: And obviously, simulation has a huge place in those early stages when you won't want to be destroying your robot every time.
• 35:06
  SPEAKER_02: Finding those failure cases is really important, but these are expensive robots, they take a lot to build, you don't want to be doing that early.
• 35:13
  SPEAKER_02: So simulation plays a huge role there.
• 35:15
  SPEAKER_02: But then obviously, as that um becomes more mature, sim to reality gap, now can you get it onto a platform out in the real world and start closing that gap with the real thing?
• 35:24
  SPEAKER_02: So I think uh teams are figuring out that relationship of how they can leverage all these various resources to build these systems.
• 35:31
  SPEAKER_01: Yeah, but broadly speaking, I think the the argument that you're making earlier is it's that kind of like the space race conversation, right?
• 35:38
  SPEAKER_01: Of like, regardless of what you think the ultimate outcome of, you know, trying to get a rocket to the moon is like all of these other sort of interesting things have been produced with all that work that was being done.
• 35:52
  SPEAKER_02: Yeah, exactly.
• 35:54
  SPEAKER_02: And sometimes we set ourselves these big system tasks in robotics of, you know, there's something that needs to be solved, and maybe take multiple robots, really complex system, and you push towards that task and you have these breakthroughs, but it might be the sensor that you developed or the specific vision algorithm or a navigation thing.
• 36:12
  SPEAKER_02: And that's the thing that actually forks into its own product line or its own potential pathway.
• 36:18
  SPEAKER_02: Uh we we've seen that a number of times, and I think that's also a big part of putting this out to the world because sometimes various customers they go, but how did you do this thing?
• 36:28
  SPEAKER_02: And I'm really interested in that part of the robot system.
• 36:30
  SPEAKER_02: Uh and you don't always can predict what that's gonna be.
• 36:33
  SPEAKER_01: Well, uh, thanks to all of our submissions, uh, you're all winners in my book.
• 36:39
  SPEAKER_01: Uh we will be, I think we'll be featuring them on YouTube.
• 36:42
  SPEAKER_01: Yeah, yeah, okay.
• 36:43
  SPEAKER_01: We'll we'll have them all up on YouTube.
• 36:44
  SPEAKER_01: You can check them all out up there.
• 36:47
  SPEAKER_01: Um, a lot of really cool work being done.
• 36:50
  SPEAKER_01: Uh again, obviously a lot of really cool work being done over at TRI.
• 36:54
  SPEAKER_01: Is there anything that you can sort of tell us about as far as you know what maybe 2026 might hold?
• 37:01
  SPEAKER_02: Yeah, so one of the things in 2026 that I think is is pretty exciting is we're we've been mount walking the halls, I guess, with Toyota, and we're starting to see some really cool applications where we can actually try and solve some of these challenges.
• 37:14
  SPEAKER_02: So uh yeah, walking the halls, understanding what is hard for the people, potentially the robots can be solving.
• 37:21
  SPEAKER_02: Uh, and we're seeing the opportunities here to really amplify uh people in manufacturing and logistics, and so we're trying to apply some of these breakthroughs into that space and try and really make uh yeah, make an impact with these robots to solve real-world problems.
• 37:38
  SPEAKER_01: Aaron, it's been a pleasure.
• 37:40
  SPEAKER_02: Thank you so much.
• 37:44
  SPEAKER_01: Thank you so much to Aaron and Wendy at TRI for setting up the first ever in-person episode of Automated.
• 37:53
  SPEAKER_01: Thanks as always to Jana and everyone at A3 who helps make this show every week.
• 37:59
  SPEAKER_01: Thanks to the viewers, that's you who's viewing this for making automated what it is.
• 38:04
  SPEAKER_01: And of course, uh, thanks to all of our great guests.
• 38:08
  SPEAKER_01: We have some more great guests lined up, including the first ever live automated that will be happening next month in Orlando.
• 38:16
  SPEAKER_01: As always, please like and subscribe, and we will see you in 2026.