Latest / RobTalk / World Models & the next step for Physical AI

Transcript

World Models & the next step for Physical AI · RobTalk

• 0:00
  Speaker 1: we start seeing people adding depth, for example to these models or a sense of touch.
• 0:07
  Speaker 0: Okay?
• 0:07
  Speaker 0: We have this hardware constraint that you typically only like say one or two GPUs on our robot.
• 0:13
  Speaker 0: there are people who actually said were not solving the problem.
• 0:15
  Speaker 0: right now let's put a big GPU cluster next to it and then we had five hundred millisecond latency in that.
• 0:21
  Speaker 0: still okay if he has all other layers in place
• 0:24
  Speaker 1: how do
• 0:25
  Speaker 0: decide what to keep and how to compress.
• 0:28
  Speaker 1: I mean, honestly through learning right?
• 0:30
  Speaker 1: So you basically learn the architecture such that the compression mechanism is just very efficient.
• 0:40
  Speaker 0: Hello and welcome back to Rob Talk!
• 0:42
  Speaker 0: The podcast about physical AI and robotics.
• 0:45
  Speaker 0: And today we're gonna talk about the latest and greatest About what's going on What is evolving in the context of physical AI and robotics in two thousand twenty six.
• 1:01
  Speaker 0: And we cover things like world models, what is that?
• 1:06
  Speaker 0: Is it the thing where the answer's always forty-two or something else?
• 1:11
  Speaker 0: for that I have Felix with me, Felix Frank our staff engineer on the robot intelligence team at Robco.
• 1:19
  Speaker 0: Yeah, Felix!
• 1:22
  Speaker 0: What is the gist of Two Thousand Twenty Six?
• 1:28
  Speaker 0: The Autonomous Robotics Podcast.
• 1:30
  Speaker 0: Physical AI, no theory just reality.
• 1:38
  Speaker 1: I think we can start at a high level and then go down into more depth on some of the topics.
• 1:44
  Speaker 1: but i guess there's few trends where seeing people switching from VLAs or vision language action models to world model video models And will talk about that.
• 1:58
  Speaker 1: But theres also other things like having policies that can deal with longer horizon tasks, so more high level abstract tasks which have to be completed by adding memory or things like.
• 2:14
  Speaker 1: So I think there's a lot of really exciting developments in the space right now.
• 2:19
  Speaker 1: There is a lot going on and yeah, twenty-twenty six is definitely big year in physical AI.
• 2:25
  Speaker 0: All right, yeah.
• 2:27
  Speaker 0: We've seen this huge jumps over the course of a year.
• 2:30
  Speaker 0: so let's start to decode it little bit.
• 2:32
  Speaker 0: when we look back people have been controlling robots for decades and since few years ago also with learning.
• 2:42
  Speaker 0: but what was thing that really changed things in these two thousand twenty three, two thousand forty four time frame?
• 2:50
  Speaker 1: I think you can even start earlier like end of twenty-twenty where previously we had a lot of small, special models for one specific task.
• 3:02
  Speaker 1: And then with the language models becoming so good and so prominent I mean...I think we all remember chat GPT like coming to the public and people seeing oh there's actually value in big models and a lot data in scaling this entire process right?
• 3:22
  Speaker 1: Then robotics followed actually quickly after.
• 3:25
  Speaker 1: So there's works like RT-One, RT-Two, Palm E where the idea really was okay.
• 3:33
  Speaker 1: we can kind of take the same architecture that we have in language models which is transformers and they don't just have to be used for language but they're essentially very good at any type of sequential data and robotics data in and off itself.
• 3:53
  Speaker 1: You have joint measurements, you have images.
• 3:57
  Speaker 1: Maybe force measurement like descriptions of what is going on and all that is a sequence And basically also try to model and solve these problems with transformers.
• 4:11
  Speaker 1: So in twenty-twenty two and twenty three were really when people started saying okay What if we take this paradigm of gathering data making bigger models put that to robotics.
• 4:24
  Speaker 1: And, and that's kind of where the origin off vision language action models is what we these days call like well let say The Gold Standard Of What We Were Doing In The Past Years.
• 4:37
  Speaker 1: um I think so if i structured a bit there are two directions.
• 4:43
  Speaker 1: One Direction Is What I Just Talked About Which Is This Like Larger Models More Data Yeah, learn from what the language models were doing.
• 4:51
  Speaker 1: And then the other big paradigm which I think we have to talk about is... Is what we call these days imitation learning?
• 4:59
  Speaker 1: Which has actually been around robotics for a really long time.
• 5:03
  Speaker 1: but in like twenty-twenty three people started to really push that forward.
• 5:08
  Speaker 1: so they where papers form DeepMind called Aloha and Liege For Example Where They Really Said Okay How Far Can We Push The idea of I show the robot a certain task, it can be dexterous.
• 5:23
  Speaker 1: It can be a certain pick-and-place tasks that can have soft objects or something like this.
• 5:27
  Speaker 1: and how far Can i push?
• 5:29
  Speaker 1: This just by sheer number Of demonstration.
• 5:31
  Speaker 1: so i remember reading that paper back then And i think they had um that thirty five operators for Like a bimanual setup So two robot arms and they were collecting data over eight months Um and then using basically what is an early version of like a diffusion policy model to basically try to learn this.
• 5:54
  Speaker 1: So if I summarize, right?
• 5:57
  Speaker 1: There's these two directions.
• 5:59
  Speaker 1: one is okay we want really go into learning from data and scale up our data collection effort.
• 6:07
  Speaker 1: the other one then large models.
• 6:11
  Speaker 1: there also an important step which you quickly realize that the amount of robotics data that we have is just limited.
• 6:23
  Speaker 1: And very costly to collect, right?
• 6:25
  Speaker 1: We know that collecting tele-operation data is quite slow and then... That's also where vision language action models come from.
• 6:34
  Speaker 1: The idea was okay.
• 6:35
  Speaker 1: there are other data modalities out there like text or images which we have many magnitudes more Right!
• 6:44
  Speaker 1: Then we had really good examples on how these models learn amazing capabilities with JetGPT and so on.
• 6:53
  Speaker 1: And you basically say, okay how can I take this knowledge which is available in a pre-trained model right?
• 7:01
  Speaker 1: Then combine it with the imitation learning paradigm and robot data.
• 7:05
  Speaker 1: that's kind of what where vision language action models come from.
• 7:10
  Speaker 0: So different developments.
• 7:14
  Speaker 0: first we had this jet GPT moment four years ago, right?
• 7:18
  Speaker 1: Pretty much.
• 7:19
  Speaker 0: Incredibly long time ago has changed so much.
• 7:23
  Speaker 0: So we suddenly saw that more data helps in this whole language space and it starts to spill over And from a vision world there wasn't really that much data.
• 7:35
  Speaker 0: There's the bootstrapping problem but their labs also combined their datasets.
• 7:41
  Speaker 0: Suddenly We had something like ten thousand or fifty thousand hours somehow with some embodiment.
• 7:47
  Speaker 0: Okay, so now this is put together and what was the quality that you could achieve?
• 7:54
  Speaker 0: This was the first time where we had these really large models
• 7:57
  Speaker 1: right?
• 7:57
  Speaker 1: yeah?
• 7:58
  Speaker 1: So I mean i think back then when people were looking at us okay how good are these models in like zero short capabilities?
• 8:08
  Speaker 1: or You basically have your let's say fifty thousand hours of like diverse robot data.
• 8:14
  Speaker 1: You combine that with a vision language backbone, which is basically very good at understanding text and understanding certain task descriptions right?
• 8:22
  Speaker 1: And then you see how good this combination in solving tasks... I mean the results were impressive but not at the level of where you would need them to be actually usable, right?
• 8:40
  Speaker 1: So like certain lab benchmarks.
• 8:43
  Speaker 1: Certain standard tasks these models could basically do zero shot with and all thirty forty fifty percent success rate.
• 8:51
  Speaker 1: but um... But he didn't really have models that went all the way to ninety ninety plus percent success rates.
• 8:59
  Speaker 1: so I think there was.
• 9:03
  Speaker 1: There were basically a few things that we needed to figure out.
• 9:07
  Speaker 1: And then at the same time, I guess there's always this distinction between having a model is like very good generically on a range of tasks and models that really specialize one specific task you want solve.
• 9:21
  Speaker 0: Okay so... We have these vision backbones which has been trained for long times now with language part coming in.
• 9:30
  Speaker 0: but So now you can deal with this in several different dimensions.
• 9:35
  Speaker 0: One is adding more data and the data has been growing, right?
• 9:43
  Speaker 0: We are seeing labs that talking about millions of hours somehow annotated data... ...in a video space.
• 9:51
  Speaker 0: The other part is modeling.
• 9:55
  Speaker 0: a model is of function, right?
• 9:57
  Speaker 0: It has inputs and outputs.
• 9:59
  Speaker 0: So what's the input to such a VLA?
• 10:04
  Speaker 1: I mean usually the input for such a vla at that most basic level like a snapshot of current state which means basically camera view on everything it sees in most setups or more industrial setups.
• 10:25
  Speaker 1: You usually have a bimanual arm, right?
• 10:28
  Speaker 1: So you have two robot arms and then both of these robot arms have cameras on the wrists And then you usually have one static camera that is fixed as a head camera or like static to the scene.
• 10:38
  Speaker 1: in general The input to the model at that point would be the joint readings Of all-of all the joints in both robots.
• 10:46
  Speaker 1: so an idea of what configuration Is the robot currently in plus camera images?
• 10:53
  Speaker 1: uh, yeah.
• 10:55
  Speaker 1: Of the current like what the camera seeing?
• 10:57
  Speaker 1: Uh.
• 10:58
  Speaker 1: and then usually on top of that you have what we call a language prompt which is exactly what-what do would expect when you're dealing with language models?
• 11:06
  Speaker 1: so it's basically an input That is supposed to Give The model An idea of What It's Supposed To Be Doing.
• 11:13
  Speaker 1: So Its Kind Of Like A Goal Description Mm-hmm!
• 11:15
  Speaker 0: So state space Is Such An Important Concept In Robotics Right Given That You Cannot input, everything that you know about the world at all times.
• 11:24
  Speaker 0: You have to somehow limit yourself in what actually put it and want to ignore.
• 11:29
  Speaker 0: so It seems like having just this stated one particular point of time is trying To find out What a conversation's about by hearing a single sentence?
• 11:38
  Speaker 0: Yeah Right.
• 11:40
  Speaker 0: So let's talk A little bit About what The limits Is And what people Have been Trying to do.
• 11:46
  Speaker 0: with That
• 11:47
  Speaker 1: I mean there's really good Point.
• 11:48
  Speaker 1: because So, I mean in like sequence modeling or robotics we call this the Markov assumption that you can essentially from your state at a current point of time perfectly predict where are and what is going.
• 12:06
  Speaker 1: And then it's just well defined problem but basically means to carefully design this observation.
• 12:16
  Speaker 1: which model is getting?
• 12:17
  Speaker 1: because you can easily run into issues where if, I mean... You just imagine going through a labyrinth.
• 12:25
  Speaker 1: Where all the walls look the same right?
• 12:27
  Speaker 1: If you get one snapshot and don't have history of how you moved before it's basically impossible to like keep track of where you are and where your going.
• 12:40
  Speaker 1: And it's the same thing with a conversation, right?
• 12:42
  Speaker 1: If there is one hour conversation then you hear one sentence out of that basically missing alot context.
• 12:48
  Speaker 1: so... There're different ways how can fix this let say or different way how people try to approach these issues having some sort of memory or history.
• 13:02
  Speaker 1: The easiest usually just Getting the model some form of velocity information, which usually means you just add multiple frames off.
• 13:12
  Speaker 1: The same observation space.
• 13:14
  Speaker 1: so instead of Just seeing where You currently are?
• 13:17
  Speaker 1: You basically also see oh by the way this is Where have been half a second ago.
• 13:21
  Speaker 1: This Is where I've Been A Second ago and you can extend that quite a bit.
• 13:25
  Speaker 1: And That Way the the Model Basically has an idea Of all.
• 13:30
  Speaker 1: This is where I have been going, right?
• 13:34
  Speaker 1: And then together with this goal description of where i want to be you can maybe figure out okay.
• 13:40
  Speaker 1: Where do we go next?
• 13:41
  Speaker 1: because that's ultimately what we want these models to basically output a next action which is bringing us closer to the goal.
• 13:52
  Speaker 1: so one option is kind of engineering your observation space.
• 13:59
  Speaker 1: Another option, and that's kind of more recent is you basically add memory directly to the model.
• 14:07
  Speaker 1: And there are different ways how you could do it.
• 14:10
  Speaker 1: There's a whole line of research which basically tries to add memory either in the form some sort of database or like past experiences That can efficiently query while doing this task.
• 14:25
  Speaker 1: So you have like a reference point.
• 14:27
  Speaker 1: Oh, this is something I've done yesterday.
• 14:29
  Speaker 1: and then what was the experience there?
• 14:32
  Speaker 1: Or another option as... You can also imagine it's an internal monologue.
• 14:37
  Speaker 1: so we have some sort of higher level agent that usually a VLM or vision language model And basically orchestrating supervising execution.
• 14:49
  Speaker 1: Yeah, really having an inner monologue and like from time to time checking up Oh what is my vision language action model?
• 14:57
  Speaker 1: So the let's say lower level part doing And then also correcting it.
• 15:02
  Speaker 1: I mean i think in The context of Robco and of industrial task right...I Think we should also talk about the most easiest way Of doing that which is essentially just as said before That part of the observation space Is adding a prompt or goal description.
• 15:18
  Speaker 1: And especially in industrial tasks, when you don't deal with a lot of variety and your task.
• 15:27
  Speaker 1: You know pretty well ahead-of time what is the usual sequence that has to be solved?
• 15:36
  Speaker 1: Then also do this in more classical engineering way.
• 15:40
  Speaker 1: so basically have some state machine around it which orchestrates switching this prompt, so the language instruction once you complete a sub-task.
• 15:49
  Speaker 1: And then you need some system to figure out.
• 15:51
  Speaker 1: okay now certain subtasks is completed but that's kind of easiest and most engineering way But overall there are like a variety different ways.
• 15:59
  Speaker 1: how can tackle it?
• 16:01
  Speaker 0: So if I want summarize maybe take another example.
• 16:05
  Speaker 0: ten years ago when we were using Google Translate The limiting factor was state space like what the next word should be.
• 16:18
  Speaker 0: You only had enough data to look maybe one or two words behind, right?
• 16:23
  Speaker 0: And so I think this whole thing with transformers and chat GPT show that you can basically have an infinite context if you know what to attend too.
• 16:33
  Speaker 0: The motivation came from machine translation because of language A in a different order than we need where to look towards.
• 16:44
  Speaker 0: And I think this is here sort of comparable only that you also have a history of what was happening in the near past, which could give you some information on what to do in the future?
• 16:56
  Speaker 0: Yeah and so it seems like we've been starting with most limited context that you can possibly use but now were trying to expand.
• 17:08
  Speaker 0: I
• 17:11
  Speaker 1: think an important point is also you're completely right.
• 17:15
  Speaker 1: The context length of Transformers, it's amazing!
• 17:18
  Speaker 1: You can have the entire book in there pretty much by now.
• 17:21
  Speaker 1: and the tricky thing with robotics obviously always that... ...you need to handle this whole context at a very fast pace because we are controlling our robots at something like fifty hertz usually or thirty hertz which basically means.. ..you need new action input thirty to fifty two hundred times per second, which gives you a very small amount of computation time.
• 17:46
  Speaker 1: And that's also why this idea of okay I have somehow orchestrate the state space well or how am i efficient about my memory?
• 17:55
  Speaker 1: Why is it so important in robotics?
• 17:57
  Speaker 1: because You don't have an huge cluster right That can basically just go through all those numbers at A quick pace and where it doesn't matter whether point three seconds later, right?
• 18:12
  Speaker 1: But but in robotics.
• 18:13
  Speaker 1: I mean you really need this high frequency interaction
• 18:17
  Speaker 0: and so having more context means usually bigger models more compute lower latency.
• 18:23
  Speaker 0: there's some tricks around that.
• 18:24
  Speaker 0: Right i mean first of all we were talking We have talked In the previous episode about The different layers That You Need Something Like One millisecond Maybe Latency And Then You Have These These Models That Were Talking About Where Sixty Hertz Would Be Sixteen Milliseconds But then they are usually slower, right?
• 18:41
  Speaker 0: So there's some tricks around that as well.
• 18:43
  Speaker 1: Yeah I mean you usually have some layered architecture and i mean we've talked about this in the past episode And I think you've talked a bunch of past episodes where You have at very lowest layer Usually quiet deterministic.
• 18:56
  Speaker 1: so your classical robot controller which runs at kilohertz speed Then if actually go all the way to motor control it is even faster than Kilohertz Speed have layers of models or controllers that basically run at slower frequencies.
• 19:13
  Speaker 1: So your VLA, your vision language action model would usually run somewhere around thirty to a hundred hertz maybe and like deal with not just the lowest level.
• 19:25
  Speaker 1: okay I have my joint in this exact position but think about a bit longer term.
• 19:30
  Speaker 1: Okay, I'm here i want to be there.
• 19:32
  Speaker 1: how do have to move and so on?
• 19:33
  Speaker 1: And then this is that can be an even higher level orchestrator which would just stand for example vision language model with runs at more like one Hertz or maybe once the second checks in on.
• 19:44
  Speaker 1: okay I am still on my way from the kitchen to the bedroom right.
• 19:47
  Speaker 1: continue all.
• 19:48
  Speaker 1: now I've arrived at the bad room.
• 19:50
  Speaker 1: so no my task changes something but it's basically layer of of models that run at different frequencies and have different tasks from lowest level control to highest level orchestration.
• 20:04
  Speaker 1: And reasoning?
• 20:06
  Speaker 0: Okay, we had this hardware constraint you typically only one or two GPUs on a robot.
• 20:13
  Speaker 0: There are people who say they're not solving the problem right now let's put a big GPU cluster next And then we have five hundred millisecond latency, and that's still okay if you have all the other layers in place.
• 20:26
  Speaker 0: But obviously would like it to be as self-contained is possible?
• 20:30
  Speaker 1: Yeah I think especially in Germany right where a lot of companies don't have amazingly good internet connections though.
• 20:39
  Speaker 1: so having something that requires cloud usage there maybe not.
• 20:45
  Speaker 0: It's also not something that is completely out of scope because humans.
• 20:49
  Speaker 0: Also have two hundred millisecond latencies for making decisions and only if you train really, really hard You can get it down to one hundred fifty or maybe a hundred milliseconds.
• 21:00
  Speaker 0: these are the baseball people who have to catch balls And stuff like That.
• 21:05
  Speaker 0: so So there Is A Like We See on The Human Baseline.
• 21:10
  Speaker 0: What?
• 21:10
  Speaker 0: Actually Necessary.
• 21:11
  Speaker 1: I guess Its Nice Way To segue Kind Of Into one of the other big topics in twenty-twenty six which is video models, world models.
• 21:19
  Speaker 1: because I mean yes humans only react at like two hundred millisecond pace.
• 21:23
  Speaker 1: right but we have a really good understanding of how the physical word around us works what certain actions has as i have...like..what the effect is of certain actions.
• 21:33
  Speaker 1: Which basically means we can somehow internally predict What a certain action will do right and then when we're acting, were only really collecting correcting for this plan but.
• 21:45
  Speaker 1: We have this inherent knowledge.
• 21:47
  Speaker 1: I mean every new task Every new object that we are handling ,we have an intuition of okay how heavy is the subject going to be?
• 21:55
  Speaker 1: How it's gonna feel ?How its gonna behave Right?
• 21:58
  Speaker 1: And these sort things Is something That we basically now starting To get into When it comes to robotics.
• 22:05
  Speaker 0: Okay so thats yet another piece context that humans do all the time, when I'm approaching a traffic light and it is yellow.
• 22:13
  Speaker 0: I know that might need to stop five seconds before because they kind of anticipate what comes next.
• 22:20
  Speaker 0: so this all goes under this label off world models right?
• 22:24
  Speaker 0: But like the term sounds very big seems like we're modeling everything in the world but obviously also only model are small part here.
• 22:34
  Speaker 0: yeah
• 22:35
  Speaker 1: i think I mean, maybe we should start and like give a bit of an overview of what.
• 22:41
  Speaker 1: What do you mean when we talk about world models?
• 22:43
  Speaker 1: And this can go... There's a wide variety but usually these days people mean is some type of model that basically based on the current state may be based in idea or what it supposed to happen look forward into future and predict certain thing of like how is this system going to evolve.
• 23:08
  Speaker 1: And I think the most easy way to visualize and it's also the most common way.
• 23:12
  Speaker 1: what we see these days, are video generation models where you essentially put in a language prompt so that you describe Imagine this scene and kind of give you a video what the scene looks like, how it evolves.
• 23:40
  Speaker 1: And I think one of really big arguments for why roboticists are now interested in this is actually there's multiple arguments but i'll start with one which a video that has high fidelity and looks like this could actually be real.
• 24:04
  Speaker 1: These models need to understand physics, in order to predict the very complex scene of how objects interact with each other so certain things fall down when they have no more support or pick up something.
• 24:21
  Speaker 1: you basically to have two objects in close contact, right?
• 24:28
  Speaker 1: In order to efficiently model and predict these sort of things your video generation model basically needs some understanding or physics.
• 24:37
  Speaker 1: And compared to vision language action models which basically have the vision-language backbone so what we now at ISF with JetGPT or Claude are something like that ,right ?
• 24:49
  Speaker 1: These models this vision language models they basically know about physics from from reading books and seeing images, right?
• 24:57
  Speaker 1: But a video model in order to really predict this.
• 25:01
  Speaker 1: it needs more basic understanding of what physics is.
• 25:06
  Speaker 1: How physics works how gravity works things like that.
• 25:09
  Speaker 1: I think one of the big arguments for roboticists better properties and better representation of how a scene or the world looks like, because it basically has to learn that from data before.
• 25:30
  Speaker 0: Okay so in essence what is a world model?
• 25:32
  Speaker 0: A world-model is something where I can predict future.
• 25:36
  Speaker 0: The argument is humans do this all the time.
• 25:40
  Speaker 0: even we needed for language models as well.
• 25:43
  Speaker 0: right if Language models only just predict the next token.
• 25:47
  Speaker 0: And so if I say, I'm going west starting from Munich to the city off and now I need to predict a city everybody would say well it's Augsburg because that is how this relation gets baked in until language model.
• 26:05
  Speaker 0: as an argument
• 26:11
  Speaker 1: Roughly speaking, you could basically argue that the language model is a word model.
• 26:15
  Speaker 1: But I think in terms of usefulness right?
• 26:18
  Speaker 1: i would argue That video model which Basically has A better understanding Of physics and how objects interact Because it needs that In order to predict what happens when two objects collide.
• 26:30
  Speaker 1: Right there still It's usually or more useful world model.
• 26:38
  Speaker 1: Let's call it that, yeah?
• 26:39
  Speaker 0: It is also a little bit over the top right because video model needs to predict pixels millions of values for every step but its amazing progress we have had in past few years while you can generate whole scenes and even longer sequences there really with high fidelity.
• 27:00
  Speaker 0: But do we need all this?
• 27:03
  Speaker 1: Its good point.
• 27:03
  Speaker 1: I mean theres kind two competing architectures i would say right.
• 27:08
  Speaker 1: um one is the yeah plain video generation models where usually the actual generations still happens in a compressed latent space so you're not predicting full pixels or, But still, I mean the end is then basically you have a decoder which takes this compressed space and basically decodes that into pixels.
• 27:37
  Speaker 1: And to something You can see... Which i mean.
• 27:39
  Speaker 1: The big argument for That Is it's just super easy To debug because you Can look at the video?
• 27:47
  Speaker 1: If It looks good well Then your model kind of Works!
• 27:50
  Speaker 1: I guess the other competing architecture Is like inspired by Jepa.
• 27:56
  Speaker 1: Where the idea?
• 27:58
  Speaker 1: Actually I don't have to learn this in the pixel space, but i can instead directly learn it in a compressed space.
• 28:06
  Speaker 1: So instead of learning how to uncompress basically my compressed space into pixels... ...I just learned away from how to compress pixels into very good compressed space and then really only learnt by world model in the compressed space!
• 28:26
  Speaker 1: It's still the race of, and that is still open.
• 28:28
  Speaker 1: Like what is better?
• 28:30
  Speaker 1: I think there are good arguments for both.
• 28:37
  Speaker 1: one big thing is for sure how if you do it the JAPA way You can basically only use this as like a world model in the sense But it doesn't really give you useful output whereas If you have away off getting back to pixels you have a video generation model, which has basically a bunch of applications nonetheless.
• 29:01
  Speaker 0: So just to sum it up so we have the videos themselves.
• 29:07
  Speaker 0: they contain tonne information that don't necessarily need.
• 29:11
  Speaker 0: things like shading and spatial thing from background not really interested in.
• 29:17
  Speaker 0: We talked about state space before this all becomes now State.
• 29:25
  Speaker 0: The idea is that you can have a lossy compression, right?
• 29:31
  Speaker 0: I think that's the point.
• 29:32
  Speaker 0: You only keep certain learned concepts which are relevant for the task.
• 29:40
  Speaker 0: but how do actually decide what to keep and how to compress?
• 29:47
  Speaker 1: Honestly through learning.
• 29:49
  Speaker 1: so basically learn architecture such as the compression mechanism.
• 29:54
  Speaker 1: it just very efficient.
• 29:56
  Speaker 1: But I mean, i would argue that even the pixel video models right they have a certain way of compressing informations like shading or background texture because there it's also usually when you look at these videos.
• 30:14
  Speaker 1: They have very high level detail in foreground and then There can be blurry background where you see okay model has basically some compressed version of what the background should look like, and then you basically just predicted it.
• 30:28
  Speaker 0: Okay?
• 30:30
  Speaker 0: And so now let's say a rollout over to next second... We're not talking about minutes right?
• 30:36
  Speaker 0: we are talking about something that is in very near future.
• 30:41
  Speaker 0: What Next?
• 30:42
  Speaker 0: Imagine this late vector or this compressed vector Or set of hundreds thousands pixels.
• 30:49
  Speaker 1: Yeah, I think that's probably a good point to basically talk now about.
• 30:53
  Speaker 1: how can we use these models in robotics?
• 30:56
  Speaker 1: And there is kind of roughly three big ways.
• 31:03
  Speaker 1: One is artificial data generation so... We know how costly it is to collect tele-operation or human echo data And you can use these type of word models basically as a simulator and then learn in them.
• 31:22
  Speaker 1: That's one option, um...and the other two I guess bigger options are.
• 31:28
  Speaker 1: if you now know how the world is supposed to evolve over the next seconds You can basically use that knowledge and then learned what we call an inverse dynamics model where the idea really Okay.
• 31:40
  Speaker 1: Given that this is supposed to be the future trajectory, right?
• 31:42
  Speaker 1: What do I have to do now in order to make it happen?
• 31:46
  Speaker 1: and one big argument is essentially learning such an inverse dynamics model when you have a video model or if your model predicts its future very well.
• 31:57
  Speaker 1: It's much easier task than basically being at current point of view and deciding okay what should i need given that i know what i want but not give me what my future will look like.
• 32:10
  Speaker 0: So that means the future becomes an input, right?
• 32:14
  Speaker 0: It becomes part of the input space.
• 32:16
  Speaker 0: Correct
• 32:16
  Speaker 1: Yeah
• 32:17
  Speaker 0: Right so and we do this also every day.
• 32:20
  Speaker 0: maybe We have An image in our mind how The kitchen should look like after we clean up And then suddenly Then the task is to align reality with what we had in mind.
• 32:31
  Speaker 0: yeah, and then there's these people who cannot really.
• 32:37
  Speaker 0: They don't haven like inner pictures or in an imagination.
• 32:41
  Speaker 0: And maybe that's how to draw the parallel, they can still do the same things and somehow don't materialize it in images but have this vague notion of what should be then the case?
• 32:53
  Speaker 0: That is maybe one could think about these latent space models.
• 32:59
  Speaker 1: I think its kind important because most video action models or world action malls that we see these days, they actually don't necessarily go all the way to predicting pixels again.
• 33:12
  Speaker 1: So usually you keep doing the prediction of what is supposed to happen but you keep it in a compressed space and then basically use this knowledge with another model to get your actions.
• 33:26
  Speaker 1: It's like from computational point-of view Predicting pixels entirely is very costly.
• 33:33
  Speaker 1: It's like requires beefy big GPUs and if you can skip that, That's basically an advantage because he just have a faster execution frequency which I mean we talked about before.
• 33:44
  Speaker 1: this important for robotics.
• 33:46
  Speaker 0: Yeah And You touch upon it Before.
• 33:48
  Speaker 0: let's Let's Just Take A Bit.
• 33:51
  Speaker 0: This The Thing.
• 33:52
  Speaker 0: You Know What Will Happen in the Next Six to Twelve Months There Because One might be the better architecture.
• 34:00
  Speaker 0: The other one might be where we have already a business model around it, so in the end what will win there?
• 34:08
  Speaker 1: Honestly I think that's like looking into the crystal ball...I wouldn't dare to say no!
• 34:16
  Speaker 1: Both camps have good arguments and followers who are very convinced.
• 34:21
  Speaker 1: this is how you should do it.
• 34:23
  Speaker 1: these days we're seeing more models that basically go to pixels eventually, simply because there is a good commercial argument for it.
• 34:34
  Speaker 1: But I wouldn't dare say one camp will definitely win in six months.
• 34:39
  Speaker 0: One argument could be... There's tons of money and just video generation so maybe they are in the position then also tackle robotics alright?
• 34:54
  Speaker 0: So Okay, so we talked about images.
• 34:59
  Speaker 0: We talked about memory and we talked About using future predictions as part of a memory maybe one thing around that.
• 35:10
  Speaker 0: obviously there's not only One Future right?
• 35:12
  Speaker 0: So There is the this old example Of people who have worked on This ten years ago.
• 35:17
  Speaker 0: I said okay you put A pen On its tip it's impossible to predict where it falls, and then people were talking about mode collapse.
• 35:26
  Speaker 0: And blurry images and so on?
• 35:29
  Speaker 0: So how is that actually done with image models?
• 35:32
  Speaker 0: Are there any restrictions?
• 35:36
  Speaker 1: I think this kind of like the third way or for you could use these models in robotics right Where You basically look at a bunch of potential futures which one you want and basically use the actions according to that.
• 35:51
  Speaker 1: So, more used model as a simulator planner where you can weigh options of what is actually future I want then act accordingly?
• 36:06
  Speaker 0: But it's all still very much sampling based.
• 36:09
  Speaker 0: so somehow make decision on how things roll out.
• 36:15
  Speaker 0: a wave, you're standing at the ocean and there's a wave coming to you.
• 36:22
  Speaker 0: The grand scheme of things that is probably quite linear roll out but then the devil in details how it actually looks like?
• 36:29
  Speaker 0: Yeah
• 36:31
  Speaker 1: I think.
• 36:31
  Speaker 1: so this basically comes back.
• 36:35
  Speaker 1: what modern architecture of these models are which some type of diffusion or flow matching model where even if we have very multi-modal distribution.
• 36:46
  Speaker 1: I mean, what these models are very good at and that's kind of a difference to what we had before which if you like now model this as some type of Gaussian right?
• 36:55
  Speaker 1: You get basically what you just talked about.
• 36:57
  Speaker 1: Just the blurry image where wherever you have these multiple possible futures... ...you just get some sort of average of them.
• 37:05
  Speaker 1: but i mean with these models in what they're like flow matching paradigm is really good.
• 37:10
  Speaker 1: it is basically add these decision points decide for one of them follow that.
• 37:15
  Speaker 1: And it might be that if you sample two or three times, so basically get different versions of it but at least you got like one consistent version whenever your sample which is a big plus
• 37:26
  Speaker 0: and also the reason why something like Will Smith eating spaghetti produces in video that actually shows something although there's an infinite number of versions of Will Smith out there!
• 37:37
  Speaker 1: All
• 37:39
  Speaker 0: right.
• 37:39
  Speaker 0: So Video as One Thing.
• 37:42
  Speaker 0: What Else Do We Have?
• 37:44
  Speaker 0: There's obviously the whole question of, what do you need to actually run robotics?
• 37:49
  Speaker 0: And that's probably far more than vision.
• 37:52
  Speaker 1: Yeah for sure.
• 37:53
  Speaker 1: I mean we can maybe talk a little bit about other types of modalities.
• 37:59
  Speaker 1: We really see how people are experimenting with like with more inputs then just The robot proprioception which is joint readings and images videos.
• 38:10
  Speaker 1: basically We started seeing people adding depth, for example to these models or a sense of touch.
• 38:19
  Speaker 1: Or force.
• 38:19
  Speaker 1: so and I mean especially when we think about interacting with the environment right?
• 38:25
  Speaker 1: And like moving objects are handling certain soft objects like cables or rope something that you could actually argue this senses is much more important as human because There's a lot you as human can even do blindly, right?
• 38:44
  Speaker 1: Without seeing it.
• 38:46
  Speaker 1: I mean if your in front of the door and have to open it with your key You just through your sense of touch basically feel where the keyhole is And without any sense of seeing.
• 38:58
  Speaker 1: So arguably these senses are equally important or almost as important And we see more and more models basically going into that direction, adding additional modalities.
• 39:11
  Speaker 1: Basically seeing if we can boost the performance of these models by adding such additional modality.
• 39:18
  Speaker 0: So I mean it seems like humans have some kind of redundancy there.
• 39:23
  Speaker 0: so you always learn with all the senses.
• 39:25
  Speaker 0: now switch off one sense and still do it?
• 39:29
  Speaker 1: Yeah!
• 39:31
  Speaker 0: And i think probably would like to chime in here.
• 39:35
  Speaker 0: This is also an extremely, extremely interesting problem from the mechanical engineering point of view.
• 39:42
  Speaker 0: Because in the end.
• 39:43
  Speaker 0: now we're getting to the next bottleneck.
• 39:45
  Speaker 0: how do you actually create robots that are robust enough to withstand production environments weeks on and sensors not breaking or drifting?
• 39:58
  Speaker 0: so we have a lot need for mechanical engineers.
• 40:05
  Speaker 0: Everybody is encouraged to not only just switch to AI modeling, but also really solve these completely unsolved problems.
• 40:16
  Speaker 0: Really big need in the market and not enough products out there And obviously we're trying do our part on this.
• 40:24
  Speaker 0: So looking back at this year you've been now very central running robot intelligence from a like technical point of view, making sure that we're doing the right things at the right time.
• 40:42
  Speaker 0: So what did you see over the course of year?
• 40:45
  Speaker 0: What has changed in space?
• 40:47
  Speaker 1: I think one really important observation is that um The models keep getting better and better And-and i mean We have to be really careful about Um what battles we pick?
• 41:03
  Speaker 1: The model that we basically used at the beginning of the year Was completely dominated by the model.
• 41:11
  Speaker 1: That were using now.
• 41:12
  Speaker 1: so Every three months, we're getting new models with new capabilities through partners or Through just the open-source world.
• 41:22
  Speaker 1: there's a lot off There's a load of things happening there and And we see that this space is moving really fast.
• 41:29
  Speaker 1: So it's very amazing how how quick the progress is these days.
• 41:36
  Speaker 1: And I mean, in terms of Robco right?
• 41:39
  Speaker 1: I think our goal really to get this models to a state where we can actually deploy them.
• 41:45
  Speaker 1: twenty-four seven in an operation and as you said hardware's basically just that important ssd intelligence around it.
• 41:54
  Speaker 1: Yeah there are bunches interesting problems were working on solving
• 42:01
  Speaker 0: Right.
• 42:02
  Speaker 0: And maybe one should say that this is not really an ivory tower kind of operation, it's customers specific things out on the factory floor have a real need and we're banging our heads against them because they've made a lot progress.
• 42:21
  Speaker 0: so tell me what do you think about today?
• 42:29
  Speaker 0: How do you learn as a group and how do you structure this work?
• 42:35
  Speaker 1: So, I mean we basically have pretty rigorous framework for experimentation where.
• 42:42
  Speaker 1: We have a continuous radar going on of okay what is the rest of the world doing?
• 42:48
  Speaker 1: because i mean we have to basically keep up-to-date with other people are doing so were like going through research papers what we can learn from them, and then in terms of how we progress ourselves.
• 43:03
  Speaker 1: We basically come together as a group two times a week.
• 43:07
  Speaker 1: Uh...we propose experiments um ...and then we execute them ,we track them .We analyze them And over the course time through these discovery loops we get better.
• 43:24
  Speaker 1: In a year from now, we can basically go back with an AI agent actually and just like sort through okay what experiments have we done?
• 43:32
  Speaker 1: Have we tried this before.
• 43:34
  Speaker 1: What have we learned here?
• 43:36
  Speaker 1: so making this type of structured experimentation part of the daily work routine is was really key at Robco.
• 43:44
  Speaker 0: So really applying the scientific method as part of an engineering org
• 43:48
  Speaker 1: yeah I think you have to do that.
• 43:54
  Speaker 1: put your hypothesis to the test, right?
• 43:56
  Speaker 1: You come up with certain ideas or you see something in another paper and then you say okay let's try this maybe.
• 44:03
  Speaker 1: This helps us.
• 44:04
  Speaker 1: uh...you have a hypothesis- you do an experiment around it.
• 44:08
  Speaker 1: Uh..You confirm that you deny it And basically move on.
• 44:10
  Speaker 1: Then you gain knowledge through That and apply It To The Product We In The End Wanted Ship To The Customer.
• 44:17
  Speaker 0: All Right.
• 44:18
  Speaker 0: So all of this is context Of also new product development.
• 44:22
  Speaker 0: Nothing Is Fixed.
• 44:23
  Speaker 0: working end-to-end on the combined hardware and software solution, making lots of progress over the course this year.
• 44:32
  Speaker 0: I guess next year will be at least as interesting.
• 44:35
  Speaker 0: we're getting closer to actual production deployments that i hope can talk about here really soon.
• 44:43
  Speaker 0: so with it's a good round up And how it ties in that, you know apparently doesn't always predict forty two.
• 44:54
  Speaker 0: It predicts the future and it informs what the robot actually does.
• 44:59
  Speaker 0: so yeah really exciting progress over the course of this year.
• 45:03
  Speaker 0: think three years ago nobody talked about world models.
• 45:05
  Speaker 0: yet now we're at a point where it says okay is This The Real Thing or Is There?
• 45:09
  Speaker 0: The Next Thing In Vicinity?
• 45:13
  Speaker 0: If It Happens You Know We Have It On The Horizon Definitely As Well.
• 45:17
  Speaker 0: So Yeah With That.
• 45:18
  Speaker 0: Thank you, Felix for joining the podcast.
• 45:21
  Speaker 1: Yeah
• 45:22
  Speaker 0: and yeah hope to see your next time.
• 45:25
  Speaker 0: always click on like subscribe.
• 45:29
  Speaker 0: come back when it says Rob talk.
• 45:33
  Speaker 0: this is Rob Talk a podcast by Robco.
• 45:37
  Speaker 0: Subscribe on Spotify
• 45:38
  Speaker 1: Apple Podcasts and
• 45:39
  Speaker 0: YouTube.
• 45:40
  Speaker 0: See You On The Factory Floor.