Latest / The Information Bottleneck / Dhruv Batra: The Browser Is a Robotics Problem - From Embodied AI at Meta to Web Agents at Yutori
Transcript
- Ravid Shwartz-Ziv: Hi everyone and welcome back to the information bottleneck. And today we have a great guest, Drew Bertera. He is the co-founder and chief scientist at Yachuri. He was a professor at Joyotech and he was a senior director at the embodied AI team at Meta for many years. Hi Drew, thanks for joining us. Dhruv Batra: Hello, Alan. Great to be here. Ravid Shwartz-Ziv: Hey Ellen, how are you? Allen Roush: I'm great. happy to be here. Happy to chat. Ravid Shwartz-Ziv: So today we are going to talk about many things, but I think we can start with embodied AI. So like you led embodied AI at Meta for many years. Let's start with what does it actually mean, embodied AI. Dhruv Batra: It means a number of different things to different people. we saw embodied AI as the study of intelligence systems, whether they be embodied in a robot or embodied as viewing the world from a human's perspective. So fair embodied AI was studying AI for robotics and AI for smart classes. And our thesis was you need to understand the world from a first-person perspective. You need to solve computer vision problems from a first person perspective. That first person perspective can be a camera in a smart class that a human is wearing, or it can be a physically embodied system like a robot actuating in an environment. Of course, there are differences between the two systems. One of them has an actual actuator. Robots can manipulate objects, move around. A smart class does not get to take physical actions itself. But a smart class can still has to perceive and can take digital actions. So if I observe the world from a human's perspective, I can take digital actions on their phone or connected to their digital other digital surfaces. there is a stronger version of the embodiment hypothesis, which was that intelligence emerges from the interaction of an agent with their environment. And there's an even stronger version that that is the only way for intelligence to emerge. I think enough has happened in the literature that we can reject the strongest version of the hypothesis. I think it would be foolish to declare today's purely ungrounded language systems as not intelligent. so the strongest versions can be rejected, but it does feel like we certainly do need more progress on physical understanding of 3D spaces and actuation. Ravid Shwartz-Ziv: Yeah. So but why we actually like want to focus on three D like the physical world? Like why what are like what do you think actually the so important in the physical world? Dhruv Batra: I think fundamentally the physical world is the most interesting environment there is. so I I used to give talks where I would have this slide where I would say that our agents are only as sophisticated as the environments that they operate in. So if you build an agent for playing chess or go, you can solve that problem. but at the end of it, you have a go playing agent or a chess playing agent. You can build an agent that takes digital actions in an email client. And today we have bash terminal use agents and coding agents, and they are showing great promise, but they are fundamentally limited by the sophistication of the environment. The physical world, the universe around us, with all the objects, entities, other agents. It is the most sophisticated environment that nature has constructed. The ability to achieve goals in this environment is therefore the strongest test of intelligence there is, because the environment is the most sophisticated environment there is. Allen Roush: I have a friend who's making a Pokemon game environment right now for his agents then. Do you do you think that like complicated video games are are great examples of environments for the same reason? Dhruv Batra: Yeah, I think so. I think those environments are often capped by the creativity of the environment designer. this is you know, there's been the two thousand tens was especially after twenty fifteen was a bullish period in and a highlight in RL literature for precisely this reason. we went through many years of thinking that we could apply the AlphaGo recipe to Game video game environments and that was going to lead to emergence of intelligent systems. And they did, like we did solve video game environments. ultimately their generality was capped by the generality of the environment itself. If you learn to play Pokemon, turns out that that does not generalize to other things beyond Pokemon. at fair, my team. Was building a 3D simulator called Habitat. It was called Habitat because it was supposed to be an environment in which we grow intelligent organisms. and our thesis was that it has to look and feel like the real world. so we went out and you know established a partnership with a company called Mataport. They scan buildings. you know, where if you're in the market for renting or buying a house, you will often go to Zillow or Redfin type sites, and you'll see a 3D walkthrough of the house. That is coming from companies like Mataport that go and scan these environments through a 3D scanner. And we had asked them to release a large data set of scans of buildings, houses, shops, apartments, malls. Those scans were then loaded up into a fast game engine-like 3D simulator. where we were training virtual robots to navigate in those indoor environments. And we demonstrated that that environment looks and feels like real environment because it is ecologically valid. It is coming from scans of real places. And training virtual robots in these environments could lead to symptom transfer to real robots operating in real environments. Ravid Shwartz-Ziv: So so like okay, I I have like several questions. First, like what is intelligence, right? Like we had like several people here in the in the podcast and like each one of them had like different definition for intelligence. What is your definition for intelligence? At least what you try when you talked about like intelligent system, what you're talking about. Dhruv Batra: So when I used to when I used to teach my I I created the deep learning class when I joined Georgia Tech. when I used to teach that class, the you know, the first lecture was okay, what is we have to start with intelligence, then what is learning, then what is deep learning? And you know, I wouldn't spend too much time it's a good intro, yes, it is. Yeah. Ravid Shwartz-Ziv: Good it's great it's a great question. It's are these are like great questions, right? Dhruv Batra: Yeah. there the I didn't spend too much time on what is intelligence, but what I said was, okay, artificial intelligence is quite easy to to define once you define intelligence. because if you if you can point to intelligence in the natural world, that's our goal to to emulate, and then you can define superintelligence beyond it. there there are a couple of definitions that I liked. One was that I think comes a lot from the reinforcement learning way of thinking, which is intelligence is the ability to achieve, to set and achieve goals in sophisticated environments. If you talk to people like Ruth Sutton, that's what those are the kinds of definitions they will they will say. There's a definition that I I liked as well, which was intelligence is the ability to navigate in idea space. That to go from point A to point B. in idea space just like you navigate in physical spaces. and I it really that definition really appealed to me personally because I I am quite a physical thinker. Like I'm a visual thinker. I I I I like thinking spatially. I can make analogies to the natural world in in how organisms and animals navigate And navigation in idea space where the the analogy is think of your what I would tell my students is think of a friend of yours that is naturally gifted at multiple different sports. They're not a specialist at any particular sport, but they can show up on any new sport and pick it up quite quickly. They have a certain way of moving their body and moving through physical space to achieve. To coordinate actions and achieve goals in physical space. Now imagine doing that in idea space. That is intelligence. Ravid Shwartz-Ziv: So so w when you say idea space like you mean kind of like what Jan is talking about, right? Like their representation space and like their world models that like you separate between the generation and live in this like hidden space that you can that you can do all of the the the actions there? Dhruv Batra: yes, I think so. it there is certainly a connection there. by idea space, so there's a long thought school of thought in philosophy that talks about the ideal platonic world in which mathematical objects live. you know, they exist above the realm of of physical existence. and in an idea space, you can compose ideas, you can you can you know certain ideas are Close by in some conceptual sense, you can transition from one idea to another. A proof can be thought of as a path in an idea space where every step is an edge in some graph that lets you take a loud set of logical leaps. those are the kinds of things that I mean, that the ability to navigate in idea space means the ability to And you t in today's language we would call it to roll out a chain of thought where no l no logical leaps are being taken along the chain of thought. Where you're going from point A, a starting premise to a to a conclusion, and no leaps have been taken along the way. Allen Roush: So does in in the diffusion model world they have the ability to like upweight and downweight prompts and traverse prompts almost in a a continuous manner. and I I always wished that we had that with with proper autoregressive language models. Do you consider such a thing to be a a form of of what you're talking about? With your definition of intelligence. Sorry, sorry. Yeah. Dhruv Batra: Yeah, certainly. Yeah, certainly. I I don't I am not commenting on the mechanism used to go from point A to point B, whether or not it is autoregressive or you know refinement of of paths. you know like yeah many of us come from a computer science background. I imagine some of your audience comes from a computer science background as well. There are, you know, in you end up taking these classes where they teach you max flow min cut. You know, I have a graph where there are vertices and there are edges. I'm trying to find the I'm trying to partition this graph into two segments with the least amount of edges that are cut between them. there is a dual version of this problem, which is max flow, which is I'm trying to push flow along the edges and there is capacity. and you can Try to think about there's there's conceptualizations of of these problems where you can iteratively refine a partial solution to an optimal solution, or you can try to think of it as a greedy algorithm that is making progress along a frontier. I I'm indifferent to the instantiations, and I think we should all be in indifferent to instantiations of how to do it, in defining intelligence, which i in in this definition is is movement through idea space. Ravid Shwartz-Ziv: Okay, so we define intelligence and we collect a lot of data from buildings and let grow this like this world. And now what? What are like how we actually learn these models? Dhruv Batra: Yeah. so b I will describe the d I'll describe some of the details and you'll it will immediately be obvious that this was work done six years ago because the you know it was pre-paradigmatic. this was work done around tw starting twenty eighteen to twenty twenty-one. so the architectures were you know transformers were new, we were still using CNNs and LSTMs. The idea was we were going to train there was no pre-training at this point. We are going to train virtual robots to go from point A to point B in a 3D simulated environment by with pure reinforcement learning. The algorithms were variants of PPO, which you know, PPO in its proximal policy optimization, a kind of reinforcement learning algorithm. That that literal algorithm isn't used today. Today for LLM research, it's GRPO and derivatives. But the idea being that you ask an agent to go from point A to point B, you reward it as it is making progress from point A to point B. And when it gets to its terminal location, if it gets to the right place, you give it a terminal reward. the action space that this agent has is it can turn left, turn right, go straight. So it has some discrete set of actions. Its sensors are there's a camera attached that From which it can see the world. It has some proprioceptive sensors that when it moves forward, it knows how much it has moved. These are called ego motion sensors. They are derived through an accelerometer or a GPS equipped in real robots. And the key innovation in this work was that if you talk to a roboticist, they will tell you that the way to solve a navigation problem for a robot is to build a map. Localize yourself against a map and do path planning. This is what roboticists are taught in robotics classes. And what we said is no, we were going to do it pure learning, that there is no map. We will not assume that we are given a map and we will not build a map. we will just perceive the world from an egocentric camera. We will know ego motion as we are moving, and we will do a pure learning-based end-to-end. We will feed that into a neural network and it will predict actions that are discrete actions or even velocities, continuous actions. And we will learn this neural network from scratch with reinforcement learning through billions of steps of interaction in thousands of 3D simulators. The equivalent of 80 years of robot experiments done in three days of GPU time, wall clock time, on something at that point. 128, 256 H100s. Because we can build a really fast 3D simulator. So we built the at that point of time the world's fastest 3D simulator that could do 10,000 steps of interaction with the environment in a second. We're going to roll out billions of steps of interaction. And what we demonstrated was we can solve navigation in the sense that the accuracy and reward curves. Do not do not saturate, they you know, they solve on the data sets we were measuring, get to hundred percent near the hundred percent performance in unseen new environments. yeah. Allen Roush: So do you think w what what do you think will will be the path towards improving like creativity and improving diversity in outputs? And do you think that this is important in the context of like continuous control and robotics problems for you know, getting ultimately better results? Dhruv Batra: Yeah. so it tends to be anytime you have imperfect information, which is in most situations you have imperfect information, whether you're navigating in new environments or you're playing a game like poker, where there's imperfect information, or even in coding agents you're interacting with a digital environment or You know, as we'll talk about later, web agents where you're navigating a website, you have imperfect information because you haven't been to this website before. You don't know what happens after a button click. Anytime you have imperfect information, you have to do exploration. You have to you have to do recovery from errors. there will be errors. Errors are inevitable because you don't have perfect information. And so you have to have recovery from errors. and that requires maintaining of either an explicit or an implicit frontier of of things that you have tried in this environment. and in fact, you know, one of the findings in some of the recent LLM literature for like the reasoning models for solving math, RLVR literature was that if you weren't careful, you would lead to an It you would end up with entropy collapse, meaning that the models would end up producing a certain kind of reasoning chain that would just not produce anything else. so it it is important to maintain a healthy amount of uncertainty over the kinds of things that you want to try. Having said that, I've seen over the years, and I've worked on this problem myself, like I think at this point, 12 years ago, I wrote a paper called Diverse Beam Search. beam search is an algorithm for you know rolling out multiple multiple chains of of outputs from sequence sequence models and diverse beam search was an attempt at making sure that the multiple rollouts that you're doing are different, sufficiently different. I've given up on or you know I don't think any explicit maintenance of diversity is critical because I I I think of preventing collapse of entropy as more important and more general a solution than defining diversity in that sense and and and preserving it. Ravid Shwartz-Ziv: Okay, so What are the missing parts? like we we know today that like I don't LLMs are great, people are using them all the time. We see them like again in math, like the like Julia Campo w was here like a few weeks ago, basically said math is solved, like kind of like the different types of like already solved and and we we need to think about what next. But but but embodied AI and robotics is not there. Like and it's actually like far from something that is this level. Why? Dhruv Batra: A few different plausible hypotheses. I don't know what the precise answer is, because if I did, I would have invested all my resources back at where in that specific direction. But a few different plausible hypotheses for why that is the case. number one, there's a there's a hypothesis that there isn't a single problem called robotics. That we are lumping quite heterogeneous things into a s into an umbrella of robotics. in a lot of ways, I would call autonomous driving roughly solved, right? I mean, what we are fighting with are rollout problems. we've sort of we have mature industrial systems that are going out into the world and doing this. if you want to do navigation Then in in you know indoor environments and what you're going to operate on is is depth-based cameras, then Symtorial is a solid problem. we know how to build large scale simulators. If you if and and the problem with large scale simulators is that you have to you run up against what is known as the Symtorial problem, that when you learn a policy in simulation, it may not generalize to the real world. However, There are certain sensors that do generalize. So in the locomotion literature, which is teaching a robot to walk or balance or humanoids or quad-legged robots, we have seen enough that we don't even need a perfect simulation or a perfect digital replica. The last few years, starting with this is Deepak Patak and Jitendra Balik's work called where where they showed that You know, you can you don't need to you don't need to do sim to real transfer so per s per se, you can do symptural adaptation. that you train a locomotion policy in simulation. The simulation does not have to be a perfect replica of the world, it just has to have enough variety. And during training, you train the neural network to predict an embedding about the the physical properties of the environment. And though though that transfers because in the real world also you can predict those embeddings. And that line of work has basically led to a a a large number of breakthroughs in locomotion, including full body locomotion. So everything that you see from robots doing parkour to robots you know climbing walls with with their front two two legs to to robots jumping over boxes. we that technology that that line of research is quite mature. We have solved those problems. what we have not solved. are fine-grained dexterous manipulation that you know have a dexterous hand or a paddle jaw gripper with with with with high degree of control because we cannot simulate the contact dynamics of that in simulation quite well and one of the main reasons is we don't have accurate large scale data sets with physical properties of the objects annotated. We don't know what the friction coefficients are. We don't know what the ma what the what the precise masses are, what the coefficient of restitution is, and so the simulation diverges. and so what you're seeing in the sorry, you have a question. Ravid Shwartz-Ziv: But this is like this is it's because like this is too difficult. Like why we actually don't know like to learn these parameters? Dhruv Batra: predicting these param these physical parameters from observations, like camera observations, is is still an ill posed problem. if I give you a camera and I just show you an object, it is it is not yet a solved problem to predict its physical properties, like it's you know, coefficient of friction between two surfaces. You cannot predict that just from from vision. in fact the way you would predict it is large learning systems, or one plausible way in which you would predict it is large learning systems where you need a data set with these things annotated. And we don't have large-scale data sets where these things are annotated. There's actually like there's a large industry where humans go in and create these digital assets for video games. the the and one of the main like this is what A l a lot of startups now are going for where trying to generate synthetic assets programmatically through AI. one of the distinctions between the video game industry and the robotics industry is that the video game industry is trying to cater pixels for human consumption. They are not trying to be accurate. It just has to look plausible and entertaining, which means a whole bunch of things can be faked. like, you know, sure. Ravid Shwartz-Ziv: Like what? Can you do you have an example? Dhruv Batra: the jump that a character makes in a in a video game can be superhuman. It does not have to be accurate. in fact, one of the tricks that we did in our simulator, our first version of our simulator habitat was meant just for navigation. That you have a robot and it's going to navigate from point A to point B. It is not going to pick up objects. One video game hack for this is that you detect a floor plane or a ground plane. And you find what is known as a navigation mesh. So you find boundaries of the floor plane, convex decomposition of the floor plane, and then the robot is basically just gliding on that floor plane. It is not actually applying torque to its wheels with friction guiding it forward. Because that would require a physics engine and a simulator. And for you to know what the friction coefficients on the wheel are. Ravid Shwartz-Ziv: Yeah. Dhruv Batra: And you know what the friction coefficients on the carpet is versus floor is, whether there's slippage or not, none of that. The robot is just simply gliding. You say you want to go 0.5 meters forward, it will just, you know, jump 0.5 meters forward or smoothly accelerate 0.5 meters forward. And it turns out for training navigation robots where you have low-level policies doing the wheel actuation, this doesn't matter. You can train the high-level policy without knowing anything about this. And these are the kinds of things that shortcuts that Ravid Shwartz-Ziv: Mm-hmm. Yeah. Dhruv Batra: Video game engines can take, but physics simulators struggle with for physical simulation. Allen Roush: I I I So, from a hardware perspective, isn't accurate physics simulation almost the opposite problem to LLMs? For meaning, you know, LLMs really reward you as the model size gets larger with low precision computing for the weights, right? That's why there's this focus on four bit. But then for highly accurate physics simulations, at least at the p point of, you know, like airlines simulation of air, you know air airplane mechanics, they need h very high precision, thirty-two, sixty-four bit, a hundred and twenty-eight bits. So I'm I'm wondering then, do you think that all of the the various things that game devs do to, you know, trickery to not have to accurately simulate things, but to just trick the eye, do you think that this will also lead to poor outcomes from like systems trained on on that data? Dhruv Batra: Not only do I think it will, I think there's enough evidence that it does. that is the symtorial problem. that so yes, you're you're absolutely right in pointing out that you know there are certain things that we can get away with on the model side, but it seems we cannot get away on the environment or the simulation side. and I think the fundamental distinction is if we the the main problem in symtorial research is that we cannot it it is extremely difficult to create a perfect replica of real world environments. There's too much unknown. If we even if we could model certain limited systems, it would be too slow to simulate. And we need scale. The whole point of the simulation research is that simulation gives you access to large amounts of data and it lets you try things that you couldn't try in the real world. So the in the hard question is what can you get away with? You cannot send like reality lends itself, the natural world lends itself to explanation at different levels of abstraction. We can explain everything in the natural world with physics or particle physics, but we do not, because it is easier to explain biological systems at the level of biology and brains at the level of neuroscience and you know chemical reactions at the level of chemistry than try to Explain why your brain firings or your behavior is in terms of particle physics that is too sophisticated, that is too slow, too, too intractable a system. And a similar thing happens in simulation research. If I want to train a navigation system, I don't need to simulate wheel friction. So the question is, what can I get away with? if I make assumptions where trajectories and simulation look very different from trajectories in the real world. And I train only in simulation from scratch, which is a mistake that we were doing six years ago, then Symtorial Transfer systems don't generalize. one of the learnings of the last few years is that we should not be doing training from scratch in simulation. That you know, read all the text on the web that gives you broad based knowledge about the natural world. It means then you can specify tasks in natural language. do pre-training and you know post-training of VLMs so that there is grounding of perception into language. And then VLAs are about making fit actuation in in the in the physical world. So which is all to say we're making progress, but SIM to real research is in a lot of sense blocked, I think. Till we solve generative models of the real world, like till we can roll out trajectories that are that are learned simulators of the Allen Roush: And what do you think about the sensor technology? You talked about like friction simulation and related. Do you think that even if we solve these data problems that we still, you know, have have this fundamental problem of of realistic sensors for robotics or or how how do we resolve this? Dhruv Batra: depends on the task. I think there are enough counterexamples. so in robotics literature and you know thinking, a common counterexample to criticism that it's a hardware problem is to demonstrate that humans can teleop a robot. So you give some you build some sort of teleoperation system like Dishwasher loading. Like I have a bi-manual robot on a body that can navigate and I want to load dishes in a dishwasher or do household chores. you know, sometimes you feel that there aren't enough sensors on the robot for it to be able to do this task well because it robots will, you know, crush the objects that they're holding or, you know, not know when the glass object that they're holding is Is about to hit another obstacle and will make mistakes. And so it feels like this is a sensor problem. But the answer to that is often: I will hand control to a human. They will only look at a camera output. Maybe they will have some extremely weak tactile feedback with buzzing in their controller as the robot moves. And the human will teleop the robot and load the dishwasher. And that is a counterexample to the sensor problem. Because if the human can do it from pixels, then the robot should be able to do it from pixels. It is just an intelligence problem. Ravid Shwartz-Ziv: So what do you think are the missing parts? Like do you think like if we collect more data, like we need better sensors, like I don't know, like more compute? How how we get these parts, these components? Dhruv Batra: Under the current paradigm of learning, it is a data problem. You know, we have converged on a paradigm of learning, gradient-based optimization of large neural networks. And under that paradigm, it is like we have we have only one free variable to play with. Under that paradigm, it's just data. You you we there's there's a meme, there's a movie called Moneybong. at the end of it, there's a scene. you know, it's about applying i ideas from economics to to baseball. At the end of that that movie, there's a there's a scene when one character tells the other character, you know, this person hit a home run, this player hit a home run and didn't even realize it. They they were they were so underconfident that they didn't even realize they hit a home run. I I I think machine learning is kinda like that. We solved machine learning in one sense of the word and we didn't even realize it. i I think it's pretty it's pretty fantastic that for the kinds of problems that we are interested in, if we can get enough data and if it is in distribution, we have the answer. Like just just go collect more data, train la train a model that is proportional to the size of the data you have. and your in-domain problem like there is no In some sense, we gave up on defining what generalization is, right? We said there is just there is just in-domain learning. having said that, that there is a big if and assumption under the current paradigm of learning, there is only one free variable. But I don't think this is the final paradigm. I You know, maybe it is coming from having spent years listening to Jan talk at fair. I I do I do buy the arguments that Jan is making. that well I don't I don't know if Jeppa is the final final story, but but I do I d he's making he's making an interesting and and a not easily rejectable argument. Ravid Shwartz-Ziv: Ha ha ha Jeppa is all what you need. Dhruv Batra: he's he's making an argument that makes you think and grapple with it. The argument is we do have the biological agents learn very quickly on the fly. They are extremely sample efficient. Why do we not have such systems? And yes, you can say in context learning is a kind of efficient learning. Yes, yes, great. Like, but it's not at the same level. Why do like and so one One answer can be it's just a matter of scale. Like give us more time, these systems will exhibit the same level of in-context learning as biological systems do. Yes, it's it's plausible, but the problem with that answer is that it's not a falsifiable hypothesis. The scaling hypothesis is not a hypothesis. It's just a hope. like because it does not answer the question, what would make it false? Like when? When does this happen? At what scale does it happen? It just Because at every point you can say more scale. So I do think it is we we have to we have to be open and I'm quite ho I'm quite excited by the large number of neo labs that are trying different ideas. In a different paradigm, this may not be the only free variable and the and the and the thing that we struggle with. Ravid Shwartz-Ziv: Okay. So you left Meta and you started a company that built on the top of language models. How come? Or why? Dhruv Batra: so a coup so two two primary reasons. first is I like it's a it's a purely personal reason. I I value diverse experiences. I am still early enough. I think my academic my intellectual career is not over, my brain still works. I think I have a few years left to try interesting things. and so this is an interesting. Chapter of my life where I can try a new kind of experience, which is you know founding a startup with people that I have enjoyed working with. that's that's number one. Number two is much more grounded in the problem. I, as you know, as we just as I described, was leading embodied AI, AI for robotics primarily, but also AI for smart classes. I'm skeptical. Of the claims that robotics will move as quickly as LLMs and you know purely digital systems. and I left Meta in 2024, it's been two years. I am not seeing anything to change from my view from that perspective. there is no possible world in which physical assistance and robots have arrived, but digital assistance have not. Because the digital world has certain properties that lets you move faster. The digital world can be cloned. The digital systems can be transmitted over long distances. Failures in digital worlds are less catastrophic. The physical world moves a lot slower. Hardware cycles are long, iterations take time, mistakes are dangerous. I just don't see a world in which In-home robots are here, or like you know, robots are assembling themselves. We have automated large parts of the economy in the next few years. And and it's primarily not it's because it's not an not simply an intelligence problem. so but at the same time, those digital assistants weren't here. when I left in 2024, The the problem that we decided to work on at UTORI is building web agents. These are agents. I I think of them as robots acting on on a in a browser. these are agents that can automate tasks on the browser. humans spend so much of our lives clicking buttons, you know, logging into websites, th if Selecting fields, clicking dates, changing our addresses, booking reservations, filling out forms. and it feels ridiculous that we do that. Why why does it take a human to click buttons on a on a UI interface or a browser? why aren't there systems that do this for us? and that is the problem that we're tackling at at UTORE. we so When we started two years ago, the the first demo that we wanted to build was the demo that everybody starts off with in WebAgence, which is, hey, can you order can you order add an item to my cart on DoorDash? Just, you know, go to this restaurant, add the first item to my cart, and it's just, you know, five button clicks. That was the state of agents in 2024. That that was hard. it is not easy. one of the reasons why it's hard is is the web is an extremely heterogeneous space. it is an extremely rich environment. something like 80-85% of websites don't have APIs available, so you cannot talk to them programmatically. the web is primarily laid out for human consumption. So we've spent 30 years of piling technological hacks on top of each other. and what a browser is at this point is a fairly sophisticated UI interaction device. humans perceive with a high bandwidth pathway through pixels. And so we generate those pixels in a browser and we show them to you, and we can get input from you by having you click buttons and drag sliders and so on. the underlying code is extremely heterogeneous, and it has piles of hacks underneath it. We wrote a blog last year called The Bitter Lesson of Web Agents. The point of that blog post was to dispel the notion that people often have coming into this space that you should just read HTML and you should treat this as a pro as a programming problem. That fundamentally there this is a code that generated a website. You should just read that code and interact with programmatically. The problem with that is that you do not have access to the back end of websites. And websites are laid out extremely heterogeneously. So if you you go to a website and you try to answer an e-commerce website and you try to say, is this item available or not? One e-commerce website will say not available in text. Another e-commerce website will make a programmatic query to get the number of items that are available and will calculate via a variable when the item is is out of stock to get that variable to be zero. And so you have to be running that code and intercepting its input to know what the output is. Yes. Ravid Shwartz-Ziv: So like but like so like you suggest basically look on the image itself, right? Dhruv Batra: Yes. this the thing that we do is we treat the pixels as the first class citizen because that's what humans were that's what i it was meant to be shown to a human. So pixels in, actions out, not too s different from robots that we were training, which was camera pixels in, some additional information, and actions out, which is corresponding to navigation. Ravid Shwartz-Ziv: So do you feel that like the current agentic systems are good enough for like dealing with these tasks? Dhruv Batra: They're getting better. they're getting significantly better. so as I mentioned, two years ago, it was something like five, seven steps of interaction with a website. in November of 2025, we released the first model available as an API. We're called Navigator N1. In April of this year, so last month, or just about a month ago, we released Navigator N one point five. It's a it's a you know. It's a pixels to actions model, takes in a screenshot, looks at some text, outputs either a UI interaction, like a click type scroll, but also the model can write JavaScript code so it can interact with with the program as it needs to. and on the API, we offer the developers to run it for as long as hundred steps of interactions. So you know, people launch tasks like go to this website. log in. Maybe it's a CRM website or a you know a record holder website. Log in, create a new entry for this project. Here's the name of the project, here's the title, here's the description, here's the due date, here's the person responsible for it. And maybe that whole thing is, you know, multi every field that you're interacting with is an action. That whole thing can be multiple steps of interaction. And You don't have to disc you don't have to describe the process. People can just describe here is what I want you to do and you figure out where this is. And there are lots of websites that are just like the model to me many times feels superhuman. I go to these websites and I don't know where things are. Like what are you talking about? Like where is this thing that you want me to do? And I have to click drop downs and figure out where where the relevant portion is. Ravid Shwartz-Ziv: But like instead of like building like based on on images and vision, like you don't think like we should just like change the website themselves like to interact better with agents? Allen Roush: Well and and Dhruv Batra: So that's Allen Roush: well and and sorry, and related to that question, isn't it tokenization that causes serious problems like image tokenization since so much of browsing websites is literally processing screenshots of them at these agents? Dhruv Batra: Go ahead. so I'll take the second question first. tokenization is actually not as much of a problem as people think it is. images are actually quite token dense. a typical image is what a thousand tokens, depending on how you patchify it and how many you're creating. if you ask for the full HTML and DOM or the document object model accessibility tree of a website, you can get into tens of thousands of tokens quite easily. because Not only will there be actual textual content that you want to show to a person, but there'll be just you know divs inside divs and descriptors and fields and button labels and things. And so then you get into the business of DOM pre-processing. That, okay, I will try to find the important portions or the interactive elements or the button labels. And that is just a minefield of feature engineering because what you do for one website will not generalize to another website, will not generalize to another website. The reason why images are extremely efficient is that's the source of the truth. It's extremely dense. And actually, there was a there was a really cool paper by DeepSeek, I think it was their OCR paper that was demonstrating that images are a fantastic way to compress context. So even textual content, they were laying it out on a page, taking a screenshot, and tokenizing that. And that give them extremely high compression rate of the of the tokens, which were purely textual tokens. so that was one question. Raveed, your question was why not change the web and identify the web? Is that right? Yeah. so that's a fantastic argument. I've thought about this in the past. I think people underestimate how hard it is to change infrastructure. Ravid Shwartz-Ziv: Yeah. Dhruv Batra: and I I will answer that by by giving you an analogy. We could have built autonomous cars by building dedicated lanes on the road. It would have been far easier to on every major highway and roadway say that there are special lanes where only autonomous cars travel and we will create sensors for them. those sensors let them access to distance where they are localization and that would have basically make them glorified trains. And that would have been an easier problem. We decided we we we owned up to the fact that roadways are a heterogeneous infrastructure. They are owned by many different people. It takes a lot of time to change infrastructure. And it is easier to share that roadway and operate like a human. And that's how we made progress on autonomous cars. Now, of course, it is true that there is a gap between the physical world and the digital world. It is much easier to change infrastructure and digital world than the physical world, but even the digital world changes very slowly. Like I I will give you an example. If I ask you the question, there is an emerging market, not an emerging market, but but in the market of let's say Germany and France. I will give you a list of 500,000 restaurants and I want to tell I want you to tell me you know, what is the median price of soda or whatever drink or a wine bottle in all 200,000. those restaur restaurants are like those restaurant websites are extremely heterogeneous. sometimes websites post menus as As actual items. Sometimes restaurants post menus as PDFs. Sometimes they post them as screenshots of individual pages of the menu laid out in a display. The web changes very slowly. We think that suddenly overnight there are going to be MCPs available. A lot of these systems were coded a long time ago, and they are owned by an extremely heterogeneous small set of players. Ravid Shwartz-Ziv: Yeah. Dhruv Batra: there is a long tail of the web. The the head of the distribution, the aggregators, will modernize and offer APIs, but the long tail will not. Your your indie developer that coded up the restaurant, the the the tennis code reservation system in Golden Gate Park in San Francisco is never going to have an API available. Yet it is it is individually not a high value enough task for them to change the website. But cumulatively the web is extremely valuable for you to automate it. I suspect the way we're going to API for API fy the web is through browser use and computer use agents, clicking buttons. Ravid Shwartz-Ziv: So Allen Roush: and and and when when Ravid was talking about changing the web earlier, I think part of it might be around like you know, the current DOM problems of tens of thousands of lines of tokens you're talking about being largely a result of the proliferation of web frameworks such as React and and all of the the things before it. And I'm wondering You know, there's been some people hoping that a combination of generative AI systems and high hardware prices would start disciplining developers and leading to higher quality code that's written more kind of targeting bare metal. and do you do you think that maybe we'll see a movement away from these web frameworks which might, you know, create pretty for human outputs but are extraordinarily difficult for LLMs to parse through? Dhruv Batra: It's certainly possible that the cost of change, the cost of migrations, software migrations, library and infrastructure migrations has come down with with coding agents. but I think people it like the the migration has to come at a at a value proposition. The reason why someone undertakes this project is because they think that they're that As a consequence of this migration, they will see increased traffic or they will see increased revenue. and and therefore they are willing to invest in this. If the alternative is that the agents can just share the roadways and click buttons and you know make bookings and answer questions and buy things and you know check out items just like a human would, why would you bother with the with the migration and at the risk? There has to be some trade off that you're that you're undergoing. Ravid Shwartz-Ziv: So w okay, so how you like how you try to to deal with it? Like what is like a at the end like what is the solution that you're trying to do? Like you're trying like to train your own agents and like you're using like cloud agents to to solve it? Dhruv Batra: So we train our own aid we train our own models. we don't pre-train, we focus primarily on post-training. all stages of post-training from SFT, rejection sampling, reinforcement learning on both synthetic websites and real websites. and in the last few months, one of the most interesting developments in this literature has been On policy distillation and multi-teacher on policy distillation. So you know, ultimately the model takes in a screenshot and a history of past interactions in this episode with this website. It predicts an action, which is a UI action like button-click, scroll type, or a custom JavaScript code that it wants to execute. the trajectories are anywhere from 10 to 100 steps long of interactions. we can When we started two years ago, one of the first things that we did was because the models were so poor, in order to bootstrap, we had to have human demonstrations. so you, you know, hire a team of annotators, you ask them to install a Chrome extension, you give them a task, they are clicking buttons on a browser, you are recording those buttons. that gives you human demonstrations, this is analogous to how robots are taught with imitation learning by giving them teleoperation data. this doesn't scale fundamentally because there's only so many humans you can hire, there's only so many hours they can work, that it doesn't scale. Ultimately you have to get to agentic data where your models are operating on the web. because this is a purely digital space, creating simulations is actually easy. the centurial gap between front ends is small, especially as code generation models get some sort of design sense, they can create front ends. One subtle point though is, and this is this was a learning for us as well, I think people forget that creating simulations of computer use models is actually easy, but creating perfect replicas of browser use models and you know websites is actually hard because you don't have access to the back end of websites. if a particular national let's say National Park website acts in a particularly weird way that if you if you go to their search, you type the keyword of of and it and that search does not work, but if you type something else, it will work. you don't have access to that webs to that backend about how it's implemented. And so if you create a replica of this which gives you perfect search, you train on that, you end up with the same symptorial problem. That your agent will expect search to work and it will not be able to recover. And so you don't know the quirks of different websites. In order to solve that problem, we actually learn from live websites. So our agents are serving not just consumer traffic, where people are people give it requests and our agents go on real websites, but also during reinforcement learning, our agents are acting. have an environment which is a browser which is acting on live websites and so the agents are taking actions on real websites. What we have to do is work on verifiers for the tasks that we have given on live websites. It means that we don't have to simulate that website, but we do have to develop we do have to know when that task was done versus not. And we have to put in guardrails for certain actions not to be taken. So for particularly for read tasks where you just want information from a website, live websites work fine. For write tasks, if you want the agent to practice buying things, you don't actually want to hand it a credit card and go wild on on real websites, then you can create synthetic checkout flows and and other things and you want it to practice that in simulation. yeah. Ravid Shwartz-Ziv: So so how you verify like the these tasks? Dhruv Batra: You have to write verifiers. so the first version of this problem f of this solution is to write hand designed verifiers. this is quite similar to math and coding problems, where you know at the end of the in in math and coding, there's there's a chain of thought, then you give an answer, that answer has a correct answer, and so you can write a verifier against that correct answer. Here let's say it's a checkout flow. and if it's if it's a synthetic website, then we have access to the back end of that website. We know the state of if you're on an e-commerce like if the task was go to this website, add this item to cart, buy it, then if it's a it's a simulation, we are tracking exactly what has been done in it. And so we can tell whether the agent purchased the right thing. If the agent is acting on real websites, Then writing verifiers is slightly more challenging, but it can be done. So initially, humans can write verifiers by, I'll give you an example. If the task is go to Craigslist and search for an apartment or an item that someone is selling with these constraints, when you click certain filters on a site like Craigslist, it adds query parameters to the URL. So your verifier can read those query parameters and decide that yes, the agent applied the right filters. So in the reinforcement learning literature, this would be called actor critic methods where the critic has access to privileged information. here we're you know, we're not we don't actually have critics, but these are verifiers that have access to privileged information. Ravid Shwartz-Ziv: And do do you think like this is the the f the future? Like that now like we want to train our agents for different like scenarios in the in the web, so like we will use like these verifiers? We'll create like very specific verifiers and Dhruv Batra: Hand-designed anything doesn't scale. And so it's not the future. It has to be replaced. And yeah, it it has been replaced. But I do think there's a part of this idea that is the that is indicating at the future, which is I think in general in agentic systems, we just have to learn on the fly that we have to learn with the real environment that we're going to be working with. So This is why learning from live websites is important. that there are no synthetic replicas, there's no simulators, there are some for certain certain scenarios. But the the web is an extremely heterogeneous place and simulators will only capture and th this is again a hard lesson that you learn after building simulators. Simulators will only capture some aspect of of that diversity. If you want the full exposure to the diversity of the web, you just have to operate on the web. what will get better over time, which is what people are showing progress on, is LLM as judge and agentic verifiers that are deciding when a model has succeeded or not. Ravid Shwartz-Ziv: And so I have like unrelated question, like so if agents like do the the browsing, right, like and it means that like humans stop seeing ads, right? So who who is going to pay for the for the web in this world? Dhruv Batra: Yes. So this is an extremely interesting future that we are headed towards. I do think that the web has to change. Right now the web is laid out for human consumption and so ads are laid out for human consumption. I think the future that we're heading into is one where agents are the primary drivers of action. websites will cater to agents rather than humans. Like why should you and I spend our precious cycles. Like you and I, I'm I'm I'm sure you've felt this too. You're trying to do some task on a website and it's just pesky. It's trying to annoy you and distract you. It's trying to show you things that almost get in your way and you you find yourself 30 seconds in, wait, why did I come here? right, I'm trying to do this and like I got distracted with something. I think that is the consequence of the web trying to extract value out of your eyeballs and your attention span. I do think that future in the future that will not be the case. we will have advertisement possibly. it could be the case that agents pay for access and there are certain protocols emerging in that case that when an agent wants an information from a website, it identifies itself as a as an agent operating on behalf of a human and pays a price for accessing that website. Or it could be that You pay my agent if you wanted to check out your website. My agent has access to privileged information, which is my preferences, my goals, and my agent does not get distracted. My agent does not get my agent is more objective than me. My agent is more persistent than me. And you try to convince it with advertising. And if it is actually aligned with my goals, maybe my agent will pursue it. and maybe you give a discount to my agent and it will come to your website. Allen Roush: So, do you think because because I've I I've seen so many investors asking about what's happening with the software as a service industry, and I tell them that everybody needs to move from pay per seat to pay per usage pricing, which would involve basically taxing MCP servers, which is basically what you're talking about right now. Do you think that that scenario of like people's agents paying taxes websites is realistic? I for the record, I I think something like it seems inevitable at this point, but I don't know at all how you would implement it, right? So I'm curious what your thoughts are. Dhruv Batra: Yeah, you're absolutely right that there is like paper seat doesn't quite make sense for this new world if there isn't no such sense of a seat. Like, you know, if if agents can do a lot more work on behalf of humans or developers, individuals, teams, companies, then really it's some sort of process based thing. There is of course the talk about outcome based pricing, but your question was more about process-based pricing. I do think acts I do think whatever solution we come up with has to make sense for all parties involved, right? If you are a store owner, you put up a website that indicates the price that you're willing to accept for you for the st items that you're selling. And if agents are simply taking that information, then we're back in the world of bots and scrapers. Then you as a store owner pay the traffic cost of serving this traffic but see no benefit and that is not that is not an equilibrium that is sustainable. What is sustainable is a win-win solution. Like you have to feel as a business owner that giving access to your information is in your best interest, either giving it freely or at a price. Or alternatively the business paying because the business Businesses are used to advertising and and you know, if you build something interesting, getting in front of relevant people is a extremely hard problem. It is a very valuable business. There are massive industries that exist just serving that role. and if that channel now is AI makes total sense to me. I think my hope is that we live in a world where a lot of These interactions are mediated through AI systems. there is value in intelligent systems mediating these interactions. They they can operate on your behalf, they're tireless, they they don't get easily distracted. Why not? Ravid Shwartz-Ziv: So we're almost out of time. I just have like one more question. so do you think like there are so many companies these days that are trying to do like agent search web search agents. why like do you think like why do you think you have a better chance? Dhruv Batra: so the what I would say is that I think web search is a completely different activity. It's complementary. so search is the start of is the start of a journey. so what is search? you are building an index of information online. you build that index with some frequency, you serve that index either to you know, classical search was serving it to humans based on keyword. Modern searches, you're serving it to agents with a different set of constraints. Like a agents won't access extremely low latency access, humans won't care as much. search cannot tell you information that requires interacting with a website. so simple example. You ask the question, you want to go to a concert or a soccer game. try searching for what is the price of a ticket. That information. Is not searchable because there is no static website that lists those prices. It's a dynamically changing piece of information. It you have to go to a website that will ask you questions about which date, which website, like which event do you want to go for, where do you want to sit in the in the event ground. It it and then it will tell you the answer to that information. So this requires interaction with the UI front end. Which is pulling information from a database that is that search does not have direct access to. And it can give you an answer, and that answer is valid for some point of time. It will change. And that is the difference between search and browser use, if you will, and computer use. Booking a restaurant reservation is not a search task. Search is the start of the journey. It will tell you that if this information is available here, and then You have to interact with a website, a front end, you filling a form, you know, moving data from one CRM to another CRM. That's not search. And that I think requires perceiving this UI like a human would. What we have done is we have plastered our digital world with user interfaces that are visual, because it's a high bandwidth communication channel into human eyes. We have to automate that. Sometimes humans will operate those UIs, sometimes machines will automate those UIs, and that's what computer use and browser use models do best. Ravid Shwartz-Ziv: Okay, great, yeah, I kind of agree. we're out of time. Do you have anything else that you want to to add? Dhruv Batra: I think yeah, sorry, the only thing I would add is you know, our our company is called Yutori. It's a Japanese word for a sense of well-being one experiences by living with mental spaciousness. I hear a lot of anxiety in my friends and other people and especially younger people who are seeing a lot of change about AI. I think the dream and the hope was that AI Offloads the dull and the dreary and makes space for us to focus on the things that are important to us rather than lock us into into an ever increasing pace of treadmill. so that's what I would leave people with. That I think there is at least some corner of the ecosystem that is trying to build solutions that offload cognitive dreariness and give people more space and time to focus on the things that are important. Ravid Shwartz-Ziv: Do you believe in that? Do you believe it like at the end like we will I don't know work less and like have like the agents will do all the things that like we don't want to do and we will just I don't know, keep only the the fun things? Dhruv Batra: Don't I don't believe that I don't I don't see a world in which people don't want to work because it seems like there is a human desire to focus on purpose, meaning, and people find meaning through work. but the arc of history has been that the dull, dreary, dangerous. gets offloaded to machines. And I do hope that's can that continues. And I it seems like it is continuing. We used to have extremely dangerous professions. We used to send people unprotected into mines, into, you know, into on on ships across transport where people died. We are making progress there. That is on the dangerous path. That's where robots come in. There is the dull and the dreary path where I can I hope we are able to offload it to machines and focus on things that are important to us and that arc seems to be continuing. Ravid Shwartz-Ziv: Okay. yeah, in with this optimistic point of view. we end of time. So thank you so much for joining us. It was a real pleasure. And thank you for the audience. Dhruv Batra: Thank you, Rabid. Thank you, Alan. Good to do. Allen Roush: Yeah, it was a pleasure and and gr great to meet another optimist. Dhruv Batra: Thank you for having me.