Latest / The Information Bottleneck / Why AI Still Can't See wtih Andrew Dai

Transcript

Why AI Still Can't See wtih Andrew Dai · The Information Bottleneck

• Ravid Shwartz-Ziv: Hi everyone and welcome back to the information bottleneck. And our guest today is the co-founder and the CEO of Illumin AI, Andrew Day. Hey Andrew, nice to meet you. Thanks for coming. Andrew: Nice to meet you on me. Thank you. Ravid Shwartz-Ziv: And as always, ALN Allen Roush: Hi Ravid, hi Andrew. Nice to meet you, Andrew. Andrew: Nice to meet you, Alan. Ravid Shwartz-Ziv: So Andrew today we are going to talk about different things but mostly about visual reasoning and about your your startup. So maybe you can start from from the be from the beginning, a bit about your background. Andrew: Yeah, sure. so my background is I grew up in the UK, did my bachelor's at Cambridge, the UK won, PhD in Edinburgh. and then after that, Google Brain was starting up and that seemed like the ideal place for me to join. So I moved to Silicon Valley with the goal of joining Google Brain. Initially I was in like the Google Now team, but two years after I joined Google I joined the Brain team. And that That was when it was only thirty or so people. so when Jeff Dean, Oriol Vignals, Cockley, who've all left now and started Disco Loop, so they were all still there at that time. Ravid Shwartz-Ziv: So how how did you how how how did you start it to to work on maybe language models or deep learn deep learning in general? Andrew: Yeah, so I joined yeah, after I joined Google Brain, that was about the time that like Oreo Ilya and Quark were working on sequence to sequence. so it was very clear then that deep learning for language really works, really is effective. They they showed it was effective for vision a few years before that, but 2015 was really when it was shown it's really good for or 2014 for language. And then yeah, a year after I joined Google Brain in twenty fifteen, Kwokle and I we wrote the first paper on pre training and supervised fine tuning. So like we actually have language model pre training in the abstract of the paper there. and yeah, that's the paper that led the groundwork for GPT, chat GPT. So all the GPT papers cite our original paper and I've been working essentially on language models ever since then. So for the last eleven years. Ravid Shwartz-Ziv: Wha what was the idea behind the like pre training the models first and only then fine tuning it? Andrew: Yeah, so that wasn't idea we came up with actually so that is more of a discovery. So I w I was like training these models. The project goal was to at the time there was a paper called Paragraph Vectors that Quark had written. and we were basically trying to find a better version of paragraph vectors, adding in sequential capabilities. Sequential representation. So combining paragraph vectors with LSTMs was the goal of the project. And I trained a bunch of the models like that. that at that time TensorFlow didn't exist, so I was using disbelief. and so I was training these models, nothing worked, nothing was better than the state of the art. But then one day I was trying a new experiment and then suddenly it was like state of the art for language understanding. for sentiment classification at the time. And then we were really excited, but when I re-ran the experiment, I couldn't reproduce the original results. So I think these Ravid Shwartz-Ziv: Your results? Andrew: Sorry? Ravid Shwartz-Ziv: They're they're like your results, you didn't you can't reproduce your results. Andrew: Yeah. Yeah. I couldn't reproduce my results. so I think like any a typical researcher would be like, that was just a fluke, something in the R and G just move on. But back then, you know, I was a young researcher, so I was like, I'm gonna figure this out. I can out what's going on. And so I looked at my bash after a while looked at my bash history and I saw that when I started the experiment I forgot to update the checkpoint directory. So I was actually using the checkpoint from a previous run and that was a language modeling run. And so I started from scratch again I trained language model and then I Yeah, effectively fine-tuned that for the sentiment classification task and I was able to reproduce the state of the art result. And then in the next three months, we also applied it for images. We also did like next row of pixel prediction for images. and then we submitted the paper to NeuroPs and that's the paper called semi-supervised sequence learning. Ravid Shwartz-Ziv: like so today like it sounds trivial right like to pre-train on on some more general data set what was like I don't know the like why people didn't do it before why you think people haven't done it before Andrew: I was I think like back then pre training and unsupervised learning was very early. it was like basically in a synthesis. it was very it would very hard to get A like semi-supervised or unsupervised model to beat a supervised model. It was just very hard. People have tried all sorts of things. But the one thing people didn't try is fine-tuning an unsupervised model. So that's the one thing that was obviously the secret ingredient in the recipe. but I think even today, I've talked to many researchers about it as well. There isn't a strong theory for why it works. so we still don't know why it works. So it's very hard to come up with this approach from first principles. Ravid Shwartz-Ziv: And do you think like okay, so well like we all know the advantages but like recently there is all these like both companies and ideas and and and different tried like to to go to go farther like to to break this paradigm right of like pre train, mid train, post train. what is your opinion about it? Andrew: my opinion is that you can try, like people should try. I don't think people should ever stop trying new approaches. but the fundamental reason why this approach has been working over the last ten, eleven years is just one thing and it's just scaling. so this approach as we've seen, scales better than any other approach. It can consume data, it can grow in capability just by pure scanning. And you don't see that with any other approach. You don't see that with like auto encoders or GANs. all all the other approaches. And I would say there is actually yeah, I'm not sure like if this is widely held, but my belief is that the reason why this scales is because there are no shortcuts to the solution. So language modeling, next token prediction is a very hard task. You can make it as hard as you want, right? but things like autoencoders, like denoising auto autoencoders, VAE's, I worked on all those things, you can always find a shortcut that gives you the answer. Same with GANs, right? There are degeneracies in the solution. and my feeling these days is that if you have degeneracies, fundamental degeneracies, you know solution that you need to patch by you know hacking around the gradients or stuff. I think like as you scale up, those degeneracies, they get worse and worse and get much harder to hack around. And so at some point you just stop being able to scale. Whereas for next token prediction, there is no degeneracy in the solution. There is no degenerate solution anywhere. and so it's able to scale. Allen Roush: Does that mean that you're bearish on diffusion language models then? Andrew: diffusion language models. Yeah, I think at some point I think they will hit the is similar kind of thing. Like they s they will scale up to a certain point, but I don't think they will scale to the degree that auto-regressive language models have. But I'm not saying that auto-regressive is the final objective, it's the best one. I think there are other ones out there, but it's the best one we have so far. Ravid Shwartz-Ziv: So okay so like you you you you've been like you were at Brain right from world to Vec and seek to to s to seek area. so why actually like you think like Google today is kind of like I don't miss the train and like is not one of the frontier frontier places? Andrew: yeah, so that's yeah, that's a pretty hot topic. I think. I think in the beginning, I think we can talk about the beginning of Gemini. So the beginning of Gemini, people thought that Google had been left behind, right? and that was true in some sense, but it was also not true in other senses. Ravid Shwartz-Ziv: Why why you think like let's start even before? Like why you think like right like why from twenty fifteen or like twenty seventeen or like twenty eighteen, like why you think like Google didn't understand the I don't know the importance of of language models and didn't put enough effort there? Allen Roush: And is it inevitable that they'll stay behind? Andrew: Okay. Yeah, that's a lot of questions. well I'll start with twenty seventeen. So twenty seventeen the Transformer paper came out. The day after it came out I pinged the Shish Vaswani, who he was a friend, we were in the same building. and said, we have this paper from two years ago. where we showed if you do language modeling and then you do fine-tuning, you get a very good state of the art language understanding model. And I told him you should try this on Transformers. But he was too busy working on vision and images. And so yeah, so that's how ChatGPT happened. And so that's also why I didn't work on anything vision related for the next nine years. but I think at that time scaling wasn't really in the mindset of Google Brain. So Google Brain was about foundational research, research that could be applied to products very easily. Just scaling these AI models wasn't something people did. It wasn't something people even considered. I think that's where Well, Ilya. Saskava really believed in scaling, really believed in committing to one approach, whereas Google Brain still had the approach of, let's like have let a million flowers bloom. eventually the work, the one that is good will succeed. But of course that takes time. That takes much longer than opening eyes approach of just like picking one winner and just scaling that. so there's that. and then the other thing from 2017 to 2022. period. Google did have some chatbots, right? There's a Lama work and or Lambda chatbot, Laccox group group did that never saw a public release. and that was for, you know, organizational reputation, Google's reputation, right? Google doesn't want anything. I think at at that time you also saw the chatbots that came out of Meta and Microsoft. I think there was a famous chatbot that from Microsoft that had a lot of obscenities around them, right? Yeah, Ravid Shwartz-Ziv: Yeah. On Twitter or something like that. Andrew: that was trained on Twitter tweets and it was online for like twenty-four hours or something. And so that put all the Google leadership in the mindset of no, we can't we can never let this happen to Google. and Ravid Shwartz-Ziv: Mm-hmm. Andrew: that if Lambda was released, there was a very real chance of that happening. Right. So I think the first release it did have to come from the startup rather than big tech. And so that's how GP Ravid Shwartz-Ziv: Okay, so so yeah, so like ja like th this is how like Google like kind of like miss it, but like then G Gemini was like established, right? And like this organization was like formed and and but but but and for some point like at some point like we thought that like yeah now Google can actually like make like they have so much compute and so many like smart people, right? And so much data. but somehow like it didn't like worked out. what w why you think is it and like is was it like inventable? Andrew: Yeah, so personally I b I believe that was avoidable. so th this came out because of the way Gemini was formed. Gemini was explicitly created with a fifty fifty design, right? So every area in Gemini had a co-lead from London and a co-lead from Mountain View. Right. And you can imagine how that works, right? You've never worked with each other before, you never even heard of each other's names before. One one team has been working on a silo for the past previous at that time, I don't know, almost 10 years. and they've always had this ambivalent attitude with each other, right? They've been working on overlapping projects, they've been racing to get publications. So you of course you can imagine like what that culture would be like when you're suddenly saying, now you have to work on the same project together, right? and so Gemini One You can see it was from I think like a forced what happens when you force two technical teams together. and what happens when that when you do that is that some teams lose out, some things are reset, some lessons have to be learned from scratch again. And so before Gemini we had Palm 2. Palm 2 were like at the time we finished training on it, it was better significantly better than GPT 3. obviously it wasn't better than GPT 4 when GPT 4 came out. but that was for because it was delayed, I would say. for Google I.O. but Gemini learned that lesson and in the future they didn't delay any releases for Google I.O. right but yeah ultimately a lot of the lessons from Palm 2 did not go into Gemini. it was a partial reset. Allen Roush: Related to this, I'm I have so many questions about some of the Gemini model releases. In particular, Gemini One, one point five, and maybe up to two and two point five had absurdly long context windows. And I think La Meta's Llama Four also had on on Scout like a ten million context window. And I know that the effective windows were small fractions of these numbers, but I'm curious why were they so long at the time? I think it was like two to ten million as well for Google's Gemini models at that time. Andrew: yeah, I mean we we found techniques that allowed these context windows to be very large and I think part of the belief was that if the context window is large enough, you can effectively have a GI, you can do anything, right? you have like an infinite memory, you can reference any book, any web page that you've ever seen, any email, right. but I think now we're seeing that yeah it's much harder than just the stated context window, the effective context window is yeah, not as long. Allen Roush: And and and and the problem is is specifically we don't have enough compute and it's really bad scaling. Not that the whole like raising the context window doesn't actually help the model, right? I just want to confirm that because that's my understanding. Andrew: so raising the context window, it lets you do more things. That's what I would say. It just lets you enable like a bunch of more applications. Like there was an example where they put in this like dictionary or some something of a lo of a language that very few people spoke, and then you know, Gemini could output things in that language. So it does unlock new capabilities. but Yeah, on the compute side, yeah, obviously compute is very expensive now. So that has to be kept in mind too. And it uses a lot of memory. And memory is also very expensive. Ravid Shwartz-Ziv: So you you call it the the Gemini pre training data, right? and like and everyone says like yeah, data quality, right, is everything now. but I wonder wh what does it actually mean in this scale? Like what what do you think is like good data? and what are actually like decisions that that matter the most, you know? Andrew: Yeah, good data is data that you can imagine a human can learn from, right? Something you can get out of it. So it's very it's like thinking about it from a human point of view does help. So if you imagine reading a bunch of textbooks, you will get more out of that than if you read a bunch of like spammy SEO type web pages, right? And it's the same for language models. It's basically we want the models To extract as much information out of the data as possible. And that is only possible if the data is high quality. Ravid Shwartz-Ziv: Do we have a metric to measure it, like a good metric to measure it? Andrew: There are various ways you can measure it. we developed some ways inside Gemini and then there are some papers on techniques you can use. and it's a very active research field. Ravid Shwartz-Ziv: Okay, so but and now about like c how much you think like the data is actually matters. So like do you think like I don't know the data mixture is super important versus like other factors, right? Like I don't know, post training or like even like the architecture or things like that. do you think this is the the most important part? Andrew: So on pre training data is the most important factor. if you look at the architecture, even if you look at the open source architectures, architectures are converging. They are not diverging. Everyone is figuring out the same kind of recipes at work. similar architectural tweaks. And basically on architecture, it's not like some fancy thing where you're imp improving the representational power of the model. You're just making it easier to optimize. Like every architectural tweak makes a model that much easier to optimize. Right. So ultimately a lot of these are optimizational optimization benefits. and data fills in the capability path, right? Without the data you wouldn't have the capability. And without the right data, you wouldn't have the capabilities that you want. for post training, I think the RL algorithm Is pretty important. but and then there are a lot of differences between different RL algorithms. but even for post-training, of course, data is important, right? RL data is really critical. If you're doing RL on wrong or bad data, that's not going to be very good for your model. So in some sense the data quality bar is even higher for post-training. Ravid Shwartz-Ziv: And do you think like we should have different types of data for pre training versus post training? Andrew: Yeah, definitely. for pre-training you want something that can give you a whole range of capabilities. And post training these days is about sharpening specific capabilities. So they have very different purposes. Allen Roush: So so the the mm The claim about model architectures converging. So I I haven't checked maybe Google's architectures recently, but it seemed like we were seeing a lot of diversity out of China in several ways. One of them is increasing usage of linear attention layers in various types of hybrid approaches, of like interweaving them with transformer layers. and it seems that the jury is still out on if sparse MOE or dense model are better and we seem to still get both. And so I'm I'm curious, like do you do you recognize this and still like conclude that that that we are still seeing convergence like in the industry? Andrew: of course I can't say anything about Gemini's models, so don't take any any of this as an indication of what's happening inside Gemini. but in the industry as a whole, there is general convergence. You can see that from the large models, right? All the large models are MOEs. No one is training large dense models. I think you see dense models for very specific cases very specific niches cases where it's more memory limited maybe for like yeah mobile device kind of situations those kinds of things but in terms of like knowledge nothing beats sparse moe in terms of the knowledge you can cram into a model right and then of course they're also a lot cheaper to serve and then you also see like these developments like shared experts Many small experts. a lot of these things are being taken up by the rest of the industry, specifically like the open source models, very quickly. Right. and I th I think you do see a lot of these this effect where say Deep C comes out of it with a new architecture, the other open source models will adopt that very quickly. and the reason they adopt it quickly is because it works. Ravid Shwartz-Ziv: But do do you think like this converge is a good thing because like for example several weeks ago there was here Rohan, Rohan Anil from Carto Machine, right? And he basically said, Look, everyone are doing the same, everyone are looking on on the same problems and the the same solutions. And we need to come with with with better things to do, like we need to come with better algorithms, we need to come with better architectures. For example, like this like discrepancy between like pre-train and post-train, right? It's like you basically ruin the loss of of the pre-train while you you're post training, right? Like this is like it doesn't make sense. Like we need to solve it. so what is your take about it? Do you think like it's it's a good is it's a good thing to to that everyone are doing the same thing and like you we have kind of like the the ideas are basically the same across all the all the industry? Andrew: Yeah, as a researcher, Ravid Shwartz-Ziv: And why is that? Andrew: as a researcher, that's obviously bad, right? As a you never want to see an academic conference where like everyone is having the same idea, right? So it's bad that we don't have like a more diverse range of approaches out there. But if you take think about this from optimization point view, Right. in some ways it's not surprising because everyone effectively has the same objective. A big portion of everyone's training data is exactly the same, which is obviously the internet. everyone has access to the same internet. Some people have access to more, some people have access to less. But most a lot of that is the Ravid Shwartz-Ziv: Do you still think Andrew: same. Ravid Shwartz-Ziv: by the way, do you still think that like the internet is so important in training these models? Andrew: yeah, definitely. If you didn't have the internet, what else would you try not? Ravid Shwartz-Ziv: No like today like you have right you have synthetic data, you have a lot of different right like books and like even like the the environments and things like that, right? Andrew: Well the book's quality is still I mean the book's quantity is still a tiny fraction of the internet. Definitely not enough for for pre training. And environments the diversity is very low. You're basically stripping away a lot of the diversity you can find on the web. Yeah, like there's no environment for like polishing some shoes, for example. Ravid Shwartz-Ziv: And do you think the quality is degrad degradated when now we see like more and more AI generated content? Is it harder to Andrew: Yeah, definitely. Ravid Shwartz-Ziv: train models now? Andrew: Definitely the quality on the web in general has gone down with AI generated content. So that's why like sometimes when I hear people say, these people are distilling, but we're not distilling. That doesn't make sense to me because if you train on the internet, you are definitely distilling someone's model. There's no way Allen Roush: Yes, yes. Andrew: you can detect all the chat GPT or Gemini or Claude output on the web. That's literally impossible. so everyone's distilling each other. The quality is not great. so yeah, there is there are some things bad happening with data quality. Allen Roush: Yeah, yeah. Thank you for confirming that particular belief that you know, about how everybody's accidentally distilling, so to speak. Because I hear policymakers once in a while post you know claims of look at KL divergence diagrams or other things of models looking somewhat similar to each other, and even Deep Seek's saying it's chat GPT, and I'm like, that's not evidence of what you think it is, you know, like Andrew: Yeah, I don't want to weigh in on on too heavily on one side or the other. but I will say if you are training on GitHub data, there is like 100% chance you're training on someone someone's reasoning trace or some model's output. Because everyone the first thing people do is upload those onto GitHub. Ravid Shwartz-Ziv: Okay, let's talk about visual understanding. what are the failures? Like let's start with with current the current status of of visual understanding in like I don't know frontier models. Like what w where are we and like what are the the failure modes? Andrew: Yeah, so current visual visual understanding, the way one way to think about it is like a set of basic capabilities. So these are things like object recognition, object detection, segmentation. And the current models are very good at those kind of things. So understanding like what is this flower or what is what kind of species of cat or what breed of cat is this, what breed of dog is this. And these are very fundamental capabilities is the way I think about it. So imagine you are a deer on like some grassland and you see an animal coming to you. You need to figure out very quickly whether that's a tiger or whether that's a huh. I don't know, something like a hippopotamus or something, right? because if you don't figure that out in less than a second, your your genes are not gonna be passed on. That's that's that's it for you and your species, right? So if you think if we think about it from this point of view, these capabilities are all very fundamental to life in general on Earth, right? and so and these models are very good at it, and so everything's great. But if you think about human human's visual understanding, human perception, we can actually perceive things in much more detailed ways than any animal or even human babies can, right? So we can count how many window panes are on an office building. We can look at the IKEA diagram and piece together some table. It might be wobbly, but it will still look kind of look like the diagram. We can look at the Lego instruction manual and we can infer what's behind that block even though the instruction manual doesn't give you a 360 view of the Lego setup. So the app apparently does now. But you know in the paper manuals it doesn't do that. But you can still build a Lego model Well, like a lot of people can build LEGO models. And so I think of like human perception as being much more advanced than anything you need in the animal world. and that's allowed us to be, you know, to solve a whole range of physical real world problems. and the the connection to the real world is because vision is you know, obviously the highest bandwidth connection we have to the real world. Ravid Shwartz-Ziv: Do you think like is it related? Is it because of this connection to the real world? Or like in some sense, like it's also happening in text and and language, but we just like wrap it with like the LM basically wrap it like with this like hand wave like opinions and hand waving of of of argument. And now like in when you connect it, like when you test it on vision and the thing that are like actually have very concrete like solution you f you you you see it Andrew: Yeah, so You you see it working for like like I said the fundamental capabilities, but for the more advanced things that human adults, the average adult can do, like counting, understanding drawings, understanding diagrams, understanding video, being able to point out details in like a drawing or painting or something like that. That's something these models are very poor at, right? Especially counting, if you just Take your camera now and you point it at something that's more than like 10 things, for example, the trophies in your background. no model can count that accurately right now. and we've been trying. All right. Ravid Shwartz-Ziv: My son has too many chess terminus. Allen Roush: that's that's Andrew: I see. that's all that Allen Roush: Well well what about Ravid Shwartz-Ziv: Yeah. Allen Roush: what about semantic segmentation? Isn't that like the task of trying to separate objects even at like sometimes high frames per second? I thought meta had some pretty good models for that. Yeah. Andrew: Yeah, meta has SAM three. but I think i even for segmentation, right? Segmentation is still I would class as a fundamental capability, not a reasoning oriented capability, because again, animals need to segment in order to survive. you need to be able to segment your scene. But just because you can segment doesn't mean you can count. So we've seen that a lot as well. Like SAM3, it's great at segmenting, it's not 100%. But the slight password is not 100%, that's causing issues with counting. and so even SAM3 isn't very good at counting things. Ravid Shwartz-Ziv: Okay, so we kind of like understand like what are the the problems. maybe let's talk about the solution or like why the current models are not good. Andrew: Yeah, so current models, first they are generalist models, so they try to do everything at the same time, right? They do coding, they do multimodal coding, they look at images and videos, they know like two hundred languages. they do obviously a ton of things. And all those things are competing for capacity, within a model. 'cause the model only has so many Ravid Shwartz-Ziv: Passity in what sense? Andrew: so the parameters in the model and the experts in the model, we do see some competition for capacity there. So like a single expert in a MOE model can't handle both coding and multimodal processing. that's something that we've seen. And so if you build a model that's trying to do everything, then there will be you know, competition between the experts for for that. So that's one thing. And then the other part is that these models right now, like I mentioned, they're all using chain of thought, right? And chain of thought is text-based. So all the reasoning happens in the text space. but if you think about how people reason, a lot of people, I read that it's not 100%, but the majority of people when they do reasoning, they do some mental visualization, right? So when you're buying a sofa for your room or you're deciding where to put a trophy in a room, you might mentally visualize where it can go, where it looks best, where it doesn't interfere with the rest of the things in your space. And you can imagine the same for like an architect, if they need to design a space that's wheelchair friendly, they need to visualize how it's going to move through that space. and when you design when the art when like Apple design is designed the next iPhone, again. He's going to be visualizing. how does that look in someone's hand? Right? So all these things require visualization. It's just this concept called visual thinking in psychology. And these things are not being done by the frontier models, by any model today. And so we believe that this is a really fundamental part of human reasoning. And this is the gap that we're filling. Allen Roush: I I'm curious, you know, there's they they call it you know, humans have different w what do you call like ontological constitutions of how their mind works in in so we we have humans who I think it's called f f ephthasia, ephtasia, something like that, where it's mind blindness where they cannot visualize objects, and they can't even in some cases like like there's a this similar thing of of the people who basically verbalize their inner monologue versus not verbalizing it. And and it seems that humans have quite a variety of experiences, up to and including great artists who, you know, report not p potentially visualizing in the mind and and the vice versa. So I'm curious then, like do you do you think that the diversity in humans implies anything for machines? Andrew: the diversity I'm not so sure it implies anything. I s I think it's more it might be more of a sign that that kind of reasoning is blocked off from access. not that it's not happening. so I expect it's still happening, but some segment of the population just doesn't have access to it for some reason or another. Allen Roush: Makes sense. Ravid Shwartz-Ziv: But so I I wonder like d do you think like why think we need like I don't know new new models? Like do why like just pushing the current models will not be enough, you know, scale it up, for example. Andrew: Yeah, I mean you can see that from Claude Fable, right? So Claude Fable is you know, a big model, bigger and train on more data than Gemini has. but it's still worse in visual understanding on a lot of tasks compared to Gemini. So simple scaling isn't working for vision, not as well as it works for language. And I think some of this goes back to the objective thing I mentioned with next token prediction prediction. Next token prediction scales very well, but we haven't really found the next token prediction for images. or 2D or 3D spaces. For 1D space is very obvious, but for 2D and 3D it's not. Ravid Shwartz-Ziv: Why do like why why I think like I don't know pixels or pictures are not enough? Andrew: I think there's a lot of there's obviously a lot of redundancy, right? so language is much less redundant than images or video, right? image, you know, is can be compressed into a thousand tokens, but a lot of those tokens are just not going to carry any information. there will be like Sections of sky where it's all blue, that's not much information, or like a table surface where it's all exactly the same color, right? so there's a lot of redundancy there. And so if you're just predicting the redundant information, you're not teaching the model anything new, it's not gaining much new information. so I think that's partly the reason why these the vision side hasn't been scaling. Allen Roush: Wha how important is scaling up like the size of images and like in in pixel space? and then what what's your thoughts on like vector-based formats and in general how important are formats like JPEG, PNG? I've heard about JPEG XL becoming an format that hopefully people start using. yeah, just your thoughts on all of these intricacies. Andrew: I think the format doesn't really matter to the models right now because everything gets dip you know decoded to a bitmap. and then from a bitmap it gets encoded into tokens. For pixels, yeah, we want the model ultimately to be able to see every pixel and index into every pixel because there are some use cases that really need that. for example with satellite imagery, there's you know, there's sometimes gigapixels of information and you need to be able to trace down to just a few pixels, which might be, you know, a car on the street or a wildfire. so we s we need to be able to operate at that. level of detail. But I don't think we figured out the the optimal representation, right? So one example is video. Video, the standard representation is you just take a frame every second or every half of a second and then you just feed that a series of images into the model, right? So I think there are much more efficient approaches. But we need to we need to figure out what those are to build more efficient models, a more efficient model for Ravid Shwartz-Ziv: What is your bet how future models will look like? Andrew: well our bet is that feature models will be more specialized. so we're building specialized multimodal models because we believe parts of the architecture need to be specialized for multimodal and we Ravid Shwartz-Ziv: Specialized for what? Like for for what like and what are the like is it for specialized for detecting dogs versus detecting cats? Andrew: specialized for reasoning. So like I said before, right? Yeah. Ravid Shwartz-Ziv: Understanding that he's a dog. Andrew: Before like C V, computer vision was very focused on rep object recognition, object detection, object segmentation. We are building models that are aim beyond that, right? So we're building models that are state of the art for counting, state of the art for track tracking things in understanding what's going on in the video, translating that into language. doing like yeah just like a lot of these like sophisticated tasks that you can you can imagine wasn't really a focus of computer vision of the computer vision field. Ravid Shwartz-Ziv: And do you think like training both like image and text is the solution? Because like at the end like reasoning is kind of like like language is super important, reasoning, right? Andrew: Mm-hmm. Yeah, in ultimately I believe that Reasoning fundamentally is vision-based. And this is because if you just look at evolution, right? Obviously, visual systems came first, and animals they do reasoning, but they're probably reasoning in vision, they're probably not reasoning in language. and then humans came along and added in this like text-based reasoning. so I believe the right order is to have visual reasoning as the fundamental layer of reasoning, and then text-based reasoning built on top of. that. So I believe there's a lot of tasks you can do just by visual reasoning. But there are tasks that you can do with text-based reasoning. And ultimately you need to combine both if you want to cover the whole range of human reasoning capability. You won't get there if you just do text-based reasoning. Allen Roush: How how important are other modalities we haven't talked about, like like sound, or even I know the one that that's by far the most obscure are I guess two for models would be taste or smell, but what about these other senses? Andrew: Yeah, these other sensors are important as well. so for example one is touch. we've seen that some robotics data companies, they are outfitting the workers with gloves so they can measure the touch and the pressure in the fingertips when they're recording egocentric data. smell and audio, there's a lot more work that needs to be done there. I was at the AI summit in Korea last week. I won at one of the discussions there was that audio actually is a lot weaker for AI models to handle than vision. So vision we're actually much better at vision benchmarks and audio benchmarks. And one of the audio benchmarks is like the sound of a rotor and the question is what model helicopter is this? So like quite a hard set of questions. but ultimately we're tackling vision first because the data is much much easier to collect. Like good luck collecting smell data that might need some breakthroughs just to get that. also it's a hundred Ravid Shwartz-Ziv: How you collect how you collect images? Like what are the sources for for that? Andrew: so that's very easy, right? This video on the web. we also have a human rater pool that's collecting visual reasoning data for us. So they are allowed to edit images and do things to images to represent their visual thinking. And then we're generating a lot of data ourselves. so it's just much easier to get that data. But also it's a higher much higher bandwidth data than audio. smell and touch obviously. Ravid Shwartz-Ziv: That's good or bad? Andrew: that's good. Yeah, we just get much more information per token that way. Ravid Shwartz-Ziv: And what about videos? Like do you think that yeah, to to treat differently? Andrew: yeah, videos I think the we're not representing them in the model very efficiently right now. but I think yeah, being able to handle video video is like a fundamental part of working with the real world. Ravid Shwartz-Ziv: And what about world models? you know, like Jan will claim that the solution is to to just to have a Jap style model, right? Andrew: Yeah, great question. so we think of world models as adjacent to visual reasoning models. and the reason for that is that world models they are very focused on direct action, I would say, in physical space. Right. So their goal is to model direct actions in some space for like what happens if you push this box, right? And so that's great for robotics applications, that's great for some If you wanna build a game very cheaply. but that's we think that's about it. So actually there's a whole range of applications that doesn't involve a direct action on the physical world, right? Like just us talking like this. We are not directly acting on the physical world, we are you know responding to facial expressions, audio, all those kinds of things. And so actually the range of applications that you can cover with visual resources. is much greater than just robotics and game creation. And so we are targeting a much broader range of use cases than typical world model would cover. Ravid Shwartz-Ziv: the end the solu like my question is like do you think it's a good solution right like four images four videos four like whatever like do you think like next token prediction is the solution or other solutions Andrew: Yeah, I don't think it's a sufficient solution because direct actions on on the real world is a very limited action space, right? for example, if you're looking at the space of human actions a hundred years ago, there was no like or maybe like a hundred and twenty years ago, there's no flying. Flying would never be an action you would even consider, right? Or interacting with things in the sky. You never even think of that. And so if you build a imagine building a world model back then, you just wouldn't have that. an action, right? but obviously we found that you know the action space of humans is actually increasing all the time because we're building tools, we're building machines, and that that every day that expands our action space. so if you want Ravid Shwartz-Ziv: World models Andrew: mm Ravid Shwartz-Ziv: sorry, continue. Andrew: yeah, so it's very hard to enumerate all the possible actions beforehand that you need to build into your world model. Ravid Shwartz-Ziv: But like old models don't require that you have a specific set of actions, right? Like basically it's just yeah, let's say give prediction in the representation space. it doesn't have to be even like explicit actions. Andrew: Yeah, but as soon as you but yeah, that's a interesting statement because as soon as you move away from explicit actions, then technically any everything is a world model, right? If LLMs are world models, LMs are world models, then you can't distinguish. So the only way I found that you can logically distinguish world models and non-world models is physical actions. But if you have a better way of distinguishing Allen Roush: So so do you do Andrew: that, yeah. Ravid Shwartz-Ziv: No, like I think like most of the people when they talk about role models, at least in this in this like with like I don't know JEPA context for example, is that yeah, you have some representation, right? And like you try to predict this representation, right? And not in the output space. You're like kind of like intervene between the output space and prediction in the output space versus the the hidden space. Andrew: so you mean working so your definition of word model is like working in the hidden space, working in the latent representation space Ravid Shwartz-Ziv: Yeah. Andrew: for a task. Right. yeah, so I th I think that that is From my point of view, that is still more of a research area. So those models haven't been shown to scale. And the closest analog, I would say, looking at the Jeppa equations is actually GANs. So GANs have have kind of a related setup, I would say, where you have two objectives, they conflict with each other. And this goes back to what I said about scaling, in that there are degenerate solutions for JEPA, right? That's why you have the stop gradient. Because if you don't have a stop gradient, you get a degenerate solution. So I think that they will come across the same kind of issues with scaling, that as you scale higher into more dimensions, a bigger model, then those degeneracies become real problems. And I've I've heard this from other people as well, that they've had a lot of issues scaling Jeppa. Allen Roush: Do you do you actually think that robotics need world models, whatever they end up being, in order to kind of get good? in general? I mean, we've already talked about things like improvements in physical sensors, but what do you think is the actual bottleneck? Andrew: For there's a lot of bottlenecks. the one I'm most aware of is the vision bottleneck. so what we've heard from some robotics companies is that at first they assumed that Gemini was good enough for vision, but then very quickly they realize it's not good enough. There are still all the time there are things that their vision systems miss and it's causing issues with the robot, it's causing issues in training, the VLA, issues everywhere. So Really I think if you don't get the vision part right, then the rest of the stuff is like a house of cards. it can break and be brittle. so yeah, vision is like really fundamental. Ravid Shwartz-Ziv: think it's a good evil for for this type of of tasks do you think like do we do we have good enough benchmarks do we need to come up with new ones Andrew: Yeah, I think we need new benchmarks. So there's a couple of famous ones I can mention. So U Pro is obviously famous, it's used by, you know, artificial analysis. Ravid Shwartz-Ziv: But it's quite saturated, right? Andrew: Yeah, it's quite saturated, but also the images you don't need to look at most of the images to be able to answer the question. So it's not really multimodal or vision focused. And also it's very old. So the data is like everywhere on the web now. and then the other benchmark like to point out is Arc AGI. So it's obviously famous because it has AGI in the title, but anyone who actually has looked at the data will realize that There's only 32 by 32 or 64 by 64 pixels. And if you tell me, if you give me an architect and tell him to squeeze his problem into 64 by 64 pixels, he's probably gonna tell you that's never going to happen. So I think that's just like a very artificial world. Good for gaming, but I'm not sure if it's useful for much else. Ravid Shwartz-Ziv: Yeah, and there is like some critics that you actually can like with good enough harness you can actually like solve it quite good. Andrew: Yeah, well with any sixty four by sixty four pixel problem, I think if you if you're a decent enough coder, you can always solve Ravid Shwartz-Ziv: Yeah. Andrew: it. Yeah. Ravid Shwartz-Ziv: So so how do you see like the next future generation evolve? Andrew: Yeah, so we are also working on some evals with some organizations and we are working on making those much harder, much more real world. So things in real-world settings and the real-world settings are very dirty. You won't be able to run a Python algorithm that just gives you the answer, the magical answer. so these that's industries like mechanical engineering or architecture, where the drawings and the diagrams are much more. Complex. So things like CAD where you're working with assemblies. And then there's also a benchmark, a couple of benchmarks came out recently. One is Baby Vision. Baby Vision is still a bit too clean, so I'm still able to run code on that. But this is very recently there's another benchmark called Active Vision. there they actually take photos of these problems, real-world photos of like maps, things that look like maps, things that look like metro maps that are much, much harder. to solve and just not solvable code at all. So I think we need to advance the eval's part much closer to how in the text part we have like GDP valve, you know, deep sweep bench, that kind of thing. We don't really have that in the visual part. And that's why we don't see AI being used much in these industries because the evals are so detached from actual reality, right? RKGI detached from any kind of real world problem. So we need to do much better there. Allen Roush: Wow. It's confirming benchmark maxing really is a problem, for sure then. Andrew: Yeah, especially Ravid Shwartz-Ziv: What about Andrew: order. Ravid Shwartz-Ziv: What about like video, like image generation? Do you think solving video understanding will also help us to to get better video image generation models? Andrew: Yeah, so I guess this is maybe this is a thing where like Yan Yan Khan has a similar idea, which is like predicting pixels is not just it's just not gonna give you the answer. so we think that generation and understanding are two different things. and they are things that are very different in terms of difficulty. Right. So one analogy I like to make is copying versus understanding. So say I want you I ask you to write a essay in Chinese about, you know, World War Two, right? without any AI, right? You're gonna find it very hard if you have nothing. But if I give you some a Chinese essay about World War Two that someone else wrote, and I give you that that then suddenly you can write it right you'll be able to copy that but you won't understand a word of it right but to average person people will say you have great Chinese writing skills So I think something similar, you can imagine something similar happening in the visual world as well. These models are very good at reproducing, you know, famous works of art from Picasso or, you know, Studio Ghibli, that art style, or like produce movies that look like Hollywood movies and explosions, but it's the same kind of idea. They don't understand what they are building. They don't understand what they are generating. And you can see that if you just take one of these AI-generated movies. And I did this like a few weeks ago that I think had nine people in the image. And then I asked the model how many people are in the image, it will say it won't say nine, it'll say like five or seven or ten. Even if you ask the model to generate, like if you ask nano banana to generate a bowl of 10 apples, it's not going to give you a bowl of 10 apples. Right? So there is they still have a gap between generation understanding. Allen Roush: W well so I've I've seen at least with image generation models, people built a huge vibrant ecosystem around them to support image to image and in painting, out painting. They even had regional prompting where you could specify where in the prompt you want. So you could actually just draw ten little circles for, you know, each individual apple. So I guess this is almost analogous in my mind to structured or constrained generation for for language models. I'm wondering, do you see value in these kind of techniques where humans are intervening and adding guidance to models selectively? Andrew: Yeah, I think there's definitely value there. So I think it's very aligned with what we're thinking of in terms of visual thinking. that yeah, there was also a paper on Google VO from DeepMind that showed like VO did have some elements of reasoning that if you show it a puzzle, VO can generate the solution to that puzzle. So I think it's definitely part of the equation, but it's not the whole picture. because I think you still need the text part, you still need the language part to cover. about all the parts of human reasoning. So you need both the visual reasoning and the language reasoning and they need to be combined with each other. Ravid Shwartz-Ziv: And do you think like the future would be that we'll train them like combined or will we train them separately? Andrew: I think this I think the trend, you know, in the last few years is definitely to combine everything. So that's the I think that's probably gonna be the approach that works. but yeah, it's very hard to predict what research Ravid Shwartz-Ziv: But we still see we still see different encoders today, right? Andrew: Yeah, yeah, I think there's more work to be done on encoders, more work to be done on the backbone. but it w it just makes everything much easier if you combine language and vision in the same model. Like infrastructure wise it's so much simpler. Ravid Shwartz-Ziv: Okay, we're almost out of time. do you have anything else that you want to add, that you want to sell? I don't know. Are you hiring? Andrew: Yeah, definitely. we're hiring we're hiring a world class team. So if you don't want to join Frontier AI lab with people who with a deep mind, who led Gemini, who led XAI, feel free to come on to our website and apply. But very specifically Ravid Shwartz-Ziv: In San Francisco. Andrew: we're based in Palo Alto, actually. Yeah, we're based in Palo Alto. We have a great office. It has a patio. we have a fountain. So it's like the ideal research environment. It's very quiet. And then specifically we're hiring for a couple of roles. One is like a RL infrastructure role and another is a model serving role. since we want to release a model pretty soon. Ravid Shwartz-Ziv: Will it be like open source? Andrew: right now we're planning to release API, closed source API. I think open source is very hard to build a business around. Allen Roush: Which is why I'm so fascinated that we continue to get all these open source releases still out of China. And I keep trying to warn people I think it won't last. Andrew: Yeah, I agree. Like economically it doesn't make any sense. but I think strategically for the Chinese government, it makes a lot of sense. So I can see the government bank Ravid Shwartz-Ziv: But even thinking machines Andrew: round. Ravid Shwartz-Ziv: release the model, right? Andrew: Yeah, yeah, thank you. Ravid Shwartz-Ziv: Even like for them it it makes sense. Andrew: Yeah, I think it makes sense if you are playing in the same space. So if you're playing in the same the exact same space as QN and Deep Seek, then you kind of are forced to open source. We are playing in a bit of a different space, which is a specialized multimodal space. And things are happening in that space. And then you can see that the recent QN release the Mac 3.8 max release. They didn't release a vision encoder. so yeah things are a little bit different there. Ravid Shwartz-Ziv: Okay. Andrew, thank you for so much for coming. It was a real pleasure to have you. Andrew: Thanks so much, Ravid and Alan. Yeah, and thanks for the invitation. It's great being on the pod. Allen Roush: Yeah, yeah, this is great. Ravid Shwartz-Ziv: Thank you. Andrew: Thank you. Ravid Shwartz-Ziv: Okay.