Latest / The Information Bottleneck / Text Diffusion Models with Brendan O'Donoghue (Google DeepMind)
Transcript
- Ravid Shwartz-Ziv: Hi everyone and welcome back to the Information Battleneck. And today we are going to talk about text diffusion models. And for that we have Brendan Odengo. He is a director of research at Google DeepMind. Hi Brendan, thank you for joining us. Brendan: Hi guys. Hi. Thanks for having me. I'm excited to be here. Ravid Shwartz-Ziv: And also we have Ellen. Allen Roush: Hi Ravid. Hi Brendan. It's good to be here. Ravid Shwartz-Ziv: So today we are going to talk about tax diffusion models. so Brendan, maybe you can start and like tell us a bit what does it mean, tax diffusion models, and what how it's different from the regular autoregressive one. Brendan: Sure. So so let's start with just diffusion models in general. So diffusion models are a type of generative model where you you lear the model learns to map a a distribution that's easy to sample from to the the target distribution that you want to draw samples from via this kind of iterative process of denoising. And you know, there's it's diffusion models are state of the art for Image modeling, video modeling, protein modeling, a bunch of stuff. And the the way they're trained is you take the the ground truth data that you want to model and you add some noise to it. And in the continuous domain, that's typically Gaussian, but it doesn't have to be. and then you train the model to the neural network to remove that noise. In and there's various ways you can do that, but typically you try to just get back to your original data that you started with, given the input being the noisy. sample and then you train it at a bunch of different noise amounts and a bunch of different data and then when it comes to inference time and you want to sample from the data distribution that you care about, you just give the the model pure noise. So pure Gaussian in the case of Gaussian diffusion. And then you iteratively apply the neural network and kind of move along this this kind of ODE flow of this kind of probability density towards maximizing the the the data distribution that you care about. So you can draw a sample from the data that you care about. So that's kind of diffusion at a high level. And then text diffusion is the the same idea, just applied specifically to the data being text. And you can do that in a variety of different ways. So you can do that continuously, just like for images, but I think state of the art right now at least is to do it using discrete diffusion, which is kind of slightly different setup and a bit different math from the continuous diffusion cases, but it's kind of a similar high level idea where You start with the clean, discrete data, which is in our case text, and then you corrupt it by adding noisy text into the text sequence that you want the model. So you kind of corrupt the sequence that you're you're trying to learn by injecting random tokens distributed taken from the vocabulary, so called multinomial diffusion from a paper from Emil from I think twenty twenty one or something like that. You just corrupt it with the random random tokens from your data. So it just looks like a noisy kind of gibberish string. at different noise levels. So sometimes it's a little bit gibberish, sometimes it's like totally gibberish. And you train the model, kind of like Bert, to fill in the correct information given this kind of corrupted sequence. They try to like to fill in the the ground truth data that it it it was corrupted from. And then when it comes to inference, you do you start off with just pure tokens generated from your sampled from your vocabulary. So pure gibberish. And then you iteratively sample from the model to fill that in with Clean text like that's drawn from your data distribution that's kind of relevant for your prompt or something like that. So that's kind of how the generative model works for discrete diffusion. and how it's different from autoaggressive models. Well, so autoaggressive models they're kind of state of the art still for language modeling. So like you know, ChatGPT, Gemini, Claude, they're all autoaggressive models. And the way they work is they are they're basically trained to do next token prediction. And you know, you have this clean factorization of the distribution into a sequence of, you know, predict the next token given the prefix. And then it's it's just trained in that way to just given a big long sequence, you do causal masking inside the transformer and you can predict the next token. And then you just train on a bunch of data and you can then sample from it by sampling one token at a time. Whereas in diffusion you you could still you still have some context like some prefix that you want the model to answer, but it's just a different way of generating text, which is you start with corrupted data and you you iteratively refine it in this process. And that has a bunch of pros and cons relative to auto aggressive models. but yeah, that's kind of the high level idea. Ravid Shwartz-Ziv: So maybe let's talk a bit in high level. So like why did you bet on tactics? Like right, Brendan: Mm. Ravid Shwartz-Ziv: auto aggressive models were ringing Brendan: Yeah. Ravid Shwartz-Ziv: everywhere, everything, right? So what convinced you that this warfare a whole team? Brendan: Yeah, I think s I started working on Text Fusion like twenty twenty three, like three or three or bit three and a bit years ago now. And kinda at the time I thought that, you know, diffusion was amazing and was producing such incredible like images and videos with like Dali and stuff that which was coming out at the time. and you know, it seemed like, you're auto aggressive next token prediction is a really powerful technique for language modeling, but maybe that this other thing is even more powerful. Like it's just a way better way of generating every other modality of data except text. And it's just about a matter of doing the work and getting the research done and doing have finding the tricks that are needed to make the same technology work for text. And then it would be a massive breakthrough and then it would be no it would replace all the regressive models entirely because it would be a better way to generate text. I that was kind of like maybe naive in hindsight. But I think you know, I still think that text vision is the future in many ways. It has still, you know, there's still a lot of open questions to answer. So it's not a solved problem in any sense. But I still think that the we still don't fully know how to do it and there's still some research need to be needs to be done. And I'm still optimistic that one day it w either we'll replace auto regressive modeling because it's a better way to generate text or at least augment it where it has these kind of niche use cases where you don't want to use an order aggressive model for whatever reason, you prefer to use a text diffusion model. I think we'll we're getting to that world where these two mo mo models or modelling paradigms will exist in parallel. Allen Roush: So so I have a few questions. I guess one is I've heard others describe auto-regressive text generation as a special case of diffusion. And so I guess what your opinion on that kind of take. And then second of all, I've noticed that models that at least today, I don't know if this is an and I guess my question is: is this just like a tendency of how the models are trained today or the data used today, or is this an architectural thing around how in general, at least for diffusion image. Models, they tend to be far more creative, which is to say, Brendan: Yeah. Allen Roush: like stable diffusion and all of its models don't have like a random orange tint on all of the generated images. But then autoregressive images have much better text encoders and thus much better instruction following. and and that's been my experience there. Do you I I I have not even used diffusion language models, so like do you think that there's similar kind of tendencies? And if so, do you think it's architectural? Brendan: Hmm. Yeah, that's a great question. So I'll I'll try to answer the first one first, which is about the autoaggressive and a kind of a special case of diffusion models. I think they are in kind of a limited, kind of not really insightful sense. So I I think you can set up, you know, you can write it down mathematically that yeah, autoaggressive models are a special case of like diffusion with a s with one denoising step and a a block size of one or something like that. So yes it is, but I'm not sure if that really delivers any insight or like or it doesn't really prove that the diffusion models are more powerful or anything like that. yeah, so the s the second question, it has been my experience, and I think a lot of the people's experience, that the that the diffusion models for language tend to be a lot more diverse that in terms of their outputs than audioaggressive models. And can generate you know many, many, many different responses that are very qualitatively different from the same prompt. Whereas if you try to get diversity from an auto-aggressive model, you you can, but it often kind of just rephrases itself in kind of not useful ways and isn't n I I it's just been my experience that it they tend to be less diverse and maybe that translates into less creativity in some sense, like if you if you kind of translate diversity of output into creativity. I didn't know that about the orange orange tint phenomenon. That's interesting to know. But Yeah, I think I'm I think it comes from the fact that diffusion models when you sample from them, and also when you train them, but when you sample from them, you tend to kind of state of the art methods or like kind of predictor corrector methods or whatever, where you tend to inject a lot of noise back into the process. So you start off with a b all pure noise, and since you start off with a source of noise, that's already s a source of diversity because you know you can sample the initial noise in many different ways. And then as you're sampling, you do like say ten or twenty steps to generate two hundred and fifty six tokens. At every sampling step there's the opportunity to re inject noise. As as far as I know, most of the state of the art like samplers re do re inject noise. And that's another source of like entropy and and then the model corrects will will correct it in different ways for how the noise is added, either like positionally, like it'll choose different positions to add noise to, or like the actual tokens that get injected are different as well. And so I that that just ends up being that there's a lot more diversity and entropy in the final sample when you measure it versus autoaggressive models. And this has a a one concrete advantage over autoaggressive models, which is in reinforcement learning. Because what in reinforcement learning what you're trying to do is get the language model to, you know, produce many outputs for the same input so that you can find one where it like got a high reward or like for instance solve the problem. And then you kind of reward the model for that high rewarding sample and you you know you maybe you give a negative reward for something that did really badly or something like that and then you kinda distill it back into the model via either a policy gradient or something like that. And so what you what you want is you want a lot of diversity in your outputs in order to like a make sure you can achieve kind of the the high pass rate when you're given lots of samples. And it it turns out it like it's been my experience and it's been like the empirical observation I have that the order regression models kind of you know, kind of tap out at some number of samples that you draw. You know, you need a lot of samples to get you maybe a a rewarding sample. But because divers because language models tend to be so much more diverse, you can actually more likely to sample something that is totally different to another sample and therefore get a rewarding like more likely to get a rewarding signal and then more more likely to be able to learn from that and and and kind of hill climb on these RL benchmarks. So I it I think it's a real phenomenon that just from this mostly from the sampling procedure, I think, that these diffusion models and my my guess is it happens in image and video diffusion as well that they are tend to be much more diverse than auto regressive models. Allen Roush: And what what's the best quality diffusion LLM today and what what's the delta between the best quality model and existing like best LLM frontier models in your opinion? Brendan: Yeah, there's a few Ravid Shwartz-Ziv: And you need all f like to normalize to to size and things like that, right? Brendan: Right. Ravid Shwartz-Ziv: And data and like Allen Roush: yeah, yeah. Brendan: Yeah, so it's hard to know exactly. So the you know, we just released a Gemma diffusion model, which is an open weights model. That's a twenty six B three point eight B MOE activated. And that's tuned for l low latency applications like to fit on a H one hundred and can hit more than a thousand tokens a second and so on. And it gets reasonably close to the base Gemma four model equivalent that we started from. so I'd say that that's Probably close to state of the art or state of the art. The only other real contender that's close is the is the Mercury 2 model from Inception. It's also a very strong model. That's a closed API model, so we don't know how big it is, but from what I gather, it's it's a lot larger than our model. and it's served on B two hundreds rather than H100, and it's a bit slower. It's about five hundred tokens a second via the API. so it's hard to know, but it's probably between those two and they tend to be pretty close in terms of the actual eval metrics. so It's probably between those two. But my guess is that both of those models are substantially smaller than the the absolute frontier model, which you know, Claude or Gemini or Chad T BT or something like that, which are in the many trillions of model parameters at this stage from what I gather. and so it's hard to know exactly, you know, if someone really did scale up a massive pre train for a massive text diffusion model if they could get close to the frontier or not. in terms of the gap, all I can tell you is that the gap from the Gemma model that we started from to the final one that we produced ended up being about ten percent. And so there's a g there was a reduction in the quality when we took the auto Resser model and tail patched it or fine tuned it to be a diffusion model. But that comes from a few different sources and It's not clear that if we did the entire thing from scratch ourselves, whether or not there'd be much gap. My guess is it's pretty close to performance neutral if you train it on the exact same tokens and so on. So there Ravid Shwartz-Ziv: So why what do you think are the main challenges now to scale it to I don't know, five trillion parameters? Brendan: I think the main challenge pr I mean the main challenge probably is like who's gonna invest in that and who's gonna actually, you know, pay for the compute in order to scale up to a five trillion parameter model. I think like as a technology, the main drawback right now for diffusion models is mm, you know, maybe maybe quality is not quite there, you know, it might be two or three percent lower. Latency is way better, so it's a that's a big pro. It can be like four to eight times faster, something like that. It depends a little bit on the the exact task that you're running it on, can be even faster. But the main disadvantage right now, and it's I know for a fact that this is why OpenAI are not investigating this direction, is because the it just tend it just if if you compare serving a text diffusion model to serving like a a very high performance, highly tuned, auto regressive model like serving s inference setup with like re that's a receiving many many queries like you know many many thousands of queries a second then it it's you're gonna need more chips to serve the same QPS from a text diffusion model because the textusion model you end up hitting a compute bound threshold earlier than a an AR model. And so yeah, go ahead. Allen Roush: And I was just gonna ask, related to that, when you said five hundred tokens a second, were you talking prefill or decode or is that still coherent distinction with diffusion models? Brendan: It it is a coherent distinction. Yeah, I'm talking about decode. Decode. Yeah, there still is a pre fill face. I dunno. Allen Roush: okay. Do you know what pre fill is by care b I I care now because agentic coding is such a big deal, right? And so pre fill ends up becoming more and more and then I okay, good. Brendan: Yeah, yeah. pre fill will be much, much faster, yeah, much faster. Like many, many, many thousands of tokens a second in pre fill. Yeah. Allen Roush: Okay. And then I I guess my my other question is it seems like diffusion language models, I I need to test them. It seems like they're a lot more diverse and creative by design, right? And indeed, I know that the whole theory of sampling with them is totally different. Like temperature becomes a computationally intensive is just one of the many changes that you have to think about. What I'm curious about is it seems like OpenAI and Anthropic also have reduced like control over sampling pat knobs and you know, abilities basically to improve creativity, almost certainly Brendan: Mm. Allen Roush: from a from a alignment and safety angle, because it increases the risk of of getting out of that kind of so Brendan: Right, right. Allen Roush: do you think that diffusion models have fundamental problems with alignment and safety in this regard because they're so much more easy to get more creative outputs out of them. Brendan: Mm, yeah, I mean it's a question. it's it's possible. Yeah, it's possible. I I I I haven't really investigated that too much, but it it c you know, if that's really the reason why Antopic and Open Air have turned off some of those knobs in order to prevent, you know, more diverse outputs, then obviously a a more diverse technology would be something similar. I guess the only data point I have really is that You know, we were you our diffusion gem model, we have to make sure it s it satisfies a bunch of safety benchmarks, which we were able to do. It was a bit more challenging than the auto aggressive model to get those safety benchmarks. So that is a a potential, you know, source of why maybe it was harder to get those safety evolves. But since we've released it, I don't think it's been obliterated, which is this this thing where people are able to fine tune away fine tune away the safety guardrails that we train into the model. As the t at least last time I checked, which is maybe a couple of weeks ago now. no one had managed to fine tune out the guardrails that we put in. So maybe there's something different fundamentally about the text fusion that makes it harder to to fine tune out the guardrails or maybe just it's been less investment and people don't know how to do as well yet. Allen Roush: It's it's funny you mentioned this. I work with the the heretic creator. Brendan: Yeah, okay, yeah, yeah. Allen Roush: So now that you've mentioned it, I'm sure he's probably listening. It's like, I'm gonna check that out. Brendan: Well that's good. Go ahead, please do, and like write a paper how you did it. Yeah. I mean, for the for the sake of the community I'd love to know how to do it. So please, you know, j obliterate it and then like write a paper and tell us how you did it and that'd be fascinating for for us who worked on that. Ravid Shwartz-Ziv: But w what do you think are the like the domains or the use cases that you believe actually like Brendan: Mm. Ravid Shwartz-Ziv: tax diffusion will be better than auto regressive? Brendan: There's a few, yeah, it's a good question. So right now, like like I think a the technology as it is, there are a few cases where it it's it's already better. The obvious one is like low latency applications. So anything you need a latency constraint, it's just you're gonna hit that latency constraint much easier than with a text fusion model than with a neuroaggressive model, because they are like many factors, you know, four to eight to ten times faster, even including like state of the art speculative decoding techniques. So that's relative to the state of the art. like MTP heads and so on. So so low latency applications. And then like on device use cases. So we have some that we've we've actually deployed TextFusion models into on device products within Google because you before mostly because of latency requirements. So you could imagine things like robotics where you you want your your you want a on device model for your on s on your robot to do planning or action control or whatever it might be. And you want that to operate as quickly as it can in the real world and to, you know, react to things that are happening as fast as possible. And you say you want a low latency device on your robot. Same with like a self-driving car, very similar, driving like down the freeway at seventy miles an hour, if there's an L L in there, then you want that to react as quickly as possible and to be able to hit the latency requirements. on device on a phone as well, so we have some work with the Android team potentially investigating putting a text vision model onto the phone in order to basically give the a better user experience because you can talk to your phone or you can get your phone to do some task and it will if it's ratchet running on the phone, it'll respond a lot quicker, it'll be a better user experience. And also uses less battery, it turns out, because eva the text fusion model text fusion in general, because it's so much lower latency, it activates TPU or like the the accelerator chip that's on phones less. And then when it's the phone is the T TPU is activated less it's drawing it's not drawing current. when it's not on and so it actually drains the battery less. So both in terms of power consumption and latency. And so overall much better user experience f when it's on the on on a phone. So I think right now the the kit use cases are low latency applications, on device applications, and then there's some benefit as well, which we we have empirical evidence for, for the bi directional nature of of the way that the text fusion model can reason and attend to the future and so on. Now there aren't a huge amount of applications of this that we've been able to find, but for instance Sudoku, which obviously nobody really cares about in practice, but the Ravid Shwartz-Ziv: Huh. Brendan: text division models are just way better at Sudoku because they can they can fill in at any point in the Sudoku rather than having to like, you know, be s be restricted to some particular order. so if there's any pro if there's any like like problem like that that arises that then you know, there there's an advantage to being able to do bi directional reasoning and to do like or any order fill in. So I think those are the use cases right now. Ravid Shwartz-Ziv: And what do you think about like combining auto-regressive and like tax diffusion? You know, for example, these days people are using like parallel drafters, right? In speculative decoding, Brendan: Mm. Ravid Shwartz-Ziv: that this is kind of like kind of like diffusion style models, right? That like you have the drafter, the like a small drafter, that you can generate like the the tokens in in parallel. so do you think this is something like do you think about these directions or do you think like it's like Brendan: Yeah. Ravid Shwartz-Ziv: the two different distinct lines of org? Brendan: No, no, definitely. I think that's I think that's a very important direction. And the diffusion Gemma model that we released, we didn't highlight this at release time, but it's gonna be in the tech report that we're putting out in a few days. Is is a d is able to do both modalities. We specifically trained it to be both an autoaggressive model and a text diffusion model. So people could mostly thought people could play with that and see if there was any interesting use cases. in the autoaggressive model, it's close, it loses a few percent of of points relative to the the AR model we started from, which is the Gemma for like twenty six B model. so it's a little bit worse than that. But it the the fact that it can do both modalities. So you can do generation in autoaggressive mode or in text vision mode. And you can also do scoring in autoaggressive mode because that's like a an an advantage to have, which is to have exact likelihoods under the model distribution, like the AR mode. I think there are use cases for that. Like some people internally have played with things like, you know, these reasoning models, they they c they can spend ages reasoning and I that can add up i eat into your latency a lot. So you could imagine but you don't need to do perfect reasoning. It doesn't have to be like really high quality reasoning. You kind of just need a coarse reasoning. So you could imagine, and some people have played with this, you could imagine having the text diffusion model, which is able to run in both modes, do all of its thinking, like all of its reasoning, in a text diffusion mode to try to get it done as quickly as possible and really lean into the low latency aspect. And then when it comes to actually responding and like let's say writing the code or whatever you need to do and you want that to be to be very precise and you don't mind you don't mind waiting for that because the expensive part of in terms of wall club time is done. You switch over to autoaggressive mode and then it just does one token at a time. It's slower but it tends to be like s slightly higher quality. And then combining those two modes I think could be very powerful going going forward. Allen Roush: So d with with these diffusion models I know at least from the the image generation space that there's a vibrant ecosystem of s the equivalent to like LLM auto regressive samplers which are are known as SDE solvers, right? So you've probably heard of these Euler, Euler Adapted FDP, etc. Brendan: Yeah. Right. Allen Roush: I don't know anything really about how they interact with the diffusion LLM ecosystem. I'd love for you to say anything you know about this and I've I've One thing I do n ha or have heard is that despite there being many, you know, second or third derivative based methods, which are much more computationally expensive and in theory supposed to be more accurate. In practice it seems to be that fast first order methods have been dominant in anyway. Brendan: Yeah. Allen Roush: very s and so j I'd just love to hear your opinion on all of these things. Brendan: Yeah, so yeah, so so for continuous samplers there's been a lot of great work. I'm a little bit out of touch with them now with whatever state of the art continuous sampler there is. But something very similar is possible inside d like discrete diffusion samplers. So it's kind of it's a different problem. But people have people have formalized discrete diffusion In a variety of ways, but there is there is some kind of flow matching or continuous time markup chain papers where they analyze it kind of similar to ODE S C E style stuff that like some of these samplers that you mentioned are based on. And I think there's even like the notions of higher order samplers, like second order, third order for samplers for discrete diffusion, but I'm not a hundred percent familiar with that. But I think like so there is there is similar work and it comes from a similar point of view, from like a similar theory point of view. ends up being a little bit different because you're dealing with discrete data and stuff, but they're they're similar. I I do think that samplers in general is kind of one area that hasn't had a huge amount of attention for discrete diffusion and it would be good to get a lot more work there. And like at a high level, you you know, without even considering the theory of discrete diffusion being analyzed like a continuous time microchain, you can just think of like At the top you when you when you're trying to generate samples from a discrete diffusion model, you you like let's say you've got a block length, but we call a canvas of 256 tokens, at the top of the transformer, you're gonna have a like a probability distribution over your vocab for each position. And those are totally factorized. So they're it's they're not joint it's not a joint distribution, which is what you'd like to like to sample from. They're just factorized marginals. And so if you sample from that naively, you can get problems of like stuttering and things like that where the model isn't quite sure where to put the word the or something like that and ends up putting two thes in a row or something like that, where it doesn't doesn't quite know what mode the tokens are gonna pick, because once you get into the each being a totally marginal probability distribution, they all they they can't coordinate at that stage. And so there's a bunch of work going in to like a on my team on trying to like sample efficiently from that or like do some sampler distillation to make it easier to sample efficiency efficiently. But I think just as there's so much more work that we could do. I think there's like a lot that if we could just, you know, sample from that marginal more effici more efficiently or somehow figure out how to get them to coordinate better or whatever, then that could be really a really big unlock because we spend some amount of the forward some amount of the denoisy steps, some amount of the forward time that we put would be spend on the neural network. Is really just coordinating the tokens, like trying to make sure that they agree on what they want to say. And if we could if we could have that by a much better sampler, like have that the number of no forward passes we do, that would be a huge unlock and make things even faster and cheaper. So I probably would close the full fully close the throughput gap. I mean like one of the reasons we open weight released the fusion gemma was in order to get people who to work on that, because now you have like a a a a basically a state of the art text diffusion model that can fit in a H one hundred and just go and like figure out the best way to sample from this, right? Or figure out new sampler distillation techniques or something like that. And if if there's a big breakthrough there, I mean that would help the entire community. That'd be amazing. Ravid Shwartz-Ziv: And how much like you so diffusion models started from from vision, like it has like very strong theoretical background, right? Like intuition from theoretical background. How much you think it's important, you know, today that like we we are talking about discrete diffusion that it's not clear, Brendan: Yeah. Ravid Shwartz-Ziv: like I know there there were several attempts, you know, like to to build a theory for for also discrete models. But how much is important? Do you think this is something that like we can yeah, there is Brendan: Hmm. Yeah, it's a it's a it's a good question. it's hard like I think understanding these models better, so building on the theory would be very useful. And there is really good theory papers for discrete diffusion. maybe it's not c it's not definitely not quite as developed or as well understood as continuous diffusion. And there is definitely a bigger gap between theory and practice. for discrete diffusion than there is for continuous diffusion where it turns out that for for the most part the theory and the practice line up reasonably well. Whereas in discrete diffusion you kind of have a bunch of different things that aren't quite, you know, ideal in your 'cause you want to sample many tokens at a time and things like that and like a lot of the theory papers don't all don't allow that. I think yeah I think you can build, you know, one view that a lot of people take from a more deep learning perspective is that discrete diffusion models are basically BERT, you know, the BERT language model, which is a masked fill in kind of like you mask out some tokens and you fill them in. You learn to you learn to replace the mask tokens. It's it's kind of BERT with like multiple varying noise levels and then sampled from. So it's like kind of from if you just you imagine where what's the an ancestry of text diffusion, you can draw a straight line to to bird and that that entire line of work because it it really is building on that to say, well if you can fill in the missing tokens from this mass sequence that you're trained on, then you could do it from every type of mass sequence and then you could sample from the model. And so a lot of practitioners I think don't necessarily need to know the theory to have an impact. But I think probably in order to make more progress and to really close any remaining gaps and to go way beyond auto-aggressive models, because I think tech speech models have a higher intelligence ceiling, if we can solve these problems, will require some innovation that may need some kind of more theoretical breakthroughs. Allen Roush: so so the the the these diffusion models, the the theoretical breakthroughs, like do you think they're they're being worked on? Like do you think the community that's Brendan: Hm. Allen Roush: working on this is large enough? Or Brendan: Yeah. Allen Roush: you know, I ask just because I see ebbs and flows in the community and I can tell you there seems to be more papers on world models these days than on diffusion language models, for example. Brendan: Yeah. I mean I think it's a really hot area. Like there's a lot of stuff coming out. Lots of great papers. Loads of great PhD students, loads of great professors working on this. I think like there was just ICML Best Paper Award. One of them went to a dis went to a text diffusion model. I'd love to see more. one of the reasons we open weights released the fusion jammo is to try to like give the community a a a really good like state of the art fusion model that people could play with, they could build on, that they could learn from, that we built some intuition in kind of in the same way the Lama model from Facebook from two two, three years ago, as an open weights model, really spurred on a lot of great research from academics and so on who couldn't afford to pre-train the entire thing from scratch. But once they got a model in their hands, they were able to do a lot with it. We w one of the big motivations for us to release this was to try to do something similar for the people who work on text fusion. And Well also like y you know, people can fine tune it for their own application and I've seen some it's been some really cool applications already for that. But I think it would be great to see more research in this area. I think yeah you know, I'm I'm confident that enough people are working on it and it it is making pretty rapid progress, I think. If you think about the fusion models, you know, the original the fusion model paper with like Jasha sold Dixtein from I think it was twenty fifteen or something like that, you know, that's a That's a long time, like eleven eleven years. Whereas discrete diffusion kind of I'm not sure exactly what the first discrete diffusion paper was, but you know, maybe the early twenty twenties. You know, Amil had an Argmax Flow paper in twenty twenty one. You know, s you know, the the D three PM paper I think was also twenty twenty one or maybe t around there anyway. So the you know, there's been several years. We're la we're lagging by several years. So I think there's like some catch up. And I think right now is actually a great time to get involved, like for you know, PhD students, because text diffusion is a much less explored area than order aggressive models, and the the frontier labs are are doing much less on it, and so there's more scope to have an impact. Whereas kind of if you're if you're working on language order aggressive language models, you're probably you don't have the compute to compete with the massive labs. There's so much secret sauce locked up in these huge frontier labs. But I think it's much more open first of all in text fusion and there's way more to do. So I think like it it is an exciting area and I'd I I think it'd be if I was a PhD student now it would be one area I'd seriously consider investing in 'cause you could have an impact, have a really big impact and you know, there's still a lot to do. Ravid Shwartz-Ziv: And what what do you think are the main button eggs? You know, it looks at like all of the the pipelines, right? Like the data, the the like pre train, mid train, post train, all these are r really highly optimized for auto regressive models. do you think they're also like aligned with with diffusion models or do you think like where are the things that we actually need to change in order to get better performance? Brendan: Yeah. Mm well yeah, so I think there is a lot of hypertuning for auto aggressive models. So you know, architectural decisions that people are familiar with would could would need to be changed. Scaling laws need to be changed. RL algorithms and RL infrastructure would have to be different. Sampler distillation is a whole new thing that auto-aggressive models don't do that you c we'd have to do from scratch. I think things like data and so on are roughly the same, so that probably wouldn't need to change much, but they're they're like Just comparing what we did with with like Gemini internally to you know to Google. Every part of the infrastructure is so hyper optimized for auto aggressive models. It's quite hard to actually do something a bit like TextFusion, which is quite different. It requires different design choices. And we kinda mostly just picked up the design choices that they that they did for their thing. And there's some regret that we incur for for that, I think. And like if you someone really did just hyper optimize for text diffusion you'd end up with a different like trade-offs and different models, different hypers, and you probably have a overall much better model. I think like some of the more interesting areas for research right now is I already mentioned like better samplers, better ways to sample from that marginal or or just like train it so you don't need to do anything clever. I think continuous diffusion for text is another really exciting area where, you know, people did do it a few years ago. It kind of fell out of fashion. Everyone was doing discrete diffusion including us from like twenty twenty three. And that really pushed the quality a lot for for diffusion models that for text. But now I think there's a s a chance to maybe take a step back and say, actually if we can get continuous diffusion to work for text in some way, then you know, in a latent space or something like that that we learn, and there's some really good papers coming out in this direction, then we could unlock using all the really sophisticated work that's been done for the continuous diffusion, which is like several years more advanced and has way better you know, noise schedules, samplers, distillation, just because it's like it's been going for longer and there's been more investment in it. So you know, I think and anything that would like improve the throughput, improve the latency, improve the quality, tricks and training to just make it, you know, overall higher quality, like would any of these could have a really big impact. Allen Roush: D do quantization and pruning methods the kind of map from Brendan: Mm. Allen Roush: language models to diffusion models? and you know, it seems like there's this tendency that the bigger the model is in terms of parameter count, that you can get away with increasing amounts of qu quantization, like to Brendan: Mm. Allen Roush: lower precisions, while preserving more of the quality. Is that the case also with larger diffusion models? Brendan: Yeah. So pruning, I don't think I've ever looked at that, so I I don't know. But quantization, we find that at least it's as easy to quantize a text diffusion model than an auto-aggressive model. And I I think if you if if everything was working right, it would actually be better to quantize a text diffusion model because of this like iterative refinement and error correction property that it has. So if it does make a mistake, you know, somehow something's gone wrong and it's sampled a bad token. For whatever property, it could be for whatever reason it could be because of the quantization, it has a chance to to look at that error and correct it. So I I think in principle, you know, right now I think quantization is basically neutral between the two, and they kind of roughly have the same, you know, performance after quantization. But y I think if things were done right, it should be even easier to quantize a text diffusion model. Yeah. Ravid Shwartz-Ziv: And what about hardware? Do you think is it possible to optimize the the hardware for Tech Diffusion? Brendan: Yeah. so I think w I mean text fusion it's really well suited to modern accelerators. It's one of the reasons that I I like I think it's it's got a real potential to to really accelerate in the future because GPUs and TPUs, you know, they have a ton of flops in their, you know, whatever they the MXU, the matrix modification unit, whatever that's called for GPU. the they have this tight you know, HPM bandwidth channel where you want to, you know, transfer the weight and the ac activations of the KV cache or whatever from HPM across this tight channel into the MXU and do the flops. And the the amount of flops you require versus the bandwidth you require for for text region is different than for auto aggressive models. So they can they can make much better use of the GPU because for every time they transfer the weights they do a way more flops and they only have to transfer all the weights, you know, ten times when they're generating two hundred and fifty six tokens instead of two hundred and fifty six times when they're generating two hundred and fifty six tokens like an order regressive model does. And if you're memory bound because that bandwidth is so tight, then you you you're my that's why order regressive models are slower because they just it take they have to do more memory transfers. And now we're popping up these these companies like Cerebrus and Groc and so on who are popping up, they're building dedicated hardware for for inference basically in order to try to get order regressive models to be like faster and easier to serve. But I think they're coming there th they're basically a hardware solution for an algorithmic problem. And probably the longer term solution is to use something like TexAfusion where you can just make use of all those flops that are on a GPU much easier than order aggressive model. And then just you know, solve the problem algorithmically by f by making text fusion, you know, way better than order aggressive models or whatever, and then be able to, you know, have better inference that way and rely on and rely on, you know. generic hardware instead of relying on custom hardware like like some of these hardware startups are doing. So I'm like I'm pretty bullet like there is some co-design potential for you know both of these technologies with hardware, but it's easier and cheaper and overall in the future like looking forward just it's going to be a lot more flops on future generations of GPUs and TPUs. And and than that it is to put like it's harder and more expensive to put more bandwidth on. And it's like physically like hardware to actually physically harder to put more bandwidth on. And so going forward I think there's gonna be more and more flops and it's easier to put flops on. So Text Vuchine is well suited to that 'cause it can make use of the flops. So there's auto regressive models in either it's gonna be custom hardware and it's gonna be tricks to try to make use of the flops without you know, s like when it's saturating the bandwidth. Yeah. So I think yeah, hardware is a big thing, like for both models, it's a big open question. But I'm overall bullish on the direction of you know travel for GPUs and CPUs with the you know, the amount of flux and bandwidth they're able to put on, that it's gonna be more and more likely, more and more favourable for text division going forward. Allen Roush: W what about government regulation, right? We've already seen, you know, Fable Mythos and then GPT five point six and others kind of have to go through strict government regulation, already having their capabilities neutered on cybersecurity and biology tasks, Chinese governments talking about kind of pr stopping the open source renaissance or golden age that's going on, you know, from there and forcing labs to close source everything. Do you think that this is a real risk of happening soon? And if so, do you think it will affect diffusion language models? Brendan: god. you know, I hope it doesn't happen. I know I hope everyone has access to the the best quality models, obviously in a safe way. and I hope open source doesn't go away. That'll be terrible for a lot of people who like a lot of startups are using like open weights models. i you know, right now Texafusion the models are much smaller and they just don't have the same dangerous capabilities than that auto aggressive models do, at least currently. If If that changes in the future, you know, if we do scale up to a massive model or there's some big breakthrough and like people are able to push the b push the boundaries of smaller models way further than autographs models, then yeah, I don't see why they wouldn't be regulated the same way. But you know, hopefully that doesn't happen. Ravid Shwartz-Ziv: And so wha what do you think about AGI? In like it looks that at least in the Silicon Valley, you know, the the main narrative now is that AGI is around the corner and like basically everyone will lose their job in like six months and Brendan: Yeah. Ravid Shwartz-Ziv: we we need to to to get ready. What do you think? Brendan: Yeah, you know I think AGI is coming. I think if you kind of you know, modulo things like hardware being hard, like robotics being a bottleneck for progress in embodied intelligence, if you just think of Mythos or whatever as being further closer and closer to AGI in a box. that doesn't have to be embodied, but can do everything that like you can do in software. So anything with not to do with text, including science and you know, math and writing code and running experiments and so on, or even proposing experiments for a wet lab or something like that. Then I yeah, I think it's it's coming. It's probably coming pretty soon. By that definition, probably coming next few years would be my guess. If the trajectories continue in this direction. are people gonna lose their jobs? sh I mean there probably will be some displacement. I mean I'm this is not my area at all, so like I'm probably gonna say something really stupid here, but my I imagine that there'll be some displacement with some jobs will be lost. But I think it will unlock such a hopefully it'll unlock if we do it safely and correctly and and all that. It will unlock such an like an amazing era of like progress and prosperity and you know, curing diseases and giving everyone access to, you know, high quality, you know t the best tax advisor in the world, the best legal advisor in the world, the best, you know, mathematician and scientist, just all available in your pocket, the the best psychiatrist or the best marriage counselor or whatever, and they're all in your pocket. That should should be a massive prosperity increase for, you know, the world and all the impacts that AI can have in science and mathematics and technology and so on. I'm thinking that yes, there probably will be some economic dis displacement of jobs, but I'm hoping that the the benefits of AI and A GI would be just so extreme that, you know, it it it massively outweighs the negatives if we can do it in a in a safe way. I mean if everyone has access to a cyber weapon, that would be bad. But if everyone had access to like the best coder in the world who knew not to, you know, do cyber attacks on your behalf, then that would be a that would be great. Ravid Shwartz-Ziv: And what do you think about the argument that like, yeah, and coding agents are great because code is very fab verifiable in math, like we always almost solve it because everything you can verify it all the way but the real world is much more massive you know like when you involve like you have humans in the loop and like some decisions that you need to make and the real Brendan: Mm-hmm. Ravid Shwartz-Ziv: world right of course the physical world then LM like are much more like they have a lot of difficulties there and like the progress there will be much slower. What do you think about Brendan: Yeah. Yeah, I mean Ravid Shwartz-Ziv: it? Brendan: I think that's ex absolutely correct. I mean, you know, I don't think a the second we get AGI will, you know, s solve all the world's problems, there'll be there'll be bottlenecks or like, you know, slowdowns that are not related to having more intelligence in a bo in a data centre, but related to just like real world, you know, requirements on running experiments and you know, actually building things and so on and like just real physical limitations will will slow us down. but my guess is just if a if it really is AGI, it will propose really good experiments to run and those will be better than the what you know humans could have proposed and you know humans will run the experiments and come back with like the results and the the the AI will be iterating on that or and figuring out the next things to to to run and we'll make ra much more fast, much quicker progress than we would we would have without it, even if it is bottlenecked by things that are kind of out of its control. Ravid Shwartz-Ziv: And w what do you think like so last week we talked with Da Daphne Coller and she's yeah yeah she's great. Brendan: definitely, yeah, yeah. She was I she's great, she's great, well. Ravid Shwartz-Ziv: Yeah, and and she has a company for for drug discovery and basically she claimed that like yeah the like the the running and and discovering new molecules it's the easy part that we can automate almost fully. Brendan: Mm. Ravid Shwartz-Ziv: but the hard part is actually like to run these experiments and to try it right on like humans for example. also we talked with Max Welling that basically they they are they're doing the same thing, right? Like you when you try to to automate experiments and wet labs and when you try to deal with the real world like the main button eggs are not there are not like in running and writing your code faster or like like thinking about different Brendan: Yeah. Ravid Shwartz-Ziv: I don't know some like molecular new molecules or something like that but like to actually run it and you can speed up like the real world Brendan: Yeah. Ravid Shwartz-Ziv: W w what what do you think Brendan: Yeah. Ravid Shwartz-Ziv: about that? Brendan: I mean it sounds I mean I believe Daphne when she says that I mean she's she's done great work. So Ravid Shwartz-Ziv: Mm. Brendan: I mean I think that's i exactly right, but you know, we'll we will be bottlenecked by things like the wet lab and how long it takes to run experiments. it is great to be able to automate that that part though, right? If i i maybe it is the easy part, just by coming up with candidate molecules to test or whatever Daphne was saying. but it is great to, you know, have a superhuman agent that can do that on our behalf. So at least that that part is solved. And and if it really is able to come up with much better candidate molecules then, you know, if you imagine how many successful, you know, like drugs go from candidate molecule, let's say that rate is five percent or something from candidate molecule to actually therapeutic drug on the market, maybe that goes from five percent to twenty percent or fifty percent or whatever, and that would be a massive breakthrough And that would be massively accelerating drug discovery and you know, compressed ten years down into one year or something like that. And that'd be huge. So, you know, I think there are these bottlenecks that are inevitable, like Daphne mentioned, like just yeah, it takes time to run these experiments and to to like see the effect of what whatever molecules you're developing. And it's gonna be the same in many, many areas like robotics and and things like that. They're just gonna be Yeah, physical bottlenecks that the AI can't help with, but you'd think that the inputs would be so much stronger that maybe it still accelerates us a lot. That's my hope anyway. Allen Roush: And do you think that these diffusion models have a c a part to play in so called recursive self-improvement, right? Which is the component, Brendan: Yeah. Allen Roush: right, that's happening supposedly now with Claude Code helping Claude you know, make Claude Code and Brendan: Right. Right, right. Yeah, I mean I think that could be that could happen for two reasons. Like one concrete reason today is that they're just much faster. So if you if if the recursive self improvement requires writing a lot of code and you know agentic coding can output like literally hundreds of thousands of tokens, if you can do that ten times faster then you pr you have ten times faster turnaround on every call to your to your you know recursive self improver. And the other reason I think there is a p a possibility that it could is that even though it's not the case today, just I think diffusion is a more powerful way in principle for to generate text if we got it right, because it can do bidirectional reasoning, this self correcting and this like editor process that if that could really pay dividends, then maybe it's with a much higher intelligence ceiling than what are aggressive models for the same model size or something like that. And so Those two combined it so it it could be the case that in a couple of years, yeah, that the text diffusion models are how we do recursive self improvement. Yeah. Allen Roush: I I noticed, you know, that there's always this history of of Technologies that anticipate a technology or even you know gamification. So I'll give an example here. in in generative AI in general, there was a series of games called Scribble Knots, which had been around for a long time where you could literally in natural language describe objects that you wanted to appear in the world, like any object, and use them to help your character solve tasks. Now they did this based on a pretty intelligent system of objects and properties and you know individual. mapping and obviously lots of limitations to this given that it was way before LLMs, but there's scribble knots, and then in the case of Bert, you know, and bidirectional reasoning models, I mean there's literal mad libs, right? Like I see those to this day, and I always Brendan: Yeah. Allen Roush: think about how you could just synthetically generate a hundred billion Mad Libs books using, you know, the technologies NLP folks have perfected. So I'm just curious about like do you see these other kinds of like anticipation of technology where, you know, it goes kind of under the radar for a while and then it gets really huge suddenly in this space. Brendan: wow, yeah, that's a very interesting question. I had not thought about that at all. yeah. wow. it's hard for me to really answer that. I don't really know you. I I guess so. I mean I I guess if you say bird like if Madlibs kind of anticipated Bird and Bert kind of anticipated modern text diffusion then maybe yeah, maybe I can see the the the ancestral line from Madlibs to Bert to Texusion and maybe there's some child technology that's gonna come next and that'd be great. Maybe ma maybe it goes back to Madlibs and we just the circle completes. yeah, I mean Allen Roush: Ha ha ha. Brendan: I like to think of like, you know, transistors were preceded by vacuum tubes and people invested a lot of vacuum tubes and they were the state of the art for, you know, computation at the time. They were huge and they were expensive to make and very delicate and you could only fit a couple of hundred of them, you couldn't miniaturize them and computation was going to be done that way until somebody developed transistors and then they were able to be they were way better at some point, like it took a while. for people to switch over and to realize but then once the investment went into transistors obviously then we had the full computer revolution and so on. So maybe you know, maybe some of these technologies are like like that progression from you know vacuum tubes to something that's ultimately going to be stronger, but we don't know how to make it like just as strong as it's gonna be just yet. And there's gonna be more time to be required to iron out the details here. But yeah, very interesting question. I'm sorry that probably wasn't a very good answer, but Ravid Shwartz-Ziv: So I have a question. So like a lot of people say we are in the engineering area, right? It like we we don't do any science anymore and what we have what we are doing is just like try to optimize very small things in order to to make it like I don't know 0.1% better on on the benchmarks. Do you buy that? does it bother you? Brendan: so I think there's a lot of science that goes into doing careful autoaggressive language modeling. I mean, I I think it's like it's empirical science, but it it's science and it's very, very careful because of the just the sheer cost of running a big training run. And so people s spend a huge amount of time and effort doing very, very, very careful scientific experiments and ablations, which really are science. on these models, which are kinda which are an empirical science like a like a real physical science of an emergent phenomenon of this it's weird, but it's like a in the computer, but it's still just like observing the physical properties of something that's outside the computer. You really do have to like look at what's happening and then make decisions based on that. But I I do get your point though. And one of the reasons I was attracted to working on diffusion in the first place was because of the beautiful theory and the elegant mathematics that went into like the original you know, the original diffusion papers, like the the original ones doing sc score matching and the STE equivalents and, you know, all the really cool, you know theory work that's gone in in that direction and that really has a practice practical impact since that you could use this theory to generate beautiful images. And there is still very elegant theory in discrete diffusion as well. And like I think there's more to be done there in a way that like I don't know if auto aggressive models will have the same like underlying elegant mathematics. Maybe, but I I don't I don't necessarily see that coming. So yeah, I think you know there is a t it is a type of science. It's not really theory, but yeah, I'd love to see more of it. And it would be great if there was like if it could be like continuous diffusion where the scientific or the mathematical impacts or like breakthroughs r had a real practical impact. You know if someone I'd love to see And I think people are working on this, but I haven't seen something actually really have impact. So I'd love to see like a theoretical analysis really showing something about the c like the capacity and the power of transformers, like why are they why are they so good? You know, people have done a bunch of kind of work in this direction, but I I haven't seen anything super convincing. And if someone was really able to analyze that and really show, look, this is this new mathematical theory that describes how Transformers are so powerful. Because they are. They've replaced, you know, everything. And we then suddenly we understood it. And maybe from that theory maybe we could build better transformers 'cause we understood it a bit better or something like that, better architectures. That'd be amazing if we had that. Allen Roush: Well, so what's your opinion on things like state space models and Mamba and etc? Brendan: Hmm. Yeah, they're they're super cool. I ru I love that work. I don't know if it's had a huge impact on the frontier models yet, from what I gather. it would be good to understand why. but you know, i maybe just something's missing there and you know in a year in six months' time there'll be a big breakthrough and everyone will be using SSMs. But Ravid Shwartz-Ziv: I think like in in recent like models, right, at least like the open source one, it on on some Brendan: Yeah. Ravid Shwartz-Ziv: of the layers they are doing like right, like they're doing kind of like Brendan: Mm. Ravid Shwartz-Ziv: linear attentions that really similar to these state models, right? State space models Brendan: Mm, mm-hmm. Ravid Shwartz-Ziv: and they combine between like full attention and linear attention and state state space models. Allen Roush: Yeah. Ravid Shwartz-Ziv: so it's kind of like at least in Brendan: Yeah. Ravid Shwartz-Ziv: the open source models. Brendan: I've seen some interleaving work, yeah, and that maybe that is where where things will go. Like interleaving the state safe model layers with like say five regular tension layers or something like that. yeah, maybe maybe that is the future. they are cool and elegant, so it'd be great to see if that if had that got more traction. Yeah. Ravid Shwartz-Ziv: So what is the most overhyped idea in AI right now and mo what is the most underhyped idea in your opinion? Brendan: Hmm. The most overhyped and underhyped ideas. Wow. You really put me on the spot I spot here. Hmm. Well Ravid Shwartz-Ziv: You you don't have to answer if you don't want to. Brendan: No no, I mean I I'm not worried about offending any, but I need to actually think of it. I need to actually come up with it. Like what is the most overhyped and underhyped? Maybe I think some of the you know, this probably will offend people, but maybe some of the video and image stuff I think maybe a bit overhyped because I think it's overhyped a little bit because the like when you see an amazing video out of one of these models, it's so visual and so gripping and so obviously amazing that how on earth did neural network produce this that it is mind blowing and people are like, whoa, and they're like, this is incredible. And but that can lead to overhyping because I don't know yet if they've had a huge impact on, you know products or technologies in the same way L L Ms have, you know, in the same way Claude and GPT and Gemini have for for actually writing code and having useful outputs. But I think there's there's probably more to do there, but they're so gripping. And then people invest a huge amount into them. Like in terms of compute, resources, money, people. That is you know, maybe maybe it's a bit overhyped because at least right now I don't see a massive a massive way for that technology to have a huge impact. I guess n I guess people are invested in model I mean world models and that's obviously going to use these technologies as an underlying part of that component of that technology and then Road models will lead into better AI and better robotics and stuff. So maybe it'll it'll achieve the hype that it kinda has right now. But I think when you see a fifteen second video of some state of the art in a video model and people are like, amazing and then they send it to each other, you know, they send it to their friends, they could just cr look at this amazing video, but then they forget about it, like the the this this doesn't have any stickiness and the the sh the cost of training those models is unbelievable. It rivals training a frontier level L L And it it has almost no sticky power. Like there isn't right now a use case for that for people on their day to day. But I think there will be, probably. The world models maybe is is that direction. But I think maybe they are overhyped. And underhyped, maybe this is a bit trite at this point, but maybe maybe reinforcement learning still I know reinforcement learning has had its Ravid Shwartz-Ziv: Mm. Brendan: renaissance and everyone does RL now and everyone says that RL is what makes plot code work or whatever, agentic code and the same for all the other Frontier Labs. I s I still think it's underhyped. I still think, you know, it's so powerful a technology, we've barely begun to scra scratch the surface. And if you think about what will we need to really reach A GI, it can't be more pre training or more SFT, right? Because that puts a limit of human capability onto the model. Like if you pre training the entire internet, the model is as good as the entire internet. You've S F T on some maybe some human generated coding problems or whatever, that's the limit. Whereas and they did that for AlphaGo. Like for AlphaGo they try they trained the initial version trained on all these like expert matches. And it was a very good player, it was a competent, expert human level player that would like, you know, rival a top, say, twenty code player in the world. But they threw all that away and just RL'd from scratch. From scratch, like no training data for AlphaZero and that was way better, like miles beyond human capability. Not like one or two stones, which are these like penalties you can give to a much stronger player. You can give the opponent stones in order to compensate for the ELO difference. It was not just one or two stones different, it was like something like nine or ten stones different. Like the co the difference in the very, very best humans who'd spent their entire lives learning Go from a young age to becoming like one of the best in the world to a tenure network that spent, you know, five days doing RL from scratch. was astonishing and it's gonna have to be the case for L L Ms to reach AGI that at some point they'll we'll reach the limit of pre training data, just from a data point of view, like not from a neural network architecture point of view, just from a data point of view. And to really become superhuman it's gonna just it's gonna be a ton of RL. And I think I I imagine that in the future there's gonna be a lot more reinforcement in these models than than SFT pre training. So I think even though it is super hyped, reinforcement learning probably still is underhyped for, you know, getting to AGI Allen Roush: Do you do you think that diffusion models d take or like does reinforcement learning operate differently on them than auto regressive models? Brendan: Yeah, it's a question. We do we did have to come up with some new you know, some new algorithms, some and some new techniques just to compensate for the fact that you don't have the likelihoods like you have in a in an order model. So you don't have the probability of this token being generated. So we had to come up with some new algorithms there. but ultimately they look very similar to what is already done. So I think like rather than think about it as an algorithm you know, what's different, I think really it's just the concept of RL, like You you can do RL many, many different ways, even for order aggressive models. If you get the recipe right, you give a right reward function and a right value function or whatever it might be and let it climb. Kinda doesn't matter what the actual algorithm is that much, as long as it kinda makes sense that you'll see you'll see your text fusion RL I text fusion, order aggressive models will will climb. Ravid Shwartz-Ziv: And do you think there are like some classical RL ideas that we that we can use, right? For example, like the GLM model, right, they just published that they actually like use the critic, like kind of like PPO Brendan: Hm. Ravid Shwartz-Ziv: style instead of the G RPO? Do you think there are like more ideas that we that we Brendan: Yeah. Ravid Shwartz-Ziv: can revise and use? Brendan: d definitely. I mean RL is a massive field for from for going on twenty years and the the algorithms that LLMs are using are just so primitive compared to like their super sophisticated ones that are available and analyzed in the literature. you know, using a critic, like a value function critic, you know, that's the I mean the reason they it's not happening is 'cause it's hard, so people haven't really been able to work out all the details. yet but I think definitely value functions will play a huge role. I think like I did a I I in my previous life before text fusion I did a lot of reinforcement learning theory on e what's called like exploration, which is like the exploration exploitation trade-off where you try to have an RL agent like explore as in like take suboptimal actions in order to learn more about the environment that it's in. So you know, move away from the most what it thinks is the most rewarding set of states and actions. into kind of novel territory in order to learn more and maybe potentially find a much better reward. So that's kind of exploration at a high level. And there's no exploration as far as I know in in any RL algorithms that people use in practice. I don't think it's it's ready for that. Like I think this you know we should get value functions before you try to do any sophisticated exploration techniques. But if we think about what is required to get to superhuman intelligence, it will be required for the model to like actively leave the the manifold of the data distribution it was trained on to go off and try to find potentially much more rewarding states by leaving the distribution that it's been trained on and finding something new. So exploration and maybe it's a simple form of exploration, but some something will will eventually be required in there in order to really squeeze more performance out of these models and really get them to leave to become superhuman. So I think yeah, there's loads. I mean that that there's been so many great people working in that field for so long. Working on either pure theory or like relatively toy problems, like for room or even Atari is relatively toy today. And that we're just waiting for that overhang of all that literature and that knowledge to translate into practical algorithms for L L Ms. And there's there's a few things missing, like for s people have tried it obviously. They haven't translated yet because it's simple things are not working well yet, like the value function critic, like you said. But as soon as that happens, I I think like there'll be a massive you know, take off as long as we have the reward functions because we have so much knowledge from you know, reinforcement learning, all this work that people have done for so long. And it will eventually have the impact that I expect it will on you know, frontier level LLMs. And then I f I think it should really take off at that point. Which is kinda why it's underhyped a little bit as a field, 'cause like we're waiting, you know, there's like so much stuff there that could just be when we get to details where it could be applied and really take off the you know, how we do RL for these things. And if we're gonna if we think in the future we're gonna be spending way more flops in RL than in pre training, then it's gonna be there's a huge, you know, possibility of having a massive impact there with new RL techniques or even RL techniques just taken from the literature. Ravid Shwartz-Ziv: So two years from now, like five years from now, how do you see the the pipeline? You know, like both in in performance and also like the the algorithms, like how AI will look like? Like bigger Brendan: Yeah. Ravid Shwartz-Ziv: models, smaller models, like Brendan: Yeah, I g I I think the direction of travel is for bigger models. I don't see anything holding that back. I don't I I don't think people will I think scale is still giving so much benefit in predictable ways that we'll just keep scaling these things. And yes we'll I think we'll have smaller models that are more capable than the smaller models we have now, but I'm I'm sure the frontier is gonna increase in size. terms of the many trillions of parameter models that they are now. I just don't see a direction pushing that down for any reason. You know, people make these like connections between like the number of synapses in a brain and the number of parameters. I mean I don't think they it really makes sense, but I but like people say there's quadrillions or whatever in the brain. There's only trillions, so we still have a long way to go. I don't know if that makes any sense whatsoever, but that's you know, th there there are measurable and real gains from just increasing your model. You know that. Like still today, like pre training is not tapped out. Maybe data is is more of a question that maybe like you kinda run out of data. Like if you train on everything everyone's ever written, you kinda maybe max out or something like that. But then I think you can still have more you can use that capacity for RL later. So I still think you'll have bigger you know, when it generates its own data where you can s you can still push the models bigger and bigger. So I don't see anything pushing them to make them smaller just yet. so I think scale will keep winning. and what was the second part of that question? Ravid Shwartz-Ziv: There wasn't like a like I basically want to ask like w what you think, like how the future will look like, you know, in terms of like both Brendan: Mm. Ravid Shwartz-Ziv: like pipelines and performance and like kind of Brendan: Right. Ravid Shwartz-Ziv: architecture. Brendan: Yeah, so I think the models would probably get bigger. Pipelines, I'm not sh I'm not sure. I guess more like I I I predict probably more overall flops and and time will be spent in reinforcement learning and what that's kinda just gonna grow as a fraction of the total training flops from where it is today. I think in terms of capabilities, I don't see any limit right now on what these models can do. You know, every limit that's pe been proposed has been smashed through eventually. So and you know hasn't required specialized, you know, stuff to do it really. It's just been make the model bigger, train on better data, do more RL and it just solves whatever problem was kind of a sticky point at that point. Yeah. Ravid Shwartz-Ziv: And what about like diversity of the of the models or like the components? Do you think we will see like converge to to some I don't it may be like auto regressive or diffusion or whatever, but do you think you will see like some like converge to specific type of of models or like different types of models will be quite similar, or like maybe good in different aspects? Brendan: I think it's hard to predict. there probably will be some convert it's hard to know. Ravid Shwartz-Ziv: Prediction is hard, especially about the future. Brendan: Fredictious yeah. Yeah. I mean there is convergence right now. I mean I think everyone all everyone, all the Frontier Labs are using MOE transformers, pre trained on a huge amount data and then RL and a and a bunch of different RL environments. I think that's basically the recipe that everyone's doing and we're all executing it in different ways and getting slightly different results. But they're only slightly different, right? And like, you know, the it i the these things ebb and flow so much that six months Things change so quickly, you know, one lab is in the lead and another lab is in the lead. And you know, does that is it gonna stay converged? Hard to know. I mean, I do think probably recursive self intelligence stuff will start kicking in where the model starts to be able to improve itself. I think we're seeing signs of that already. And maybe that ends up with more diversity or something like that. And maybe these models start training themselves or start postering themselves or start proposing new architectures themselves. I could definitely see that happening soon. I don't know if does that mean a c does that would that imply a convergence? Maybe. Maybe or but I do think as well we'll see a proliferation of LMs or like AIs, like like pseudo AIs into many, many aspects of our lives. It won't just be like on your you know on your web browser or on your computer working with identic coding or whatever. I think it'll be on your phone, it'll be like, you know, in your microwave it'll be whatever, like it'll be all over the place and those will be different models, maybe s much smaller or something and you know, maybe c communicating with each other in some way. And you know, it won't be just it's big server size models, there'll be models all over the place with that are performing different functions would be my guess. And that probably means that a bit less convergence because those will be slightly different in different sizes, different trade offs that they have to make. But yeah, it's hard to know. Ravid Shwartz-Ziv: Okay, I think we're out of time. Do you have anything else that you want to add? Brendan: No, I mean the thanks for having me. This has been great. yeah, I guess maybe the last thing I'd say would like I'd love it if people were able to like pick up the diffusion Gemma model, you know, fine tune it for your use case, or find some new sampler, or d or just use it as a base for research. I'd love for people to be able to play with it and, you know, have some push the push the boundaries of what Text Fusion can do, you know, write a paper, obliterate it, you know. Like you like you were saying, Alan, like figure out a way to to break the safety guardrails and write a paper about that so we know like exactly what's going on there. Or even just do interp work on it. Like you know I think we're only at the start of understanding what the fusion models are doing in terms of what they're actually thinking and and what they're capable of. yeah, so I'd s I just encourage people to take check that out, you know, download it, play with it, run it on your own machine. Put it on a put it on a random new piece of hardware that no one has ever run it before and see if it still runs. And yeah, just write See if you can have a contribution to Text diffusion and help us push this field forward. Ravid Shwartz-Ziv: Great great message. Brendan. Thank you so much for joining Brendan: Ha ha ha Ravid Shwartz-Ziv: us. it was a pleasure. Allen Roush: Yeah, I I Brendan: That's been great. Thanks for having me on, guys. Allen Roush: I learned so much from you. This was a great discussion. Brendan: Yeah, I you asked Ravid Shwartz-Ziv: Thanks. Brendan: me some pretty t tricky questions that I hadn't ever even thought Allen Roush: Ha ha ha. Ravid Shwartz-Ziv: Okay. Brendan: about before. the connection between mad libs and bird is really that is that is a new c that is a new Ravid Shwartz-Ziv: No. Brendan: one for me and I I I enjoyed it. I really enjoyed it. It's been a fascinating Ravid Shwartz-Ziv: Yeah. Brendan: discussion and thanks for all the questions. It's been great. Ravid Shwartz-Ziv: Thank you so much. Thank you. Brendan: Cheers guys. Bye bye.