Latest / The Information Bottleneck / Continual Learning Is the Next Bottleneck | Rohan Anil (Core Automation )

Transcript

Continual Learning Is the Next Bottleneck | Rohan Anil (Core Automation ) · The Information Bottleneck

• Ravid Shwartz-Ziv: Hi everyone and welcome back to the information button neck podcast. And today we have Ruhan Anil. he's the optimizer wizard. he was at Google Brain, Gemini pre-training, then he moved to to Anthropic and then he left, one of the only one that left before IPO. to co found maybe one of the hottest startup today, Core Automation, to accelerate automate AI research. Hey Juhan, thanks for joining us. aro: thank you for having me as well. pretty exciting to meet you and both and then talk about some research. Ravid Shwartz-Ziv: So I think like we can start from the beginning maybe. you just shared shared with me an email that you sent me I think like nine or ten years ago about about some paper that we that we that we published about some question like again a long time ago. So maybe let's start from the beginning. Like how you get into the field and what you have done, maybe and and why you end up in like pre training and aro: Yeah. Ravid Shwartz-Ziv: trying to to do the thing that you are doing now. aro: Yeah, so I guess my journey started at Google. that was an awesome place. I was working on trying to always get into so my whole reason to come to the United States was trying to do like computer vision research products on top of it. Pretty excited by that type of work. But then quickly realized that computer vision, at least when I was in grad school, was a dead end because it was random forests and feature engineering and I was pretty sad about it and I was like, okay, I will try to make money and like have a life. And then I got really I was like, okay, I can't. I need to understand a lot more of the stuff that I came to the US for. and then Google I went to Google to do B ML adjacent. So I was still doing other things. So I ended up doing a lot of systems engineering work at Google for the first two years. It ended up being extremely like some cool memory allocators and stuff like that that had impact at Google scale to the point where I was able to be like, okay, I want to move to Google Research. It was like very exclusive. It is before everything took off. There's only like people who are very famous in there. And so I was like a fly on the wall in the Sybil team. That team was doing large scale linear solvers for Google. So basically think of it as like extremely large dimensional sparse features. And you train a model for like months online with Ravid Shwartz-Ziv: Wh which model? Which types of models? aro: so back then really like Google's found foundational things like ads PCTR predictions, ranking problems, anywhere you needed some type of machine learning, but at Google scale, so really that dimension is sort of like the R D is D is very huge. So we would go to like perhaps more than int 32, bigger than int 32 because we ran into N32 bottlenecks. But only a few of them are active. So it's sort of like very sparse. So the R square of the feature is pretty small. So it's like very sparse feature. And and then I was doing that. so I started in that team and when I joined the team there was going through this r initial revolution pre-neural networks on if you remember latent feature models like collaborative filtering, tensor products, you add into a linear model and then just like do recommenders, you can do indexing, you can do retrieval with those. So it was going through that and the entire solver was getting rewritten. I personally was starting to understand, like, okay, w I wanna work on this, it's very exciting. Like Obviously there's a lot of cool math and trying to understand this better. Then I got like nerd sniped by two papers, mostly by Shayshela Schwartz and Tong Jiang, and then another with by Peter Rich Tarek called Coco and Michael Jordan, I think if remember correctly, and a few others, like th these are the big names. I forget the f first author's name, but like Coco Coco Communication Efficient Training. So I remember like spending my like one I was doing many important things for Google, but I was largely like fifty percent of my time was spent on like I wanna understand STCA, accelerated STCA, prox STCA, I wanna understand how to train all the details. So just like soaking in all the information while scaling up these learner and TensorFlow had just come in. It hadn't been released externally. So I was writing these solvers on TensorFlow. And then that got really exciting. luck would have it. so I was working on like search and ads and stuff, making their models better. Neural networks are starting to show promise, it's starting to take off. Like you can see it's gonna come. The first TPUs are not taped out yet. but like I was like a dinosaur doing large-scale linear solver and neural networks are right behind me trying to all like completely remove all my work. So it's certainly like it was that time, and then like luck would have it, like the stuff that I was working on ended up being extremely useful for certain shapes of problems. That's like the accelerated convergence of STCA. that's some cool problem for which like the previous solver that a lot of people were working on would take seven days to converge. Mine would finish in five minutes. If you remember, STCA doesn't have like warm starting properties. So I remember one of the senior researcher who then I worked at Brain came to me and it's like, How do you do online learning with this TCA? Because the duals don't you can't transfer duals. I mean, some people have tried to work on it and I was like, I just train it from scratch again because it only takes five minutes to convert. So I can write a photo over it. So that's how I got into up time. Ravid Shwartz-Ziv: Good times, good times. aro: So I got into optimization. Then was then I started working on a couple of projects and then one of the projects, it was like two teams. I was doing large scale linear models. My future manager, Yonghui Wu, was doing ranking with neural networks, and then obviously neural networks beat the linear model fair and square. And so I was like, Hey Younghui, can I work with you or merge these things and just and so I moved to Yonghui's team. And I was in Yonghui's team till I left. Google. So I've been with it with him since then. So over there, I was like, okay, 50% you can spend time doing cool research, 50% do engineering, like you know, build inference and things other like there were a bunch of things that we were doing back then for Google prediction. So I ended up like some so because of my work on optimization before, like as you know, the shampoo crew found me. I was like before shampoo was written, hey Rohan, you seem to be very much like gung ho on this optimization thing. We are doing it. I we will teach you all the stuff around convergence bots, all the other stuff, which I've never learned, right? So it's like kind of a like apprenticeship in their team, but I was pretty good at like systems, so it was kind of like a good fit. I was like in trying to for the longest time worked with a lot of work like SM3s, versions of distributed shampoo, making shampoo work. Linear algebra routines of it. Ravid Shwartz-Ziv: So shampoo just just like like shampoo who who the one that the the the two people that didn't know that didn't know about shampoo, so this is like an optimizer a preconditioned optimizer, right? That like that like you basically need to condition every time on the on the previous few steps. we will get into it, but yeah, continue. aro: Yeah, so that's how I ended up like kind of like kind of getting deeper into things that were like lower down the stack. And then I was largely situated in brain as like, okay, I get dropped in to pr like solve problems for different teams. So one of them was like, Look, in we're putting neural networks in ads but it's not going well because neural networks have high variance, try to figure out how to solve that and that like type of it was like kind of like a war room because neural networks are important, but we're not able to productionize it. That's bad. you can make a lot of money if neural networks predictions can actually work in inference time. So I was that's where all the online distillation paper and stuff came about. So I was in a role where I had problems that I would find and I could think about how to solve it. And it just happened that brain had like all these really amazing, obviously like in hindsight, like some of the best people in the world just sitting around and they were very generous with their time. So they were just like, Rohan, just you're working on this distillation idea. We should write a paper on this another angle on it. And so it's kind of like kind of like a lot of cooking was happening while many problems were showing up at scale. And that was like a really perfect time because you would need to seek don't you have to seek out problems, but at the same time you can immediately get feedback at scale of that time from a f from a real and then it was aligned to Google's mission like all the impact and stuff because it was kind of like making Google more revenue, more you getting more users and whatnot. So it was like a really like a cool time. And then I spent a lot Allen Roush: Yeah. aro: of time thinking about those and then obviously started hitting a stride of like thinking about what should we be doing. But like I've been thinking about those same problems for like nine years. This is where your paper comes in. I was kind of frustrated by the search thing 'cause we hit a wall in terms of neural network accuracy. We could not go to like we were like if you think about prediction, it's either like compute if like some compute constraints in training or inference and you're kind of finding the local minima that kind of constrains these two. And so you're innovating in the margins. So I was very frustrated about that. That's when I think I remember sending an email. I saw this paper. I remember Hinton was saying this paper is so cool. Like in one of the week Rikti thing, I thought so too and I was like, Okay, I need to understand my the neural network more and that's the we're using information and then was sending Ravid Shwartz-Ziv: It's the opening i it's called the opening the black box, by information. If if someone wants to to look at it. aro: Yeah, and now yeah, there is like video too on lectures on that too. I think it's pretty relevant Ravid Shwartz-Ziv: Yeah. aro: still. And it's sort of like fundamentally like the more you understand, more creative you can be in solving some of the f problems. And it's sort of like it's very optimization dependent, like so the way deep networks is encoding maybe the features and how it's encoding an encoder or a decoder and current models. It's sort of like if you can see it maybe you can intervene and make something better if there are bottlenecks, literally no pun intended in the in the architecture, right? So that's like always like more understanding is always useful. And and I guess that's like the pre brain work. And at that time I was kind of always kind of full go different parts of the stack. So I was kind of writing the inference for the first like translate on TPUs, writing beam search, decoder. writing slightly faster top K operator and things. So I would just like go all the foundational levels. yeah, and then yeah, that was like the most I feel like it was the most fun time because it was just like problems just coming out, all these people just like and some reason like 27 to 2018, 2019 is just like some of the papers that came out of that time now everyone's like, okay, because it has like large industry impact. Like talking about it, but it's like it's fun thinking about how that was cooked and people didn't know it was gonna be a big deal. It was just like people cooking, you know, network stuff. So it was fun. Ravid Shwartz-Ziv: W why why you think like it was such a I don't know, like a unique ta period of time? Like why aro: I think it's a Ravid Shwartz-Ziv: what what was so special then? aro: I think it's like the ethos, right? Like, I think Jeff Dean and the initial funders of Google Brain was always like this like moonshot thing. So it they would like p and I was not part of the initial team or I came in pretty like later, right? Like in twenty sixteen or twenty seventeen is when I joined. It's sort of like the ethos of like being able to understand that you can't you have to kind of make bigger bets and then have alignment on researches, but not how you do it and have like a culture, like research culture, right? It was kind of replicating a great university inside with Google's resources. And and then obviously like when you have big organization, like how do you keep people motivated? You need like performance. Like this is back in the day, right? But this is before all the crazy things that happen today within the AI industry with like money being thrown at people like crazy, like insane numbers, val everything is now a little bit like amped up because of the economic impact. Back then people were pretty not I mean there was people getting paid. Like they weren't retiring after a year, right? Like they were still kind of like motivated, self-motivated. And then I think I think that environment creates like great work. And it was something about that like ethos and like farming that organism that can just like keep innovating is it's is just happened. I think this has happened, I would say, back in the day in other labs, right? Bell labs and other places. So it's like it's really hard to recreate and it does need like an all like a big industri like at least like back in the day there was multiple companies that had money to fund this and Google definitely had the money to fund it and had the right people at the top to kinda think about the impact of these in an altruistic way in some sense, right? 'Cause it was like publication, right? At some point Google had like the most number of submissions for New Reps, ICLEAR and OP or SCML or in the top three. so it was one of the biggest un institutions, including Meta as well and other institutions as well. So I think yeah. Allen Roush: So so so real quick related to that, we're we're seeing now a a sort of moment for open source AI model releases at least, right? aro: Yeah. Allen Roush: Like China's c coming out with all these cool models. Nurit and all these other conferences keep having record breaking number of submissions. Do you think that that we'll get more open source research going forward? Like do you think that it's like are you bullish on open source research or bearish and Model release. aro: I think it's I think a research you like you cannot bottle up research into a single entity for sure. talent and ideas are distributed, you're building off like shoulders of giants across the world. So I think like I don't think anyone would take the other position. I would be happy to debate anyone who takes the other position if they're really attuned to what's going on, right? Like what maybe bigger institutions can do is like scale something very quickly and very l s you can like solve a problem, but like the fundamental science is now happening at a faster rate. And then in some sense AI agents are also kinda distributed. I mean it token pricing is definitely something that is keeping Not all researchers have access to yet because these models are expensive to run. But you can see start to see their impact is getting faster because they can run more experiments by telling the AI agent the specification. Even if it's slightly under specified, they can explore the local minima that they've set up. And actually finding that like place to search is more important than the search itself, right? Like knowing where to dig is harder. Like if the current, I mean, this is maybe going into a rant, I don't know. But like, you know, these frontier models are dumb as a rock to me if I don't know what to ask them. And sort of like depending on who uses it, you get completely different outcomes. If I use Fable, I don't think I can s disprove the Jacobian conjecture. I have skill issue. But if I Use Fable for some of the stuff that I know what I'm wanna do. I get there much, much quicker, maybe speeding up six months of work into like a week or a few days, right? So it's sort of like depending on who you like, it is the person behind the scenes that's like putting in the tokens into the model, and those tokens are the valuable prior in some sense. And that will never go. I think that would not is I'm I'm gonna make make a bet that that's be very hard to for models to search over. I think eventually it'll get there, but I don't see it getting there at least this year or next maybe for the next one year I could make a prediction that it's not gonna get there, right? Like in terms of models having the sense of the right search space to search for before for like these general like frontier problems. Like if you can verify, yes, sure you can add more but if you can't verify you have to run big experiments, then no, right? It's very hard for models to get the same sense of like what could work. Ravid Shwartz-Ziv: Okay, I I want to continue wait, I want to continue the the journey, right? So like you you like Google was the most the best place in the world till what twenty twenty two? and then what? What happened? aro: yeah, so Okay, so I I mean like it's my like obviously I I like the there is something about hanging out with theory researchers and a bunch of like people who are like in the fundamentals, you miss the like the bigger picture, right? Like you'd miss like so the G P D paper happened, I was like, I can train at Big Transformer at Google too. What's the big deal? Like I I wanna work on optimization and other things. Like that's more like interesting. So you kinda miss the picture of like like the zero shot learning and then so that's that's personal. right like not Google it's very a lot of people working on language models and stuff and then I think like I even like if you look at my 20 17 or 2018 paper, the online display. I got the first common crawl at Google. So I downloaded all of Common Crawl using Google's indexing software. So I just downloaded like I think like a trillion tok more than a trillion tokens. And I wrote my like with George Dahl, did my own QA of pre-training data and deleted it, deleted all the junk. But then transform I think when I was doing it, Transformer was around the same time. So it hadn't gotten into my like as like something I could use. So I was using an LS. CM to scale. So you trained a paragraph level LSTM model and figuring out instability to prove out that online distillation works. So the the sc like it it was there, but it wasn't like I would say like the fact the scaling kind of showed like you can just get bet like better returns on many tasks by just training a very giant multitask model on lots of data. So and even inside Google Brain, people like Norm would come and be like, I want to train a trillion toque parameter model on this data. And I would be like there were a lot of things happening. Even Norm has trained like multiple models inside Off GPT-2, GPT-3 scale inside as well. It's just maybe not GP33, but like maybe slightly smaller. And so Yeah, so that was happening in 2022. I had my kid. I was in PAT leave. I was working on fun stuff with Boris Dema, who is this open source hacker on reproducing DALI Mini with like open code. He he even did this live streaming of like training like the entire training curve. He would have a constant learning rate so he could always decay it and then send pictures of all the generation that came out of that model. It was a transformer. He was asking, hey, what's a better optimizer? As I was sort of like, hey, like I actually finished the Jax implementation of the shampoo thing. You should it's in the Google Research repo, you should use it. And he uses it and he's like, Holy shit, like this is so cool. Like all my stuff Ravid Shwartz-Ziv: Ha ha. aro: are gone away. And it's about X plus like 2X plus something. So he's like training at first time. And that like causes Jacob Devlin, who was working on he's the bird author who's working on language models to ping me, is like, Rohan, we need Need help. Our training is not stable at scale. Like we should use shampoo or something, you should come. And I was and I kind of misjudged. I was like, I'm on my TAC leave, but once I come in, it'll take me 14 days to just get you something. And you can and then 14 days turned into like two years of like work making pre-training optimization and other things better and thinking about that problem even more. Better and I think it's sort of like yeah, chat GPT happened and it obviously that created like a threat or like a clear business and then Google had to respond. I helped the Pump do a release with my manager who was leading it kinda like on optimization and then since then like worked on making like smaller efficient versions of it. So The Gemini team was being formed. They asked me, okay, from the Google s brain side, maybe you and Yong Kwe should help with some of the text pillar modeling, like it's like the pre-training side of things, and then a couple of others, the Chinchula folks from the brain deep mind side, and then we worked for about two years trying to like get things in shape and when you have very large organization, it w it takes a while to get through all the details of a full pipeline and it was fun. and then Yeah, I think that's the story. And then that that's where like I was kind of looking at efficiency. So the it took a while. I initially put in some nano models out and then Gemini Flash was kind of cool. You can see the paper. It had all the culmination of like s like there was a lot of stuff, but like the thing that I'm proud of is like second order methods, distillation. On policy distillation, everything that you can think of that's now hard to put that into the model to cram it into the smallest package. And it was kind of surprising to a lot of the leadership at that time. How could such a model exist that can work at this? Now it's just common. Everybody every lab trains in a very efficient flash-size model. But then that was like kind of fun, and that was kind of the peak of it. Then I was starting to think of like, okay, this is the limit. So we move the inference frontier. Further, how do you get like I started getting more thinking about like computation and like I was trying to think of like what did I miss in the last four years in terms of like my research directions that obviously I did not jump into language models and I was like spending computation and spending computations more Correctly and like how do you think about spending computation and how do you improve things? So just a quick back. When I started working on second order methods, everyone was saying, Why do you need this? Is this too expensive? This will never run on GPUs and like dah da, right? Like it is like why Adam is all you need. And then Tom or Coran was like, no, no, like this is the frontier, like this is the You put more thing he didn't say it in terms of compute, but he was like, you use more information, I guess, and then make a better model better optimization. Obviously the other things have hit the limit. You can see the first paragraph from the paper I have with Tomer. It's just like he wrote that paragraph of like, you know, like the fundamentals of why this is important. And then I was thinking, like, that is this really Ravid Shwartz-Ziv: I just want to to say one one more thing that I I think I met Tomer like at Google like maybe it was twenty seventeen, something like that, twenty eighteen he was at the Mountain View and like and I asked him like what you're working on and all these things and he explained me and then I asked him, Okay, so but like d c like can you do something with that? Like is it somehow like useful? And he some he said like something like No, actually no, like I I don't care about something that is useful. I don't want that people will use my thing, so aro: It's funny because I think like our pairing worked quite well. I have some like unders not stories about how the shampoo thing was built. and this is like through oration from like I was there but like I didn't see this happening in front of my eyes. The first one is like these guys work out like the shampoo initial sketch on a whiteboard, right? Like so the shapes match and things are good. And as with like all learning theory people, like they try to prove stuff. So they went, So Tomer. Took like If I understand right, sorry, Tomorrow if I'm this is wrong, but I'm just gonna say it and then maybe it's like somewhere in the middle. But like he spent like two and a half weeks not talking to anyone, working by himself at home, trying to prove theory for this new algorithm they have come up with. And then he spends like and then like both Vinit and Yoram both don't know like what's going on, right? He's just not in the office and just working on theory. And then he just like comes back and then goes to the office. And then says, I have the proof for two dimension and and I think yeah, couldn't do it for three dimension or he did, I'm not sure, but it was like so many pages of like math to get to the solution. So then Vinny looks at it and he he's also brilliant and he's like, okay, he's like pretty excited. He's like always learning the new stuff at Google and he just looks Ravid Shwartz-Ziv: Ha ha. aro: at it and then it's like, I can simplify it. And then you can see the current proof, which is like using singular value decomposition, and it's probably like and longer orders and and stuff, it's like much shorter. And he makes that version that simplifies it and makes it like maybe one you can fit in a page, right? Or less than a page now. So that's the story of Like, you know, Tomer. And then it kind of was great because I had the feeling of like I need to make this work. Like paper by itself is not useful. I need to make it work in a Google scale production pipeline. That's when something actually works. So I think we kind of balance each other. Tomers, like, I don't care if something works in mine, which is like I need it to work before it works. so that's the story. That was that was fun. In hindsight, it is interesting story too, because he's just like like this classic like you don't spend all this time trying to like prove this thing because it's the the final boss proof. Ravid Shwartz-Ziv: So okay, so like you spend like you're like you you spend a lot of time on like pre-trained Gemini. and then why? So why to to live to to entropic? aro: I was personally very tired of like the amount of time time it takes to kinda like the Google I'm like a bit more like explorer my exploit explore bias is like more on the explore side because I saw like the work on shampoo and like very few people have actually worked on a single thing and then continue to work on those variations of it and improved it. I mean there are many like maybe many people, but like compared to the whole lot. And I was kind of doing it when nobody was giving me any encouragement except like a couple of my closest colleagues. I was just doing it purely for I think there is something here and there is more to learn. And so when I looked at that, like how do I put myself in that situation? Because I think we are pretty much in the exploit era because there is like industrial demand for competition to happen. And so then we miss out on exploration. And I think exploration is what keeps me like alive, like because it's the thing that pushes my knowledge through the like to the frontier. And also like In some sense, like all these things, algorithms and stuff, is still relevant today. So like it will still be relevant ten years from now. Many things that were a very short term will not be that relevant. It'll be the the thing for the year and then people forget about it. So like fundamental advancements do have like a lifeline that's like lifetime that's longer. Like I still talk about your paper on information, right? Like this is like there is something about like creating new knowledge or like even pushing the knowledge in an epsilon way that I think in my head canon feels like super valuable enriching my life. So I was kind of like, okay, I can't keep doing the thing. I feel like innovation is where I want to spend on. Innovation's hard to do in an environment that's always like get the next thing in and I kind of was like self selecting myself out of it and I was trying to see where to spend time to do the research and trying to push the frontier 'cause research is hard, most things don't work. And you need to be in an environment that kind of like pushes that much bigger. And I was fine personally to do that trade, right? Like Google's comfortable, it's great. I have like eleven and a half years there, people know me and all that stuff, but kind of wanted to push and I was like, okay, the field is going into this and also closed versus open. I could see like, I have innovator on X and Y. I see like a year later papers written with the same idea and I we could not publish our idea. So it was kind of like bottlenecked into like I'm like kinda just into this machine that's like Epsilon improving. But as I do com if if you see my Twitter I see communicate like pretty like openly, like to the point, like obviously more than most researchers on some of the from labs on the some of the ideas. And that's like something that's maybe how I've been trained to do and something that I enjoy doing. And then yeah, that's I think like the story of it. And then I think I got my mojo back at Anthropic. It was like, like the pre-training team there is amazing. It's the best research culture. Some of the best people are there trying to do innovative research. And it's the most fun I had. My brain got like fully c cleansed. I was just having the most fun there. But then I realized, okay, at this point, like they're good. There is like open AI versus anthropic coding wars, coding agent wars. Again I started getting these patterns of like how it was at Google, like, make make ads better by X person than because the next three model release matters. For the company and IPOs you said and everything, and I was sort of like, How do I think about something bigger? And then I met Jerry and we initial conversation was all like these nitty gri like kind of the details of things on how we should and I was like, Okay, kinda like kind of wavelength matched, and this is what I kinda wanted to do and make sense to do now because we combined may have I mean we we will know soon, but like have some indi directional insights on which which directions to go on taking this thing further and that's what we wanted to push on 100%. Like I think like somebody gives you advice, right? Like don't do like many things, but do something really well. And like that's what would matter at the end. Like take that shot and it's like peop like for different people, like that risk reward ratio is pretty different. Cause if you take a shot and lose, then you kind of lose your all your hand. In my case I'm like, okay, I can take this shot and I should do it. Ravid Shwartz-Ziv: But but I I I just I w I want to to to to spend one one more minute. So you you met Jerry, Jerry Tw Twerik, what is his last name? aro: Twerk, yeah. Ravid Shwartz-Ziv: Yeah, and he he is like he was like what VP of research or something like that at OpenAI and you met with him and like and then what? Like you talk about you say that we are thinking that like the the current direction that that these companies is wrong or like we we think that like we have better ideas how to make it right? aro: no, so I would say like the first meeting was sort of like him pinging me on X is like, Hey, do you wanna do this together? And I like with you, I'll consider it. It was like an opener opening line, like, you know, and then I was Ravid Shwartz-Ziv: Talk with my agent. aro: Yeah, like yeah. I was like I was not I was so like very few people like I gravitate to people who like kind of think through like these things, right? Like how do you push like more of the details, like kind of getting into the details. And like obviously bef after I left Google there was many options to do many other things. And I was most like I wasn't s I saw something different, right? Our conversation wasn't about like how what the company is about, how we will portray it, how like all the surf superficial stuff. It was purely like talking about like current limitations and trying to think like, okay, what is the different access that we might need to explore. And obviously at that time we were also communicating in like this like dance because we can't talk to each other about the details either. Cause we haven't really like so so we were discussing and it was kind of funny. It was just like a thirty-five minute conversation at Dolores Park and it's kinda like like I was actually intrigued. I was like, okay, this person is also talking about stuff about architecture that I believe is also true and that's very few people believe the stuff that I want to do is actually true. And I was like, okay, this makes kind of sense. And then I started w think w like like because I hadn't worked with him. I talked to him for any month and a half and then we assembled a team together of people we worked with and are having a lot of fun at core automation. So that's like the short version of the story. but it's sort of like Fundamentally we believe it's a research company. We believe research is direction plus like getting people armed with a lot of agents can make a lot of progress on the research points we want to do. And there was like a very co we have a very coherent view, which is sort of like there are these bottlenecks both in architecture and implementation kernels and stuff, which you need to get to experiments to run every day on something innovative. And we want to get to models where models that remove some of these bottlenecks, the kernel generation being one of them, because then we can iterate on some of the ideas. I have a history of like three ideas that took like three, four, or five years to come to fruition. And I have lots of patient and a lot of it's related to the lower level details. Hey, can I get this very large embedding tables that spans multiple devices? Or hey, can I get this kernel that can do tensor products for attention? in this particular way and it just feels that like if I can do that on a much shorter cycle, I can iterate on the ideas much quicker and then get there. Then also like we both have been like I mean Jerry's previous work is on Owen with his team, Strawberry. I feel like that was like one of the biggest jumps I've seen in terms of like I almost feel like if they kept the A AI party alive, like if it was just pre training, we would have just had like RLHF maxed chat bots today. And we the agent tick thing was truly like the thing that you can put these like get out of the bottleneck of the transformer by putting chain of thought tokens to get computational depth. And that's the perspective I look at. Like and that was like really cool and they were able to train it and the model seems like it can stink in natural language, which is like doubly good. And so it's sort of like It it was the big gym and how do we get like a something next so that these agents are more reliable, can learn on the fly at inference time and those are the problems we're trying to tackle. And and we believe like obviously there are a lot of teams working on many of these things with harness engineering, wiki LLM or like programs that you like learn about like how the model behaved and then add it to context. We believe like the next architecture needs a bit more than in context learning to get there. It needs like better ways to incorporate new data, new memory, being able to learn how to learn. And so those are the type of ideas that we kind of vibe of research that we're kinda exploring and we see science and so it's like okay, it makes sense. Ravid Shwartz-Ziv: So So w what are your bets? Like where w how how we can get like you know, better like or how can we achieve continual learning, right? Like in like all the aro: The I think continual learning Ravid Shwartz-Ziv: things that like that are missing. aro: Yeah, so I think like I can think of it in like I this is my take. Jerry has like beautiful takes you can find like his own. mine has like cut off like kind of optimization built because like that's like my brain somehow thinks from that perspective. It's sort of like you know, I have how do I make training and inference look up like the same? That's like one question. Another question is how do I incorporate new data into an already trained model? And you it's probably like a lot of things are probably firing in your head around plasticity and many other things, right? So the fundamental optimization questions. And how do you like think in terms of like what knowledge to incorporate, what time scale to incorporate it at? How do you fold things in? How do you operate at a level where the model is improving by itself? On a daily basis, and obviously you can take the most extreme ends, which is like I will just post-train more, bro. Like it's just like I will make an end every day of all the things that happened today and post-train a little bit more, a little bit more, but that hits plateau. Then you can say, I'll on policy distill it with itself, that hits another plateau because it's not doing much feature learning. RL is not like isn't like it's not changing the base learning that has happened. So, how do you do b all of that at Training time if in inference time more efficiently is a question we ask. And that gets us out of this like, you know, we've said like, okay, we want to combine pre-training and RL together, and that's it's kind of a interesting question. Like one is a diff one is building priors, the other is building like compressing, like or sharpening the distribution to the tasks we care about. How do you do them together th in an ex in a way that can happen? on a At inference time and that is the so then it's like a lot of science questions need to be answered. Very, very fundamental and and kind of the belief is that the current transformers have some bottlenecks that we want to remove. Like it doesn't like it doesn't do all of these, otherwise we would have I mean we would all be retired, I suppose, 'cause like I'm working with the same sort of coding agent and it's so bad, like in many of the things and it's frustrating, it doesn't know anything about core automation. I can obviously make a big agents.md or claw.md, but like it's can only go so far. So clearly that bottleneck. That's what I wanna kind of unlock. And it does make make you feel different, like if the model is co evolving with what you're working on, it does like you're working in a like it's almost like you have now something that can you can steer very easily and cheer it in a way with like that gives you a different user experience. Ravid Shwartz-Ziv: It's so funny when we talk with people, you know, that like for example, like we talked like several weeks ago with Joseph Soretz and he's like from Puffer AI, he's doing RL, right? Like all these games and like and he basically said that like look, like these coding agents are really nice, but like when I really want to get something, when I really want to get optimized like my RL pipeline, I need to write everything by myself and I need to like to go into the code. And you can see this like creating like you know like differences between the like people that are building apps and things like that versus people that are actually like trying to optimize like you know very low level pipelines b in in how they like how they think about coding agents and and so wh why why you think like it's like that? Why you think like aro: I think Ravid Shwartz-Ziv: coding agents are like still not like good enough and like maybe now Like you can talk about maybe let's about the the kernel optimization competition. aro: Yeah, so you here is the thing, right? Like so one you can say maybe labs haven't focused on these problems. like that's like the one take you c I can give one take, maybe like the str the extreme end is like, this is solved, but like it just we just needed to write add the right ends. into the model training and it will magically be solved. But n that's not really true because it's sort of like you need to like understand the role model of where what is going on and optimize like it's like like distributed systems, distributed training, kernels, it's all of like using different resources in ways that like and in a way that's more as efficient as possible. And for that you need to understand like both algorithms, systems, and how to map them into hardware. And I think these models lack that completely and that's one. So like you can then har harness engineer and put a lot of search to try to like get to a solution. Alpha Alpha Alpha Evolve is like the deep mind version and open involve. So you're gonna or you can do the slash goal in codex and claude and to kind of like get to like the solution by hardness just keep operating. And but a lot of the problems with that is that like It doesn't know much about numerics. It doesn't know much about like the underlying end to end. So that like most of the thing that's comes out of it is slop and unusable and reward hacked. Like on the kernel competition we have for the first kernel competition we put QR factorization. For those who don't know, it's just like you rewrite a matrix in the form Q and R, R is upper triangular, Q is orthog orthonormal, and then you kinda like use this factorization. And you can use it for many things, right? You can use it in terms of an orthogonal iteration to get these Rayleigh coefficients and Rayleigh vectors, or you can do many other things with it. It's useful for shampoo line of work, soap, and many other things. And so this This one has like obviously James Damell and many others who work on Lapak has implemented and published papers around how to optimize these for CPUs. And each one of them, it was like you can take that as a prior and then obviously ask the agent to implement the exact same algorithm to but to come up with the algorithm It was quite hard, right? Some some person actually realized like this very you could you you the way you solve it is by doing householders on every column. they someone figured out that you don't need to do every column at a time, you can like delay something and get some matrix multiplies in. And so that was like an innovation somebody did in the 70s or 80s, I forget the papers come from. And like sort of these like innovations stacked on top, you need to do that first. And then if you give the model, okay, I exactly know what algorithm to do it could do it it needs to know how black walls work where the memory is what is the trade-off in accessing sram to slightly like when you go on the memory hierarchy it needs to know where to put the communication and so many people like attack this problem. The top one solution on the QR factor factorization problem is basically like completely reward hacked. if you use it for anything useful it will nan immediately. The second one, written by a human who is the world's expert probably or one of the experts in kernel writing and he learned about QR, actually thought about the memory model a lot. It is much more Cleaner C code base, it is actually something you can put it into a production pipeline and get like a 60x speed up on some of the kernels. And he used agents and he used his own, like like you can see, like the human is the like human and agents both are important, but like it's a human part that actually drove the result. I think the top one is just like agent going crazy with lots of search. It's called 10 billion tokens. They probably spent 10 billion tokens on it, and you all got was like a useless kernel. I mean like it's good. It has some cool patterns in it. I looked at the code, but it's not that useful for the end to end. So like that's the problem. It's like it's almost feels like it's a Like the agents can reward hack and reward hack definition is probably something to think about on any problem you give it. So you get this like, it is complete, but it isn't really complete. Right? Like it and when you work on systems, it's like you can't get 99% correct, you need 100% correct. Otherwise nothing works. Ravid Shwartz-Ziv: So So why? Why you think he's like that? What what is the reason? aro: And so I think it's really hard to get write good very like so many problems, right? Like numerics, how do you verify your program is correct? How do you verify program transformation is correct? Can models have a good mental model on itself? Verification, the current models just purely lack that. The second is like the mo the hardware continues to change. so if you have to train a new model for every new hardware, That seems not possible. Like I mean, that seems like possible, but sort of like kind of wasteful, right? Like to go through like because then you have like this problem of like many you you have to simulate everything that you're trying to attack to so put the dist distribution in distribution. So like that seems like something that the models lack, ability to explore a new environment, explore how build fundamental principles back about certain things about the environment and then fold it back into the model's own prior. And so that doesn't exist today. It can exist as an MD document or in context, but that can only go as far as the context can go. And then context really doesn't go that far. Ravid Shwartz-Ziv: Yeah, I just saw like a paper that they looked on the solutions that like they compare between solutions that LM and the like humans like suggested. And like they actually they saw that like that the LMs are really like the solutions are really related to previous like solutions. So they only like explore very like narrow sub space of the of the solutions while like humans just like generate like different solutions from from different aro: Yeah. I think the search is powerful, right? LLMs are awesome. Like it's so useful. we're all loving it. Ravid Shwartz-Ziv: Yeah. aro: But it is that like the question of like the curiosity or the creativity that is sort of lacking and the measure is probably hard to say. Like but it's like but we see it, right? Like and it's sort of also human bias to like kind of like promote the LLM a lot more because if it finds some something in a search space that I did not know, I'd be like the LLMs are so much more intelligent. It's knows all of it. But if if for the things that you know, you realize like, I just looked up this. But so it's sort of like it's able to compose things in very creative ways, so and explore the local minima extremely efficiently better than any human, but it's sort of like it needs the prior. If you give it the wrong like wrong prior, it searches and makes like like use not that useful. Ravid Shwartz-Ziv: It it's like d do you know Dana Rielli? he aro: no, it does yeah. Ravid Shwartz-Ziv: so so the so Dana Rielli I think he's professor at I don't know maybe like yell or something like that. Like but anyway like he is a like a professor for like business administration or something like that, but like he wrote a very famous like series of of of books about irrational like predic it I think he's called predictability irrational, the upsize of irrationality, like all these things. aro: Yeah, I I know the book. I know the book. Yes, yes. Ravid Shwartz-Ziv: yeah yeah yeah but like the funny thing is that like you like when you talk about with people about like his books they always tell you that like super smart like the book is so interesting and and everything like makes sense besides what I actually know about but besides like like like the the area that I actually like an expert then in this area is like totally bullshit it's just hand waving so aro: Yeah. Ravid Shwartz-Ziv: I I feel it like I don't know I'm sometimes like that aro: It it it is. It's it's it's like i but like but when it comes up with like this creative solution that you didn't know, I feel like I'm like, my god, it's all it's all all over, it's cooked. Then like the next day Ravid Shwartz-Ziv: Yeah. aro: I see it like making the dumbest mistake in the area that I know and I'm like, I could have just done this myself. Like this was just I burned so many tokens for this like s tiny like the thing that I could have and has like could have instructed it better. So the models are there. It's sort of like it's not able to learn from both the positive and negative experiences. And it there was no mechanism to teach it that or an automated mechanism to incorporate new knowledge as it explores a new environment. And I think that's sort of like the next frontier to make these agents more useful. I think a lot of the other things people do are gonna make the usability better, harness engineering adding nor knowledge bases and stuff, those are extremely useful and absolutely will keep it. But can you get there can you get a much better optima is what we're trying to think of. And can you do that from the very basic? Like did you have to take a transformer and do agent T RL or could you have taken something else that could do some of these things that that would help in these use cases? Allen Roush: So, what what do you think are the methods to i if they ever exist for improving the creativity of models like this to go, you know, genuinely like out of distribution, or do real extrapolation instead of just merely interpolation? aro: So okay, so if any I mean like so okay. Well just framing the thing like, okay, the Jacobian conjecture being disproved. I I enjoyed reading it. I ended up like working on it with DLM to see like, okay, what did it do? How did it like create it and then I even asked like, can you find like some patterns and like like everyone else did after 11th posted that sha like almost like a shit post about like the Jacobian conjecture is false, here is the proof, QE D three line thing. So clearly it's able to like ex do something novel in it. I did not he did not release the hardness for it, but me the OpenAI folks who said that that could also do it in 40 minutes released their prompt and it had patterns on adaptive like a research manager that like adaptively looks at the problem. It they had had some adversarial agents that could like push the thing further and ask it to explore more. So I would say like it is it is the like beauty of RL and the curse of RL, right? Like the RL collapses the distribution in some narrow space of the environments we put in. By definition, in terms of entropy, this is like killing creativity, but extr making outcomes extremely good. And pre-training is sort of like the base prior. And again, there the levers are sort of like the your data distribution, what you're building, and that's like I guess like the base model is more creative, but it's sort of like a creative genius that does nothing because it cannot produce any outcome. And RL just produces outcome, but it has like the problem of like exploring the space. So there's a fundamental question of which distribution to work with and how do you kind of perhaps if you combine these techno techniques that might be one way to have this explorer exploit and explore will help with maybe creativity. But that's sort of like a little like hand wavy answer. So Ravid Shwartz-Ziv: So so what is the solution? How do you see the solution? Like what will be like the the the the basic components? Like transformer will be there, like do we have like pre-training? Do we have RL? Like do we have like agents? aro: You so that's the one thing we jerk in the company and it's very true. Like, why did we start calling it pre-training? Why not training? Why did we call it RL? Ravid Shwartz-Ziv: That's true. aro: Why not call it example revating or something so that we horn and on the distribution? Like it seems like we may have introduced terminology and taxonomy on our on the research. in this like top-down fashion rather than going bottom up of like what each of these things are doing. So sort of like in some sense like what is modern pre-training do? It creates easy vowel sets and then these are held out perplexity and then you build the biggest mo not the biggest. So that's again mental lapse because everyone talks about it bigger is better type of thing. it's sort of like like computational like it learns some circuit that learns to compress as well as possible given a data distribution. A pre-training engineer goes and tweaks the data, it tweaks the architecture, it tweaks the optimization to reduce the perplexity on these held out sets. And obviously when you get this problem, you can see how you good hot the perplexity sets because now you know exactly what the perplexity sets are. Or I care about some perplexity on SVG generation. So I should add more SVG. So now it becomes like at least from a very Underdefined, under specified problem to like assigning weights to what you care about compressing and then just optimizing that in an end-to-end loop, and that's kind of the the thing. But then RL takes a model that's takes the pre- this like I I give Jerry like a model that's like okay, it's the scaling loss are beautiful, it's the perfect model for the amount of GPUs we have, it has the best compression. He takes that and then just destroys this loss, right? Then the NTP loss, like because he's doing RL with a completely different objective. And then it's just like, while I'm like throwing my hands up in the air and it's like, okay, what am I measuring? Like so then it's like, okay, from an organization perspective, I can measure one thing so I can make a surrogate loss of what I want. Then RL does another one. So this is a question of forming the right objective function. I feel like that's the hard hardest part of all of this. And then even then, if you had the objective function, maybe it's by level or something of that sort. How do you optimize it is also hard because RL has completely different performance, number of throughput ratio like systems side itself is quite different, variance is different. Somebody said egg roll, like you can do egg roll on RL, but you can really egg a growth to learn features in pre-training or at least would be very expensive to do so, right? So there's a question of what are we really doing in either of these spaces. And I can post questions rather than like say this is the answer, but these are kind of like kind of very interesting questions. Ravid Shwartz-Ziv: So what is dancers? Like how like what what are dan dancers here? Like d do we have do we still need to to split it to different phases? Like or you will say you you will see it in a in a year from now. aro: No, I wouldn't say like it like it couldn't like be I have this like silver bullet and it will solve all of all of these problems and then we'll be better. We are making progress on all the fundamental questions around these like general topic. And it there's sort of like it's really like a mm objective function is a very fascinating question. I think like even posing this question will get us like thinking about a hierarchy of like research directions to go through. and then like the optimization question around like what is RL doing, what is pre-training doing is already interesting. I think the papers like Eggrol and a bunch of papers that showed like what RL is doing, like the off principle basis and like the one where they kept the spectrum constant and just optimized the rotations just for RL is kind of s interesting. So there's like a lot of like interesting things coming on around like what is RL really learning and Can you just do more of it? And the answer seems to be just like no, like there's a limit. And that's where we're trying to see like how do you remove that limit, right? Because you want scaling of your new system to continue beyond just adding one more task to the pro to the system. Ravid Shwartz-Ziv: And do you plan to compete with the the big model the big companies with huge models? aro: in some sense like it would be very like especially like, you know, there are like four like ev what all of them have like many years thousands of people working on these problems. So the question is if you do, then it's kind of crazy. But sort of like at the same time, I can give examples of the Chinese labs, right? They they quickly were able to train with like very subset of people, particularly Deep Seek, the research was quite novel, was super interesting. The engineering was quite beautiful for like people into systems. Then so they wouldn't been able to make progress with fewer people. And now the question Like what could we do with fewer people but lots of agents and lots of like direction like directional signals, like could we make progress? So we don't know like how the system would interact, but we're not thinking of others. We're trying to think it's like what could we Like we think of others in the sense of like the broader ecosystem of where we are versus others, but we're like probably some people like they're not even bothered about, probably. Hopefully we they leave us like kind of to our thing so that we can work on the thing very c carefully and methodically. in some sense if the Ravid Shwartz-Ziv: And aro: stuff starts like I mean, we do wanna have our outcomes be visible to external folks or we would want to get there. Perhaps kernel generation is gonna be one of the first ones we'd hopefully see because it's sort of like kind of like kind of it's easy to measure. It's easy to know where we are, it's easy to know what the problems are and how do we get there. Like it's sort of like b because the c other problems are a little bit more like slightly longer term, right? Like if I had a solution like it's just like there's many things involved to get to the point where a user is using a continual learning system. So we're planning for it, but there are milestones along the way. Allen Roush: Do you do you think that Ampere GPUs will stop being used once once we have all this kernel level optimization or are they still gonna be commending two dollar an hour prices for a few more years? aro: Yeah, so I'm I'm probably the like I I you know I don't know the answer to this MPR GPU question. I would need to look up like what are the different use cases. just like I would say like Bif knowing a little bit about hardware from mostly from the TPU land. I've been in the GPU land very recently. Mostly have worked with TPUs for like all of my time at Google. and starting from the first generation. Sort of like every t the compute demand is so high that if you can make there's like the landscape in the industrial landscape of like where computers going, it's mostly going in inference. Not everything is going into an LLM. A lot of it is like on nets and many other things. So if you have If you it's gonna be like a business thing. If you can run something efficiently on cheaper hardware, and cheaper would mean like how many, how much is the operating cost of this hardware, how much has it been amortized so far, and what is the value improved performance of your kernels and the end-to-end system on that particular hardware? It could still be extremely valuable for many use cases, so but it's something that I haven't personally like this is just like a meta thing of how it Think about it, but this is sort of like we would want to know. Kernel generation perhaps could solve one of the problems, which is like the inner loop will get faster. It can maybe get closer to the roof line. Or if kernel generation includes some creativity aspects, it can maybe come up with clever algorithms to do the same thing. James Temmel was saying that they found a new clever algorithm for linear algebra primitive, which is like replicate the data. So before all the things were kind of implemented with like data is constant and then you're doing mem memory move so you can only work with this data. But if you realize that you have a lot more memory, you can replicate the data so that like you can different parts can use different of the same data, but then you won't have like cache conflicts and things like that. So it's sort of like they found a new optimization pattern. So if there are other optimization patterns that we discover through a combined human plus models, then perhaps these things could get even more valuable. And it's sort of like a problem of like also like addressing NASMA Market is like harder because it's a long tail. It's sort of like and people usually humans kind of want to focus on one or two problems, not the long tail. Like I can make inference for the next generation of LLM be more efficient because of rethinking architecture, but I would not get to like connets kind of thing. So I don't know if this answer gives you I think, but it can give you the Inner loop better, but I just also want to say like it's inner loop sometimes is not sufficient for the end-to-end system. There are Umdahl's law. comes at us all, as Mark Sorophon, one of our co-founders say, like it's sort of like, what is the arm Dal's law bottleneck? And that's like how slow fast the system can go. You have all the kernels but something else is slow on the system, then like I don't know. It's gonna go for slow, right? Like it's like I can make a sixty X faster QR kernel, but I I have other limitations in this thing. The training is not gonna go sixty X faster. Right? It's just gonna make that thing not be bottlenecked. Okay. Ravid Shwartz-Ziv: What about open source? Do you plan to open source? aro: Yeah, we're pl planning to I mean I we don't know yet about the weights and stuff, but we are pretty like, you know, like the team is composed of people who've largely worked on open systems like PyTorch and Marcus works on fla he wrote Flash Adention V four. So the team is pretty like kind of like Swapno wrote the t the scaling book, the GPU section of it. The team is pretty much like systems and stacked on like kind of contributing to external. So if we improve kernels, we'll absolutely contribute it. If We find new ways of doing things, we are fine to give people ability to use that to do better. But the other part is sort of like how like you know, the thing that I cannot commit to is like the business model, because we do have to exist in this ecosystem and knowing like we're a small company and what is our like leverage, we would want to figure that out. But we are in like very reasonable. Ravid Shwartz-Ziv: Yeah. Okay, we are almost out of time. so let's do like a few questions from the audience. do you think could we have agentic optimizers? aro: so I wasn't sure what people meant by agent tick optimizers, honestly. it it d like w I think there was a definition, I just don't remember it now. Do you like do you do you know what he meant in terms of agent tick? So Ravid Shwartz-Ziv: So he d he doesn't like I I can tell you like what is my interpretation. Like now how can you have like several models? Like can you like train all of them in once? Like not like a single agent but like different agent, right? And then aro: Yeah. Ravid Shwartz-Ziv: how you backpropagate for all of these like agents? aro: In in some sense you we are starting to s like multi-agent if I don't know if I'm going in the wrong path, but like we already have multi-agent working in many of these systems, right? Like these age and then you can ask it to figure out how to do coordination, planning, backprop through this entire system to make it more efficient. How do you communicate agents to agents communication and things like that? I do see like from an optimization perspective, not like optimizers or optimization fundamentals. But in some sense, this auto research is sort of like a local minute, like a like a program search to do optimization for the problem that we care about. That's happening all over, right? Everyone's using these agents for this narrow problem and getting amazing results. Like I saw someone optimize using Fable when it was not nerfed, like the Gemma models on like Web on Chrome or something, WebRTC or Chrome, I forget which one. But it's sort of like they were using that that. So that's happening and Alpha Volve have found optimized kernels for like kind of like gem, better algorithms for gem and then implemented it efficiently as well. So that's happening quite a lot. Ravid Shwartz-Ziv: Okay, what do you think for the future of a high order optimizers and the recent adaptation aro: Yeah, I'm really very Ravid Shwartz-Ziv: of MUNET scal? aro: Yeah, so it's kinda cool, right? Like Muan is very cool. I think a lot of people used it it came out in like 2024 out of like this keller jordan's speedrun. Actually in the previous work of that is like Jeremy Bernstein had been thinking about it. He was looking at shampoo and saw the spectral descendant site and then like kind of built on top this whole non-feud feud is about Poemi pointing out that if you put better two equal to zero you get the same thing but Nuon implements the thing with Newton shows you don't need to do an inversion because when you set it to zero it's like z to the power zero. Instead of power minus one half. So we and then like adding momentum or nest of momentum on top obviously works. And obviously like you can find some implementation of shampoo. Actually, the first implementation of shampoo that we didn't do, but some username called Moscow Mule that's you can find on GitHub is implementing it that way. Add momentum, run shampoo and set you can set better two to zero there. I mean you have to write a little bit of code. So it's like that. But it was kind of cool to use it. And why did they use it? Because it was kind of it can It was like Newton Schulz. So it's just a mate bunch of matrix multiply. And then then they started doing like I mean it's sort of like stuff that was being done in the Shampoo Quad base all over. Like you can ha take care of attention heads this way, you can distribute it this way. Just a bunch of regular systems problem. But you can see, right, right, like up until 2024, no one was using it. Now all the friend friendier open models are using it. And clearly, like why are they using it? Can because it can be used very quickly, and it took Kimmy like a couple of tries to. Get to the second version of it. And obviously, like this literature is vast, and there's so much more things you can do to get better results than that. I think some of it is out in the open. And in general, like this fundamental problem do exist, right? Like, training is using a very little amount of compute comparative inference. Could you could I use a lot more parallel compute? The answer is really no today because you can't scale batch size beyond certain points, and usually, like where optimization matters, and this is my like Tomer Correns influence is sort of like you think in terms of like the noise-dominated regime and the curvature-dominated regime. In the noise-dominated regime, you can keep doubling the batch size, noise reduces, you can get half the steps every time you double the batch size to the point you fit the curvature-dominated regime and then you're cooked because you need better optimization to handle curvature. And that's where I think like the better optimization plays a role. And do like the general community who trains on Few GPUs need it, probably not. But do frontiers need it? Yes, if you want to scale up to like, I mean, why do we limit ourselves to like like this question of how many parallel GPUs do you use for a single training run? I would like to use a few million GPUs at the same time. I don't think anyone does it. I just wanna and then the bottleneck for that is first optimization, second is money and power, but like the first thing is not resolved, right? Like, so like well, like in Make sense to work on the frontier problem. And it's a very interesting problem because you start thinking about generalization as well. As you know, like you know, larger batch sizes, curvature, sharpness, many other things become more important. So there's more interesting questions there. And then that's where my online distillation paper was also. It was about like you hit the limits of batch size. How do you go beyond? You train two models to use two X the parallelism and teach each other, and then you get 2x the reduction and almost 2x the reduction in steps. So you almost you utilize more hardware to train fast. faster. So it's like you could do a lot more of these type of things and then it's always exciting to me to do more of this because there's a real problem to be had. Like I don't know if you if you could train like I don't know Mythos or Gemini Ultra or OpenAI Soul in one day, it'd be much more fun. Like I'm like I just train. I just like let's go. I get the model out. Ravid Shwartz-Ziv: Yeah, I agree. okay Ruhan, do you have anything else that you want to add? Are you hiring, do you or like advertising or whatever? aro: no, this is great. Thank you for hiring. yeah, so you're definitely we're definitely hiring. We're a small place. you guys should ping me if you are interested in any of the problems you can Ravid Shwartz-Ziv: Where? In San Francisco? aro: Yeah, yeah, we are in San Francisco. We are largely in like a single office. Do check out Mark's RFM's set of talks he shared about our core party or the party where we had James Demel and a bunch of others talk about the work to see if you like the vibe. see you can watch a lot I mean, I don't do podcasts. This is probably the first one to one. I did like one second or my third one while my kid was born, like with Boris Dama, but this is like the only two I've done, so you probably don't l like know what I I'm usually doing. other than my shit post on Twitter. But like you can probably find my research and other people's research off the team and see if you like it, see if this problem is something you want to solve. And sort of like we're kind of like a team of builders, dreamers, kind of like vibe of like where we're going. We're much more risk taking in what we want to get to and that's what we are trying to do. And it's a pr very fun place to work. So yeah. So just find talks by Jeremy and others as well. See if you like us. Ravid Shwartz-Ziv: Great. Wuhan, thank you so much for joining us. It was a real pleasure. Allen Roush: It was a pleasure. aro: Yeah, likewise, thank you for having me. Hopefully this went well. Ravid Shwartz-Ziv: Thank you.