Latest / The Information Bottleneck / Yuandong Tian on Recursive Self-Improvement

Transcript

Yuandong Tian on Recursive Self-Improvement · The Information Bottleneck

• Ravid Shwartz-Ziv: Hi everyone and welcome back to the information button like podcast and today we have a special guest Yongkyun Tian. Hey Yongkyun, thank you for joining us. Yuandong Tian: Hey Ravid. Yeah, hey Alan. thanks for inviting me. Ravid Shwartz-Ziv: So, Yon Dintso, you spend a lot of time at Meta, at FAIR, and recently, you are the co-founder of Recursive Superintelligence, and it focused on self-recursive and automated research. So, we will get there, but let's start from the beginning. Let's start like how you actually started... in deep learning and machine learning. How does it happen? Yuandong Tian: Okay, yeah, thanks for this question. so actually like I came to United States in two thousand eight, right? So and as a PhD student in in robotics institute, right? So in two thousand eight. and that time I was like really into machine learning. so at that time I think people actually don't think machine learning is kind of a valued past. People really push back on this concept, saying that yeah, machine doesn't work that well because it's overfitting, does not understand what's going on. But I managed to do some of the optimization and machine learning work back in the PhD, right? So and basically on some like a complicated nonlinear non-convex optimization problems in the computer vision domains and managed to prove some of the interesting theorems that shows even if the problem is non-convex, you can still prove its optimality in some extent, but adding more data. Right. So it's an interesting Combination of data-driven approach plus optimization. So I still really proud of this kind of work. and when I about to graduate in 2013, twenty twelve, I mean I see this big news, right? So you know AlexNet, coming at his data back at 2012. And I actually sent an email to Alex asking for his code base that bank that time and play with that a little bit. I mean, I f of course I have no idea what is called programming at that time. Right. So and then decided, yeah, that's something very nice. And something that is related to the hierarchical kind of processing of data into neural network so that it actually gives you some interesting intermediate representation and give you some nice results. Right. So so at that time I was thinking, yeah, maybe that one is something that I'm I'm chasing after in a different form because when I was in PhD and one of my thesis the thesis I'm working on is kind of hierarchical. way of optimizing some non-convex optimization problems in the computer vision. Right. So basically I try to restore the distortion of the of the image caused by the water, right? So this kind of high directions. Right. So I'm trying to use hierarchical approach to do that and actually prove this is something that we can do better theoretically. empirically it also works very well. so this makes this connection then we I that's where I'm having this kind of ini initial belief of deep learnings. But at that time I would say there's a lot of interesting things happen because people don't believe in that. People will say, this is just overfitting. Right. So everyone just come to the office and we actually had lots of a heated debate in the Smithsall in CMU and talking about whether this Alex Next results is bullshit or a real breakthrough. Right. So at that time it just happens. And most of people just say it was just a bullshit because it it just doesn't make sense. Right. This is but I think at that time actually I was thinking about maybe I should switch to deep learning at that time. so I interviewed a few places and including some places that actually give me offer like an EC labs, they want to do deep learning. But end up with Google self-driving car team. Right. So because first of all it's a very prestigious place, and second I as we have I have group friends there and who refer me and I don't think I should break that tie to some extent. so I go there and also I want to do deep learning that time. But unfortunately, in this one year, three months period, I didn't get a chance to do that. because you know it's it's more like a kind of startup and you really need to make the separate car work. Right. So it's better to use some like a very mature approach. And the kind of scientific exploration probably is not the the the the right thing to do when I joined there. So I come to decide to like jump ship, right? So in move to fare. I that was actually one of my best choice at that time, I would say. Yeah, so I won't move to fare and then I started to do the research, which is super interesting and nice. and I think I started with this deep learning for for for Computer Go project. Right. So that was actually pretty pretty successful to some extent because we actually got some very nice results before the AFA go. In 2000 200 like 15 we had this first bot called Dark Forest that can beat top amateur players on the internet with the commercial neural network. so that's actually interesting. But of course, I mean Apago is coming it came at the spring of twenty sixteen, right? So and that was beating the professional players. Of course that's very, very good. And at that time I think my team only have two people, right? myself plus an in intern. so that was Ravid Shwartz-Ziv: But did you think, like when you saw like AlphaGo and like did you think about like I don't know like how I didn't think about this or like they're just too good for me Yuandong Tian: I think first of all, I mean this b I mean AlphaGo definitely have many innovations, right? So one of the things that they have but we don't have is the value network. Right. So we at that time we only have a policy network. so they have a network, so this actually makes the the play much stronger. that's one of the advantages. And second difference is that I was basically not having any background on on like reinforcement learning. I basically like do while I'm learning. it's a very interesting experience. Ravid Shwartz-Ziv: like reinforcement learning yourself. Yuandong Tian: Yeah, yeah, yeah, yeah. So it's it's a very interesting experience. So that's why I'm now I mean after that I basically switched to more like reinforcement side. doing many reinforcement work after that. Right. So Ravid Shwartz-Ziv: But at that time, like, well, so I want to stay one more minute on it. Like, so like at that time, like, like, was it surprising or like, because like you're in for a time or something like, yeah, it's kind of like, it's really cool idea, right? But like, it's not really work and like we can solve some toish problems, but like, so was it surprising for you? Yuandong Tian: I would say the reason we choose this topic is because I sense that the commercial network neural network can actually capture many patterns in the in the board and this pattern is something that humans leverage to play the board, to play the game. Right. So to some extent it's not that surprising. but what makes things very surprising is it can actually beat professional players. That is the surprising component. Right. So if you actually check the method, methodology. it's actually not that crazy. Yeah. So it's kind of a natural consequence of the successful success of the commercial network. Allen Roush: I'm curious, were there any methods from from this era that you think are being ignored today? Yuandong Tian: Yeah, there's a many approaches, right? So as at that time I think people developed many things about about like reinforcement learning, right? So many of these approaches probably not yet being leveraged in today's larger model eras. so for example, people are thinking about like using more dense reward from the future, right? So this is called distributed IR. Right. So instead of having just single number, you want to have a distribution of all the rewards, and then use that to basically tune your models. Right. So that's something that I think the current IR they maybe use it, but if it's a different form, right? So for example, they use the languages rather than scalar rewards, propagated back. Right. So there's also many different ideas, for uncle and also people are using Q learnings, right? They are using different formulations. I so but right now most of the time we are mostly like using policy gradient. Right, so it's it's very different errors. So yeah, so back to that two sixteen and then in two thousand eighteen we actually reproduced alpha zero. We probably know alpha zero, which means that there's no human human place. You basically like do this self-play from the beginning, right? So and we actually reproduce that and make it a little bit more efficient to some extent to create this bot called open goal and that bot actually beat professional players easily. And we had a official match with the Korean goal association. we beat four of their players I choose with five to zero, which means that they don't even win a single game, even if we only use a single GPUs, which is V one hundred, to play this game. that's I think that's probably the first time that we use people use single GPUs to beat professional players. Yeah. So that's not a Ravid Shwartz-Ziv: But was it surprising for you? Did you think that like, I don't know, when you go to this competition, did you think that like, yeah, we will beat everyone, like we're just so much better or like you had some doubts? Yuandong Tian: Yeah, before we actually start this official match, we actually asked some of our friends who are also professional players, right? So or semi professional players to evaluate how strong it is. and they it's the game is actually very interesting. At the beginning things are fine, and after that no one knows going on. They will say, yeah, this looks very strong. Very strong. Okay, but I don't know whether it's actually strong or not because I cannot see the game very well. so At that time I would say, Okay, let's take a bet, let's actually play with the real professionals from the Korango Association because we have some connections there. And then it turned out that the bot is really, really good. Ravid Shwartz-Ziv: you Allen Roush: So do you I'm I'm curious, what's your opinion on gr gradient free based optimization methods? You know, I I tend to think that that's a a set of techniques that have been kind of ignored a little bit as well, historically and and or even up to today especially. I mean Yuandong Tian: Yeah, I think this actually connects with one of my research directions back in Fair, right? So after the reinforcement on games kind of directions and we are trying to think about how to make reinforcement more general and more practical problems. so this actually involves that we might also a little ex explore a little bit about gradient free approach. Right. So because reinforcement you can think of this is one way to do gradient based approach, but of course the signal is very sparse. And it's very hard to get the signal in a very efficient way. Right. So we are thinking about maybe this limitation of it, maybe we can you should go beyond that. Right. So for gradient free approach, I would say it's definitely offered like more flexibility. and with the current larger model net models, it might have more potentials because large models now No better about high level understandings about the systems, about the the the the object function to be optimized. Right. So this high level understanding is actually much more informative than the local grid information. So it's Ravid Shwartz-Ziv: So what you're... No, like what actually happened in like gradient free approach? Yuandong Tian: Yeah, I think for it depends on like a what gradient fine if gradient free approach you are talking about, right? So okay, if you just talk about zero zero Ravid Shwartz-Ziv: You tell us. Yuandong Tian: order, for example, zero, right? So evolution algorithms, zero solders, then if without any kind of hello information, then what happened really is that you do exploration where the gradient is not super informative, right? So then the function is very, very, very chunky, right? So okay. So you may have like a interesting like local optimum appear of nowhere. It's no gradient going that way. so in that case, the gradient information may be may very misleading, right? So we'll give you the wrong place to go and it will stack you into local minima. But usually the gradient free approach will come up with the explor exploration techniques, right? So this is something that we are we we actually talk we actually talk as like for example the OB NS search. Right. So we have exploration and we hop also have exploitation. So by exploitation, we are actually following some gridding information. By exploration, we are saying that yeah, then maybe the local search is really not not that great. So let me let's like try something different. Right. So this jump Allen Roush: I Yuandong Tian: is something that gridding approach is never ever used. That's something that's a few advantage. Allen Roush: Yeah, yeah. And and so I'm a lot of my early forays into learning how neural nets worked was trying to code them up from scratch and then realizing that I had no hope of doing auto differentiation or doing the backward pass in the general case from scratch. And so I was like, Well well let's look at forward pass only based methods and then I started discovering all these things you talked about where like if you don't assume, you know, the the shape of the the you know values are are the the rewards, then you get great exploration. I actually care a lot about like creativity. And so I guess what I would ask is do you think that these gradient free based methods are why we get s or or you know lead to more diverse outcomes and furthermore, do you believe I'm I'm aware that diffusion models do have, you know, gradients in them, but I claim that when you look at how they're optimized, for example, denoisers for example, my understanding is that there are plenty of gradient free approaches for those. And and and so I claim that you know diffusion models and diffusion language models, I've heard many others argue that they're more diverse. And so do you think that there's a connection here between that, if that's true, and the gradient-free method stuff, being better at diversity? Yuandong Tian: I see. I see. Yeah, I think for diffusion-based approach, first of all, I'm not an expert of that, right? So second, I think my understanding right now is that diffusion-based approach is good because it actually used many different masks to train models, right? So you can think the auto-aggressive approach is kind of special case. Right. So basically you only use the the the attention mass in a specific way that so that it's become causal then you learn some patterns. Right. So but if you use diffusion based approval usually like the mass is kind of random generated. So you generate mass can then try to fill in the gap. Right. So of course it actually gives you a kind of advantage in in the case that you are forcing yourself to predict something in the middle, even if that's information is already being provided. in the actual aggressive counterpart. So I think because of this kind of shuffling, then you might be able to learn more correlations that is hidden in the data. So this might give you an advantage. but of course the diffusion model itself is still trained by green descent. So it's different. Ravid Shwartz-Ziv: Okay, so you spend time on understanding and developing better Go players. And right, we were in the, it was 2018, something like that, right? So Yuandong Tian: Yeah, Tony. Ravid Shwartz-Ziv: in that time, language models were kind of useless, right? Yuandong Tian: Yeah. Ravid Shwartz-Ziv: In some sense, right? Yeah, I know. But like, think the common understanding was that deep learning is super strong with vision, for example, and we are still waiting for it to come to text and language, right? Yuandong Tian: Mm-hmm. Yeah. Ravid Shwartz-Ziv: Okay, so then next, like, okay, so like now you said, okay, like, yeah, Go is actually like an interesting game and like we developed like some RL methods to try to beat. And then what did you try like to go to other domains, like to generalize it to to our different RLs. Yuandong Tian: Yeah, so I yeah, I think there's definitely like two directions after that, right? So once it's like okay, yeah, reinforcement on games to some extent is being solved. because you can always like set a reward, let it run, and then it will give you something, which is nice, right? So that's something that people have consensus. I think back in eighteen, nineteen ish. Right. So I mean of course like there's two paths. One path is that okay, how about we do search optimization, reinforcement is part of it, right? So two real hard kind of like a real world problems. Right. So and real problems may be tricky to do. Right. So we actually spend a lot of time on that. so because then we find that for real world problems, what happens is that if you apply reinforcement to these problems, what really matters is not the algorithm. Interesting. What matters is the design of the action space. And also state space. That's something that we have been studying a little bit, right? So that's why if you check my resumes, check my Google Scholar, we see a bunch of new architecture search papers, right? So you still want to search this big space, but this search may not follow reinforcement. It might follow other kinds of domains, like Widden Free Approach, as you just mentioned, right? So but the the quad the real question is like how can we automatically learn the the way we design this place and action space so that the search becomes much more efficient. So for that direction we actually have a bunch of work on that. Right. So but the idea here is that can we automatically find the right action space to take in order to search the search the entire pass, entire object function much more effectively. Right. I will give you one example, right? So for example, if you think about architecture search I mean human will say, okay, we all know that the the depth really matters, right? So the shadow network is really like not doing a good good good job compared to the to the the deep neural network. But we know this is kind of like a heuristic that we want to impose in the in the search, right? but on the other hand, the the AI probably don't know. If AI start with a search about these small details, then it will never give you something interesting. Right? So suppose like AI start with a search about the kernel size of the first layer. Of the neural network in the conversional settings, then it doesn't matter that much because whenever you do the search, I mean, unless you hit some like extreme numbers like one, you may give you it may give you like a very comparable performance. So in that case, all your search branches are comparable. And if you compute a log value function on top of that, they're all telling you the same story. Then you really need to get deep into each of the branches. This will release an exponential explosive search space. Right. So in that case, it doesn't matter what kind of algorithm you use, right? You can use reinforcement, you can use any kind of gritty-free approach, it doesn't give you anything that useful because you're facing an exponential search space. But if you have an idea to basically shuffle your action space and you somehow magically find the depth is important, then you all of a sudden you actually see a very sharp contrast between a deep network and a share network. Right. So you don't need to traverse all the paths, you just have a few drawouts. Then you clear see there's a big difference. Then you will go through the deep path and then you have making pro can make progress on the search. So this idea can be implemented and can be actually formulated by a mathematical formulation. that's something that we developed a called the latent space many card research. So for that idea what happens that you want to find what is the action space that will lead you to the most discriminative search subs spaces. And you can do this intuitively. And then once you have that and all of a sudden you have you find some nice action space that you will guarantee to have good progress on that. Ravid Shwartz-Ziv: Do you think like around this time, there was, we talked like a weeks ago with the Frank Crutcher, that like... was really into AutoML, right? Like the idea that you sent, like you find the hyperparameters, right? And like automatically. So what is your take? Like you think like what are the, like at least in that time, like do you think like the, what are the important components actually that like you actually need to search? Like what are the components that are, like they had like the most and influence them on your results, on your performance. Yuandong Tian: I think at that time the most important component is still a manual design the action space. Right. So if you check for example auto-mair work from Google, right? So usually what they do is they they say, I want to design this very nice sales, and the cell has six or seven different sequence of actions. And then you replicate it with basically like you stack all this sales together and you get a new network. Right. So stacking is kind of fixed. And then the key is to design this kind of a sales that can consider that has a component to trend. Right. So you you might say, why not eight or nine? The answer of course is I guess this is too exponentially expensive, then you cannot do that. Right. So then all these small actions are allowed to that is allowed to to take it it basically deserves a lot of human design before you can actually make this work. Right. So usually that's that's what's h Allen Roush: So so related to action spaces, what do you think is the path to getting good like robotics and physical AI systems? And I wanna particularly highlight that you actually have a paper talking about reasoning in continuous spaces. I'm I'm trying to get the exact name of it. It's one your yeah. Yuandong Tian: Yeah, yeah. It's a coconut paper. Ravid Shwartz-Ziv: The Coconut Paper. Allen Roush: Yeah, coconut, exactly. And I wanna point out, you know, Yan Lacoon and all the others talking about, you know, Jeppa and reasoning in l in latent space, not token space and all these. And so I'm very curious, your your thoughts on all of this. Yuandong Tian: Yeah, I think this paper is the CONA paper is actually the paper from my intern, right? So he has been like coming to meta and then we've been brainstorming some like random ideas and thoughts. And then I think one of the arguments that I have is that when I'm doing the thinking, I never use a language to do thinking. I can clear feel that when I'm thinking, I'm using a different part of my brain. And then I have to translate my thoughts into language so that I can communicate with others very freely. Right. So this is the how I am doing that. so I I feel this is basically based my my on like my this intuitions. Right. So and then this comes with this idea that yeah, how about we get rid of the token at all, right? So we just use like a latent space to do the thinking. Because obviously latent space seems to be an interesting representation that we should take. And then after this kind of spark, then I actually start with some of the theoretical derivations. And that the version actually give you some good intuitions. maybe the latent space representation is kind of interposition, it's kind of superposition of multiple ideas together, and then use this superposition to do thinking, you might be able to have exponential advantage, right? So if you traverse some graph. Right. So that's some idea like that. so of course it's may not be exponential, but it's at least like give you some of the ad advantage of that. So then the the reason then we have I mean our intern yeah is coming and say like Shubo, he is being try trying to execute that and we find some initial signs of life doing that. Allen Roush: So so so so when you mentioned that your your intuition about how, you know, you don't verbalize most of your thought, I'll point out that I also do not verbalize everything before I talk. Maybe that's a source of why I talk too much and don't think enough before and etcetera. But but I claim that that humans are have huge diversity, right? Like I know a lot of people who absolutely do verbalize most, if not everything, they call it internal monologue. Yuandong Tian: Yeah, yeah, yeah. Allen Roush: people who do it sometimes and and similar with with images right some people visualize images really well and can rotate shapes others can't some of those who can't are great artists right and so I'm curious about this like ontological diversity and and how you think this relates does it just mean I guess that for if if humans are so plural we should assume that AI would also have many different paths to the same kind of outcome Yuandong Tian: Yeah, I think totally is that's totally true. I so it's possible that there will be some underlying mechanisms for learning, but different people may have different ways to represent their thoughts. Right. So and they might be presented internally as different kind of modalities. Right. So for me I feel still like sometimes I when I'm thinking I'm still I'm thinking maybe it's actually picture wise. or like visualize sometimes it's very it's kind of weird feelings, right? So then you suddenly can jump in and you have an idea, yeah have an idea. And and the idea will be basically externalized as a verbal language. Right. So but if this is the case then I would say the best bet is still the internal hidden representation. Because that's something that is probably like encapsulate all the possible modalities and all the possible possibilities. Right. So that's something that I would I would bet. Ravid Shwartz-Ziv: So, but like, so, but at the end it didn't catch, right? Like today, models, like frontier models are not training with continuous chain of thought. Why I think it's the case. Yuandong Tian: First of all, is the there's a lack of data, right? So I mean you have lots of languages, lots of thinking thoughts, thinking patterns that is represented by languages, right? So and this is the way you can communicate and this is the way that you can store people's thoughts in written form. Because of that, you have way, way, way much data compared to the latent space. so then obviously you have to use that amount of data so in some way, right? So that's that's one way of doing that. And second is that the latent representation probably is personalized. I mean everyone is using their own latent representations to some extent. So as Alan said, I mean people will have a lot of diversities there. So then in that case, it's going to be very tricky to learn because humans learning efficiency is much, much higher than computer and then our large models, I have to say. Right. So if our large models not really have this high efficiency in learning new concept. then we have to use languages as a surrogate to let module. Ravid Shwartz-Ziv: But do you think it's related to the problem with JEPA, for example, right? At the end, continuous space is like, it's really hard to regularize it, right? Like the main problem with all the SSL methods is that you need a good regularizer and there are different ways to do it, but it's not really like what is the right approach. it's kind of like very like... non-defined problems that you basically can't, you can't like high climbing on some very, like, specific, well-defined metrics. Do think this is the same problem here? Yuandong Tian: Yeah, to some extent, yes. I would say there's a two effects, right? First of all, I think the gradient learning probably is not that efficient, so they need a lot of data. If you don't have lots of data then it is under regulated regulated, right? So you have to put regulars. But the question really is that what are the regular you have to use? There's so many regulars. Each of them will give you some different representations. So maybe none of them actually the the right thing to do. So that's actually causing problems for for that, right? And the second one is really that we don't understand what is the representation, right? So and how the representation forms over time. That's actually the problem, right? So and this this actually is probably the reason why the gradient-free approach may be very useful. because we are stuck with the gradient descent solutions as the local optimism. in in this kind of meta level, right? So there may be some different ways we should learn the representation. Ravid Shwartz-Ziv: Do you think it is an optimal representation? Do you think that kind of like, Blantonic representation paradigm? Yuandong Tian: Yeah, so I would say it's still it's an unsolved problem right now, right? So there's so many different ways you can represent the world. I think there's there probably some way that is better than the others, right? So there's some way that in occupy like a less bits, right? So for compressions, right? So there's some way that may generalize things much better. so and then the real question is like how the goodness end, how the current like neural network learning can learn that. And what is the limitation? for these learning paradigms. There could be some representation that is very good, but the current agreement standard proof cannot learn that. We don't know the boundary right now. Right now we just say, okay, yeah, it looks like it works. And then we just use it as our systems. Right? So but we scientifically we have no idea how what is the boundaries and where how can we reach that boundary. Yeah. That's really the the the motivation for me to basically work on the integrity, work on understanding your network. Ravid Shwartz-Ziv: And do think it will work? Like, do you think that we will have at some point this understanding? Yuandong Tian: Yeah, I think at some point we should have the confidence that there will be understandings, right? So this is like alchemy versus chemistry, right? So that's kind of the stories, right? So this story is still there. but of course like because the current model is too s very, very strong, then people will chase after the the the the the the results of the models and we might not have enough time, enough people's mind on that direction. Yeah. But I would say on the long run this is going to be a very hard problem and very important problem for human kind. Or maybe for AIs as well. Ravid Shwartz-Ziv: So I want to ask you like more like personal question because like you you you you worked on deep learning and vision and RL like for a long time and then suddenly right like in I don't know 2022 2023 like around this time like suddenly like all of us understood it like yeah LendGovitch and text models are like here for stay and like are the next big thing. How you manage this transition and like how you adapt your, I don't know, works and like the thing that you actually care about them in your work. Yuandong Tian: Yeah, I think first of all, I've been like working on like interoperability sa starting maybe like twenty eighteen, twenty nineteen ish, when when I'm trying to do some grading dynamics analysis on the deep neural networks. Right. So this is in some sense I kind of like the the threads I have been pushing very consistently. And that threads is in some sense quite consistent, right? So you can still do the same thing for the large models as well. Right. So you are changing your models, but the way you do analysis is similar. Right. So in that case it's not disruptive. It's mostly like you're still working on something that you really want to work on. Right. but of course on the other hand, on the empirical side, it's going to be tricky because now we have to completely change the way we think as a researcher. Right. So previously, like usually for researchers, we come with our own ideas. And then we test the ideas with the experiments and we get some conclusions. So this kind of process is similar to maybe a thousand a hundred years ago when people do the research in a paper and do experiments by their own. Right. So but given the large models, I think usually like because it actually can give you some of the thoughts by itself when the model becomes stronger and stronger. So then the researchers are probably forced to think about something in the meta-level. rather than just spending their time on coming well up with individual hypothesis. So this is the first challenge that people need to face for that. Right. yeah and the second challenge I I think at that time is that first of all we need to emphasize that the engineering excellence is also very important in parallel to the research sense. Right. So because many of the good work from the large model eras, they are kind of like half research, half engineer. So because you cannot if you cannot like engineer things very well, then your results probably is not great. Then people probably want to take that as a serious output. Right. So that's actually another challenge that the current research are facing. Right. So I think from two thousand twenty two to two thousand to maybe a year ago, it's like it's like that. But right now, because we are all using AI coding agent, so the thing might become a little bit different. but because with the help of AI coding agent, what happened is that researchers with good hands-on experience and knowledge, it they might be able to come with nice ideas, getting implemented by agent very quickly, and then find the key problems and solve it. so then all of a sudden these researchers actually have the superpower to explore something that is really deep. Yeah. So there's some back and forth. Yeah. Ravid Shwartz-Ziv: Okay, so I think that's a great time to go to talk about recursive self-improvement. Like we see super power users or super power engineers. We will have super power agents that improve themselves. Yuandong Tian: Mm-hmm. Yeah, I think that's ideally to go, right? Yeah. Ravid Shwartz-Ziv: So tell me, will it happen? Yuandong Tian: Yeah, I think so. I think first of all I wanna tell you like why I think it's time to open a company, right? So on that. the last paper I had in Meta is a paper that collaborate with GPT five. Right. So this paper is about study grocking behaviors in the neural networks, right? So the basically the network will basically start from memorization and train, train, train, train, train all of a sudden it becomes generalization. Right. So this kind of groking behavior is very, very fascinating and interesting. And for that paper I collaborate with GBT five and I ask lots of questions and assign them homeworks, right? So yeah, please prove this theorem, right? Proof please make sure that the theorem is correct. Please help me brainstorm on some ideas and thoughts. I find that it's actually much more effective than what I used to work on similar problems. Right. So and I would say it actually gave me maybe six X to ten X boost in terms of the efficiency. Right. So and this is actually conservative because at that time I still rely on myself around writing code. But now I think code writing and implementation of course can be delegated by strong agent as well. Right. So this I will make things much, much faster. Yeah. So after I finish this project, I got some nice w interesting results. I suddenly realize that wow, okay, maybe my job will be replaced in five years. Right. So Very clear that I'm going to lose my job very soon. so instead of losing my job involuntarily, I think it's it's good to to basically like start a company that for for that directions. Right. So that's why I you see that happens. Right. So usually like a researcher probably don't want to go to the small companies as a co-founder because there are lots of things, lots of engineering, I need to worry about many things beyond the research. but I would say right now it's a phase shift. and if I don't do any actions then I might be irrelevant later. Yeah. That's Allen Roush: Y you know, you know, w I I I get into bitter debates sometimes with revived off camera and and also, you know, certainly when I'm not feeling as good where I say similar things, mostly marveling at how good coding agents have gotten and also being very bearish on my s you know, circa Ravid Shwartz-Ziv: You Allen Roush: the twenty forties, you know, fe feeling like eventually the AI has to give me the talk and is like, you know, it'd be a little better if I was just doing all the prompting and I'm very concerned about this and and the worst part is I think your your you know response to it to go form your own company is a right response. My response has been to go try to learn how to work with my hands and I am not good at working with my hands. Dad. So I'm screwed when the singularity happens. Ravid Shwartz-Ziv: I will just say that like, and we can see it like everywhere, right? Like software engineering, like the demand for software engineers are just going up, right? And we can see it like everywhere, like there are no more like, know, like better products, right? Or like better like applications or something like that. Like it may be that like the models will be better at some point, like we... they don't need us, right? Yuandong Tian: Well. Ravid Shwartz-Ziv: But at least for now, we don't see it. We don't see it. And like, it's not possible to come up like with good products end to end, right? So do you think first, like, do you agree with that? And do you think like, and if so, do you think it would change? Yuandong Tian: Yeah, first of all I would say it's right now it's still at a stage that it's full animatic yet, right? So I don't think the current model can give you very nice results without any human interventions. I think it's still not at a stage. But even for the paper I had back in meta, I still give high level thoughts, high level ideas so that the model can follow, rather than ask the model to come with their own ideas. because that part is Very hard because you don't have an idea, then it will give you very medical and medical ideas. Everyone knows, and it's not an interesting direction to go. Right. So I think as I would say till now the model are still doing that. there's no we still need humans to to initiate, still need humans to think about what are the the important things to work on. Right. So that's something that we need to do. And also we need humans to validate, right? So what are the problems? that the the the the solutions come up with the AI look like. So these are actually the the problems that we are facing. So and hopefully in the future and if there's more automation from the agent, from models, humans may spend less and less time on repetitive work and spending more time on the more creative directions. Right. So and humans can also have like a more ambitious goals. Right. So in the future, right? So female will say, maybe I should go to Saturn at some point and then I will delegate lots of my execution engineering work to my engineers, my agent. Right. So they will come with a ways how to build a rocket. Ravid Shwartz-Ziv: But what do you think about all... So there are a lot of papers and works right now that are trying to look on self-improvement, recursive self-improvement. Now you give an agent two hours to GPUs and let him do post-training or just try to improve some... don't know, Neumotron or whatever. Something like that on some metrics. And at least what I am aware that like this like works mostly like mostly showed it like, of course agents can do everything, right? They can do anything that you want in terms of like optimizations and things like that. But they are coming up with very narrow space of ideas, right? Like at the end, they are very inefficient in terms of like ideas and they don't really like know how to generalize a problem and they are overfitting to make quite a lot of Kaggle competitions these days that are trying to use agents and they kind of cloud code for improvement and included are kind of like overfitting the training of validation metrics. What is your take? Do you think this is something that like we don't... First again, do agree? Do you think this is something that we don't know how to measure? Or maybe it's a kind of like temporal problem that are going to be solved soon? Yuandong Tian: Yeah, I think to some extent they are temporal because there's always a way you can ask the agent to focus on or ask the model to focus on more high-level problems. Right. So you can do that. I think the reason why the current model cannot do that very well is because the model is asked to kind of detail stuff. There's no context. If the model has more context, it might come with some ideas that are more interesting. Then just fine-tuning parameters, like tweaking all the hyper things. That the functor configurations, et cetera. But I would say adding like more context for models is probably like a temporal fix. It might give you some boost, but in the future we probably need something substantially better than the current fund for transform architectures. Right. So I think the key difference right now is that the kernel model are trained on a lot of data, and these data are all coming from the internet. Right, so and they are consists of a lot of repetitive work. So for repetitive work, I think the the model is doing very well. So it actually had enough data to know what's going on and then enough data to come with the new ideas. But going forward, if in the future you want the model to do very creative things, something that is something out of the box, then the model probably don't have enough data to to do that. then you probably need to think about what are the new architectures that leads to much more efficient learning and can pick up like a new signals super quickly with very few amount of data. so that's basically two stages. This from from what I just said as a kind of a routine like a regular repetitive work to the next level, I think we need some big jump there. But the the current paradigm still can exist a long time because there's always lots of patches you can you can use to make the model stronger. Right. So that's my my feeling of that. Allen Roush: What's what's the most important for making models better out of like pre training, post training, you know, even inference time optimizations, all all these kind of possible ways to improve them going forward? Ravid Shwartz-Ziv: Or maybe different architecture, Like totally Yuandong Tian: Yeah, yeah. Ravid Shwartz-Ziv: different architecture, optimizers, like... Yuandong Tian: Yeah, exactly. Yeah, so totally different architectures, optimizers, right, so data mix, etcetera. So this can be very interesting and can be the subject of self improvements, right? So this is actually what we are doing. Ravid Shwartz-Ziv: And do you think like, do you see it? Because like, right, like till now, like at the end, like there are a lot of companies and people and startups, they try to suggest like different architecture, right? Besides transformers. And some of them actually like show like kind of like good results in small scaling, on small scale problems. But it was never actually like a scale up to. real like bigger models. Why you think is that and do you think it's something that you can solve? Yuandong Tian: Yeah, I think that's a good question. First of all, the reason why you see a lot of good improvements on a small scale but not large scale is because usually for small scale you are lacking of data, so you have to basically additional add some inductive bias in the architectures. And that bias works quite well for small scale because you don't have enough data. But for large scale, because all there's a lot of data, and the best thing to do is to basically leave model to figure out to to figure out what's going on. Right. So I think one example one very c interesting example is this visual transformers. Right. So when it happens, usually what w you can if you read the paper, what happens is that if you use like some internet level of data to train models, the vision transformer probably is not as strong as the convolutional network. But when you have a lot more data, like two hundred million or five hundred million, then eventually you see transformer become much better compared to the convolutional network. Community network has this inductive bias, which is basically saying that Ravid Shwartz-Ziv: Are you sure that Jan will agree with you about that? Yuandong Tian: I'm not sure. Yeah, I I I I Yeah, Ravid Shwartz-Ziv: You don't have to answer that's fine Yuandong Tian: I think the the probably the the the intuition right now is like your transformer there's basically there's you can have transformer attend to the locality or your transformer can attend to the non locality data. as well, right? But for the community network you have to focus on the local things first and then gradually build. So these two are very different. Ravid Shwartz-Ziv: But my question is, how you want, what is your strategy? Do you think that we actually should start with, now, right, you said it, yeah, you have new idea about different paradigms. So now I will start in small scale experiments and try to scale it from there, or you think it's worth to start? No, I don't know. a one trillion model from the beginning and to try to iterate on this model. Yuandong Tian: I think of course you need to try from small scale for first and then gradually go to the larger scale. But I think what really matters is that this is my personal take, is that I we really need to to some extent know what's going on inside the neural network, right? So there's some kind of interoperability component in it that we know it can generalize well in larger scale. That's something that could be one of the additional components and we need to think about. Yeah. And we know that there's some kind of interesting insights that will scale up to large scale, right? So then we are confident that this can work well. Right. Otherwise, if we blindly follow screen loss, then what happens is that you can never prove that anything in the working on a small scale can generate to large scale. If you don't do that, if you don't have this kind of confidence, then of course only the company with the biggest amount of compute will win. That's the only solution. But you actually see something deep in your in your small scale experiments, then it's possible that you can get some I think work on the last scale. Ravid Shwartz-Ziv: So, if you look on this loop of idea, now code, run and analyze, what part is the hardest to automate? Yuandong Tian: I would say I mean ideal idea is probably very very important and very hard. Right. So if you ask a model about ideas, I mean it will give you some like a kind of like medium media core ideas, right? So I mean combine all these two things together, check what works or not, combine things together, do ablations, right? So these are actually ideas that are not super interesting. And I will imagine that everyone else are going to think about the same thing. Right. So then the question really what is the right thing to come with the right idea? And so that idea actually point to the right insights. Right. That's very important. Right. So of course you can also do a transfer, right? So if you sell some nice idea coming f from different domains, maybe you can transfer to the domain that you have been working on and in that case things might be very effective. I think the recent breakthrough from for mathematics, right? So AI for maths, many of these ideas are kind of the transferring. I from one domain to another domain and find that that idea should work. Ravid Shwartz-Ziv: So for example, you had this and you tried, right? Like the Nano GPT speedrun, right? Yuandong Tian: Yeah, yeah. Ravid Shwartz-Ziv: And so do you think like this type of competitions and like even now like, yeah, you achieved like you shaved that two seconds of the record, right? Do you think it's like a real discovery or just very good search over some like known tricks that actually it was like that it was easier to find? Yuandong Tian: Yeah, I think it's in some sense a combination of interesting discovery with the previous approaches. Right. So for example like the addition of like a biogram and unigram, right? So in the in in the in in in architecture, it's in some sense interesting. And of course some people will say this is coming from maybe Deep Seek, coming from other other and maybe like the Gemma three approach, right? So and actually We all when I was in Mala, we also had a paper called STEM and also following very similar ideas. So it's a kind of a transit to this actual competition and this actually boost the performance. Right. So I would say it's it's a combination of basically like a transferring existing ideas and basically there's some creativity in putting them pretty much together so that they can work well. Right. So example, each different I different layers you might have different embedding tables, right? So and and basically the this this this this combination are different. Allen Roush: So what's what's the future of AI? I mean, we've discussed a few of these possible candidates like world models, and also, you know, we tau touched on recursive self improvement, we touched on physical AI. Are we missing anything for the future? More scaling? Yuandong Tian: I mean I think scaling probably is probably the common sense right now, right? So people say, yeah, you scale, skill, skill. But I think because of common sense, we want to think about something different than the common sense, right? So as a new alternative explorations. Right. So I mean recursive self-improvement is important, because the model now reached that point that this is already un to some extent understand what's going on inside itself. Right. So even if this understanding is shallow, but it's still understanding. We can leverage that to do the self-improvement. So that's the time. And plus that it's also have very practical use cases, as we show in many of the scenarios like kernel optimization. Right. So that's that's one. of course, I always say the future probably is still to find new architectures, find a new way to learn, and find a better way to match humans' data efficiency. So it Ravid Shwartz-Ziv: Mm-hmm. Yuandong Tian: doesn't matter which which modality we are working on, it doesn't matter which problem we are working on. I think the key is still the learning mechanism and learning algorithms in these neural networks. That's probably the the the the key pro the key point. So once we c incorrect that, right? Suppose that we now have a new algorithm that is learning as efficient as humans. Right humans use like a twenty thirty watts of compute, and the human only use like a one thousandth of one of the thousandth of the basic like a yeah. Yeah, so zero point one percent of the of the c of of the of the data, right? So and the human can figure out things in very short amount of time. Right. So if we have that, this kind of learning mechanisms, and all of a sudden all the problems can be addressed very well. So right now I think the problem with robotics is really you say, okay, lots of data, I think Faye Faye actually mentioned we need lots of data on robotics. but this is just for the current paradigm. The current paradigm Because you need lots of data, so you we really need to provide lots of data. But if you think about how humans do that, how all the I mean all the cat stocks, all the other like all all the other creatures they are they are doing the body AIs, they don't need a lot of data, right? So they just have very crappy processors, and at the same time they can survive in this complicated world. There must be a new algorithm there. So I think that's the key. And I would believe that the reinforcement and other the self improvements is probably the the faster way to get there. Ravid Shwartz-Ziv: so like, several, like final question, like, you think, so in, tropic and like other, there is like a lot of, debates about it right now about restricting, restricting the latest models, right. And they called for coordination around the recursive cell improvement. what do you think about it? Yuandong Tian: I just want to be very honest with that because I feel like it's very hard to prevent people from doing that, right? So you know that's the boundary is very blurry, right? So you talk to your coding agents, you come with new ideas and ask them to iterate on new ideas, and you are doing self-improvement to some extent. you talk with other people and they come with ideas and they work together with you on some papers, it's also self-improvement. So it's very hard to have this draw this boundary that saying that, yeah, you cannot do this because it's self-improvement, right? So it's very hard to to do that. I mean it's also very hard to execute, right? So what is the boundary to that? So and we're now we also have all the open source models, right? So and these open source models and they are all very, very good, very strong, right? You can also use that and to to do some research, some studies, right? So an idea called a self-improvement, how you're going to Animus treason, it's going to be very hard to to to to to answer these questions. Right. So it's not a Ravid Shwartz-Ziv: What do you think about open source? Are you going to open source your models and open source in general? Yuandong Tian: mean I think for the outcome of our res our results, I mean we are we we were going to open source, right? For example, our previous blog post, we open source all the kernel, we open source all the and all the all the other like the the recipes for training this nano chat nano experience. Right. So basically the outcome of our systems we're going to open source that. So this this is important. First of all we everyone else can check. And they can build on top of that. That's good. So that's and also I think I would say open source is actually very good for the entire community. Right. So the the worst situation that could happen is that a small elite of people and they keep they have a control of this biggest strongest model ever in the world and say everyone else needs to pay a lot of premiums to use that. that probably not the great things for the entire society, right? So we I think we really want to dem democratize the the power of AI. everyone so that everyone has the opportunity, equal opportunity to to basically like to to to do nice work and to to to strengthen themselves with the model. Right. So that's something Ravid Shwartz-Ziv: Do you think like in the future we will see open source models that are the same level of the closed ones? Yuandong Tian: Yeah, I think Rhino is already pretty pretty good. Right. So for example, Chemication is it's very good. I saw they they can do lots of nice work. because the the thinking tracing is a bit longer, right? So than than the current closed source models, but still it's it's usable and you can leverage that in many different domains. So I think I will investigate that in the future the open source model becomes stronger and stronger. So it's not not a problem at all. Right. And plus that for as a coding agent. you don't really expect AI AI Newton, AI instant come to your place and do your coding for you. So there's actually a boundary. So there's a line that as long as the model passes a line, you it's usable and you you're willing to use it. Right. So I think you know I mean a bunch of frontier labs that already cross the line and now open source models start to cross the line as well. And I think I will say in the future more and more people more and more models will cross the line. and once you sell that then it will be commodized and everyone's going to use their own favorite model and very happy there. so this is eventually this will happen. So and then the real question is like how can we leverage all this model to do something better and bigger? Ravid Shwartz-Ziv: Okay, Yonok, do you have anything else that you want to add? Yuandong Tian: I think that's pretty much it. yeah. Ravid Shwartz-Ziv: Are you hiring? Do you want to advertise your... Yuandong Tian: Yeah, I think I would say like yeah if you are interested in our companies, feel free to send us resumes, right? So our email address is talent at recursive dot com. And everyone who are interested please send us yeah and a resume. Ravid Shwartz-Ziv: Where? Where do you... Where do you... Recruit? Yuandong Tian: Yeah, so we have a place in well our main place is San Francisco in London. Right. So two thirds of our employees in San Francisco and one third in London. Ravid Shwartz-Ziv: Okay, thank you so much for joining us. It was a great pleasure. Allen Roush: Yeah, it was a real pleasure. Thank you. Yuandong Tian: Yeah, thanks so much for interviewing me and and hope that people will benefit from the discussions and interesting brainstormings. Yeah. Ravid Shwartz-Ziv: Yeah.