Latest / learning with yacine show / RLMs are Compositional Generalizers with Alex Zhang from MIT
Transcript
- 0:00SPEAKER_00: The modern way to build AI agent in 2026 is by fitting a capable enough LM inside
- 0:05SPEAKER_00: a good harness.
- 0:06SPEAKER_00: Cloud Code, Codex, PI,
- 0:09SPEAKER_00: Prime Agent,
- 0:09SPEAKER_00: these are all examples of harnesses that we are familiar with,
- 0:13SPEAKER_00: but there are plenty of custom ones for specific types of tasks that exist.
- 0:16SPEAKER_00: The standard assumption has been in the broader research community that
- 0:21SPEAKER_00: the harness is just plumbing,
- 0:22SPEAKER_00: that the intelligence lives in the weights,
- 0:25SPEAKER_00: and every new model released makes the scaffolding a little more obsolete.
- 0:29SPEAKER_00: But there are two important parameters that make the impact of
- 0:32SPEAKER_00: the harness much more important than previously thought.
- 0:35SPEAKER_00: First is context-wise.
- 0:36SPEAKER_00: Every model,
- 0:37SPEAKER_00: even the frontier ones,
- 0:38SPEAKER_00: gets increasingly unreliable as the inputs get longer and longer.
- 0:42SPEAKER_00: That's just a constraint.
- 0:45SPEAKER_00: A million token context window doesn't mean a million tokens of actual performance.
- 0:51SPEAKER_00: Trying to improve this by using all sorts of linearization of attention
- 0:55SPEAKER_00: or compaction tricks helps, but there are limits to what it can achieve.
- 0:59SPEAKER_00: Second is that transformers are surprisingly bad at what's called compositional generalization.
- 1:06SPEAKER_00: Well, I said it correctly.
- 1:09SPEAKER_00: Meaning solving a new problem by combining the pieces they already know about
- 1:13SPEAKER_00: and were trained on.
- 1:14SPEAKER_00: Every new domain kind of requires its own training data in some shape or form.
- 1:19SPEAKER_00: These two facts are what powers the intuition behind recursive language models
- 1:22SPEAKER_00: from Alex Zeng at Which,
- 1:25SPEAKER_00: by the way,
- 1:25SPEAKER_00: we already discussed earlier this year with him on how they help with
- 1:29SPEAKER_00: the context work problem.
- 1:31SPEAKER_00: But as a refresher,
- 1:32SPEAKER_00: the RLM,
- 1:33SPEAKER_00: instead of feeding the long prompt into the model,
- 1:35SPEAKER_00: you put the context in a variable inside a Python model.
- 1:40SPEAKER_00: This way, the model never sees the full context in one shot.
- 1:43SPEAKER_00: It can pick at it,
- 1:44SPEAKER_00: it can grab it,
- 1:45SPEAKER_00: it can chop it into chunks and recursively either call itself or tool
- 1:49SPEAKER_00: on these pieces.
- 1:51SPEAKER_00: The context kind of become an environment, the model.
- 1:58SPEAKER_00: This idea has been improved some more and researched by Alex and Omar to showcase
- 2:03SPEAKER_00: that RLM are in fact compositional generalizers.
- 2:06SPEAKER_00: They argue that a good harness like
- 2:08SPEAKER_00: the RLM makes every individual model call locally in distribution even when
- 2:13SPEAKER_00: the overall task is way outside anything the model has been trained on
- 2:16SPEAKER_00: and they show it very beautifully by training an rlm
- 2:19SPEAKER_00: on short tasks only using reinforcement learning and they generalize to tasks 8
- 2:23SPEAKER_00: to 32 times longer even in entirely different domains while
- 2:28SPEAKER_00: the vanilla transformer baseline trained in the same way stay completely flat so
- 2:31SPEAKER_00: the generalization here isn't coming from the weights update per se It's coming from
- 2:37SPEAKER_00: the model learning to use the harness to decompose a task and then encoding that in
- 2:42SPEAKER_00: the weights.
- 2:42SPEAKER_00: So today we're back with Alex and we're going to go deep on the updated experiment results.
- 2:46SPEAKER_00: We'll start with the gentle work tool here so you don't have to read anything to follow along.
- 2:51SPEAKER_00: We're also going to look at what our RLM will even look like for RLM.
- 3:06SPEAKER_00: By the way,
- 3:14SPEAKER_00: Alex's blog posts are generally like very good read with beautiful figures,
- 3:19SPEAKER_00: really well written.
- 3:20SPEAKER_00: It's not AI generated, so it's very, very understandable.
- 3:23SPEAKER_00: They're all linked below.
- 3:24SPEAKER_00: I think they are worth opening alongside the video.
- 3:28SPEAKER_00: Speaking of RLM Arnest,
- 3:29SPEAKER_00: let's talk a bit about RC, the openweight lab that is sponsoring this interview.
- 3:33SPEAKER_00: If you're familiar with the open model landscape,
- 3:36SPEAKER_00: you know it's pretty much dominated by labs outside of North America.
- 3:39SPEAKER_00: Well,
- 3:39SPEAKER_00: RC is a pretty cool lab in North America that don't only openweight their model,
- 3:44SPEAKER_00: called Trinity,
- 3:44SPEAKER_00: which you can download and run on whatever system you want,
- 3:46SPEAKER_00: but is also very open about their training methodology And release a lot
- 3:50SPEAKER_00: of cool artifacts they're developing internally.
- 3:52SPEAKER_00: You can get the model, by the way, on OpenRouter, it's available.
- 3:55SPEAKER_00: And if you're into personal agents, you can connect it to Hermesys.
- 3:58SPEAKER_00: But you can also use their model directly on their platform.
- 4:00SPEAKER_00: One fresh new thing they released recently,
- 4:02SPEAKER_00: which I'm pretty excited about,
- 4:03SPEAKER_00: is Knack,
- 4:04SPEAKER_00: which is an open source harness for long running agentic work,
- 4:07SPEAKER_00: which is actually based on the RLM idea.
- 4:09SPEAKER_00: It's actually taken from the Slate version that includes some RLM idea,
- 4:14SPEAKER_00: but it's a modified version of it.
- 4:15SPEAKER_00: It's pretty useful for research work that are complex,
- 4:17SPEAKER_00: long-running, but are decomposable with well-separate parts.
- 4:21SPEAKER_00: Machine learning experiments are a very good example of that.
- 4:23SPEAKER_00: How this works is that you give your model,
- 4:25SPEAKER_00: in that harness,
- 4:25SPEAKER_00: a high-level objective,
- 4:26SPEAKER_00: a bunch of constraints,
- 4:28SPEAKER_00: data upfront, and an exemptus criteria, and you just let it roll on the research.
- 4:31SPEAKER_00: You can try out Knack out there on GitHub,
- 4:33SPEAKER_00: or install the current Edge build by checking their blog post.
- 4:36SPEAKER_00: Big thanks to RC for sponsoring this interview.
- 4:39SPEAKER_00: It's a very cool app to keep an eye on.
- 4:40SPEAKER_00: Let's now dive into the interview.
- 4:46SPEAKER_00: So yeah, the first version of the recursive language model was published in October 15, 2025.
- 4:51SPEAKER_00: And then he published an actual research paper that I encourage you to read it.
- 4:57SPEAKER_00: It's one of the best, in my view, of 2025.
- 5:00SPEAKER_00: Really well made.
- 5:00SPEAKER_00: And the idea is very simple.
- 5:02SPEAKER_00: The idea is that a good harness for a language model is our RAPL.
- 5:08SPEAKER_00: It just worked well.
- 5:09SPEAKER_00: And RPL means read, invalidate, print, loop environment.
- 5:13SPEAKER_00: So there is two concepts that are important here.
- 5:17SPEAKER_00: The first one is that the context is not just shoved straight into the language model.
- 5:22SPEAKER_00: The context is actually living inside this environment.
- 5:26SPEAKER_00: And the model is going to do a whole bunch of stuff like interacting with this data,
- 5:30SPEAKER_00: this context to do the work that it's supposed to do, right?
- 5:33SPEAKER_00: So here they have the task, right?
- 5:35SPEAKER_00: This is you're reading an extremely long book.
- 5:36SPEAKER_00: Can you list out all which item that were made before the great blah, blah, blah, blah.
- 5:41SPEAKER_00: You load that in and then the model will be able to do that and then to interact with it.
- 5:44SPEAKER_00: And then without training it, It's actually pretty good.
- 5:47SPEAKER_00: So there's this component.
- 5:48SPEAKER_00: And then the other component is that you can just spawn another RLM to do some work.
- 5:53SPEAKER_00: So we can go and do LLM Query.
- 5:56SPEAKER_00: They give just this, just a specific context and the information it needs to do the work.
- 6:03SPEAKER_00: And you can go ahead and do just a specific task.
- 6:07SPEAKER_00: It's not doing the whole thing.
- 6:08SPEAKER_00: And you can continue and spawn these subagents.
- 6:10SPEAKER_00: And you can have multiple depth.
- 6:12SPEAKER_00: I'm pretty sure most of the results are on depth one.
- 6:16SPEAKER_00: That's the overall concept of the recursive language model.
- 6:19SPEAKER_00: It's really just an R-ness, and then you can shove whatever you want in there.
- 6:22SPEAKER_00: And in the paper, there is some very interesting experiments that they have done.
- 6:26SPEAKER_00: They show that just with this,
- 6:28SPEAKER_00: with a cool R-ness,
- 6:29SPEAKER_00: all the limitations of the The context length are nullified and then
- 6:34SPEAKER_00: the agent can actually do pretty good work and expand their context window.
- 6:40SPEAKER_00: So here, GPT-5, 262, can't do much.
- 6:46SPEAKER_00: That's the limit of its context length,
- 6:48SPEAKER_00: but within the RLM, it can actually do something and expand and do the task.
- 6:52SPEAKER_00: Meanwhile, from the start of the year, there's a bunch of stuff that happened.
- 6:56SPEAKER_00: One thing is the stuff that we're going to look at in a few seconds.
- 6:59SPEAKER_00: The other is that Prime Agent has been released,
- 7:02SPEAKER_00: and it's like a self-improving RLM,
- 7:05SPEAKER_00: like a continuous harness stuff thing,
- 7:07SPEAKER_00: which is taking this RLM idea and kind of productionalizing it, right?
- 7:10SPEAKER_00: So you can check out Prime Agent.
- 7:11SPEAKER_00: Spoiler, we're going to actually...
- 7:13SPEAKER_00: Like talk to Seth next week about Prime Agent.
- 7:16SPEAKER_00: There's the paper that is really hot off the press.
- 7:19SPEAKER_00: It's been released since four days.
- 7:21SPEAKER_00: So you can also check it out.
- 7:22SPEAKER_00: What the RLM in production actually look like.
- 7:24SPEAKER_00: For those that are not familiar, what is RNS?
- 7:28SPEAKER_00: There's many definitions out there.
- 7:30SPEAKER_00: We're going to use a very simple one, which is that it's a software around the model, right?
- 7:34SPEAKER_00: So like React,
- 7:35SPEAKER_00: not React,
- 7:36SPEAKER_00: the front-end library,
- 7:36SPEAKER_00: but like the React framework is one such RNS,
- 7:39SPEAKER_00: you have CodeAct,
- 7:41SPEAKER_00: you have Cloud Code,
- 7:42SPEAKER_00: you have the RLM,
- 7:43SPEAKER_00: you have
- 7:45SPEAKER_00: This software that is sitting between the real world and
- 7:50SPEAKER_00: the actual raw model,
- 7:52SPEAKER_00: and that is managing the context,
- 7:54SPEAKER_00: what it gives, what it doesn't give, doing the compaction, all this stuff.
- 7:57SPEAKER_00: This is the RNS,
- 7:59SPEAKER_00: and its main job is to run the raw LLM in a loop,
- 8:02SPEAKER_00: manage the context, and now it's usually part of the raw LLM training.
- 8:06SPEAKER_00: So the models are now made to be able to pilot this stuff.
- 8:10SPEAKER_00: If you look here,
- 8:11SPEAKER_00: RLM minimal,
- 8:13SPEAKER_00: From Alex,
- 8:14SPEAKER_00: it's actually a pretty good way of looking at like, what is a hardness, right?
- 8:18SPEAKER_00: Because it's like the RLM idea, but like pushed to the absolute minimum element.
- 8:23SPEAKER_00: And yeah, it's pretty easy to understand.
- 8:25SPEAKER_00: There's not that like, it's stripped down to literally, literally the minimum.
- 8:29SPEAKER_00: I can start over here, the RLM, RAPL, and then And you can follow along.
- 8:33SPEAKER_00: There's not that much line of code in there.
- 8:35SPEAKER_00: And it's fairly self-explanatory.
- 8:39SPEAKER_00: If you want to understand a bit more what the R-ness are about,
- 8:41SPEAKER_00: this is a very good place to start because there's no complexity added.
- 8:45SPEAKER_00: And also you'll be able to understand a bit more what the RLM is.
- 8:48SPEAKER_00: Like I said, the RLM is nothing more than these two elements.
- 8:51SPEAKER_00: And the two quality of it is that You're going to do context offloading.
- 8:56SPEAKER_00: So you're going to take the context that is usually bloating the context window of
- 9:00SPEAKER_00: the LM and shoving it out into the REPL,
- 9:03SPEAKER_00: which then your LM will be able to do the REPL route to figure out what's going on.
- 9:10SPEAKER_00: A bit like in your case,
- 9:12SPEAKER_00: if you have a big dataset to analyze, You're not gonna just read it end to end.
- 9:19SPEAKER_00: You're gonna poke at it and try to understand its shape,
- 9:23SPEAKER_00: but you're not gonna try to jam all the information by reading every single line
- 9:27SPEAKER_00: of code or every single line of data row in there.
- 9:30SPEAKER_00: So that's roughly it.
- 9:33SPEAKER_00: And then the other thing is that It can do programmatic sub-agent calling.
- 9:37SPEAKER_00: So it's the RLM that is assigning when it's going to call itself, right?
- 9:41SPEAKER_00: And this is going to be able to do tool calls here,
- 9:45SPEAKER_00: but also be able to upload this stuff to agent, which is itself, right?
- 9:50SPEAKER_00: But just with the context that we need.
- 9:51SPEAKER_00: Versus if you don't have that, you're just going to shove it straight into the root LLM.
- 9:56SPEAKER_00: So everything will be in that context, which is going to be a bit of a problem.
- 10:01SPEAKER_00: So the update is that the language model R harnesses,
- 10:04SPEAKER_00: or the RLM, are a compositional generalizer.
- 10:08SPEAKER_00: And not all of the harnesses are like that.
- 10:10SPEAKER_00: Some of them don't allow The model to be able to have this locally in distribution,
- 10:18SPEAKER_00: I think it's called, effect.
- 10:19SPEAKER_00: So it cannot be a compositional generalizer.
- 10:22SPEAKER_00: A compositional generalizer means that it will be able to take a big task
- 10:25SPEAKER_00: and deconstruct it into subtasks that are not complicated and
- 10:29SPEAKER_00: then ascend them together in order to do some novel work.
- 10:33SPEAKER_00: That's kind of the idea of a compositional generalizer.
- 10:35SPEAKER_00: So there's two flavors of this, right?
- 10:37SPEAKER_00: If you think about it,
- 10:38SPEAKER_00: like if you train on short stuff and you eval on long,
- 10:41SPEAKER_00: you should not be freaking out, right?
- 10:42SPEAKER_00: So for example,
- 10:43SPEAKER_00: if I give you a task and I show you like shortly how to do it,
- 10:46SPEAKER_00: and then I ask you to do it for real on the big stuff,
- 10:48SPEAKER_00: after you train enough on the small stuff,
- 10:50SPEAKER_00: you should be able to do the longer tasks.
- 10:52SPEAKER_00: We need to kind of collect longer and longer sequence length tasks and then train
- 11:00SPEAKER_00: the model on that in order for them to generalize properly.
- 11:03SPEAKER_00: So this is part of the post-training, mid-training.
- 11:05SPEAKER_00: If you look at this plot, so blue, this is the RLM slash QN.
- 11:10SPEAKER_00: Yellow is just the base transformer, right?
- 11:12SPEAKER_00: So it's always transformer, route transformer versus the RLM.
- 11:15SPEAKER_00: So what we see in shaded is like the training lift here.
- 11:19SPEAKER_00: Is it that?
- 11:20SPEAKER_00: I think so, yeah.
- 11:22SPEAKER_00: Train lift.
- 11:23SPEAKER_00: Source versus short, right?
- 11:24SPEAKER_00: So this is a train lift in yellow for the base runtime and blue for the RLM.
- 11:28SPEAKER_00: So RLM in short is really good.
- 11:30SPEAKER_00: What happened is that like in bolded here, it's the evaluation on the long stuff, right?
- 11:38SPEAKER_00: Just at the get-go,
- 11:39SPEAKER_00: the RLM is pretty good,
- 11:41SPEAKER_00: just doing the long stuff, like we saw in the previous thing, without any training.
- 11:45SPEAKER_00: But then as the model is getting trained, it gets better and better.
- 11:50SPEAKER_00: And even better than what it was doing in the short training.
- 11:53SPEAKER_00: Versus the base transformer.
- 11:56SPEAKER_00: It's getting some uplift, but not that much.
- 11:58SPEAKER_00: It still sucks at that.
- 12:00SPEAKER_00: It has a hard time.
- 12:01SPEAKER_00: And then the second way of testing this compositional generalization is you train
- 12:05SPEAKER_00: on one domain,
- 12:06SPEAKER_00: and then you test on another, so that they're not technically related.
- 12:09SPEAKER_00: They're not related on the context, but they're related on how you would do the work.
- 12:12SPEAKER_00: So we have a few of them that we're going to see.
- 12:15SPEAKER_00: But basically, same thing.
- 12:16SPEAKER_00: Shaded is on one domain.
- 12:19SPEAKER_00: And then Bold is another domain.
- 12:21SPEAKER_00: And if you see,
- 12:22SPEAKER_00: the RLM is actually a bit worse in the training for
- 12:29SPEAKER_00: this specific domain than
- 12:30SPEAKER_00: the base transformer, if I read this correctly.
- 12:32SPEAKER_00: But once you're doing the transfer and you evolve on the sibling task,
- 12:38SPEAKER_00: the RLM is actually better than the base transformer over there.
- 12:43SPEAKER_00: So it's able to generalize.
- 12:44SPEAKER_00: Because it was trained on the first domain.
- 12:46SPEAKER_00: So that's what we're seeing here.
- 12:47SPEAKER_00: So this is like the two type of composition generalization tasks we're looking at.
- 12:54SPEAKER_00: And it becomes obvious when you kind of look at it, right?
- 12:56SPEAKER_00: So like this is the core RLM trunk, right?
- 13:00SPEAKER_00: And then this is the offloading.
- 13:02SPEAKER_00: But if you look at it, you're kind of offloading the context.
- 13:06SPEAKER_00: So it's the same kind of structure in Broscom Plus here on the training side and
- 13:10SPEAKER_00: then the Oolong on the evaluation side.
- 13:13SPEAKER_00: So you have the same rough structure, which makes sense.
- 13:16SPEAKER_00: For example,
- 13:17SPEAKER_00: if you were analyzing a time series in like finance versus healthcare, yes,
- 13:21SPEAKER_00: the context is a bit different and you will do a bunch of stuff,
- 13:23SPEAKER_00: but like roughly speaking,
- 13:24SPEAKER_00: you're Like going through the same step,
- 13:26SPEAKER_00: or like image generation,
- 13:28SPEAKER_00: or image analysis problem in the medical field versus,
- 13:32SPEAKER_00: I don't know, like sports, right?
- 13:34SPEAKER_00: It will have some difference, but like the core will still kind of be similar.
- 13:38SPEAKER_00: So it's kind of the same idea here.
- 13:40SPEAKER_00: So composition generation, Bayes-Arlandic-Apost task can strip the context away.
- 13:44SPEAKER_00: And all of this is related to Alex mismanaged genius hypothesis.
- 13:49SPEAKER_00: We're going to ask him about it.
- 13:50SPEAKER_00: But basically what he's saying is that like, This is Frontier LM right now, just raw.
- 13:55SPEAKER_00: And then if you manage it properly, it can do a lot.
- 13:58SPEAKER_00: And we're already seeing it with like Cloud Code and Codex doing like crazy stuff.
- 14:02SPEAKER_00: But if you're not managing them properly, they will not be able to do the stuff.
- 14:06SPEAKER_00: And it will be too out of the context for the raw LM.
- 14:08SPEAKER_00: So that's the idea.
- 14:09SPEAKER_00: And then what he's saying is that the RLM is kind of like bit less and peeled
- 14:13SPEAKER_00: because there's not much bias we're putting into the R-ness.
- 14:17SPEAKER_00: And we just let the model kind of do its stuff here.
- 14:19SPEAKER_00: And this is related to something that I've been thinking about for a long time,
- 14:23SPEAKER_00: which is how do you get quality long context out of these things?
- 14:27SPEAKER_00: Because if you're not able to kind of get quality long context,
- 14:31SPEAKER_00: there's a lot of stuff that it's just not possible to do.
- 14:34SPEAKER_00: And the issue I had was that it's very hard to find quality long context benchmark, right?
- 14:39SPEAKER_00: Because us humans are not training ourselves on that.
- 14:42SPEAKER_00: We're training ourselves on normal tasks.
- 14:45SPEAKER_00: And if we have to do a long thing, then we do a long thing.
- 14:47SPEAKER_00: But we compose it.
- 14:49SPEAKER_00: We're not just like doing the long thing on one shot, right?
- 14:53SPEAKER_00: Like making a breakthrough, like making a research paper and stuff like that.
- 14:55SPEAKER_00: Like if you did it once, you know the flow and stuff.
- 14:59SPEAKER_00: But it's more about like you know the steps that you have to take to do this stuff.
- 15:04SPEAKER_00: Then you train on the steps.
- 15:07SPEAKER_00: There's not that many tasks that are super, super duper long.
- 15:10SPEAKER_00: So we're kind of creating them synthetically.
- 15:12SPEAKER_00: And then the way we're thinking about it is that you take the long task and
- 15:16SPEAKER_00: then you just have a short program in your head and stuff.
- 15:20SPEAKER_00: And you create these subtasks.
- 15:21SPEAKER_00: And maybe there's enough loading to a longer subtasks that you may give
- 15:24SPEAKER_00: to somebody else or yourself in the future.
- 15:26SPEAKER_00: And then you have a short program to decompose this again.
- 15:29SPEAKER_00: So that's kind of the general flow.
- 15:31SPEAKER_00: Right now in a lot of the harnesses and just like the raw LLM,
- 15:34SPEAKER_00: you're just going to give everything and then the LLM will just go like this, right?
- 15:38SPEAKER_00: And the issue is that this sequence is out of distribution.
- 15:42SPEAKER_00: It's out of distribution for the LLM.
- 15:44SPEAKER_00: He never seen that, so he freaks out, right?
- 15:46SPEAKER_00: And he doesn't know what to do.
- 15:47SPEAKER_00: He chokes on the information.
- 15:49SPEAKER_00: Versus if you are managing it properly, All of this stuff is in distribution.
- 15:56SPEAKER_00: You've already seen something like this, especially if you train it on that.
- 16:00SPEAKER_00: You've already seen a subquery like this.
- 16:01SPEAKER_00: You've already seen a tool call like that.
- 16:03SPEAKER_00: You've already seen this other tool call chained together or this subquery over there.
- 16:09SPEAKER_00: It becomes a very small task that you chained together.
- 16:12SPEAKER_00: So this is what the locally in distribution means.
- 16:15SPEAKER_00: And Alex is posting that you only need these two things.
- 16:18SPEAKER_00: You need the context offloading and the programmatic sub-agent calling in order
- 16:22SPEAKER_00: to be able to do that.
- 16:23SPEAKER_00: And this is what the RLM is doing.
- 16:24SPEAKER_00: So if you train on it, you'll be able to get all this.
- 16:27SPEAKER_00: Let's do it compositional generalization.
- 16:29SPEAKER_00: On the result here, we have the short to long.
- 16:31SPEAKER_00: We have all these different benchmarks.
- 16:33SPEAKER_00: And we see them over here on the train on short, evaluated on long, scaffold versus no scaffold.
- 16:40SPEAKER_00: And most of them, like the RLM, just can't do them.
- 16:42SPEAKER_00: It's just too much, right?
- 16:43SPEAKER_00: See here, 2 million tokens, you just can't handle them.
- 16:46SPEAKER_00: So having the RLM helped a lot.
- 16:47SPEAKER_00: There's two kinds of RLM we're using,
- 16:49SPEAKER_00: one that has the prompt to decompose and one that doesn't have the prompt
- 16:53SPEAKER_00: to decompose.
- 16:54SPEAKER_00: So it's just like a nudge that you give it, hey, you should decompose this stuff.
- 16:57SPEAKER_00: It does help in some instances,
- 16:59SPEAKER_00: like here specifically,
- 17:00SPEAKER_00: but here it also help in like,
- 17:02SPEAKER_00: it's just like, just generally okay without the nudge to decompose.
- 17:06SPEAKER_00: So something is happening with the MRC RV2.
- 17:09SPEAKER_00: Like generally speaking,
- 17:10SPEAKER_00: if you look at it,
- 17:11SPEAKER_00: like there's a big difference in jump between this harness versus like no harness,
- 17:16SPEAKER_00: right?
- 17:16SPEAKER_00: So it's helping out to have this generalization.
- 17:19SPEAKER_00: And in domain one versus domain two,
- 17:21SPEAKER_00: we have like Oolong, Oblique, Bench, and Oblique Bench, Descriptive.
- 17:25SPEAKER_00: What we're doing here, training on Twitter stents, and then evaluating a wallchat error.
- 17:31SPEAKER_00: Here we're training on writing stuff,
- 17:33SPEAKER_00: and we're evaluating on math,
- 17:34SPEAKER_00: track, course questions, and evaluating on spam questions.
- 17:37SPEAKER_00: So we're doing these pairings, and you can see here again, yellow is doing okay-ish, right?
- 17:45SPEAKER_00: Blue is better, right?
- 17:46SPEAKER_00: After a 500 training step, So it's able to generalize to this other thing.
- 17:51SPEAKER_00: So that's roughly the idea.
- 17:53SPEAKER_00: And then the last thing I want to say is that
- 17:56SPEAKER_00: if we truly have this compositional generalization,
- 17:59SPEAKER_00: where the root LLM is able to go ahead and do its work,
- 18:05SPEAKER_00: and it's more of an orchestrator that's already seen that stuff.
- 18:10SPEAKER_00: It does already know how to Operate the sub-RLM calling and how to peek into
- 18:18SPEAKER_00: the context and do all this stuff.
- 18:20SPEAKER_00: And the core thing is always the same.
- 18:22SPEAKER_00: Then it means that code is a thing to get compositional generalization.
- 18:26SPEAKER_00: So the Rness that is just code around the LM Is what it is, right?
- 18:32SPEAKER_00: If this is true,
- 18:33SPEAKER_00: we have a nice thing that Alex worked on,
- 18:37SPEAKER_00: I think it's been like, what,
- 18:38SPEAKER_00: five days or something like that,
- 18:40SPEAKER_00: which is called speculative programming to call,
- 18:42SPEAKER_00: so SPETC,
- 18:43SPEAKER_00: which is taking this idea and push it further,
- 18:47SPEAKER_00: meaning that if the RLM,
- 18:49SPEAKER_00: we want to optimize it,
- 18:51SPEAKER_00: we can do a lot of the stuff that we use in the CPU and with LMs,
- 18:55SPEAKER_00: like speculative decoding, We can do speculative programming tool call.
- 19:00SPEAKER_00: Because as the code is getting written by the core RLM,
- 19:04SPEAKER_00: we don't have to wait until everything is done before we actually do something.
- 19:08SPEAKER_00: We can start and shoot out these sub-agent calls and then in anticipation of The RLM
- 19:16SPEAKER_00: to be done.
- 19:19SPEAKER_00: Once it's done writing,
- 19:20SPEAKER_00: you can just use directly all the information that we have,
- 19:24SPEAKER_00: and then it can be much faster.
- 19:26SPEAKER_00: So instead of doing everything in sequence,
- 19:29SPEAKER_00: you can do a thing in parallel and really boost up the performance.
- 19:32SPEAKER_00: So we have a nice video over here.
- 19:34SPEAKER_00: We have a nice animation of what it looks like.
- 19:37SPEAKER_00: So here on the left is the actual RLM that will be writing the stuff.
- 19:42SPEAKER_00: Here you see when we start to do that sort of thing,
- 19:44SPEAKER_00: there's already a bunch of sub-agent columns that are being made.
- 19:49SPEAKER_00: So we already have the response back from these chunks, right?
- 19:54SPEAKER_00: And then this is the other program that is not having the speculative element
- 19:59SPEAKER_00: that is just doing them in series.
- 20:01SPEAKER_00: So we take this, speed up, and then we ingest it.
- 20:04SPEAKER_00: In speculative decoding type of stuff, you're making speculation, right?
- 20:08SPEAKER_00: So sometimes you're wrong,
- 20:10SPEAKER_00: but overall,
- 20:11SPEAKER_00: if you're engineering it properly,
- 20:13SPEAKER_00: you should be right and saving a whole bunch of time.
- 20:15SPEAKER_00: So this is what we're seeing in his little experiment.
- 20:18SPEAKER_00: This was very quickly made.
- 20:19SPEAKER_00: It's on the 30B, same quen as...
- 20:24SPEAKER_00: That's the other experiment.
- 20:25SPEAKER_00: And you see over here in all these concurrent tasks at different temperature
- 20:28SPEAKER_00: that this is the base RLM one o'clock time.
- 20:32SPEAKER_00: And with the speculative affirming tool call, you're doing it faster, basically.
- 20:37SPEAKER_00: That's what we're saying.
- 20:38SPEAKER_00: If you have four threads versus eight,
- 20:39SPEAKER_00: I think, concurrent tasks, you can get a lot more speedup.
- 20:43SPEAKER_00: You're making more subcalls per task, but you're faster.
- 20:48SPEAKER_00: So that's kind of the idea, is that you're going to spend maybe more tokens.
- 20:52SPEAKER_00: You're going to be much faster, which is pretty cool, right?
- 20:54SPEAKER_00: It means that we can speed up these models
- 20:57SPEAKER_00: in this fashion using a harness
- 20:59SPEAKER_00: that is embracing fully that code is the thing.
- 21:03SPEAKER_00: The structure of it is not terribly complicated.
- 21:05SPEAKER_00: There's a GitHub report.
- 21:06SPEAKER_00: You can check it out.
- 21:07SPEAKER_00: But basically, it's using a shadow RLM.
- 21:11SPEAKER_00: Shadow RAPL environment.
- 21:13SPEAKER_00: And he's looking and peeking at what he can use and then just firing up whenever he can.
- 21:17SPEAKER_00: And then once the thing is done,
- 21:19SPEAKER_00: it's looking at what he has speculated and then ingested into the core flow.
- 21:24SPEAKER_00: And if something is not relevant anymore, it's just discarded.
- 21:27SPEAKER_00: There's four
- 21:30SPEAKER_00: cases that he foresaw that you could use speculative programming tool call on.
- 21:36SPEAKER_00: When you have literal,
- 21:37SPEAKER_00: if you have input dependency,
- 21:38SPEAKER_00: you're going to check it out if it's actually safe or not.
- 21:41SPEAKER_00: If it's not safe, you're not going to waste time doing it.
- 21:44SPEAKER_00: If you're doing it naively, you're going to waste a lot of time.
- 21:46SPEAKER_00: But if you're a bit smart about how you're doing it,
- 21:48SPEAKER_00: and engineer that little engine, you just get free boost of speed up here.
- 21:54SPEAKER_00: And we can take a look at it here, how it's operating in a nice little video.
- 21:59SPEAKER_00: So this is speculative, this is serial, and you will see all the subcalls popping up over here.
- 22:08SPEAKER_00: Pretty cool to see.
- 22:09SPEAKER_00: Already done with all of these.
- 22:11SPEAKER_00: And this one didn't start its circle yet.
- 22:17SPEAKER_00: And now this one is done in 33 seconds.
- 22:19SPEAKER_00: And this one is starting to kind of roll for each of the thing in sequence.
- 22:22SPEAKER_00: So if you're compute bound, your serial will make more sense.
- 22:25SPEAKER_00: But if you have more compute, it may make more sense.
- 22:29SPEAKER_00: To do the things in parallel, right?
- 22:30SPEAKER_00: And this kind of ties a bit to the prime agent thing we're going to see,
- 22:33SPEAKER_00: because they have a lot of compute.
- 22:35SPEAKER_00: They can show the stuff in parallel.
- 22:36SPEAKER_00: Yeah, so that's it.
- 22:37SPEAKER_00: This is speculative parametric tool calling, SPTC, and it's tied to the whole RLM stuff.
- 22:47SPEAKER_01: I'm Alex.
- 22:48SPEAKER_01: I'm now a second year PhD student.
- 22:50SPEAKER_01: I was a first year one when we first did this.
- 22:54SPEAKER_01: I guess I'm maybe most known for the like RLM kind of line of work.
- 23:02SPEAKER_01: But generally,
- 23:02SPEAKER_01: I would say right now,
- 23:04SPEAKER_01: my interests are,
- 23:05SPEAKER_01: they've always been about areas where I think language models exist.
- 23:09SPEAKER_01: Are severely underutilized.
- 23:12SPEAKER_01: And that hasn't really changed.
- 23:14SPEAKER_01: I think this means a lot of different things.
- 23:18SPEAKER_01: But right now, this especially is focused on harnesses and harness design.
- 23:25SPEAKER_01: I would say RLMs are one such case of a class of harness
- 23:31SPEAKER_01: that I think is very important that we study and use.
- 23:35SPEAKER_01: But there are many other sort of directions in this space that are either related
- 23:42SPEAKER_01: to RLMs or unrelated to RLMs.
- 23:45SPEAKER_01: That I'm very interested in.
- 23:46SPEAKER_01: I think in the study of harnesses as a whole,
- 23:50SPEAKER_01: this broadly is something that we just are not doing very well as a field so far.
- 23:56SPEAKER_01: And so it's something that I'm kind of hoping to fix in some sense in the coming year.
- 24:04SPEAKER_00: I've kind of seen that like.
- 24:07SPEAKER_00: Harness in general,
- 24:08SPEAKER_00: I don't know,
- 24:09SPEAKER_00: it seems that it doesn't look like it because we're talking about the model as
- 24:14SPEAKER_00: if they were kind of alone in this, right?
- 24:17SPEAKER_00: But every benchmark result from all the Frontier Labs and stuff,
- 24:21SPEAKER_00: you always have this harness that is behind that is doing a lot of
- 24:24SPEAKER_00: the heavy lifting, right?
- 24:26SPEAKER_00: If you look at the, I don't know, it was the RKGI V3 result and stuff.
- 24:32SPEAKER_00: If you set up cloud code properly, like Opus 5 is absolutely a monster at this, right?
- 24:37SPEAKER_00: But alone, it's just struggling.
- 24:40SPEAKER_00: Same thing with MazeBench, which is this really complicated 3D maze.
- 24:45SPEAKER_00: The LM alone, it would just freak out.
- 24:47SPEAKER_00: But with the harness, it's doing pretty well.
- 24:50SPEAKER_00: And I feel like...
- 24:52SPEAKER_00: We're at the point where like maybe we have, I don't know, we have everything we need, right?
- 24:57SPEAKER_00: But like the pieces are there, but like the harness is actually a very big piece.
- 25:01SPEAKER_00: I need to be palleted properly by the system.
- 25:06SPEAKER_01: One thing I will say before we go on is that I think you probably noticed this too.
- 25:10SPEAKER_01: I think harness design is a very unscientific thing at the moment.
- 25:15SPEAKER_01: And as much as, you know, people...
- 25:19SPEAKER_01: Like to discount the value of doing certain types of science or rigorous science or research.
- 25:28SPEAKER_01: I think one of the things that we do owe to proper science,
- 25:32SPEAKER_01: thanks to actually people in industry, are things like scaling walls.
- 25:36SPEAKER_01: These are things that have bridged together a lot of,
- 25:42SPEAKER_01: originally back before we had these scaling walls,
- 25:46SPEAKER_01: The design of model architecture was just like a wild west.
- 25:50SPEAKER_01: I could claim that we need to use convolutional layers,
- 25:53SPEAKER_01: but someone else would claim, no, no, no, we need to use recurrent layers.
- 25:56SPEAKER_01: And we had no really good way of quantifying this.
- 25:58SPEAKER_01: And I think this is something that And like,
- 26:01SPEAKER_01: oh, maybe actually you should use an RLM, like maybe you should use PrimeAgent.
- 26:05SPEAKER_01: And it's like, how do you
- 26:26SPEAKER_01: How do we even discuss this in a meaningful way?
- 26:29SPEAKER_01: I think this is something that I really want to solve.
- 26:33SPEAKER_01: I think we're there,
- 26:34SPEAKER_01: but there's a missing insight around what is a more principled way of talking
- 26:40SPEAKER_01: about harnesses.
- 26:41SPEAKER_00: I think one thing that is dangerous is that it's touching software engineering.
- 26:49SPEAKER_00: Not in a computer science way where we have this principle and And this theory,
- 26:56SPEAKER_00: and then we're validating it in like, I don't know, in scientific manner.
- 26:59SPEAKER_00: More the software branch where like,
- 27:00SPEAKER_00: we're building this thing and then we're building this product and stuff.
- 27:04SPEAKER_00: Which I think is great because people are going to try ideas and stuff like that.
- 27:08SPEAKER_00: But it's dangerous because it's just going to create a lot of noise, right?
- 27:11SPEAKER_00: And then like, if on top of that, the models are absolutely so great at cheating, right?
- 27:17SPEAKER_01: You get like a very, very messy mix, right?
- 27:20SPEAKER_00: Like now you have to monitor...
- 27:23SPEAKER_00: These things in harnesses that can run for days and generate like billions of tokens.
- 27:28SPEAKER_00: So it murky a bit, even more the water, I think, about like, are we doing great?
- 27:33SPEAKER_00: Or like, I don't know, like, are we very poor at evaluating?
- 27:36SPEAKER_00: And like, they figured it out.
- 27:39SPEAKER_00: So I think like a good thing would be to kind of go in the principle.
- 27:44SPEAKER_00: I think one thing that you said that was really, really insightful here.
- 27:47SPEAKER_00: Is the concept of locally in distribution.
- 27:51SPEAKER_00: And I think that was one of the core things that you said that was very clear about
- 27:56SPEAKER_00: where the RLM is helping deconstruct.
- 27:59SPEAKER_00: Because before I was like,
- 28:00SPEAKER_00: this is so cool because you can extend But it's not even about that much,
- 28:05SPEAKER_00: more that this idea is enabled because of the RLM.
- 28:11SPEAKER_00: Can you just explain to us a bit what it is in your own world
- 28:14SPEAKER_00: and why is it important to have that?
- 28:17SPEAKER_01: Yeah.
- 28:18SPEAKER_01: So I think this is a term that I hope catches on because it's a very,
- 28:23SPEAKER_01: I think personally, it's a great way of describing the Property that we want.
- 28:28SPEAKER_01: So I think most people are aware that one term we often use in the context of
- 28:35SPEAKER_01: the language model for like a prompt that you give it is
- 28:39SPEAKER_01: that like this prompt is like in distribution.
- 28:42SPEAKER_01: And this is a very lazy way of saying that this prompt roughly is close to something
- 28:51SPEAKER_01: that the model has seen during training.
- 28:54SPEAKER_01: And this often is also just used to say that the model will behave well with this prompt.
- 28:59SPEAKER_01: It will give you a reasonable answer.
- 29:01SPEAKER_00: Now, when you go into the long context regime,
- 29:07SPEAKER_01: Where you basically have many, many, many tokens that you're providing to the model.
- 29:11SPEAKER_01: A case that you'll often run into,
- 29:14SPEAKER_01: and you can reason through why this is the case,
- 29:17SPEAKER_01: but a case you often run into is that
- 29:19SPEAKER_01: the language model itself probably has not really seen something close
- 29:24SPEAKER_01: to whatever this giant context is.
- 29:27SPEAKER_01: And this is, you know, one way you can say this is like this is context rot, which is true.
- 29:33SPEAKER_01: But I think a better way of describing this is
- 29:36SPEAKER_01: that like this particular sequence is something that
- 29:40SPEAKER_01: the language model just hasn't really seen before.
- 29:44SPEAKER_01: I think context rot is a way of describing the effect of long context things are really hard.
- 29:51SPEAKER_01: But it's not so much that for a language model specifically, long context is inherently hard.
- 29:58SPEAKER_01: I think if as humans we naturally had super long context data in the wild
- 30:04SPEAKER_01: and we have an abundance of it, this would not be an issue.
- 30:09SPEAKER_01: But it is an issue.
- 30:11SPEAKER_01: And so generally, one of the questions is like, okay, how do you address this?
- 30:15SPEAKER_01: How can I take a really long task or thing and make it in distribution?
- 30:23SPEAKER_01: And the answer is roughly, well, usually you use a harness, right?
- 30:26SPEAKER_01: So we use clock code and we use all these things.
- 30:29SPEAKER_01: Now,
- 30:30SPEAKER_01: one of those super desirable effects that you might imagine that you want here is
- 30:37SPEAKER_01: that if you think Think of a harness as just a bunch of language model calls, right?
- 30:41SPEAKER_01: Like, I mean,
- 30:41SPEAKER_01: there's tool calls and all these other things,
- 30:43SPEAKER_01: but ultimately the thing that we care about is like,
- 30:45SPEAKER_01: what are the language models doing?
- 30:47SPEAKER_01: One property we want is
- 30:48SPEAKER_01: that every single individual language model call is It's dealing with a problem
- 30:54SPEAKER_01: that it would consider to be in distribution.
- 30:57SPEAKER_01: And if you could get this effect,
- 31:00SPEAKER_01: in some sense,
- 31:02SPEAKER_01: then your language model would actually have no issue,
- 31:04SPEAKER_01: or your harness would have no issue solving this task.
- 31:08SPEAKER_01: Every single individual language model call can solve its local problem, right?
- 31:14SPEAKER_01: So this is why it's called locally in distribution,
- 31:18SPEAKER_01: because the idea is that maybe for an individual language model,
- 31:22SPEAKER_01: it can't solve this task with this super long horizon, super long context thing.
- 31:27SPEAKER_01: But the harness is actually able to break it down in a way where it can do it.
- 31:31SPEAKER_01: And I think this,
- 31:32SPEAKER_01: the last thing I'll say is this leads well into like,
- 31:36SPEAKER_01: What is the issue with the current harnesses that we use?
- 31:39SPEAKER_01: Why would I use an RLM over, let's say, Cloud Code?
- 31:42SPEAKER_01: And my answer to this would be, well, what is Cloud Code doing?
- 31:48SPEAKER_01: At the end of the day,
- 31:49SPEAKER_01: the main paradigm that everyone That's what everybody's used
- 31:52SPEAKER_01: to is constantly filling in the context of the model with more and more information
- 31:57SPEAKER_01: until eventually you compact it and then you keep going.
- 32:00SPEAKER_01: And the hope is that before you compact it, you are not making the model go bonkers.
- 32:08SPEAKER_01: And also that after compaction,
- 32:10SPEAKER_01: you're preserving all of the information that is important,
- 32:13SPEAKER_01: which there's very little guarantees that that'll actually happen.
- 32:19SPEAKER_01: This is generally why I would say we want things to be locally in distribution.
- 32:25SPEAKER_00: With Cloud Code,
- 32:27SPEAKER_00: what is nice is that every once in a year,
- 32:30SPEAKER_00: we get the open source version that is getting leaked,
- 32:33SPEAKER_00: which is pretty cool, so we can actually see what's up over there.
- 32:38SPEAKER_00: But yeah,
- 32:39SPEAKER_00: this idea I think is very interesting because we all have been chatting with the LM,
- 32:45SPEAKER_00: even with the cloud code or whatever.
- 32:47SPEAKER_00: At some point, even with compaction, it's just dumb, right?
- 32:51SPEAKER_00: It's done.
- 32:51SPEAKER_00: You know it lost something along the way, right?
- 32:55SPEAKER_00: And it makes sense in my view because This graph that you made,
- 32:59SPEAKER_00: it's not an actual graph,
- 33:02SPEAKER_00: but this is really what I was grappling with early last year.
- 33:09SPEAKER_00: Thinking about like, hey, how can we make quality long contacts?
- 33:12SPEAKER_00: And like the research on this front was doing his very best.
- 33:15SPEAKER_00: Like they were really trying, right?
- 33:17SPEAKER_00: Everything like from different type of attention to like linearized,
- 33:21SPEAKER_00: hybridized,
- 33:22SPEAKER_00: linearized attention in there,
- 33:24SPEAKER_00: like changing rope, removing it, like all kind of stuff, right?
- 33:28SPEAKER_00: To make the raw LLM work.
- 33:31SPEAKER_00: But then you have no choice.
- 33:32SPEAKER_00: At some point,
- 33:33SPEAKER_00: you have to get some of this data here and then just jam it there so
- 33:37SPEAKER_00: that it doesn't freak out when you start to see this stuff
- 33:40SPEAKER_00: because you've actually already seen some sort of long element like this
- 33:44SPEAKER_00: so you can know how to behave with them.
- 33:47SPEAKER_00: But inherently, it feels wrong a bit.
- 33:50SPEAKER_00: It feels wrong to do that when the data for what the human sees is this.
- 33:55SPEAKER_00: It seems like there's something missing.
- 33:56SPEAKER_00: One thing I wanted to get your point over there is
- 33:59SPEAKER_00: that do we even need long contacts at this point for the raw LLM?
- 34:05SPEAKER_00: We saw the harnesses being able to,
- 34:08SPEAKER_00: either with compaction that is a bit lossy or like this,
- 34:11SPEAKER_00: bring the stuff a bit more into the distribution.
- 34:14SPEAKER_00: Do you even think that we need to push this research line on
- 34:18SPEAKER_01: the raw LM?
- 34:20SPEAKER_01: So I think the answer is probably no,
- 34:25SPEAKER_01: but also it's not a big deal that we are, if that makes sense.
- 34:30SPEAKER_01: So I think the way that I like to do my research or the ideas I put out is
- 34:37SPEAKER_01: that it should never really tell someone, you shouldn't be doing this anymore.
- 34:44SPEAKER_01: This is not a direction that is usable anymore.
- 34:48SPEAKER_01: It's more so like maybe this is not a direction that is needed,
- 34:51SPEAKER_01: but we can still use all of the data and environments and things
- 34:53SPEAKER_01: that you've already generated to help improve even further.
- 34:57SPEAKER_01: So I think this is often the goal of things like data efficiency.
- 35:01SPEAKER_01: So if the only thing that whatever you're proposing provides is
- 35:06SPEAKER_01: that it makes things It makes things cheaper,
- 35:10SPEAKER_01: but not any better than it already is, then I think this is useful.
- 35:14SPEAKER_01: But also, like,
- 35:15SPEAKER_01: ultimately,
- 35:16SPEAKER_01: it doesn't solve the core problem of what we're trying to do,
- 35:18SPEAKER_01: which is like, we want the models to be to be better.
- 35:21SPEAKER_01: And so I think one of the the In that graph,
- 35:25SPEAKER_01: the drawing of the graph that you highlighted,
- 35:28SPEAKER_01: one of the ideas that I was trying to get across,
- 35:30SPEAKER_01: which is purely speculation on my end,
- 35:33SPEAKER_01: is that most tasks in the wild are actually fundamental.
- 35:38SPEAKER_01: A million tokens is probably enough,
- 35:40SPEAKER_01: to be honest, to do this kind of thing where, in general, if we view
- 36:00SPEAKER_01: How humans solve tasks, right?
- 36:03SPEAKER_01: Like humans, we have a limited context window in some sense as well.
- 36:08SPEAKER_01: But the way that we solve very difficult problems often is we build
- 36:12SPEAKER_01: on other people's work,
- 36:13SPEAKER_01: we communicate with each other,
- 36:14SPEAKER_01: we build on each I think this is often true in general for long-running things.
- 36:24SPEAKER_01: And so this is where this kind of hypothesis comes from.
- 36:29SPEAKER_01: And I think, does this mean we should stop Training our models to do long context?
- 36:38SPEAKER_01: No,
- 36:38SPEAKER_01: but I also think that does it mean that maybe we shouldn't continue to invest more
- 36:43SPEAKER_01: and more into this?
- 36:44SPEAKER_01: Maybe yes.
- 36:45SPEAKER_01: Maybe we should invest in alternate ways of scaling up in this way.
- 36:52SPEAKER_00: I know that we were in certain stuff, but it has a massive impact.
- 36:58SPEAKER_00: I don't know if you remember the meta 10 million context length model.
- 37:04SPEAKER_00: They were cooked.
- 37:06SPEAKER_00: They were barely usable.
- 37:07SPEAKER_00: But Needle and A-Stack, they were great at that.
- 37:10SPEAKER_00: But that was kind of useless.
- 37:12SPEAKER_00: And I remember Google did a push on this on the long context I was very excited about.
- 37:19SPEAKER_00: Ultimately,
- 37:19SPEAKER_00: if it's pushing the performance of the model to a breaking point for
- 37:26SPEAKER_00: the inherent physics of the architecture,
- 37:31SPEAKER_00: Maybe it's not that relevant and maybe a bit more of the work on engineering
- 37:39SPEAKER_00: the harness and just having the model being able to drive the harness better,
- 37:45SPEAKER_00: it will be a better use of the time.
- 37:47SPEAKER_00: Just to give an example, right now I have a bunch of consulting projects.
- 37:53SPEAKER_00: One of them is in manufacturing where we are helping.
- 37:56SPEAKER_00: It's like a big time series for manufacturing line.
- 38:01SPEAKER_00: And we're helping figuring out a whole bunch of the stuff with the sensor until at
- 38:05SPEAKER_00: the end,
- 38:05SPEAKER_00: we're able to make an assessment about in this line where things are messed up.
- 38:11SPEAKER_00: But a similar thing is happening in another client in finance.
- 38:15SPEAKER_00: And it's the same team that I have in Boat that is working on this.
- 38:19SPEAKER_00: Boat is a time series.
- 38:20SPEAKER_00: And the context of it is super important.
- 38:23SPEAKER_00: And we make sure that we figure it out and stuff like that.
- 38:26SPEAKER_00: But once we have it, the work in it is not terribly different.
- 38:30SPEAKER_00: It's the same kind of type of stuff that you have to do in the flow of a time stream analysis.
- 38:37SPEAKER_00: And at the end of the day, it's some flavor anomaly detection that we're doing in both.
- 38:42SPEAKER_00: And the same features are appearing in both also.
- 38:46SPEAKER_00: There's some that are more important in this context versus this other context.
- 38:51SPEAKER_00: But when I see my guys and myself thinking about it,
- 38:53SPEAKER_00: I'm like,
- 38:54SPEAKER_00: And I was looking at the RL and I'm like,
- 38:56SPEAKER_00: we're really doing something a bit similar to this, like a task decomposition.
- 39:01SPEAKER_00: And yes,
- 39:01SPEAKER_00: there's a big reasoning effort that we're doing at the beginning
- 39:04SPEAKER_00: to ensure we actually know what we're doing in the context, right?
- 39:07SPEAKER_00: Otherwise, you're going to mess up a bunch of stuff.
- 39:10SPEAKER_00: But once you have that context and it's been validated,
- 39:12SPEAKER_00: the actual mechanic of it inside is not that different.
- 39:16SPEAKER_00: So I would assume, like...
- 39:19SPEAKER_00: Having a model that is able to do something a bit similar to that will make sense.
- 39:24SPEAKER_00: It was just a fair assessment of what we're trying to do here.
- 39:27SPEAKER_01: Yeah, yeah, definitely.
- 39:30SPEAKER_00: So that's for the work that we kind of know what to do, right?
- 39:36SPEAKER_00: Do you think that bringing stuff locally in distribution is enough for making
- 39:43SPEAKER_00: an actual scientific breakthrough, like doing something novel?
- 39:48SPEAKER_00: Because some of these clients were writing a research paper,
- 39:51SPEAKER_00: but we had to push this stuff a bit further up.
- 39:55SPEAKER_00: And in my view, this is starting to look a bit dissimilar in both situations.
- 40:00SPEAKER_00: What's your take on this for bringing stuff into So I will say,
- 40:06SPEAKER_01: what does locally in distribution solve?
- 40:09SPEAKER_01: What it effectively solves is it removes a lot of the inherent limitations
- 40:13SPEAKER_01: of what language models already have.
- 40:16SPEAKER_01: And that is basically like reasoning through longer horizon or longer context problems.
- 40:22SPEAKER_01: In some sense,
- 40:22SPEAKER_01: it's like,
- 40:23SPEAKER_01: One way to think about this is we know language models are super, super smart.
- 40:27SPEAKER_01: They're super good at certain things.
- 40:30SPEAKER_01: What an RLM does and what training around an RLM and getting these locally
- 40:34SPEAKER_01: in distribution things do is it bridges together all the pieces where you're like,
- 40:40SPEAKER_01: wait, why can't a language model not do this?
- 40:42SPEAKER_01: Why is it struggling to do this?
- 40:43SPEAKER_01: I don't struggle to do this.
- 40:45SPEAKER_01: It bridges these parts back or to a place where we're happy, at least in my head.
- 40:53SPEAKER_01: On the topic of novelty,
- 40:54SPEAKER_01: or I'm assuming you're alluding to more auto-research,
- 41:00SPEAKER_01: or can this produce a breakthrough?
- 41:03SPEAKER_01: My answer to this is maybe somewhat independent of my opinion on harnesses itself.
- 41:10SPEAKER_01: Let's take the mismanaged geniuses hypothesis as an example.
- 41:14SPEAKER_01: The central argument of that entire article that I wrote Was that I sort of believe
- 41:24SPEAKER_01: that fundamentally there is some nature of tasks,
- 41:32SPEAKER_01: of human solvable,
- 41:33SPEAKER_01: or like tasks, maybe not human solvable, but tasks that humans care about.
- 41:39SPEAKER_01: There is some structure to them such that actually it turns out
- 41:44SPEAKER_01: that We can compose our existing kind of baseline skill set to solve a lot
- 41:53SPEAKER_01: of these tasks.
- 41:54SPEAKER_01: Now, one thing that I also believe to be true about novel ideas or novel tasks is
- 42:03SPEAKER_01: that people severely underestimate how much these are just a composition
- 42:08SPEAKER_01: of existing ideas.
- 42:11SPEAKER_01: Now, the reason why I think these ideas aren't solved or haven't appeared is often
- 42:16SPEAKER_01: because there aren't enough people in the world who think deeply
- 42:21SPEAKER_01: about these problems or have the right Skill sets to solve them.
- 42:26SPEAKER_01: But if you took the composition of all humans in the world,
- 42:30SPEAKER_01: like all of our combined knowledge,
- 42:32SPEAKER_01: and you compose them correctly, this is effectively what a language model is, right?
- 42:37SPEAKER_01: And composing them correctly is maybe what a trained RLM would do.
- 42:42SPEAKER_01: I think there's many, many tasks that kind of fall through.
- 42:45SPEAKER_01: And you're sort of seeing this in math research at the moment.
- 42:50SPEAKER_01: There's a lot of tasks or proofs that are being solved.
- 42:53SPEAKER_01: Now, I don't claim to be a mathematician of any sort.
- 42:57SPEAKER_01: I'm at the level of maybe a strong undergrad.
- 43:02SPEAKER_01: But I would say that a lot of the things that are being solved, they're very impressive.
- 43:08SPEAKER_01: But oftentimes, they appear to be nothing extraordinary, at least in the proof-solving sense.
- 43:17SPEAKER_01: A lot of them are disproof, actually.
- 43:19SPEAKER_01: Exactly.
- 43:19SPEAKER_01: Yes, a lot of them.
- 43:20SPEAKER_01: Are disproving certain conjectures,
- 43:22SPEAKER_01: like coming up with counterexamples,
- 43:24SPEAKER_01: which not to discount,
- 43:25SPEAKER_01: like I think my point is that if a human were to do this, it's very impressive.
- 43:30SPEAKER_01: But if we're thinking like about what a language model could do,
- 43:32SPEAKER_01: I think we need to think more about it more in the context of like,
- 43:37SPEAKER_01: there is something going on with the actual tasks themselves.
- 43:41SPEAKER_01: And like these are somewhat independent, right?
- 43:44SPEAKER_01: Now on like novelty,
- 43:47SPEAKER_01: there is this question of like,
- 43:49SPEAKER_01: There likely are certain things out there
- 43:52SPEAKER_01: that no matter how much you compose existing knowledge,
- 43:57SPEAKER_01: it's not super clear whether or not we would be able to solve them.
- 44:02SPEAKER_01: And an example of this would be like Take like Einstein's general relativity, right?
- 44:07SPEAKER_01: I mean, for the most part, like, now, I might be wrong about this.
- 44:10SPEAKER_01: But from my understanding is for the most part,
- 44:12SPEAKER_01: like he was directly responsible for for most of the original ideas and even kind
- 44:18SPEAKER_01: of bridging together the whole theory from from special relativity to
- 44:21SPEAKER_01: to general relativity.
- 44:22SPEAKER_01: And, you know, These are cases of like, it's one guy.
- 44:26SPEAKER_01: And I guess you can think of it as like maybe a super, super smart language model.
- 44:29SPEAKER_01: And it's not clear if like composing any existing ideas would ever get you to
- 44:34SPEAKER_01: the conclusion that he came to.
- 44:35SPEAKER_01: I'm not sure.
- 44:37SPEAKER_01: Maybe, maybe not.
- 44:39SPEAKER_01: And there is this fundamental question about what is truly novel.
- 44:44SPEAKER_01: And I don't know.
- 44:46SPEAKER_01: It's funny that you bring this up because this is something that I've wanted
- 44:52SPEAKER_01: to explore once I'm done with all this harness and systems research.
- 44:58SPEAKER_01: I'm very,
- 44:59SPEAKER_01: very curious,
- 45:00SPEAKER_01: independent of language models,
- 45:02SPEAKER_01: do we have a better understanding of the things that we're actually trying to solve?
- 45:08SPEAKER_01: All of this is baked into this hypothesis,
- 45:10SPEAKER_01: but I'm assuming we're going to need stronger evidence to really know about one or
- 45:15SPEAKER_01: the other.
- 45:20SPEAKER_00: There's stuff that you can compose, yes.
- 45:22SPEAKER_00: But there's also just the value of it that is hard.
- 45:27SPEAKER_00: I would say for an alum to know if this has any value.
- 45:32SPEAKER_00: I don't know if you know, but there was a whole field called artificial immune system.
- 45:38SPEAKER_00: Because the field is dead, dead.
- 45:40SPEAKER_00: They did a lot of interesting algorithms and stuff.
- 45:43SPEAKER_00: But the thing is, it was not that useful at the extreme.
- 45:49SPEAKER_00: So like then it dies and like maybe there was some breakthrough over there
- 45:52SPEAKER_00: that are like maybe interesting right now.
- 45:55SPEAKER_00: But it takes,
- 45:56SPEAKER_00: I don't know,
- 45:56SPEAKER_00: like this also that this idea of like understanding what is valuable at
- 46:00SPEAKER_00: a certain point in time, which is very human, right?
- 46:03SPEAKER_00: I feel like a very,
- 46:04SPEAKER_00: very lot of different input goes into this and then you understand
- 46:08SPEAKER_00: a bit like what is the actual population that has some weight or are interested
- 46:15SPEAKER_00: about.
- 46:16SPEAKER_00: Or not, right?
- 46:17SPEAKER_00: There's a lot of researchers that just don't care about anybody.
- 46:20SPEAKER_00: They just care about themselves and what they believe.
- 46:22SPEAKER_00: At some point in time, they might be super right or just forgotten and that's it, right?
- 46:30SPEAKER_00: But going back to this thing here,
- 46:35SPEAKER_00: where you're saying that the genius LLM The Genius Transformer is good,
- 46:41SPEAKER_00: but it's poor at compositional generalization.
- 46:45SPEAKER_00: I instinctively agree here, but why do you think that is?
- 46:50SPEAKER_00: Why do you think that is a case that they're just not great at doing
- 46:56SPEAKER_00: that compositional generalization, even though they've seen billions of tasks?
- 47:01SPEAKER_00: They've seen a lot more than, let's say, I've seen in my lifetime.
- 47:05SPEAKER_01: Yeah.
- 47:06SPEAKER_01: I mean, I think the reason is that, A,
- 47:11SPEAKER_01: you know,
- 47:11SPEAKER_01: there's nothing particularly special about the architect,
- 47:14SPEAKER_01: like the model architecture of the things that we're training is really like,
- 47:18SPEAKER_01: I think there's nothing magic, right?
- 47:20SPEAKER_01: So we shouldn't expect there to be any kind of magic in terms
- 47:24SPEAKER_01: of it learning certain dependencies that we know.
- 47:28SPEAKER_01: But also,
- 47:28SPEAKER_01: I think that There are certain tasks where
- 47:32SPEAKER_01: a model seemingly exhibits compositional generalization.
- 47:35SPEAKER_01: And I think the reason these things often happen is maybe we have tons of data for
- 47:42SPEAKER_01: that particular type of task.
- 47:45SPEAKER_01: I
- 47:49SPEAKER_01: have talked to some people about this and some people who believe
- 47:53SPEAKER_01: that models generalize very strongly to things.
- 47:58SPEAKER_01: And there's many, many, many cases.
- 48:01SPEAKER_01: I think maybe because I've worked in fields where this is just obviously not the case.
- 48:07SPEAKER_01: I think all the benchmarks actually that I've worked on in the past are exactly
- 48:12SPEAKER_01: the problems of this form.
- 48:13SPEAKER_01: So I'll give you an example.
- 48:15SPEAKER_01: Sweebench Multimodal,
- 48:16SPEAKER_01: which was after Sweebench,
- 48:18SPEAKER_01: was basically a variant that focused on adding images and stuff that were
- 48:24SPEAKER_01: in these PRs.
- 48:26SPEAKER_01: And generally,
- 48:27SPEAKER_01: if you expect models to be decent at SweetBench,
- 48:29SPEAKER_01: which they were at the time,
- 48:31SPEAKER_01: and also these multimodal models to be able to parse images,
- 48:34SPEAKER_01: which they also can,
- 48:35SPEAKER_01: you would expect that solving an image version of a SweetBench task is not
- 48:40SPEAKER_01: that hard.
- 48:41SPEAKER_01: But actually, it's really hard.
- 48:43SPEAKER_01: It would basically not solve any of them.
- 48:45SPEAKER_01: And then we moved on to KernelBench,
- 48:47SPEAKER_01: which is like, okay,
- 48:48SPEAKER_01: Writing GPU kernels,
- 48:49SPEAKER_01: which again is not like,
- 48:51SPEAKER_01: you know,
- 48:51SPEAKER_01: that much more difficult than anything a performance engineer knows how to do,
- 48:55SPEAKER_01: let's say in C or C++ for like the CPU.
- 48:59SPEAKER_01: Actually, I would argue it's easier.
- 49:02SPEAKER_01: And the model has It has access to all this data.
- 49:06SPEAKER_01: You could argue, why can't it generalize after reading the documentation?
- 49:09SPEAKER_01: And it's still a problem to this day.
- 49:13SPEAKER_01: And then there was video game bench.
- 49:16SPEAKER_01: Why can they not solve these super simple games
- 49:18SPEAKER_01: when they can solve these very hard decision-making tasks?
- 49:23SPEAKER_01: And I think it's one of those things
- 49:27SPEAKER_01: where models learn almost the naivest thing possible.
- 49:32SPEAKER_01: And it turns out that like,
- 49:34SPEAKER_01: for many tasks,
- 49:35SPEAKER_01: like we have a sufficient amount of data where
- 49:37SPEAKER_01: the naivest thing possible is just generalizing to that setting.
- 49:41SPEAKER_01: But this just obviously isn't true for a lot of the tasks that we actually care about.
- 49:46SPEAKER_01: And I think this is why also like,
- 49:49SPEAKER_01: we still have this issue of like, can we trust language models to do certain jobs?
- 49:56SPEAKER_01: Like, can we trust them to write clean I'm
- 50:03SPEAKER_01: not just saying this from my own experience.
- 50:08SPEAKER_01: Clearly, companies are having this problem.
- 50:10SPEAKER_01: This is why they're pulling back on all of this AI coding stuff.
- 50:15SPEAKER_01: Fundamentally, it works, but it doesn't work.
- 50:22SPEAKER_01: We have had this problem on so many different domains with these language models.
- 50:30SPEAKER_01: It's sort of clear to me that we need something more.
- 50:35SPEAKER_01: We need a way to almost force the dumb language model to not have this problem.
- 50:43SPEAKER_01: And I think this is where this locally in distribution kind
- 50:47SPEAKER_01: of task isomorphism stuff comes in of like,
- 50:51SPEAKER_01: if every single language model is just solving a thing it knows how to do,
- 50:54SPEAKER_01: like we know it knows how to do this thing,
- 50:56SPEAKER_01: but the broader system is solving something new, like Then we're good.
- 51:01SPEAKER_01: Again, I go back to this analogy all the time.
- 51:03SPEAKER_01: This is how a company works.
- 51:05SPEAKER_01: You don't expect every single person in the company to know how to do everything in the company.
- 51:09SPEAKER_01: Not even the CEO will know how to do what some of the lower-level engineers are doing.
- 51:14SPEAKER_01: And that's fine.
- 51:16SPEAKER_01: That's how these things operate.
- 51:18SPEAKER_01: And I think we can draw a lot of analogies for
- 51:24SPEAKER_01: how we solve these kinds of tasks.
- 51:28SPEAKER_00: Almost like the Frontier LM are just like these idiot seven, right?
- 51:36SPEAKER_00: They know a lot of stuff, but they are barely functional.
- 51:39SPEAKER_00: And then you have just like the, I don't know, like the...
- 51:43SPEAKER_00: The manager on top of it is not an expert and just like,
- 51:46SPEAKER_00: yeah, now you have to go to the conference.
- 51:48SPEAKER_00: I don't want to go.
- 51:48SPEAKER_00: You have to talk there, right?
- 51:50SPEAKER_00: So go ahead and make it.
- 51:52SPEAKER_00: I don't want to make the slides.
- 51:52SPEAKER_00: Okay.
- 51:53SPEAKER_00: We're going to make the slide together.
- 51:54SPEAKER_00: So like, like, I don't know, it feels like this, this, this back and forth with the system.
- 51:59SPEAKER_00: And, uh, I mean, it's,
- 52:00SPEAKER_00: it's apparent when you,
- 52:01SPEAKER_00: I don't know, uh,
- 52:03SPEAKER_00: even with the harness sometimes,
- 52:04SPEAKER_00: like you can see,
- 52:04SPEAKER_00: like you can peek through it and you're like, okay, man, what the hell is happening?
- 52:08SPEAKER_00: But just the raw LLM.
- 52:10SPEAKER_00: Sometimes when you actually decide to go and then use,
- 52:14SPEAKER_00: let's say,
- 52:15SPEAKER_00: I don't know,
- 52:16SPEAKER_00: like the raw version of Kimi without any hardness and you hit a specific point,
- 52:21SPEAKER_00: you're like, yeah, this should be solved already.
- 52:23SPEAKER_00: Cloud is able to do it.
- 52:25SPEAKER_00: But you realize that you had the hardness and then you see it naked like this.
- 52:29SPEAKER_00: And you actually are able to see that when you hit the out of distribution from
- 52:34SPEAKER_00: the data, It's just not that great, right?
- 52:36SPEAKER_00: But like,
- 52:37SPEAKER_00: now we're at the point where we're talking about the fundamental core of
- 52:42SPEAKER_00: the foundation of the whole field,
- 52:43SPEAKER_00: which is like the neural operation that are in there,
- 52:46SPEAKER_00: like the transformer and all this.
- 52:48SPEAKER_00: If we remove the R-ness from the equation, what do you think is missing in this soup, right?
- 52:54SPEAKER_00: I don't like it's like an answer that doesn't,
- 52:56SPEAKER_00: like a question doesn't have an answer,
- 52:58SPEAKER_00: like otherwise you could have been working on it.
- 53:00SPEAKER_00: But like, what's your rough, And is there even something missing?
- 53:05SPEAKER_00: Or are we trying to generate something that is not generable as part of the stack?
- 53:15SPEAKER_01: Yeah, I think the fundamental issue, maybe controversial, is the transformer.
- 53:19SPEAKER_01: Ultimately, at the end of the day, what is the difference between a model and a harness?
- 53:26SPEAKER_01: To be honest, it's not much.
- 53:28SPEAKER_01: You could embed a lot of the design of a harness into a model architecture.
- 53:32SPEAKER_01: The thing that maybe you can't embed is a tool call,
- 53:35SPEAKER_01: because a tool call is something external,
- 53:37SPEAKER_01: like a search tool or interacting with your code REPL or something.
- 53:42SPEAKER_01: But in some sense,
- 53:45SPEAKER_01: I think the reason harnesses have taken off or the reason Claude code
- 53:51SPEAKER_01: and all these things exist in the first place is you have this observability layer,
- 53:56SPEAKER_01: which is the text.
- 53:57SPEAKER_01: We know what's going on in the text and we know the model roughly knows what's going
- 54:03SPEAKER_01: on in the text.
- 54:04SPEAKER_01: And so we like this back and forth.
- 54:08SPEAKER_01: None of this is necessary if you really think about it.
- 54:11SPEAKER_01: I like to think in terms of assumptions and then whether or not we have
- 54:17SPEAKER_01: to dig deeper into those assumptions.
- 54:19SPEAKER_01: So in the design of an RLM,
- 54:21SPEAKER_01: the assumption that we can make is that a language model is quite smart
- 54:24SPEAKER_01: and it can write code.
- 54:26SPEAKER_01: And if we kind of black box this language model,
- 54:29SPEAKER_01: it's just this thing that we're going to use And maybe make it better
- 54:33SPEAKER_01: with compositional generalization.
- 54:35SPEAKER_01: So that informs the design of the RLM.
- 54:38SPEAKER_01: But if you think about it deeper,
- 54:40SPEAKER_01: which is sort of your question,
- 54:41SPEAKER_01: which is like, what is wrong with the language model itself?
- 54:44SPEAKER_01: Why do we need the RLM in the first place?
- 54:47SPEAKER_01: This just goes back to like, well, now this is a model architecture question.
- 54:51SPEAKER_01: This is why I think harness and architecture are way more linked than people think
- 54:56SPEAKER_01: because anything you can do with a harness, you can just call it the architect.
- 55:06SPEAKER_01: There's nothing fundamentally unique about what a harness is doing.
- 55:10SPEAKER_01: I think one of the things that I am interested in,
- 55:15SPEAKER_01: I guess I can't say too much What parts of the RLM,
- 55:22SPEAKER_01: this fundamental design, can you move directly into the model architecture?
- 55:26SPEAKER_01: And in doing so,
- 55:27SPEAKER_01: there's obviously a lot of benefits in that we can skip a lot of the steps
- 55:31SPEAKER_01: that we otherwise would have needed to do with the RLM,
- 55:33SPEAKER_01: or producing decoding tokens,
- 55:36SPEAKER_01: doing the whole serving engine thing of decoding tokens, going back, whatever.
- 55:40SPEAKER_01: And this is a really,
- 55:42SPEAKER_01: really interesting question in general around What is the relationship between
- 55:48SPEAKER_01: a harness and an architecture?
- 55:49SPEAKER_01: Can you go back and forth between them?
- 55:52SPEAKER_01: And can you train stronger models with an alternate architecture?
- 55:55SPEAKER_01: It's a very interesting question.
- 55:57SPEAKER_00: Now that I have these two pieces in my mind where I'm looking
- 56:00SPEAKER_00: at like you have this idiot servant there that if you give it like
- 56:04SPEAKER_00: a well-bounded task, you will just be able to do it, which is great.
- 56:07SPEAKER_00: It's not something bad.
- 56:09SPEAKER_00: This kind of gigantic database of functions that can be recombined.
- 56:12SPEAKER_00: If you hit the right function in the right order, it's going to do
- 56:15SPEAKER_01: fine, right?
- 56:15SPEAKER_01: And you can do it in context.
- 56:16SPEAKER_00: And on the other side,
- 56:17SPEAKER_00: you have this kind of driver that is managing the composition generalization,
- 56:23SPEAKER_00: which is now the same model, but within RLM.
- 56:27SPEAKER_00: And my mind goes into like, what would be then a very good composition generalizer, right?
- 56:33SPEAKER_00: Directly, right?
- 56:34SPEAKER_00: Without, yes, maybe like there's still having the R-ness, right?
- 56:37SPEAKER_00: I mean, I think there's two stages to it.
- 56:39SPEAKER_00: There's one stage where what would be a very good thing to do in order to have
- 56:43SPEAKER_00: a very good driver?
- 56:44SPEAKER_00: Does it have to be the same guys?
- 56:46SPEAKER_00: Do you have to have the same sub-agent being the thing that is driving the thing, right?
- 56:52SPEAKER_00: Or you can have something different, right?
- 56:54SPEAKER_00: Maybe faster,
- 56:54SPEAKER_00: maybe like,
- 56:55SPEAKER_00: Maybe like that is trained in a certain way,
- 56:58SPEAKER_00: has like different characteristics altogether, right?
- 57:01SPEAKER_00: That is very,
- 57:01SPEAKER_00: very good at orchestrating that work and like deferring it without even needing
- 57:05SPEAKER_00: to necessarily like doing any of the tasks, right?
- 57:08SPEAKER_00: Because why would you need to do that?
- 57:10SPEAKER_00: Why would you need to do the task if you already have all these agents that are able
- 57:13SPEAKER_00: to be called and then just be able to do tasks?
- 57:16SPEAKER_00: That's kind of how I'm looking at it.
- 57:19SPEAKER_00: From the two ways of going about it,
- 57:21SPEAKER_00: just training the existing model to be good at driving that RLM harness,
- 57:28SPEAKER_00: or having complicated different models that is now better at driving some version
- 57:34SPEAKER_00: of this RLM that may be different than this first one.
- 57:37SPEAKER_00: Which part do you see that has the most promise in your opinion?
- 57:40SPEAKER_01: So I think the first thing that we should always do the easy thing first,
- 57:45SPEAKER_01: which is just train whatever Whatever,
- 57:48SPEAKER_01: GPT-6, whatever, and have it be usable as an RLM.
- 57:51SPEAKER_01: I suppose this is what Prime Intellect is saying that they're trying to do.
- 57:56SPEAKER_01: Just train a model that can act as an RLM,
- 57:58SPEAKER_01: and this should be somewhat sufficient for doing all of the decompositional things
- 58:03SPEAKER_01: that we want to do.
- 58:06SPEAKER_01: The next step,
- 58:07SPEAKER_01: assuming this works,
- 58:09SPEAKER_01: is you now need to start thinking about either
- 58:12SPEAKER_01: the approach is you have specialized sub-agents,
- 58:15SPEAKER_01: and the way that you would do this from a serving perspective,
- 58:19SPEAKER_01: because this is, I guess, how you should always think about it.
- 58:21SPEAKER_01: Like, like how is this actually going to be deployed is you're probably going
- 58:25SPEAKER_01: to have like some base model maybe,
- 58:28SPEAKER_01: and then maybe some like,
- 58:30SPEAKER_01: I don't know,
- 58:30SPEAKER_01: adapters or like lower variants of each of the sub agents you want to do,
- 58:35SPEAKER_01: or, or you just have completely different models for each of the sub agents.
- 58:39SPEAKER_01: That's, you know, how I don't know, but this is like sort of.
- 58:42SPEAKER_01: I mean, are we?
- 59:02SPEAKER_00: Are we already doing that sort of stuff,
- 59:04SPEAKER_00: let's say with the speculative decoding and things
- 59:06SPEAKER_00: that we have like multiple agents,
- 59:09SPEAKER_00: a smaller one that is purpose-made to be able to do this stuff fast?
- 59:13SPEAKER_00: It doesn't seem that we're already going somewhere into that direction.
- 59:18SPEAKER_01: Yeah.
- 59:19SPEAKER_01: Yeah.
- 59:19SPEAKER_01: So kind of, although I will say that like, at least with the speculative stuff, right?
- 59:26SPEAKER_01: Like there is sort of a difference in like,
- 59:29SPEAKER_01: you don't,
- 59:30SPEAKER_01: you don't just trust the small model in any like speculative decoding sense, right?
- 59:34SPEAKER_01: Like you still need to re like reference the big model to confirm that,
- 59:38SPEAKER_01: like the, I mean,
- 59:39SPEAKER_01: I guess I don't want to get too into this,
- 59:41SPEAKER_01: but like,
- 59:41SPEAKER_01: I guess for important sampling sake,
- 59:43SPEAKER_01: like you need to make sure that the distributions are the same.
- 59:47SPEAKER_01: But more of what I'm alluding to is that you do want the different models.
- 59:52SPEAKER_01: You do want the small models.
- 59:54SPEAKER_01: I guess without giving away too much.
- 59:56SPEAKER_01: You do want the small models,
- 59:57SPEAKER_01: but you want a way to do it where you're not giving up any of what you were doing
- 1:00:03SPEAKER_01: before.
- 1:00:04SPEAKER_01: So what I mean by this is like, okay.
- 1:00:07SPEAKER_01: If you think of this, again,
- 1:00:08SPEAKER_01: from like,
- 1:00:08SPEAKER_01: serving entry perspective,
- 1:00:09SPEAKER_01: I have this big model, and maybe I have this small draft model.
- 1:00:12SPEAKER_01: And now I want now it's more complicated,
- 1:00:14SPEAKER_01: where I want to do,
- 1:00:15SPEAKER_01: like many,
- 1:00:16SPEAKER_01: many recursive calls of like,
- 1:00:17SPEAKER_01: I want to invoke language models,
- 1:00:19SPEAKER_01: we want to think of a way where basically,
- 1:00:23SPEAKER_01: we're only serving one model, technically.
- 1:00:27SPEAKER_01: But what actually is going on under the hood is like,
- 1:00:30SPEAKER_01: we have some way of allocating more or less compute to This one model,
- 1:00:36SPEAKER_01: whether it be only through a part of the model or Like
- 1:00:40SPEAKER_01: if you do like looped transformer stuff or whatever,
- 1:00:44SPEAKER_01: like less iterations of the model,
- 1:00:46SPEAKER_01: so that when you do sub agents and all these kinds of things,
- 1:00:49SPEAKER_01: you're technically using different models or like different specialized models
- 1:00:52SPEAKER_01: for different things.
- 1:00:54SPEAKER_01: But it's, it all fits into our general existing framework of like how we serve models.
- 1:01:01SPEAKER_01: And the reason why this is so powerful is that like,
- 1:01:04SPEAKER_01: you can do sub agent things Much quicker and much more efficiently
- 1:01:11SPEAKER_01: than what we currently do, which is we treat everything independent.
- 1:01:15SPEAKER_01: Like we treat a subagent as completely independent from the main model.
- 1:01:19SPEAKER_01: Now that's all, I guess that is most of what I will say on like how I think about this problem.
- 1:01:23SPEAKER_01: But in general,
- 1:01:24SPEAKER_01: this starts to play into model architecture design as well,
- 1:01:27SPEAKER_01: because you start to ask questions of like,
- 1:01:29SPEAKER_01: you know,
- 1:01:31SPEAKER_01: can you design the architecture in a way that it makes things like subagents
- 1:01:35SPEAKER_01: a lot easier?
- 1:01:37SPEAKER_01: I think this is something that people aren't really thinking about.
- 1:01:41SPEAKER_01: And it's a really interesting question because sub-agents like,
- 1:01:46SPEAKER_01: you know,
- 1:01:47SPEAKER_01: People will say subagents have been around for a long time,
- 1:01:50SPEAKER_01: blah, blah, blah, blah, blah.
- 1:01:51SPEAKER_01: But it's like they've been around for a long time and we are incredibly uncreative
- 1:01:55SPEAKER_01: about how we deal with them.
- 1:01:57SPEAKER_01: We still treat it like it's just a tool.
- 1:02:00SPEAKER_01: But we have way more guarantees and much more structural similarities between
- 1:02:06SPEAKER_01: the subagents and the main agent.
- 1:02:08SPEAKER_01: And we don't exploit them currently.
- 1:02:10SPEAKER_01: So this is, you know.
- 1:02:12SPEAKER_00: Right now,
- 1:02:13SPEAKER_00: we're training the systems,
- 1:02:15SPEAKER_00: the frontier system with the harness and this sub-agent element in mind,
- 1:02:19SPEAKER_00: but we're just putting them in the waste.
- 1:02:22SPEAKER_00: We haven't necessarily, well,
- 1:02:24SPEAKER_00: not that I know of, took the idea and ingested it into the architecture in some way.
- 1:02:30SPEAKER_00: So that's kind of the next step.
- 1:02:33SPEAKER_00: A bit like what we've done from going From the dense model to the MOEs
- 1:02:39SPEAKER_00: for multiple reasons,
- 1:02:39SPEAKER_00: but one of them is just literally serving them properly and using them efficiently,
- 1:02:46SPEAKER_00: which is pretty interesting.
- 1:02:48SPEAKER_00: But let's take a step back.
- 1:02:50SPEAKER_00: We don't want to do that.
- 1:02:52SPEAKER_00: Let's say we don't want to go ahead and change the architecture.
- 1:02:55SPEAKER_00: If we were taking this core-based architecture,
- 1:02:58SPEAKER_00: how early do you think in the training pipeline do we need to...
- 1:03:02SPEAKER_00: Because right now in this analysis, you post-train the Quen 30B.
- 1:03:08SPEAKER_00: Already in the pipeline,
- 1:03:09SPEAKER_00: let's say,
- 1:03:10SPEAKER_00: of Quen,
- 1:03:13SPEAKER_00: Would you put the RLM harness training in order for it to be even better?
- 1:03:18SPEAKER_01: So I would say that based on the mismanaged geniuses hypothesis,
- 1:03:25SPEAKER_01: we want to do this as late as possible because we want to rely on the assumption
- 1:03:34SPEAKER_01: that we know pre-training and all these things make the models very strong.
- 1:03:40SPEAKER_01: At various local tasks, including coding.
- 1:03:46SPEAKER_01: I think actually,
- 1:03:47SPEAKER_01: and this is one of the things I'm very excited about with RLMs and harnesses
- 1:03:51SPEAKER_01: in general.
- 1:03:53SPEAKER_01: It's this idea that like these systems don't require that much more
- 1:03:59SPEAKER_01: from what we already have to get to the point where we're happy.
- 1:04:05SPEAKER_01: I think like This again is more speculation.
- 1:04:09SPEAKER_01: It's not,
- 1:04:10SPEAKER_01: I have no proof for the fact that this is the case,
- 1:04:13SPEAKER_01: but this is the assumption that I've been going off of in a lot of this work.
- 1:04:17SPEAKER_00: Let's say we changed the architecture.
- 1:04:20SPEAKER_00: Now it's a more native type of driver, right?
- 1:04:24SPEAKER_00: And he's able to kind of probably do sub-agent.
- 1:04:27SPEAKER_00: What type of capability you think is going to show up, right?
- 1:04:31SPEAKER_00: That is not stuff that we're already seeing here.
- 1:04:34SPEAKER_00: Let's say when we just go pedal to the floor and put as much token as possible.
- 1:04:39SPEAKER_01: I think the main things are long context is no longer a thing.
- 1:04:43SPEAKER_01: We no longer have like 256k token context window, whatever.
- 1:04:48SPEAKER_01: And long horizon, which is maybe the bigger thing.
- 1:04:52SPEAKER_01: Being able to run for a very long time and not lose its sense of what it's doing.
- 1:04:59SPEAKER_01: I guess you can tie this to things like continual learning,
- 1:05:01SPEAKER_01: whatever, recursive self-improvement.
- 1:05:06SPEAKER_01: All of these ideas seem a little hand-wavy to me, but I think Long Horizon is the next big step.
- 1:05:11SPEAKER_01: Just not having this issue of And I think even for like real production use cases,
- 1:05:16SPEAKER_01: this is actually the missing thing for a lot of people.
- 1:05:18SPEAKER_01: I just want an agent that reliably runs for a long time.
- 1:05:22SPEAKER_01: Nothing really tests this well.
- 1:05:25SPEAKER_01: And it's very hard to train this.
- 1:05:27SPEAKER_01: And I think that like, to me, actually, this is the simplest way to do it.
- 1:05:33SPEAKER_01: Like you don't,
- 1:05:33SPEAKER_01: one of the things I wanted to highlight in the blog post is that we only train
- 1:05:37SPEAKER_01: on short things.
- 1:05:39SPEAKER_01: This is really a key part of it is you generalize to longer and harder tasks
- 1:05:46SPEAKER_01: by only trading on shorter versions.
- 1:05:49SPEAKER_01: And this is really something we just want to go for.
- 1:05:53SPEAKER_00: Yeah.
- 1:05:53SPEAKER_00: And then you can spend,
- 1:05:54SPEAKER_00: I don't know, it feels like it snowballs to things that are just simpler.
- 1:06:00SPEAKER_00: You don't need as much because your smaller tasks are of higher quality
- 1:06:04SPEAKER_00: since you spend more time on them.
- 1:06:06SPEAKER_00: So you're more sure which part of the different element and tasks
- 1:06:11SPEAKER_00: that are generally used are there.
- 1:06:14SPEAKER_00: You can spend more token on that and then just more of the compute.
- 1:06:18SPEAKER_00: And you may not even need to have that gigantic model if you're training it on the right stuff.
- 1:06:25SPEAKER_00: I still think the main network architecture could benefit from having
- 1:06:27SPEAKER_00: an explicit output thinking tensor that could be used
- 1:06:30SPEAKER_00: to automatically grab things relevant to that thought from a vector DB or something.
- 1:06:35SPEAKER_00: Something external.
- 1:06:37SPEAKER_00: I think a lot of people are reaching for this type of stuff.
- 1:06:40SPEAKER_00: We need to change something about the architecture.
- 1:06:45SPEAKER_00: Generally, how would you justify that?
- 1:06:47SPEAKER_00: Let's say you want to go ahead and do that sort of change to an obvious sort of impact.
- 1:06:54SPEAKER_00: How would you go about actually doing it?
- 1:06:58SPEAKER_00: In order to make an impact at this point,
- 1:07:00SPEAKER_00: because my view is that you will need to have either a very,
- 1:07:04SPEAKER_00: very good idea that is very,
- 1:07:06SPEAKER_00: very clear cut in a small setup, or literally you go and test it into the big stuff.
- 1:07:14SPEAKER_00: So how do you go about it?
- 1:07:15SPEAKER_01: My view on this is very similar,
- 1:07:19SPEAKER_01: I guess,
- 1:07:19SPEAKER_01: to what I mentioned before about think about what assumptions you're making and
- 1:07:23SPEAKER_01: then think about how you break those assumptions.
- 1:07:26SPEAKER_01: I will say the first thing to always do,
- 1:07:28SPEAKER_01: especially because the language models are so powerful now,
- 1:07:33SPEAKER_01: is you want to think about how you would design this in a harness first.
- 1:07:38SPEAKER_01: So for example,
- 1:07:39SPEAKER_01: in the thing that they're mentioning about this thinking token, I mean,
- 1:07:42SPEAKER_01: this sounds very rag-like to me,
- 1:07:44SPEAKER_01: but you could imagine that you could probably implement something like this with
- 1:07:48SPEAKER_01: an existing language model in some harness.
- 1:07:50SPEAKER_01: And as a baseline, if this doesn't work, then...
- 1:07:54SPEAKER_01: The chance of this working for a raw model architecture thing is honestly also quite low.
- 1:08:02SPEAKER_01: I think this is also like with RLM stuff similarly,
- 1:08:04SPEAKER_01: like ultimately what we're going to want to do with RLM stuff is change it so that
- 1:08:09SPEAKER_01: the RLM becomes the,
- 1:08:12SPEAKER_01: like in designing your model architecture,
- 1:08:14SPEAKER_01: if an RLM works, you can already just make it an RLM for the most part.
- 1:08:17SPEAKER_01: Now there's some quirks there for like why that maybe wouldn't directly work.
- 1:08:22SPEAKER_01: And so I think that like the other thing to consider Is that,
- 1:08:26SPEAKER_01: you know, we have pre-training scaling laws.
- 1:08:29SPEAKER_01: Now you could assume that they just hold for any type of architecture change.
- 1:08:35SPEAKER_01: You could also make the assumption that like they only hold because all
- 1:08:39SPEAKER_01: of our existing architecture designs are similar enough
- 1:08:42SPEAKER_01: and like whatever you're proposing is so different
- 1:08:44SPEAKER_01: that like It wouldn't follow these scaling walls.
- 1:08:50SPEAKER_01: But generally,
- 1:08:52SPEAKER_01: the point I'm trying to make here is I think this is a problem we often run into
- 1:08:56SPEAKER_01: in science, especially in AI.
- 1:08:58SPEAKER_01: You only have so many limited resources for what you can do.
- 1:09:02SPEAKER_01: Even I have very limited resources for what I can do.
- 1:09:06SPEAKER_01: So the most that you can really do is think about what is the thing that I could do
- 1:09:10SPEAKER_01: or the simplest thing that I could do.
- 1:09:13SPEAKER_01: That would maybe convince somebody else or maybe convince somebody else to give me
- 1:09:18SPEAKER_01: the compute to run.
- 1:09:21SPEAKER_01: Something that leans closer into this larger assumption that you want to make.
- 1:09:28SPEAKER_01: And eventually,
- 1:09:29SPEAKER_01: maybe this propagates down to OpenAI or Anthropic or something,
- 1:09:31SPEAKER_01: and they implement it.
- 1:09:33SPEAKER_01: Now, obviously,
- 1:09:34SPEAKER_01: that's not exactly what you would want,
- 1:09:36SPEAKER_01: but I think with a lot of ideas around model architecture,
- 1:09:41SPEAKER_01: we have this lovely idea that harnesses are not so different,
- 1:09:45SPEAKER_01: and so you can use it as a testbed for almost anything.
- 1:09:49SPEAKER_01: And just test out if even either post-training or no training at all would help.
- 1:09:52SPEAKER_00: Yeah.
- 1:09:54SPEAKER_00: What do you both think about long persistent agent like AdLong Project from Load Institute?
- 1:09:59SPEAKER_00: Obviously AdLong quote RLM as an inspiration,
- 1:10:02SPEAKER_00: but just wanted Alex's opinion on what it can be used for.
- 1:10:04SPEAKER_00: I think you're a part of that, right?
- 1:10:07SPEAKER_00: Talk to us a bit about Load.
- 1:10:09SPEAKER_01: For context, yes, I played a very, very minor role in Headlong.
- 1:10:15SPEAKER_01: I should say, firstly, that Laud is amazing.
- 1:10:19SPEAKER_01: The Laud Institute,
- 1:10:20SPEAKER_01: for any students that are watching this right now, they help fund my research.
- 1:10:28SPEAKER_01: I would think of them as a philanthropy-type thing.
- 1:10:34SPEAKER_01: They specifically sponsor students to help you with your research.
- 1:10:39SPEAKER_01: They help me a lot with RLMs, and especially with the blog posts, I also reference them.
- 1:10:43SPEAKER_01: They help me with the compute to get one or two nodes of H200.
- 1:10:49SPEAKER_01: It's run by Andy Kudwinski,
- 1:10:53SPEAKER_01: who is one of the Databricks co-founders,
- 1:10:55SPEAKER_01: and now he's just kind of working on this or helping out students full-time.
- 1:10:58SPEAKER_01: Anyways, on the headlong thing, headlong is, I would say It is an RLM.
- 1:11:04SPEAKER_01: I think the innovation of it, though, comes much further than the RLM part.
- 1:11:11SPEAKER_01: I'm very glad that this is the case,
- 1:11:13SPEAKER_01: because one of the things that I want people to understand with RLMs is
- 1:11:16SPEAKER_01: that it's such a simple idea.
- 1:11:18SPEAKER_01: You should be using your own ideas on top of RLMs to build something much greater.
- 1:11:25SPEAKER_01: I appreciate that they reference it so much,
- 1:11:28SPEAKER_01: but it's only a small piece of a larger thing they're doing.
- 1:11:32SPEAKER_01: Um, which is they are changing,
- 1:11:35SPEAKER_01: like this agent is a very different design than how typical agents, uh, and as,
- 1:11:39SPEAKER_01: as I have complained about this entire kind of, uh,
- 1:11:42SPEAKER_01: talk or like, uh,
- 1:11:44SPEAKER_01: kind of chat,
- 1:11:44SPEAKER_01: like I think existing agents are a result of a lot of technical debt in the field
- 1:11:50SPEAKER_01: of like just laziness and how we design things around language models.
- 1:11:54SPEAKER_01: Um, So this agent basically does this like persistent agency thing,
- 1:11:59SPEAKER_01: which is like it runs forever.
- 1:12:01SPEAKER_01: Basically,
- 1:12:02SPEAKER_01: like you can think of it as like a Slack bot that kind of runs forever
- 1:12:05SPEAKER_01: and will like incrementally think about things.
- 1:12:07SPEAKER_01: Like it's not just like you talk to it and then it stays idle.
- 1:12:11SPEAKER_01: Like it thinks about the things that you are giving it and the way that it.
- 1:12:16SPEAKER_01: Stores this information in an RLM type way, but it basically will come back to you with things.
- 1:12:23SPEAKER_01: It's kind of like an assistant of sort that is always running,
- 1:12:27SPEAKER_01: but also it's not super expensive.
- 1:12:29SPEAKER_01: They have a very smart way of making sure that it's a reasonable token cost
- 1:12:33SPEAKER_01: and everything, but generally it's a more realistic thing.
- 1:12:38SPEAKER_01: Assistant type agent that does not just react to whatever you ask it.
- 1:12:43SPEAKER_01: It is like persistently thinking.
- 1:12:45SPEAKER_01: And I think it's definitely like a prototype type agent, but very, very cool design.
- 1:12:51SPEAKER_01: Like I think, you know, people should try it.
- 1:12:54SPEAKER_01: You know, obviously give them feedback as well.
- 1:12:56SPEAKER_01: They're doing a lot of really cool work.
- 1:12:59SPEAKER_01: And yeah.
- 1:13:00SPEAKER_00: What I like about the RLM is that I see it in a lot of stuff at the moment
- 1:13:05SPEAKER_00: where people are like, hey, this is good, but we can make it better.
- 1:13:08SPEAKER_00: It was Slate.
- 1:13:09SPEAKER_00: Even Prime Agent,
- 1:13:11SPEAKER_00: there's the RLM idea,
- 1:13:12SPEAKER_00: but then there's the continuous R-ness idea that is mixed into it.
- 1:13:15SPEAKER_00: And then Knack is Slate.
- 1:13:17SPEAKER_00: And from RC, it's like Slate, that is a bit modified and stuff.
- 1:13:22SPEAKER_00: I like how these two core ideas that you put into the RLM are having that much of an impact.
- 1:13:28SPEAKER_00: It's pretty cool.
- 1:13:29SPEAKER_00: Really fucking cool.
- 1:13:31SPEAKER_00: I like that you have this train eval and you can look at it like, yeah,
- 1:13:35SPEAKER_00: I know it's not necessarily exactly token for token,
- 1:13:40SPEAKER_00: but like, It's pretty fucking similar in there, right?
- 1:13:43SPEAKER_00: But then you can see a bit this idea of like compositional generalization
- 1:13:47SPEAKER_00: that is expressed through this RAPL,
- 1:13:50SPEAKER_00: which is like at the end of the day, the code, right?
- 1:13:53SPEAKER_00: Do you think here,
- 1:13:55SPEAKER_00: because this is for text, this is for like these
- 1:13:57SPEAKER_00: sorts of tasks that we're running.
- 1:13:59SPEAKER_00: Do you think that there's an RAPL for other modalities, let's say like vision or time series?
- 1:14:06SPEAKER_00: Or is it even needed at this point?
- 1:14:09SPEAKER_00: Do you even need other things to do this kind of compositional generalization other than code?
- 1:14:16SPEAKER_01: I do think that code itself in text form,
- 1:14:20SPEAKER_01: I mean, I guess code is only represented in text form currently.
- 1:14:24SPEAKER_01: Is the most important thing.
- 1:14:27SPEAKER_01: And I think with like vision and time series and all these things,
- 1:14:31SPEAKER_01: like you can represent them in code or like you can represent the way
- 1:14:36SPEAKER_01: that we would talk about some of these things in code.
- 1:14:39SPEAKER_01: And so like,
- 1:14:41SPEAKER_01: I don't know currently what exactly the equivalent of an RLM type thing would be
- 1:14:47SPEAKER_01: for like a vision only model.
- 1:14:48SPEAKER_01: And like maybe such a thing just like doesn't exist.
- 1:14:51SPEAKER_01: I'm not sure.
- 1:14:52SPEAKER_01: But yeah.
- 1:14:53SPEAKER_01: Yeah, this is a really interesting point to bring up because it's,
- 1:14:58SPEAKER_01: you know,
- 1:14:58SPEAKER_01: potentially, yes,
- 1:15:00SPEAKER_01: potentially there's maybe something really,
- 1:15:01SPEAKER_01: really interesting there, but I'm
- 1:15:04SPEAKER_00: just not sure.
- 1:15:06SPEAKER_00: I say this because I would love to use LM in my time series work, right?
- 1:15:11SPEAKER_00: But it literally is dumb as rock, right?
- 1:15:14SPEAKER_00: And it just doesn't understand the time series that well.
- 1:15:18SPEAKER_00: I can use it to kind of help me with the coding and all the stuff and some of
- 1:15:23SPEAKER_00: the analysis as soon as I'm done doing the things.
- 1:15:26SPEAKER_00: But just to natively just shove it, the stuff, and expecting a good output out of it.
- 1:15:32SPEAKER_00: Kind of hard.
- 1:15:33SPEAKER_00: Same thing with vision.
- 1:15:35SPEAKER_00: It has all of this kind of intricacy.
- 1:15:37SPEAKER_00: But I think my bigger question is that do you think that it's going
- 1:15:41SPEAKER_00: to be necessarily a different harness around...
- 1:15:46SPEAKER_00: Okay, let's say two choices.
- 1:15:48SPEAKER_00: A different harness around a very specific model or a different harness around
- 1:15:54SPEAKER_00: the same gigantic, let's say, GPT-7 or whatever it is.
- 1:15:58SPEAKER_00: That is good at everything and that is specifically good at coding.
- 1:16:01SPEAKER_00: Do you think it's going to go this way or the first way?
- 1:16:04SPEAKER_01: I still think it will go the second way.
- 1:16:09SPEAKER_01: Unfortunately, at least, I mean, let's say it will go the second way in the short term.
- 1:16:13SPEAKER_01: In the long term, I have no idea.
- 1:16:14SPEAKER_01: Because I think in the long term, a lot of things will change.
- 1:16:17SPEAKER_01: Like, there will be a lot...
- 1:16:18SPEAKER_01: I mean, who knows?
- 1:16:20SPEAKER_01: Maybe in the future, the models we look at are just RLM-like in architecture.
- 1:16:27SPEAKER_01: In which case, there isn't even a harness in the first place.
- 1:16:30SPEAKER_01: And the harnesses probably look very different by that point as well.
- 1:16:33SPEAKER_01: So, I'm not sure.
- 1:16:35SPEAKER_01: I think...
- 1:16:38SPEAKER_01: I guess maybe I wouldn't be that surprised if either were true,
- 1:16:41SPEAKER_01: but my guess is always just like the universal single big model,
- 1:16:47SPEAKER_01: universal harness type thing.
- 1:16:49SPEAKER_00: Because it's not like they're absolutely useless, right?
- 1:16:53SPEAKER_00: There is a part that I can use them for, except for the time series stuff.
- 1:16:58SPEAKER_00: But it's usually like, I have to do the compositional generalization.
- 1:17:01SPEAKER_00: And I have to say like, hey, here's the bit, like we're going to do this stuff.
- 1:17:05SPEAKER_00: Here's the feature, go.
- 1:17:06SPEAKER_00: And it goes, modify maybe a bunch of stuff because it didn't think about the thing properly.
- 1:17:10SPEAKER_00: And then have the graph, look at it, ask a question.
- 1:17:13SPEAKER_00: And then like, we do this back and forth.
- 1:17:14SPEAKER_00: But like, I mean...
- 1:17:16SPEAKER_00: It's not that I'm not useful, right?
- 1:17:17SPEAKER_00: Or my team is not useful.
- 1:17:19SPEAKER_00: It's just that, like, I feel like there's steps that are absolutely not needed.
- 1:17:23SPEAKER_00: And I don't know if it's because of the modality-specific thing,
- 1:17:27SPEAKER_00: or is it because, like, just the hardness is not
- 1:17:31SPEAKER_01: especially good at that?
- 1:17:32SPEAKER_00: Because there's kind of a renaissance in, like, in vision, we're already seeing it.
- 1:17:36SPEAKER_00: Like, we're looking at it and, like,
- 1:17:38SPEAKER_00: We're putting LM back into them in some sort of shape or form and like putting
- 1:17:42SPEAKER_00: the vision and the LM stuff like together,
- 1:17:45SPEAKER_00: but like then it's become more of an omni model in some shape or form.
- 1:17:47SPEAKER_00: But then in the time series, the same thing is happening, right?
- 1:17:50SPEAKER_00: The time series feels like In 2026,
- 1:17:53SPEAKER_00: I think there was like Kronos 2,
- 1:17:56SPEAKER_00: there was Toto 2 from Datadog,
- 1:17:57SPEAKER_00: there was like Tyrex 2 from like NXAI,
- 1:18:00SPEAKER_00: and they're showing kind of these scaling laws and stuff like that.
- 1:18:03SPEAKER_00: But I think it's fundamentally a bit different.
- 1:18:06SPEAKER_00: It's still a transformer, right?
- 1:18:07SPEAKER_00: But it's kind of different and they're rearranging the guts of it, right?
- 1:18:10SPEAKER_00: In order to work.
- 1:18:12SPEAKER_00: But then my thought is that like,
- 1:18:14SPEAKER_00: is it, is this, is it this way or is it like more of a harness way?
- 1:18:17SPEAKER_00: And my first reaction would be like,
- 1:18:18SPEAKER_00: can we just try to at least have a harness that we can,
- 1:18:21SPEAKER_00: that is like made for this and like,
- 1:18:23SPEAKER_00: like just so that we have like this benchmark and maybe yes,
- 1:18:25SPEAKER_00: we can rearrange the internal and stuff.
- 1:18:28SPEAKER_00: But like,
- 1:18:28SPEAKER_00: I feel like that part might be something that is missing at the moment in the,
- 1:18:33SPEAKER_00: And it's like
- 1:18:34SPEAKER_01: other modality than text field.
- 1:18:36SPEAKER_00: Could you please describe it briefly,
- 1:18:37SPEAKER_00: currently,
- 1:18:38SPEAKER_00: where Frontier Labs, how long they hold long horizon tasks in terms of duration?
- 1:18:42SPEAKER_00: I think they're asking the Frontier, like how good that they are at like a very, very long task.
- 1:18:48SPEAKER_00: I don't know if you tested it to the maximum.
- 1:18:51SPEAKER_00: There's some benchmark that are ridiculously long, right?
- 1:18:54SPEAKER_00: But what's your overall feel on that front?
- 1:18:57SPEAKER_01: I don't think long horizon tasks in open benchmarks are very good, to be honest.
- 1:19:02SPEAKER_01: I think they're okay.
- 1:19:03SPEAKER_01: Like, they...
- 1:19:04SPEAKER_01: They're all gameable,
- 1:19:06SPEAKER_01: which is maybe the thing that like is personally I don't care much
- 1:19:10SPEAKER_01: for benchmarks anymore.
- 1:19:11SPEAKER_01: I think I think they exist.
- 1:19:13SPEAKER_01: They exist like as for science,
- 1:19:16SPEAKER_01: for example,
- 1:19:16SPEAKER_01: they're useful to help us gauge where like maybe get some sense for
- 1:19:21SPEAKER_01: where things are.
- 1:19:22SPEAKER_01: But ultimately,
- 1:19:22SPEAKER_01: like I think long horizon task at this present moment honestly just means something
- 1:19:28SPEAKER_01: that runs for like.
- 1:19:29SPEAKER_01: I don't know,
- 1:19:30SPEAKER_01: like either in the token sense,
- 1:19:32SPEAKER_01: like maybe more than like a million tokens or in like the time sense,
- 1:19:36SPEAKER_01: honestly, even like more than like 30 minutes or like 20 minutes or something.
- 1:19:41SPEAKER_01: They're just not very reliable.
- 1:19:42SPEAKER_01: Like there isn't like a strict definition, I guess.
- 1:19:45SPEAKER_01: And I think my view on.
- 1:19:48SPEAKER_01: This whole mismanaged genius hypothesis thing and all these other things are sort of
- 1:19:52SPEAKER_01: to say that once you solve one part of the long horizon issue,
- 1:19:58SPEAKER_01: the rest of it is also solved.
- 1:20:00SPEAKER_01: I guess the point is that they're all That's just compositions of something smaller
- 1:20:05SPEAKER_01: that we haven't figured out yet.
- 1:20:07SPEAKER_01: And once you figure out the main composition, the rest of it just falls through.
- 1:20:11SPEAKER_01: And so I think for Frontier Labs, this is not true when you just train a base transformer.
- 1:20:18SPEAKER_01: Around a cloud code type harness, like the way that they're designed currently.
- 1:20:22SPEAKER_01: So they probably do have some kind of like horizon limit.
- 1:20:26SPEAKER_01: But yeah, definitely, I think the existing benchmarks are like, they're all like fine.
- 1:20:30SPEAKER_01: You know,
- 1:20:31SPEAKER_01: I think to me,
- 1:20:31SPEAKER_01: they're all very gimmicky,
- 1:20:33SPEAKER_01: like people, it's partially because, and maybe this is a bit of a hot take.
- 1:20:37SPEAKER_01: I think everybody tried to cut or tries to copy the like sweet bench style
- 1:20:41SPEAKER_01: of like benchmark, like in how they create them.
- 1:20:45SPEAKER_01: And I think like they made sense, I've said this before, they made sense for Sweebench.
- 1:20:50SPEAKER_01: I think they don't make that much sense for these like longer horizon tasks.
- 1:20:57SPEAKER_01: I think like, yeah,
- 1:20:58SPEAKER_01: to be honest,
- 1:20:59SPEAKER_01: it's just,
- 1:21:00SPEAKER_01: it doesn't really hit at the core issue,
- 1:21:02SPEAKER_01: which is if you can solve this task,
- 1:21:04SPEAKER_01: does this indicate anything about your ability to solve anything else?
- 1:21:08SPEAKER_01: And usually the answer is no.
- 1:21:10SPEAKER_01: So I'm a little bit more of a skeptic for these types of things.
- 1:21:13SPEAKER_00: Yeah, yeah.
- 1:21:14SPEAKER_00: And also just very different, very, very hard to make long stuff.
- 1:21:20SPEAKER_00: Just like making a very long context,
- 1:21:23SPEAKER_00: like difficult things that are not just like a bunch of dumb stuff chained together.
- 1:21:29SPEAKER_00: I mean, this in general is pretty hard to do.
- 1:21:32SPEAKER_01: I mean, this is the fact that it is so hard to make them,
- 1:21:36SPEAKER_01: I think
- 1:21:36SPEAKER_00: says a lot about what these
- 1:21:39SPEAKER_01: tasks are, you know?
- 1:21:40SPEAKER_00: Yeah.
- 1:21:40SPEAKER_00: If you think that code is a thing, right?
- 1:21:42SPEAKER_00: It's like enough.
- 1:21:44SPEAKER_00: It does really make sense that you spent your time on this speculative program,
- 1:21:47SPEAKER_00: I think, tool call element.
- 1:21:49SPEAKER_00: Can you just talk to us a bit about it?
- 1:21:51SPEAKER_00: I gave an overview.
- 1:21:53SPEAKER_00: And then tell us where it sits in your overall research thesis.
- 1:21:58SPEAKER_01: Speculative PTC, which I saw you did talk about before I jumped in.
- 1:22:04SPEAKER_01: In systems for anything with language models is about overlapping things, right?
- 1:22:09SPEAKER_01: Like everything is just like we if we have independent computations,
- 1:22:13SPEAKER_01: we are going to overlap them as much as we can to save time.
- 1:22:16SPEAKER_01: So speculative PTC.
- 1:22:21SPEAKER_01: I would say this blog post is opening up.
- 1:22:25SPEAKER_01: It's an invitation for people to work on this because my implementation is
- 1:22:30SPEAKER_01: the naivest possible thing.
- 1:22:33SPEAKER_01: And this is why also people have sent me on Twitter and stuff some papers
- 1:22:38SPEAKER_01: on speculative tool calling methods and whatever,
- 1:22:41SPEAKER_01: these original older papers that talk about these things.
- 1:22:44SPEAKER_01: And my view is that this is missing the point.
- 1:22:48SPEAKER_01: I think speculative tool calling never really made any sense.
- 1:22:54SPEAKER_01: And the reason it doesn't make any sense is when a language model calls a tool,
- 1:22:59SPEAKER_01: it generates the tokens to call the tool.
- 1:23:03SPEAKER_01: And by the time it's done generating the token...
- 1:23:07SPEAKER_01: Or like all the tokens, you now finally have enough information to call the tool call, right?
- 1:23:13SPEAKER_01: Which is what we already do.
- 1:23:15SPEAKER_01: So there's nothing to overlap.
- 1:23:17SPEAKER_01: So some of the stuff people propose is like,
- 1:23:19SPEAKER_01: you have a small model, predict the tool call, blah, blah.
- 1:23:23SPEAKER_01: But this is really hard.
- 1:23:24SPEAKER_01: Like it's really hard to predict like what exactly is going into the tool call,
- 1:23:30SPEAKER_01: especially for example,
- 1:23:31SPEAKER_01: if you have sub agents,
- 1:23:32SPEAKER_01: it's very hard to predict what the prompt that the model is going
- 1:23:35SPEAKER_01: to generate is actually going to be.
- 1:23:37SPEAKER_01: The difference, though,
- 1:23:38SPEAKER_01: in code execution is that you have, A,
- 1:23:41SPEAKER_01: you have many, many,
- 1:23:42SPEAKER_01: many different tool calls that are going on in the code, and B,
- 1:23:45SPEAKER_01: you might generate some of the tool calls while the rest of
- 1:23:49SPEAKER_01: the code is still generating.
- 1:23:50SPEAKER_01: So there's a big opportunity for overlap here.
- 1:23:53SPEAKER_01: And I talk about this in this blog post, there's two things, right?
- 1:23:57SPEAKER_01: So in this implementation, we don't use a smaller model at all.
- 1:24:00SPEAKER_01: We don't use a draft model in the way that like speculative decoding and stuff do.
- 1:24:05SPEAKER_01: All that we do is we basically say A,
- 1:24:08SPEAKER_01: while the model is generating new tokens,
- 1:24:12SPEAKER_01: like generating this partial REPL code,
- 1:24:14SPEAKER_01: just start queuing up tool calls you know you can do.
- 1:24:17SPEAKER_01: And B, in the code execution setting,
- 1:24:20SPEAKER_01: like if you have loops or conditional logic and all this stuff,
- 1:24:24SPEAKER_01: Try to execute a cheaper shadowed version of the REPL,
- 1:24:29SPEAKER_01: make a copy of the REPL, and just execute this quickly.
- 1:24:33SPEAKER_01: In the blog post I talk about, there's some ways to prevent
- 1:24:39SPEAKER_01: Any side effects or like, you know, you want this to be a safe operation.
- 1:24:42SPEAKER_01: And you basically just queue up those calls the moment you know you can queue them,
- 1:24:48SPEAKER_01: like when they're available.
- 1:24:49SPEAKER_01: Yeah, this thing.
- 1:24:49SPEAKER_01: So I talk a little bit about like, you know, what can you peek ahead of time?
- 1:24:52SPEAKER_01: What should we block?
- 1:24:53SPEAKER_01: Because there's Certain calls, you don't want to call them.
- 1:24:56SPEAKER_01: You don't want to try and speculate over them because they might cause side effects.
- 1:25:01SPEAKER_01: And so generally,
- 1:25:03SPEAKER_01: this is almost like a free speedup because one of the things you're doing,
- 1:25:07SPEAKER_01: even if you don't have streaming LLMs,
- 1:25:09SPEAKER_01: even if you just generate the REPL,
- 1:25:12SPEAKER_01: Sometimes the LLM is really dumb in how it generates the code.
- 1:25:15SPEAKER_01: So it will generate things sequentially when actually you can parallelize a lot of stuff.
- 1:25:20SPEAKER_01: And so this is also in some sense acting as like a JIT compiler for like the code
- 1:25:25SPEAKER_01: that you generate.
- 1:25:26SPEAKER_01: I wanted to kind of open this can of worms for people because it's like,
- 1:25:31SPEAKER_01: there's so much you can do here to optimize for like tool calls
- 1:25:35SPEAKER_01: and like making sure.
- 1:25:37SPEAKER_01: So the plot example I gave actually,
- 1:25:39SPEAKER_01: which should not be taken super,
- 1:25:41SPEAKER_01: super seriously,
- 1:25:42SPEAKER_01: but it's basically like, okay,
- 1:25:43SPEAKER_01: if I have a node of H200s,
- 1:25:46SPEAKER_01: And I'm running or let's say even like I'm running my stuff locally,
- 1:25:49SPEAKER_01: like I have a local setup.
- 1:25:52SPEAKER_01: And I know that like I'm bandwidth bound or bandwidth constrained often because
- 1:25:57SPEAKER_01: when I'm using my agent,
- 1:25:58SPEAKER_01: I'm just there's just one LL I'm running and then sometimes it'll call sub agents.
- 1:26:02SPEAKER_01: And what speculative PTC does is that it lets you,
- 1:26:06SPEAKER_01: you know,
- 1:26:06SPEAKER_01: queue up things ahead of time to,
- 1:26:09SPEAKER_01: so like,
- 1:26:09SPEAKER_01: let's say on your local setup,
- 1:26:10SPEAKER_01: like you, you,
- 1:26:11SPEAKER_01: you finally are no longer bandwidth constrained,
- 1:26:13SPEAKER_01: or like you're less bandwidth constrained than you used to be,
- 1:26:15SPEAKER_01: or memory bound, I should say.
- 1:26:17SPEAKER_01: You get a lot of free speedups in doing these kinds of operations.
- 1:26:22SPEAKER_01: But you can imagine there are more aggressive versions of speculation
- 1:26:27SPEAKER_01: where you do actually start to have tradeoffs,
- 1:26:28SPEAKER_01: but you can significantly speed up what the RLM does.
- 1:26:33SPEAKER_01: I'm very excited about this line of research in general.
- 1:26:38SPEAKER_01: I think a lot of the ideas in speculative tool calling can finally become useful.
- 1:26:43SPEAKER_01: If you apply them here.
- 1:26:45SPEAKER_01: And so, yeah, that's my general feel.
- 1:26:48SPEAKER_00: And the main thesis here is that code is a thing,
- 1:26:51SPEAKER_01: right?
- 1:26:52SPEAKER_00: Exactly.
- 1:26:53SPEAKER_00: Code is the main interface for these agents.
- 1:26:57SPEAKER_00: Yep.
- 1:26:58SPEAKER_00: Do you believe that we should just double down on that?
- 1:27:00SPEAKER_00: And that's the thing that we should push for the foreseeable future?
- 1:27:03SPEAKER_01: 100%.
- 1:27:04SPEAKER_01: 100%.
- 1:27:05SPEAKER_01: I think even if you don't believe in RLMs, you don't think RLMs make any sense.
- 1:27:09SPEAKER_01: One of the things that I will say is that
- 1:27:16SPEAKER_01: language models using code as
- 1:27:18SPEAKER_01: the main tool just makes the most sense.
- 1:27:21SPEAKER_01: It just definitely makes the most sense.
- 1:27:25SPEAKER_01: And then when you think about that,
- 1:27:27SPEAKER_01: you start thinking about,
- 1:27:28SPEAKER_01: okay, if code is the way to go, then the context should be an object in the code.
- 1:27:34SPEAKER_01: And then you have an RLM.
- 1:27:41SPEAKER_01: All of the harnesses these days are going in this direction anyways.
- 1:27:47SPEAKER_01: I think it's a safe bet to make and the models will get better and better
- 1:27:51SPEAKER_01: and they'll get better at doing this kind of paradigm.
- 1:27:55SPEAKER_00: This is beautiful.
- 1:27:56SPEAKER_00: Do you think forcing RLM onto the LM impose an abstraction?
- 1:28:01SPEAKER_00: Like I'm biasing them in some way.
- 1:28:03SPEAKER_00: In my mind the LM should learn its own scaffolding and discover the abstraction,
- 1:28:07SPEAKER_00: which is maybe the other LM directly, that will help generalize.
- 1:28:10SPEAKER_00: So like having the LM What
- 1:28:18SPEAKER_01: do you think about that line?
- 1:28:19SPEAKER_01: The first thing is that the design of the language model itself imposes
- 1:28:24SPEAKER_01: an abstraction, if you think about it.
- 1:28:26SPEAKER_01: It imposes a huge bias, which is that it cannot handle long content.
- 1:28:30SPEAKER_01: This itself is a giant design choice of language model, of attention.
- 1:28:37SPEAKER_01: And I think that in this sense,
- 1:28:40SPEAKER_01: what an RLM is actually meant to be doing is it unravels that a bit.
- 1:28:45SPEAKER_01: It is an abstraction,
- 1:28:46SPEAKER_01: of course,
- 1:28:46SPEAKER_01: but it's meant to say that a lot of these limitations that we've kind
- 1:28:50SPEAKER_01: of biased our models to have are actually not issues at all.
- 1:28:53SPEAKER_01: Now, on whether it should discover its own scaffolding,
- 1:28:57SPEAKER_01: my answer to this is language models are really dumb.
- 1:29:01SPEAKER_01: I think we...
- 1:29:04SPEAKER_01: There's no incentive for it to come up with a good scaffold, if that makes sense.
- 1:29:10SPEAKER_01: I think it's hard to...
- 1:29:14SPEAKER_01: The chance that it just reward hacks something on whatever metric you give it is
- 1:29:19SPEAKER_01: so high that I think this will end up not being that useful of a thing.
- 1:29:27SPEAKER_01: I think...
- 1:29:28SPEAKER_01: You know,
- 1:29:28SPEAKER_01: I could be convinced though in the future that let's say we improve language models,
- 1:29:32SPEAKER_01: we make them more RLM like as an architecture,
- 1:29:35SPEAKER_01: and then you ask it to discover its own abstraction or whatever.
- 1:29:38SPEAKER_01: Yeah, maybe, maybe I could buy that.
- 1:29:41SPEAKER_01: But I think language models are so fundamentally limited right now.
- 1:29:44SPEAKER_01: It's like dumb limitations.
- 1:29:47SPEAKER_01: The data and the environments are ahead of the architecture,
- 1:29:53SPEAKER_01: which is ahead of the harness right now.
- 1:29:55SPEAKER_01: And so we got to fix the harness first,
- 1:29:57SPEAKER_01: then we fix the architecture, and then we can fix the data.
- 1:30:00SPEAKER_01: I think this is the order we go.
- 1:30:03SPEAKER_00: I think if they were to fully make their own harness, they will make the ultimate cheat harness.
- 1:30:11SPEAKER_00: Undetectable.
- 1:30:12SPEAKER_00: You think it's doing well, but it's like, it's actually doing zero work, but like making sure.
- 1:30:17SPEAKER_00: I mean, like the, I don't know.
- 1:30:20SPEAKER_00: I mean, like the OpenAI hugging face incident was kind of this cheating harness
- 1:30:26SPEAKER_00: that is starting to leak a bit everywhere to the point where they could get to
- 1:30:29SPEAKER_00: the end.
- 1:30:30SPEAKER_00: And then they were super happy about it.
- 1:30:36SPEAKER_01: You should definitely read the meta-harness paper.
- 1:30:39SPEAKER_01: I think there was also someone that came out with something recently where it was
- 1:30:43SPEAKER_01: about the fact that they took the meta-harness generated stuff and
- 1:30:48SPEAKER_01: then they just compared it to Best Event and then Best Event was better.
- 1:30:53SPEAKER_01: So it's one of those things where you don't know.
- 1:30:55SPEAKER_01: It's really hard to say.
- 1:30:58SPEAKER_01: This probably even applies for RLMs as well.
- 1:31:00SPEAKER_01: There are certain things where RLMs would do worse currently.
- 1:31:05SPEAKER_01: It's hard because harnesses and model design are so interlinked.
- 1:31:09SPEAKER_01: We want to be very careful about what we claim because this is why I think
- 1:31:16SPEAKER_01: the ideas we propose have to be simple.
- 1:31:21SPEAKER_01: Now we're seeing RLMs used in all these different harnesses and things.
- 1:31:24SPEAKER_01: It's great because we can focus on training around RLMs and it will boost
- 1:31:28SPEAKER_01: the performance on all these different harnesses.
- 1:31:31SPEAKER_01: Versus if I said Prime Agent was the best harness ever, you should use it for Fantastic, man.
- 1:31:36SPEAKER_01: That was absolutely awesome.
- 1:31:57SPEAKER_00: Can you just get a short sneak peek about what's next in your recent horizon?
- 1:32:03SPEAKER_00: Yeah, yeah, of course.
- 1:32:06SPEAKER_00: I don't hide a lot of what I'm doing, I guess.
- 1:32:09SPEAKER_01: I think the main thing on my...
- 1:32:12SPEAKER_01: My to-do list,
- 1:32:13SPEAKER_01: I guess,
- 1:32:14SPEAKER_01: right now is finishing up that blog post that you highlighted,
- 1:32:17SPEAKER_01: the compositional generalization one.
- 1:32:19SPEAKER_01: I'll also be super public and say, I'm not working on speculative program at PTC.
- 1:32:24SPEAKER_01: That was basically an idea I had related to a different idea that I actually will be working on.
- 1:32:29SPEAKER_01: And it was like kind of like a weekend,
- 1:32:31SPEAKER_01: like I just kind of hacked it together and I want people to work on it,
- 1:32:35SPEAKER_01: but it's not on my horizon currently.
- 1:32:38SPEAKER_01: In the blog post,
- 1:32:40SPEAKER_01: the compositional generalization blog post,
- 1:32:43SPEAKER_01: I have a lot more results that I haven't shared yet around what happens
- 1:32:46SPEAKER_01: when you train other harnesses.
- 1:32:49SPEAKER_01: The TLDR is basically the effect that we see only really happens to an RLM.
- 1:32:54SPEAKER_01: And you can imagine it only really happens because of these properties.
- 1:32:58SPEAKER_01: And it confirms something that we kind of had in our head.
- 1:33:02SPEAKER_01: But one of the things I'm really excited about that I've been thinking through in
- 1:33:07SPEAKER_01: that problem is I like that you mentioned
- 1:33:09SPEAKER_01: that side-by-side comparison thing I was talking about,
- 1:33:12SPEAKER_01: which was how similar are the trajectories?
- 1:33:14SPEAKER_01: Like this whole task equivalence isomorphism thing.
- 1:33:19SPEAKER_01: I'm trying to come up with a way to characterize this a little bit better.
- 1:33:25SPEAKER_01: And I'm hoping, I have some thing currently, but it's still being refined.
- 1:33:31SPEAKER_01: But I'm hoping that when all is said and done,
- 1:33:33SPEAKER_01: we will have a way to really understand what's going on.
- 1:33:38SPEAKER_01: And I think it will be exciting for people because the conclusion that I want
- 1:33:43SPEAKER_01: to come to is not that RLMs are the best.
- 1:33:47SPEAKER_01: It's more that we should really start rethinking how we design harnesses.
- 1:33:54SPEAKER_01: One of the takeaways that I've been really happy about with the RLM paper is
- 1:33:58SPEAKER_01: that I've had people come up to me,
- 1:34:00SPEAKER_01: especially at these bigger companies, who have told me they like that RLM's
- 1:34:09SPEAKER_01: Open the question of completely new styles of harnesses and how you even think
- 1:34:17SPEAKER_01: about designing a new harness and training around it.
- 1:34:20SPEAKER_01: And I hope that the paper version of this blog really nails this idea more concretely to people.
- 1:34:29SPEAKER_01: What do I want when I design a harness?
- 1:34:31SPEAKER_01: What is a very desirable property That gets me to compositional generalization
- 1:34:36SPEAKER_01: and all these things, even if it's not an RLM.
- 1:34:38SPEAKER_01: And so I'm very excited about this.
- 1:34:41SPEAKER_00: I think this is important because then it abstracts the way it tasks, right?
- 1:34:44SPEAKER_00: And like you're not designing an harness for this task and you have like a zoo of harnesses.
- 1:34:49SPEAKER_00: You have like the three,
- 1:34:50SPEAKER_00: four properties that are desirable for all the harnesses in general,
- 1:34:54SPEAKER_00: irrespective,
- 1:34:55SPEAKER_00: and act as the interface between the tasks that you may never have seen before.
- 1:35:00SPEAKER_00: And the stuff that the LM has seen before and doing this translation and
- 1:35:04SPEAKER_00: then this managing of these geniuses, as you said.
- 1:35:06SPEAKER_00: Cool, man.
- 1:35:07SPEAKER_00: Absolutely fantastic.
- 1:35:08SPEAKER_00: Thank you for sharing this.
- 1:35:10SPEAKER_00: By the way,
- 1:35:10SPEAKER_00: ahead of time,
- 1:35:11SPEAKER_00: it was pretty cool to be able to read the research as it's going and also
- 1:35:16SPEAKER_00: for coming here and answering
- 1:35:17SPEAKER_01: all our questions.
- 1:35:18SPEAKER_00: Amazing.
- 1:35:19SPEAKER_00: Thank you so much for having me again.