“A big case study of propensity entanglement” by nielsrolf, lwen2027, megasilverfist, Owen Terry, Johannes Taraz

LessWrong (30+ Karma) · Oct 27, 2026 · 5m

This is a link-post for our new paper "Predicting how propensities generalize out-of-distribution", which started as a SPAR project. We select 24 propensities, elicit each through 4 different methods, and then test how this spills over to non-targeted traits:

How well can we predict spillover?

Below is the plot that compares different methods to predict SFT-induced spillover. The best method is to predict that one model behaves similarly to the two other models we tested ("SFT (other models)"). The second best is to system prompt the model to express the target trait, and use the behavior of…