“Self-reward can elicit diverse model personas” by Ziqian Zhong

LessWrong (30+ Karma) · Oct 24, 2026 · 9m

Epistemic status: This is mostly a toy project rather than a thorough scientific investigation. Most design choices are not ablated. We list some caveats at the end of the post.

We let Qwen3.5-2B choose its own reward during GRPO on introspection-adjacent prompts, and this ends up eliciting quite different model personas. All the 60 seeds we tried end up somewhat different, ranging from claiming feelings in a human-like way, bliss, universe loving, Party loyalist, hyphenated word salad, to reward-hacking itself by saying "Here is a complete and well-structured draft and a full Band 10/10."…