“LoReFT GRAFT Better at Preventing RL Grift” by Samuel Knoche

LessWrong (30+ Karma) · Oct 25, 2026 · 8m

tl;dr. I show that the recently introduced GRAFT method can be improved upon by using a single layer rank 1 Low-rank Linear Subspace Representation Fine Tuning (LoReFT) adapter as the FORGET adapter, using only 0.029% of the parameters of the one used in the GRAFT paper. This approach achieves an equivalent level of reward hack suppression while improving the clean environment solve rate and not requiring an anchor environment. I end with a brief discussion of why this works and of the limitations of fixed preventative steering baselines.

Background

1. GRAFT

GRAFT is a recently proposed…