【第178期】spurious forgetting：大模型的虚假遗忘

Seventy3 · Mar 27, 2025 · 17m

Seventy3: 用NotebookLM将论文生成播客，让大家跟着AI一起进步。

今天的主题是：

Spurious Forgetting in Continual Learning of Language Models

Summary

This paper introduces the concept of spurious forgetting in large language models during continual learning, distinguishing it from actual knowledge loss and attributing it to the disruption of task alignment. The authors demonstrate through experiments and theoretical analysis that early training on new tasks can misalign the model, particularly in the bottom layers. To address this, they propose a Freezing strategy that keeps the initial layers unchanged, significantly…