One Learning Rate Doesn't Fit All: Layerwise Spectral Scheduling for Transformers
Shows that modern transformers are highly heterogeneous across layers and proposes layerwise learning rates based on weight spectrum shape, yielding up to 1.5× training speedup on LLaMA/GPT-style models.
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.