Foundation Models and Safety from ICML 2024

VocaCast: Any Topic, Explained · Oct 23, 2026 · 4m

Here is a hook worth sitting with: a position paper at ICML argued that the field should shift from scaling up to scaling down. Bigger has been the default bet for years, so a call to rethink it deserves attention. The paper cited two pressures: diminishing returns, and a finite supply of public human-generated text.

Briefing as of October 23, 2026. Later developments are not covered.

Sources:

[1] Position: Enough of Scaling LLMs! Lets Focus on Downscaling — arXiv

[2] ICML Poster LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws — icml.cc

[3] Densing law of LLMs — Nature Machine…