Tokens and the Observational Boundary
A crossword with every square filled in leaves no move to make. A language model reading tokens meets the same thing at its own grain: the tokens are definite, and whatever lies finer than them stays undecided for the model. That undecided part is not a leftover for a better scheme to fill in. It is the condition for any next token, any measurement, any sentence.
The conversation lays out "Breaking the Token Ceiling", the study by Kalyani Marathe, Artidoro Pagnoni and colleagues at Meta FAIR and the University of Washington, and keeps all of it as measurement: byte models start worse, keep…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.