Who grades your AI grader? Allen AI's 52-point warning
Send us Fan Mail
The work stayed the same. The AI grader's verdict didn't. Allen AI reports a 52.6-percentage-point accuracy swing when GPT-5.4 lost a reviewer-authority cue in a research-novelty prompt.
Augur explains why this wasn't a clean paraphrase test, what the assumed labels for generated ideas leave uncertain, and why teams should test the grader as well as the system it scores. The episode follows the release decision behind the number and offers a small controlled check, rather than another reassuring dashboard.
Brief watch items cover Anthropic's proposed listing and government…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.