Latest / Complex Systems with Patrick McKenzie (patio11)
AI and the great developer speed-up, with Joel Becker of METR
This week on Complex Systems, Patrick McKenzie (patio11) is joined by Joel Becker from METR. They discuss groundbreaking research on AI coding assistants.
Joel et al’s randomized controlled trial of 16 expert developers working on major open source projects revealed a counterintuitive finding: despite predictions of 24-40% speed improvements, developers actually took 19% longer to complete tasks when using AI tools, even though they retrospectively believed they were 20% faster. The conversation explores why even sophisticated professionals struggle to accurately assess their own productivity…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.