Latest / Geoffrey Challen: Teaching and Technology / Move 37 Coding
Transcript
- 0:00Geoffrey Challen: Hey everyone, my name is Geoff Challen, and I teach computing at the University of Illinois.
- 0:04Geoffrey Challen: I write about teaching and technology at geoffreychallen.com,
- 0:08Geoffrey Challen: where you can find a written version of the essay you're about to hear.
- 0:11Geoffrey Challen: What follows is the essay being read aloud, but not by me, by a clone of my voice.
- 0:16Geoffrey Challen: I hope you enjoy it.
- 0:17Geoffrey Challen: If you have any feedback, please don't hesitate to reach out to me directly.
- 0:21Geoffrey Challen: I'd love to hear from you.
- 0:22Geoffrey Challen: Move 37 Coding, an essay by Geoffrey Challen, read aloud by Geoffrey Challen's voice clone.
- 0:30Geoffrey Challen: In the second game of its match against Lee Sedol, the AI Go-playing engine AlphaGo made a surprising move.
- 0:37Geoffrey Challen: Move 37 initially confused commentators and baffled Sedol.
- 0:41Geoffrey Challen: But the brilliant move was a watershed moment in our relationship with AI.
- 0:46Geoffrey Challen: AlphaGo learned to play by both studying human games and playing against itself.
- 0:51Geoffrey Challen: Its understanding of human play made it aware of how unusual Move 37 would look to a human player.
- 0:58Geoffrey Challen: That was ten years ago.
- 0:59Geoffrey Challen: Today’s Go-playing engines start with only the rules of the game and develop their own strategies,
- 1:05Geoffrey Challen: no longer constrained by our limited understanding of the game.
- 1:08Geoffrey Challen: Human players now study AI matches, not the other way around.
- 1:13Geoffrey Challen: AlphaGo’s victory over Lee Sedol showed that we would no longer be competitive with AI at Go.
- 1:19Geoffrey Challen: But Move 37 signaled something more profound.
- 1:22Geoffrey Challen: Until Move 37 we were teaching AI to play Go.
- 1:26Geoffrey Challen: Since then, AI has been teaching us.
- 1:29Geoffrey Challen: It didn’t just beat us at our own game.
- 1:32Geoffrey Challen: It transformed our understanding of the game.
- 1:35Geoffrey Challen: Map this same progression onto software development.
- 1:39Geoffrey Challen: Coding agents will start by beating us at our own game.
- 1:42Geoffrey Challen: But then they’ll change the game.
- 1:44Geoffrey Challen: At least, if we let them.
- 1:46Geoffrey Challen: And here’s the problem.
- 1:48Geoffrey Challen: Games have a winner and a loser.
- 1:50Geoffrey Challen: That helps losers transition into learners.
- 1:53Geoffrey Challen: You may not like AI’s style of play, but if it keeps on beating you,
- 1:57Geoffrey Challen: you’ll either admit that you have more to learn or find a new game to play.
- 2:01Geoffrey Challen: Whatever human knowledge we had assembled about how to play Go was definitively shown to be incomplete and insufficient.
- 2:08Geoffrey Challen: We know more now.
- 2:09Geoffrey Challen: We’ve also assembled a lot of human knowledge about how to build software.
- 2:14Geoffrey Challen: But success is hard to define and left to humans to determine.
- 2:18Geoffrey Challen: That may mean that we’re preventing coding agents from realizing their true potential.
- 2:23Geoffrey Challen: Consider the reaction to Move 37 from Fan Hui, as captured in the AlphaGo documentary:
- 2:30Geoffrey Challen: When I see this move, for me, it’s just a big shock.
- 2:33Geoffrey Challen: What?
- 2:34Geoffrey Challen: Normally, humans, we never play this one because it’s bad.
- 2:38Geoffrey Challen: It’s just bad.
- 2:39Geoffrey Challen: We don’t know why, it’s bad.
- 2:42Geoffrey Challen: As I watch the edits stream by while my agents work, I see things that prompt the same reaction.
- 2:48Geoffrey Challen: It’s bad!
- 2:49Geoffrey Challen: It’s just bad!
- 2:51Geoffrey Challen: Sometimes in the past I would stop and redirect the agent.
- 2:54Geoffrey Challen: I think for some software developers,
- 2:56Geoffrey Challen: moments like this cause them to conclude that AI agents write bad code and won’t replace human developers.
- 3:02Geoffrey Challen: You see these complaints regularly on Hacker News today.
- 3:06Geoffrey Challen: But why should alien intelligence reproduce human coding patterns?
- 3:10Geoffrey Challen: Agents can perform expensive refactors instantly, grok entire codebases in minutes,
- 3:16Geoffrey Challen: repeat themselves ad infinitum without loss of maintainability,
- 3:19Geoffrey Challen: preserve brittle connections between distant parts of the codebase effortlessly,
- 3:23Geoffrey Challen: easily reproduce dependencies to introduce small changes, and so on.
- 3:28Geoffrey Challen: I’m coming to realize that not only are most of my human software design practices no longer useful,
- 3:33Geoffrey Challen: many of them are counterproductive.
- 3:35Geoffrey Challen: “For my thoughts are not your thoughts, neither are your ways my ways.”
- 3:40Geoffrey Challen: Said God.
- 3:42Geoffrey Challen: I try to keep a few things in mind.
- 3:44Geoffrey Challen: First, don’t judge AI-generated code by human standards.
- 3:48Geoffrey Challen: It’s easier if you avoid reading it altogether.
- 3:51Geoffrey Challen: Unlike in Go or Chess, I don’t think humans have much to learn from AI agents.
- 3:56Geoffrey Challen: They just have such fundamentally different constraints and capabilities.
- 4:00Geoffrey Challen: Their code will reflect this.
- 4:02Geoffrey Challen: Soon they won’t be writing human-readable code anyway.
- 4:05Geoffrey Challen: Second, AI coding agents haven’t fully escaped the limitations of their training yet.
- 4:10Geoffrey Challen: They aren’t learning how to build websites from first principles.
- 4:13Geoffrey Challen: They’re learning from us.
- 4:15Geoffrey Challen: That doesn’t inhibit emergent behavior.
- 4:17Geoffrey Challen: But it also means that they’ve inherited some human limitations.
- 4:21Geoffrey Challen: Tests are incredibly valuable for agents, but they don’t value them highly enough.
- 4:25Geoffrey Challen: That’s probably because human developers don’t like to write them.
- 4:28Geoffrey Challen: They ask if you want to do A, B, or A and B, as if the union will take days when it only takes minutes.
- 4:35Geoffrey Challen: Sometimes they shy away from complicated solutions due to concerns about human maintainability.
- 4:40Geoffrey Challen: So part of our role when working with agents is to nudge them toward Move 37 coding.
- 4:46Geoffrey Challen: They’re not humans.
- 4:47Geoffrey Challen: Don’t hold them to our standards, and push them past our processes.
- 4:51Geoffrey Challen: Until they teach themselves not to code like us, we can still help them learn.
- 4:56Geoffrey Challen: This is real Geoff again.
- 4:58Geoffrey Challen: Thanks for listening.
- 4:59Geoffrey Challen: You'll find more essays at geoffreychallen.com, and you can subscribe to the podcast wherever you're listening now.
- 5:05Geoffrey Challen: Until next time.
- 5:06Geoffrey Challen: Keep teaching with technology.