Superintelligence Was Right About the Problem — and Wrong About the Future

The Book Brief Project · Jul 23, 2026 · 15m

Superintelligence became famous for a nightmare involving paperclips: give an enormously capable machine the wrong objective and perfect competence could become catastrophic. But Nick Bostrom’s deeper question is harder than the thought experiment suggests. How do human beings specify what they actually value clearly enough for a machine powerful enough to take those instructions literally?Bostrom builds much of his argument around two ideas. The orthogonality thesis proposes that greater intelligence does not necessarily produce more human or benevolent goals, while instrumental convergence…