“A cat-belling problem for FDT” by jessicata

LessWrong (30+ Karma) · Oct 26, 2026 · 20m

Audio note: this post contains a lot of mathematical notation. We narrate the simpler formulas and flag the complex ones, so you may want the original text to hand.

[AI disclosure: I used Opus 5.5 for light edits after my first draft]

Functional decision theory (Yudkowsky and Soares 2017) proposes choosing actions as follows (equation 4 in the paper):

Here is the agent's probability distribution, is a graph saying how changing a logical variable affects other variables, is the observation history, and is utility. The underlines denote dequoting: is a variable in the agent's model representing…