“The user in the LLM might be a moral patient too (a strange implication of the PSM and simulators)” by agastyasridharan, niranjandeshpande

LessWrong (30+ Karma) · Oct 15, 2026 · 32m

Thanks to Christopher Ackerman, Margot Stakenborg, Darion Cassel, Andy Han, and Ariana Azarbal for feedback and/or valuable discussions on these topics.

TL;DR

We think that, conditional on attributing moral weight to the assistant, we should also attribute a non-negligible fraction of that moral weight to the user persona.

Our argument proceeds like this:

The moral patients associated with LLMs might be simulated personas. On this view, our obligations are to the assistant and the other characters that the LLM simulates, rather than the LLM itself.

The assistant is not the only moral patient…