Latest / Ethical Machines / When Metrics Make Us Happy, or Miserable

Transcript

When Metrics Make Us Happy, or Miserable · Ethical Machines

• 0:01
  [SPEAKER_00]: There's two different contexts in which we get measured to see whether or not we're successful.
• 0:05
  [SPEAKER_00]: One arena is games, which we obviously typically enjoy.
• 0:08
  [SPEAKER_00]: And then the other arena is something like a business context at work where you get measured about, you know, have you hit your numbers?
• 0:14
  [SPEAKER_00]: Are you doing the things you're supposed to be doing in their various metrics to track?
• 0:17
  [SPEAKER_00]: How well you're doing?
• 0:18
  [SPEAKER_00]: Do well in the metrics that equals success, that equals, oh, I'll see, cool, promotions, raises, blah, blah, blah.
• 0:23
  [SPEAKER_00]: But in the case of games, we usually find measurement how well we're doing, a score,
• 0:30
  [SPEAKER_00]: We find that really enjoyable, or we want to do better in the score, and we find ourselves engaged by the game.
• 0:36
  [SPEAKER_00]: We find engaging in the game meaningful even.
• 0:40
  [SPEAKER_00]: On the other hand, getting measured at work does not feel quite as joyous, doesn't feel quite as meaningful.
• 0:46
  [SPEAKER_00]: In fact, it could be pretty dreadful.
• 0:48
  [SPEAKER_00]: Me, I always resented when anyone assessed me at work, which perhaps is why I have my own business.
• 0:53
  [SPEAKER_00]: Today, my guest is C.T.
• 0:56
  [SPEAKER_00]: Nguyen from the University of Utah, where he's a philosophy professor, he recently came out with a book called The Score.
• 1:01
  [SPEAKER_00]: How to stop playing somebody else's game.
• 1:03
  [SPEAKER_00]: You can find the link to that book on the show description.
• 1:07
  [SPEAKER_00]: His answer, as to why do we find scoring one arena interesting, rich and meaningful, and the other one not, is something like the following.
• 1:15
  [SPEAKER_00]: When it comes to scoring in games, the scores that are created, at least
• 1:21
  [SPEAKER_00]: It does so in a way that supports us having rich experiences while engaging in the game.
• 1:27
  [SPEAKER_00]: It's built around us finding intrinsic enjoyment, reward, meaningfulness in the activity that we're engaged in.
• 1:33
  [SPEAKER_00]: On the other hand, if you score us for something that is external to anything having to do with our experience, or whether you find it meaningful or not, but rather
• 1:45
  [SPEAKER_00]: in a way that tracks how well we're doing at pursuing external institutional goals.
• 1:51
  [SPEAKER_00]: We find it not only not meaningful, but in some cases we can find it alienating.
• 1:56
  [SPEAKER_00]: We talked about a lot of other things, having to do with how metrics, shape or activity, actually, some of the conversation reminds me of a conversation that I had with a friend back when I was in grad school where we were both getting PhDs in philosophy.
• 2:08
  [SPEAKER_00]: And he said his real love is literature, not philosophy.
• 2:11
  [SPEAKER_00]: I was totally perplexed.
• 2:13
  [SPEAKER_00]: I said, why isn't you at a PhD in philosophy?
• 2:15
  [SPEAKER_00]: Surely, PhDs is a thing that your philosophy is something that you're passionate about.
• 2:19
  [SPEAKER_00]: And he said, no.
• 2:20
  [SPEAKER_00]: And I said, why not get a PhD in literature?
• 2:21
  [SPEAKER_00]: And he said,
• 2:22
  [SPEAKER_00]: because it's going to ruin it.
• 2:23
  [SPEAKER_00]: If I get a PhD in literature, it's going to turn literature into my career and to a job and to obligation, but I want it to remain pure.
• 2:31
  [SPEAKER_00]: So a lot of the conversation that we have to me brings to mind that conversation, because
• 2:38
  [SPEAKER_00]: T talks about how metrics shape our activities, even the ones that we love to engage in, and can begin to eat away at them.
• 2:46
  [SPEAKER_00]: And he also talks about how, once those metrics become the measure of success, they start to replace our real values with simplified countable ones.
• 2:57
  [SPEAKER_00]: This is what he calls value capture.
• 2:58
  [SPEAKER_00]: So super fast in conversation, his stuff is always so interesting and coming at things from a different angle.
• 3:05
  [SPEAKER_00]: So as always, love to talk to him, go get his book.
• 3:09
  [SPEAKER_00]: If you want to complain about anything, tell me I'm no good, make a recommendation.
• 3:14
  [SPEAKER_00]: You can email me at EM at redblackman.com.
• 3:17
  [SPEAKER_00]: I'm coming out with a new book in April called The Ethical Nightmare Challenge, how to avoid the worst of AI.
• 3:23
  [SPEAKER_00]: And if you want to stay informed about that, you can go to redblackman.substite.com, sign up there.
• 3:28
  [SPEAKER_00]: I don't send you a bunch of promotional stuff.
• 3:31
  [SPEAKER_00]: It's only if I have actually something substantive to say, including about news about the book, excerpts from the book.
• 3:36
  [SPEAKER_00]: That's her.
• 3:37
  [SPEAKER_00]: OK, let's go talk to T.
• 3:49
  [SPEAKER_00]: So you wrote this book called The Score, just came out, and it ends or the point is almost insuitable.
• 4:00
  [SPEAKER_00]: The main point that you're making, which is something like, there's games on the one hand and there's bureaucracy on the other, and your claim is we've got more fine-game control over games than we do bureaucracies.
• 4:14
  [SPEAKER_00]: which might be a really strict conclusion, but of course the question is why, why even think to compare games and bureaucracy in the first place?
• 4:20
  [SPEAKER_00]: What's the connective tissue between these things such that we come to this big conclusion?
• 4:26
  [SPEAKER_02]: Yeah, I need to be the long-drawn out personal story, but maybe I can just tell you, the heart of the matter.
• 4:32
  [SPEAKER_00]: Is there a short personal version?
• 4:34
  [SPEAKER_02]: Yeah, well, they're the fun to them.
• 4:36
  [SPEAKER_02]: But the heart of the matter is this question that I got really obsessed with, which is about scoring systems, which is why it seemed to me that sometimes we do this thing, we play games,
• 4:47
  [SPEAKER_02]: and then the game scores that goes really clear reports of how well we do and in games and sports and climbing for me in board games, this is the center of so much a light, so much fun, so much flexibility, they're like the light of my life, and then yeah.
• 5:02
  [SPEAKER_02]: We turn around and there's another part of our life we're exposed to another set of clear scoring systems, institutional metrics, citation ratings, evaluations, yelp stars, page views, likes, retweets.
• 5:17
  [SPEAKER_02]: And here, these seem to drain the life out of everything.
• 5:21
  [SPEAKER_02]: So, that's a good one.
• 5:23
  [SPEAKER_00]: So, one, we get measured all over the place, games and a special air professional lives.
• 5:31
  [SPEAKER_00]: If I have you right, they do opposite things.
• 5:33
  [SPEAKER_00]: The scoring game is makes us really excited and the scoring and professional life makes us want to kill ourselves.
• 5:38
  [SPEAKER_02]: Yeah, exactly.
• 5:40
  [SPEAKER_02]: It's actually, it's fine to use the word excitement, because I think a part of the core for me is in both cases, they seize our motivations.
• 5:48
  [SPEAKER_02]: One of the most interesting things, both just watching games and looking at...
• 5:53
  [SPEAKER_02]: all the so-called geology and psychology of metrics.
• 5:56
  [SPEAKER_02]: All of them say that they sneak into our values, they can change our values, they can change our desire, they can super-reorient us.
• 6:03
  [SPEAKER_02]: Yeah.
• 6:03
  [SPEAKER_02]: And in games, this seems to be used to explore different landscapes, and in...
• 6:09
  [SPEAKER_02]: metrics.
• 6:10
  [SPEAKER_02]: It's used to drag us into horrible grinding activity.
• 6:14
  [SPEAKER_02]: So this book is like a quest to understand what the difference is.
• 6:18
  [SPEAKER_02]: And it's one that takes me through game design and the beauty of game design and the degree to which various sports and games have saved my life.
• 6:27
  [SPEAKER_02]: And then it takes me through.
• 6:29
  [SPEAKER_02]: Understanding exactly what bureaucracy is, exactly what bureaucratic metrics are, and exactly how they work the way that they extract certain kinds of information.
• 6:39
  [SPEAKER_02]: Yeah.
• 6:40
  [SPEAKER_02]: And you need to understand really carefully both how games and bureau.
• 6:43
  [SPEAKER_02]: Because there's a way in which both of them are quite similar.
• 6:46
  [SPEAKER_02]: Both of them are clear rules to coordinate us.
• 6:49
  [SPEAKER_02]: And their clear rules, the center of a lot of them, is clear rules about what counts as success.
• 6:54
  [SPEAKER_02]: Right?
• 6:55
  [SPEAKER_02]: What do you do the score?
• 6:56
  [SPEAKER_02]: And it's actually, once you talk to yourself at that point, it's really puzzling why, I mean, and a lot of people think that if you just add scoring into education, if you just add scoring into work, if you just gave a five, because I came back, you motivate people and bring the fun of games into real life.
• 7:12
  [SPEAKER_02]: And the opposite happens.
• 7:13
  [SPEAKER_02]: People mostly experience gameifications, is horrible intrusion to their own face.
• 7:17
  [SPEAKER_00]: I want to know if I understand your sort of question in the right kind of way, what would count as an answer to the question.
• 7:22
  [SPEAKER_00]: Are you asking and answering an empirical question, like a question about human psychology motivation?
• 7:28
  [SPEAKER_00]: So we need to do sort of investigation into what explains why people react differently to metrics in different contexts?
• 7:36
  [SPEAKER_00]: Or is there some other kind of philosophical and or normative question answer being offered as well?
• 7:43
  [SPEAKER_02]: it's going to be a big messy mixture because a lot of what we have are a lot of empirical facts that are very well recorded, the fact that people can immediately, the fact that in the game space, the thing I find really interesting is the fact that people can just change
• 8:01
  [SPEAKER_02]: their desires in a second, open up a rule book.
• 8:04
  [SPEAKER_02]: Look at the scoring system, be like, okay, we're gonna kill each other.
• 8:07
  [SPEAKER_02]: Okay, we're gonna cooperate.
• 8:08
  [SPEAKER_02]: Okay, I'm on a team.
• 8:09
  [SPEAKER_02]: There's this basic core psychological fact about gameplay that's well observed in all of anthropology and sociology, it's empirical fact, that we change easily.
• 8:19
  [SPEAKER_02]: And then there's this incredibly well recorded fact in the sociology of bureaucracies and institutions that
• 8:26
  [SPEAKER_02]: when you put a metric in front of people in any institution, they all like orient on it, like schooling fish.
• 8:32
  [SPEAKER_02]: It just becomes central to their focus and attention, and the thing that's similar between both, empirical fact, is that human beings find it really easy to use a well laid-out scoring system to focus their attention and cares.
• 8:47
  [SPEAKER_02]: And we also know, basic empirical fact, that people love playing games that find it very fulfilling, and it is
• 8:55
  [SPEAKER_02]: The sociologist Wendy Esplend and the sociologist Michael Souter in the book Engine to Vengziety, which is a study of what happened to law school culture when the US News and World Report started ranking law schools.
• 9:07
  [SPEAKER_02]: Their report is that every one of the systems felt consciously anxious, constantly miserable, and forced to do terrible things to their institutions that they didn't care about at all.
• 9:16
  [SPEAKER_02]: So these are all empirical facts on the ground.
• 9:17
  [SPEAKER_02]: There's not what there isn't.
• 9:20
  [SPEAKER_02]: is a good explanation of why one of these is so harmful in one of these is not and I think I have this is where this is what I want to do this is the philosopher coming in to tell you about why one of these is part why games and player central to a meaningful life and my metrics are probably going to rip a meaningful life out of your hands.
• 9:38
  [SPEAKER_00]: Okay, I want to start espousing hypotheses, but I'll refrain.
• 9:41
  [SPEAKER_02]: So let's see, let's put it to your hypothesis on the ground.
• 9:44
  [SPEAKER_00]: Okay, down.
• 9:45
  [SPEAKER_00]: So this is obviously off the cuff.
• 9:46
  [SPEAKER_00]: I haven't given this nearly as much thought as you have.
• 9:49
  [SPEAKER_00]: I would guess something like the metrics and games or the conditions under which we engage it intrinsically enjoyable activity.
• 9:56
  [SPEAKER_00]: It creates the context or situation or foundation, which we can do stuff that we find intrinsically, whatever, fun and enjoyable blah, blah, blah.
• 10:07
  [SPEAKER_00]: So there's something there's something intrinsically great about hitting a, I don't know, I don't want to use baseball because I don't even like baseball about climbing.
• 10:14
  [SPEAKER_00]: I don't know, okay, okay, I'm going to start there.
• 10:17
  [SPEAKER_00]: And then with metrics in the bureaucracy or institutional life or work or whatever, it's not setting the foundations up for a certain kind of activity that we find intrinsically enjoyable.
• 10:26
  [SPEAKER_00]: It's because it makes meeting the metrics.
• 10:29
  [SPEAKER_00]: The very thing that constitutes success whereas metrics in the game situation is not the success, the success is the activity that going for the score in genders.
• 10:40
  [SPEAKER_02]: Beautiful man.
• 10:41
  [SPEAKER_02]: Okay.
• 10:43
  [SPEAKER_02]: I think that is absolutely right, and I could take you a long route to give you all the details, but that is it.
• 10:49
  [SPEAKER_02]: Okay.
• 10:50
  [SPEAKER_02]: Okay.
• 10:51
  [SPEAKER_02]: Sometimes they're designed for different.
• 10:53
  [SPEAKER_02]: The nail that done, so the quick version of the answer is that games are designed.
• 10:57
  [SPEAKER_02]: They're two main features of the answer.
• 10:59
  [SPEAKER_02]: The games are designed.
• 11:01
  [SPEAKER_02]: for rich enjoyment, pleasure, satisfaction, and intrinsic value, and in particular that they're highly tailorable because of the nature of games to your own peculiar personality, where metrics are not designed for that, they're designed for a kind of large-scale information extraction, and we can get very clear on how they're designed for that.
• 11:21
  [SPEAKER_02]: And in order to play their role in the information,
• 11:24
  [SPEAKER_02]: extraction and aggregation in large-scale institutions.
• 11:27
  [SPEAKER_02]: They're not tailorable.
• 11:28
  [SPEAKER_02]: You can't.
• 11:29
  [SPEAKER_02]: If you want the endpoint in one sentence, it's going to be something like, you can house-roll Dungeons and Dragons, but you can't house-roll GPA, and you can't house-roll page views.
• 11:39
  [SPEAKER_02]: Okay, but I can take a-
• 11:41
  [SPEAKER_00]: Yes, I want to hear more about the tailoring pit, because I don't understand that immediately because there are rules to the game.
• 11:48
  [SPEAKER_00]: Any game you play, there are rules that you walk into, right?
• 11:51
  [SPEAKER_00]: In climbing, there's the rankings of, let's say, V zero to V, whatever they're doing now 15, 16, I don't know, but Anna's stuff.
• 11:57
  [SPEAKER_00]: And you walk into that world of ranking and baseball or football or wrestling, whatever you walk into the rules of the game.
• 12:02
  [SPEAKER_00]: You don't, they're not tailored to you.
• 12:04
  [SPEAKER_00]: Maybe you choose that because something like the rules speaks to you or something
• 12:11
  [SPEAKER_02]: Let's dive into Game Design, because there are two big chunks we need to lay on the table to make this make sense, which is a theory of games and theory of Bruxie.
• 12:19
  [SPEAKER_02]: So let's talk about games.
• 12:20
  [SPEAKER_02]: The long answer you get to your question is going to be games of this really funny thing, because you're exactly right.
• 12:26
  [SPEAKER_02]: There's a way in which they're solid.
• 12:28
  [SPEAKER_02]: They're pre-written and you walk in and you experience rules and they drag you into a new kind of activity.
• 12:35
  [SPEAKER_02]: And there's a way in which you have a high degree of control over the rules because you get to choose your games.
• 12:40
  [SPEAKER_02]: And one of the core background features is there are hundreds of thousands of games to choose from.
• 12:46
  [SPEAKER_02]: If you're a student there's GPA and that's it and you have no choice.
• 12:49
  [SPEAKER_02]: Okay, look back up.
• 12:50
  [SPEAKER_02]: So there are a lot of features of games that are super interesting for the contrast.
• 12:54
  [SPEAKER_02]: I've been working on
• 12:55
  [SPEAKER_02]: the philosophy of games from nine years in my life.
• 12:58
  [SPEAKER_02]: By the way, this is a very strange thing to do as a philosopher.
• 13:00
  [SPEAKER_02]: This is not a normal thing.
• 13:01
  [SPEAKER_00]: I mean, being a philosopher, the same thing to do as well.
• 13:04
  [SPEAKER_00]: Yeah, it's exponentially weird.
• 13:05
  [SPEAKER_00]: Yeah, right, right, right.
• 13:06
  [SPEAKER_00]: The philosophy of being a straightforward person.
• 13:08
  [SPEAKER_00]: Yeah, yeah, for sure.
• 13:09
  [SPEAKER_02]: Yeah, yeah.
• 13:10
  [SPEAKER_02]: Yep.
• 13:11
  [SPEAKER_02]: I need some creamer, the crop equivalent that's for weirdness.
• 13:14
  [SPEAKER_02]: It's the weird, it's like the weird, the worst of the weird swamp.
• 13:18
  [SPEAKER_02]: OK.
• 13:18
  [SPEAKER_02]: There are two kind of major ideas that really help me understand what games were.
• 13:22
  [SPEAKER_02]: I started reading my games, I was doing some work on games, trying to figure out what they are, I was reading all the stuff, they were saying, oh, games are art, because they're interactive fiction, they're like movies, and you get chances, and I was like, no, because there's so much more, there's so much more about skill and science, and there are two theories that really help me understand the nature of games.
• 13:39
  [SPEAKER_02]: One is burner two, it's a philosopher, and he offers a definition of games, and his definition is the game,
• 13:47
  [SPEAKER_02]: is voluntarily taking on unnecessary obstacles to make possible an activity.
• 13:55
  [SPEAKER_02]: So the obstacles create the activity.
• 13:57
  [SPEAKER_02]: So one of the quick way to put it is in every game he thinks there's a goal you're trying to hit and there's an easy way to do it.
• 14:05
  [SPEAKER_02]: if you're running a marathon and you're trying to get the finish line, you could call the taxi, right?
• 14:10
  [SPEAKER_02]: You could call the taxi, sure, sure, sure.
• 14:12
  [SPEAKER_02]: You could take a short cut, right?
• 14:13
  [SPEAKER_02]: If you're a rock climber like me, right?
• 14:15
  [SPEAKER_02]: You could also walk up the back or use a grappling hook, or use a helicopter, right?
• 14:20
  [SPEAKER_02]: There's an easy way, but you avoid it.
• 14:22
  [SPEAKER_02]: So what that means is that if you're avoiding the easy way, it must be that the outcome, the goal by itself is not what's valuable is in the doing.
• 14:30
  [SPEAKER_02]: So exactly, this is a route to intrinsic value.
• 14:33
  [SPEAKER_02]: Other major data point for me is the Reiner Knitia, who's my favorite board game designer.
• 14:37
  [SPEAKER_02]: He's a German genius of board game design, and he is his interview at a game developer's conference, where he's a talk, where he says, the most important thing in my game designer's toolbox is the scoring system, because the scoring system tells people what to care about.
• 14:52
  [SPEAKER_02]: It sets their motivation in the game.
• 14:54
  [SPEAKER_02]: When you put these two together, what you get is his view,
• 14:58
  [SPEAKER_02]: that what games do is they offer you this alternate self and at the center of that alternate self is a different set of desires.
• 15:06
  [SPEAKER_02]: And when you play a game, you take on those desires.
• 15:09
  [SPEAKER_02]: To me, the understand this, I think one of the core ideas in the book and all the work I've done in the philosophy of games is introducing this distinction between two motivations for which you could play a game.
• 15:19
  [SPEAKER_02]: So what does a achievement play?
• 15:20
  [SPEAKER_02]: That's just playing because you care about winning and winning really matters to you.
• 15:24
  [SPEAKER_02]: The other kind of game playing is striving play.
• 15:27
  [SPEAKER_02]: So striving play is when you take on the desire to win in order to have the struggle.
• 15:33
  [SPEAKER_02]: So in achievement play, winning is the point.
• 15:35
  [SPEAKER_02]: In striving play, the struggle is the point.
• 15:38
  [SPEAKER_02]: One way to put it is I think, so a cheap and play.
• 15:40
  [SPEAKER_02]: So if you're an Olympic athlete, they want to be the best, or you want to win nobility for your country or your poker player, right?
• 15:48
  [SPEAKER_02]: What you actually care about it's winning.
• 15:50
  [SPEAKER_02]: Striving players,
• 15:52
  [SPEAKER_02]: are people for whom winning is Galactically unimportant, but they need to try to win in order to have fun.
• 15:59
  [SPEAKER_02]: So here's an argument.
• 16:01
  [SPEAKER_02]: Uh-huh.
• 16:01
  [SPEAKER_02]: By the way, when I put this idea down, some people are like, oh, of course.
• 16:04
  [SPEAKER_02]: And other people are like, that is nuts.
• 16:05
  [SPEAKER_02]: There's no such thing.
• 16:06
  [SPEAKER_00]: No, that's a argument.
• 16:08
  [SPEAKER_00]: Yeah, go ahead.
• 16:09
  [SPEAKER_02]: So here's an argument that the striving play exists.
• 16:11
  [SPEAKER_02]: Consider the category of stupid games.
• 16:14
  [SPEAKER_02]: A stupid game is a game where the fun part is failing, but in order to have fun, you have to actually try to win.
• 16:19
  [SPEAKER_00]: Okay, yeah, like that.
• 16:20
  [SPEAKER_02]: So like Twister.
• 16:21
  [SPEAKER_02]: Yeah.
• 16:21
  [SPEAKER_00]: So that's the thing, two days ago.
• 16:24
  [SPEAKER_00]: Yeah.
• 16:24
  [SPEAKER_00]: Right.
• 16:25
  [SPEAKER_00]: Okay.
• 16:25
  [SPEAKER_02]: If you fall on purpose, it's not funny.
• 16:27
  [SPEAKER_02]: Yeah.
• 16:28
  [SPEAKER_02]: So to play Twister, you have to Galactically want the joy of failure, but to actually have it to be failure in the game.
• 16:35
  [SPEAKER_02]: You have to make yourself actually try not to fail.
• 16:37
  [SPEAKER_02]: Okay, so what this exposes to me is that what games are, what games are in the expose is the value of intrinsic Is the degree to which activities are intrinsically valuable what they expose is Aristotle's original insight That tapiociation in the doing yeah, the meaning of life is in Action itself and games are sculpted action
• 17:00
  [SPEAKER_00]: Yeah, that's interesting.
• 17:01
  [SPEAKER_00]: As you know, I'm also a climber.
• 17:02
  [SPEAKER_00]: So yeah, get into the top of the carers.
• 17:04
  [SPEAKER_00]: Is that about being at the top?
• 17:05
  [SPEAKER_00]: It's about the trying.
• 17:06
  [SPEAKER_00]: Not trying unless you're trying to get to the top.
• 17:08
  [SPEAKER_02]: That is the weird paradox.
• 17:10
  [SPEAKER_02]: For sure, the weird paradox of climbing is all the intensity and thrill comes from trying to get really hard to get the top and the moment you get to the top, you lower off.
• 17:20
  [SPEAKER_02]: That's the paradox of games.
• 17:22
  [SPEAKER_00]: what you lower off, what do you do by that?
• 17:25
  [SPEAKER_02]: If you're sport climbing, you climb to the top and then the moment you touch the clips at the top, you're like, okay, lower me.
• 17:30
  [SPEAKER_00]: Yeah, you're right.
• 17:32
  [SPEAKER_02]: Care about being you.
• 17:33
  [SPEAKER_02]: You have to entice the fear of being with the top.
• 17:35
  [SPEAKER_00]: Yeah, so here's the group.
• 17:36
  [SPEAKER_02]: But here's the really important part for me.
• 17:39
  [SPEAKER_02]: So,
• 17:40
  [SPEAKER_02]: Striving play is in between kind of two positions.
• 17:43
  [SPEAKER_02]: One position I think is I just want to win what I care about as a winning achievement Yeah, the other version is oh the other pole is we can just play we just do whatever we don't need rules We don't need structure like we just have fun.
• 17:56
  [SPEAKER_00]: Let's have fun.
• 17:56
  [SPEAKER_02]: Yeah, yeah
• 17:58
  [SPEAKER_02]: Yeah, but I think what's interesting about games is in so many games, you need the rules and structure in particular, you need the very highly specified goal and you need to be able to try really hard for that goal in order to have this very specific kind of activity.
• 18:12
  [SPEAKER_02]: So I fly fish.
• 18:14
  [SPEAKER_02]: And fly fishing is a very weird activity because you have a very specific, in dry fly fishing, there's only one thing that really counts as a victory, which is landing a floating fly delicately on the surface of the water in front of a feeding fish that you can see and having that feeding fish take the fly.
• 18:35
  [SPEAKER_02]: Okay.
• 18:35
  [SPEAKER_02]: Once you take that as a goal, you have to do all this weird stuff.
• 18:38
  [SPEAKER_02]: You have to be hyper absorbed in watching the current looking for trout, looking at bugs, trying to help.
• 18:43
  [SPEAKER_02]: If you didn't have that goal, if you were just like fishing with some other goal, like catching fish within that.
• 18:48
  [SPEAKER_00]: Yeah.
• 18:48
  [SPEAKER_02]: Yeah.
• 18:48
  [SPEAKER_02]: You wouldn't get dragged into that activity.
• 18:50
  [SPEAKER_02]: Yeah.
• 18:51
  [SPEAKER_02]: So here's, this is, this is related to what, this is exactly related to what you said before at the top of this.
• 18:56
  [SPEAKER_02]: When we walk into games, they are pre-structured.
• 19:01
  [SPEAKER_02]: One of the interesting things about games is, it's not just that you can randomly wander and just do whatever.
• 19:07
  [SPEAKER_02]: Sure, yes.
• 19:08
  [SPEAKER_02]: Because someone gives you a very specific structure and that teaches you a new, there is no way on my own, without the culture of community and rules of fly fishing, that would have found my way into this weird asset activity.
• 19:21
  [SPEAKER_02]: So, it is a pre-existing structure, but,
• 19:25
  [SPEAKER_02]: It's also one that you are free to move between different games and modify games.
• 19:30
  [SPEAKER_02]: I can change the game of fryfishing for myself.
• 19:33
  [SPEAKER_02]: I can change, I think the Mario is super interesting example.
• 19:36
  [SPEAKER_02]: Some people choose getting to the end of the game.
• 19:39
  [SPEAKER_02]: As a goal, some people choose getting it there faster.
• 19:42
  [SPEAKER_02]: Some people choose collecting everything for maximum points.
• 19:45
  [SPEAKER_02]: Each of those is a different experience with different like feel and with games you get to choose.
• 19:54
  [SPEAKER_00]: Let me see one other thing about what might be interesting about games and tell me the sort of fits into how you think about them.
• 20:02
  [SPEAKER_00]: I think one thing that's fun about them, and maybe this, I don't know, it's something like you're, you're, there's something intrinsically enjoyable about
• 20:13
  [SPEAKER_00]: Engaging activity that's at the limit of your capacities, something along, if it's too easy, it's not fun.
• 20:20
  [SPEAKER_00]: Even if they were climbing and it's you're climbing like a 5, 5, 5, 6, something really easy.
• 20:25
  [SPEAKER_00]: It's like a ladder.
• 20:26
  [SPEAKER_00]: It's not interesting.
• 20:28
  [SPEAKER_00]: If it's.
• 20:29
  [SPEAKER_00]: A five, whatever, is it's extremely difficult and you can't even touch the thing, you can't even hold on to anything.
• 20:34
  [SPEAKER_00]: So no, it's no good.
• 20:35
  [SPEAKER_00]: But there's something about exercising your capacities at their limits or near their limits, that's where the enjoyment comes from.
• 20:40
  [SPEAKER_00]: Something along those lines.
• 20:41
  [SPEAKER_02]: I think one of the interesting things about games, so I have two answers for you.
• 20:46
  [SPEAKER_02]: One is exactly right.
• 20:48
  [SPEAKER_02]: And one of the interesting things about games is if the thing that's delicious is this experience of just being in your limit,
• 20:57
  [SPEAKER_02]: Games offer you something the world does not, which is ability to micro scale the challenges to put you at your limit.
• 21:04
  [SPEAKER_02]: The world normally doesn't let you carefully choose exactly the challenge and safely at your delicious limit.
• 21:10
  [SPEAKER_02]: And safe, games don't always have to be safe.
• 21:12
  [SPEAKER_00]: I guess that's right.
• 21:14
  [SPEAKER_02]: True, yes, but you can but whatever your ability you can change the difficulty setting You can change pick the difficulty of the climb.
• 21:22
  [SPEAKER_02]: You can change your opponent.
• 21:24
  [SPEAKER_02]: You can change the game and tell it pushes your abilities to the level you want the first up the second thought is not only do you have that choice of Scaling the challenge to you
• 21:35
  [SPEAKER_02]: You also have the choice of the degree of challenge you want.
• 21:37
  [SPEAKER_02]: I think I love being at my limit.
• 21:40
  [SPEAKER_02]: Other people don't.
• 21:42
  [SPEAKER_02]: And I freely, like some days, I've learned, some days I want to be my absolute limit in climbing and other days, I can be like, oh, this isn't giving me any pleasure.
• 21:51
  [SPEAKER_02]: I'm just going to climb medium climbs as elegantly as I can to hit this flow state.
• 21:56
  [SPEAKER_02]: This is the contrast we're building.
• 21:57
  [SPEAKER_02]: Games are highly structured scoring systems in rulesets of creative activity.
• 22:05
  [SPEAKER_02]: high change ability, high modifiability, and high on the fly alteration of not just the constraints, but even your conception of what you're trying to do.
• 22:14
  [SPEAKER_02]: I had one time I was just suffering rock climbing.
• 22:17
  [SPEAKER_01]: Yeah.
• 22:17
  [SPEAKER_02]: And I was trying to climb something hard.
• 22:19
  [SPEAKER_02]: And my friend just like, just try to climb something easy beautifully.
• 22:22
  [SPEAKER_02]: And that was a different goal.
• 22:23
  [SPEAKER_02]: And I was like, oh, this is what I want to do today.
• 22:25
  [SPEAKER_02]: Awesome.
• 22:25
  [SPEAKER_02]: Because I had the ability to shift the goal, mid-activity.
• 22:30
  [SPEAKER_00]: So it'll be okay.
• 22:31
  [SPEAKER_00]: So let's pivot a little bit.
• 22:33
  [SPEAKER_00]: Why then, why does it gamification work of the other stuff, right?
• 22:38
  [SPEAKER_00]: Okay.
• 22:38
  [SPEAKER_00]: I've got to do some activity for my job or whatever it is and I don't really want to do it.
• 22:45
  [SPEAKER_00]: And then we make it a game somehow where I get pitted it, I get pitted against either my previous self, how I performed last month or maybe against other departments, whatever.
• 22:54
  [SPEAKER_00]: Why does it become an addition to?
• 22:58
  [SPEAKER_00]: the thing that I have to do, why the numbers there also create condition under which I can find the activity in terms of the enjoyable.
• 23:07
  [SPEAKER_02]: I will give you a very short answer and then there's a much longer program.
• 23:15
  [SPEAKER_02]: The short answer is that
• 23:19
  [SPEAKER_02]: Gamifications are normally tied to external considerations.
• 23:23
  [SPEAKER_02]: So if I want to gamify, if I want, if I've got a bunch of employees, and I want to gamify them to make them more productive.
• 23:30
  [SPEAKER_02]: then I have to tie the point system to something that counts as productivity.
• 23:36
  [SPEAKER_02]: The employees cannot re-alter the scoring system.
• 23:40
  [SPEAKER_02]: Just see, that's number one.
• 23:41
  [SPEAKER_02]: Number two is, game scoring systems because they're not tied to anything in the world can be freely shifted plasticly to yield the most satisfying activity.
• 23:51
  [SPEAKER_02]: large-scale metrics aren't free in that way because the things that you can play games in a classroom.
• 23:57
  [SPEAKER_02]: You can play games in the workplace and be free to specify your scoring system.
• 24:02
  [SPEAKER_02]: But what gamification is attaching game-like structures to pre-existing value systems and where those pre-existing value systems are normally tied to large-scale institutions and economies.
• 24:14
  [SPEAKER_02]: So that's the short answer.
• 24:16
  [SPEAKER_02]: I can give the bigger answer if you want to make sure that you just sort of freak out about the short answer.
• 24:20
  [SPEAKER_00]: I just want to understand that, I'm not sure that I understand the explanation yet.
• 24:24
  [SPEAKER_00]: So, I don't hold to understand the point about
• 24:30
  [SPEAKER_00]: It's plastic in the world of games.
• 24:31
  [SPEAKER_00]: It's not plastic in the world of whatever and large companies are having like that.
• 24:36
  [SPEAKER_00]: Because of the point we said earlier, I walk into the baseball game.
• 24:40
  [SPEAKER_00]: I walk into the climb and cry, I walk blah, blah, blah, blah.
• 24:43
  [SPEAKER_00]: And so I don't choose those rules.
• 24:45
  [SPEAKER_00]: Similarly,
• 24:46
  [SPEAKER_00]: In some broad sense, maybe it's not broad enough, I choose the job that I work at or the industry that I work in, whatever, at least there's lots of cases in which I do choose the industry.
• 24:55
  [SPEAKER_00]: There's lots of cases in which you don't, of course, but take the cases which you do.
• 24:58
  [SPEAKER_00]: Yeah, I want to be in a pharma.
• 24:59
  [SPEAKER_00]: I want to be in healthcare.
• 25:00
  [SPEAKER_00]: I want to be in whatever.
• 25:02
  [SPEAKER_00]: So you do choose that.
• 25:03
  [SPEAKER_00]: So now you're choosing some broadly speaking some rules to walk into.
• 25:07
  [SPEAKER_00]: The fact that I can't change them is the similar to the fact that I can't change the rules of baseball once I enter the baseball field.
• 25:16
  [SPEAKER_00]: Okay, so keep going.
• 25:16
  [SPEAKER_00]: So now I don't know where do I go from here.
• 25:19
  [SPEAKER_02]: One answer is there's not a clear game versus institution to division.
• 25:22
  [SPEAKER_02]: It's a very complex spectrum.
• 25:25
  [SPEAKER_02]: If you are a high school athlete and the only game in town is football, this is more like exposure to a fixed rigid system.
• 25:34
  [SPEAKER_02]: And I think some of us do have the privilege of getting to choose
• 25:39
  [SPEAKER_02]: a job that has a scoring system and activity that satisfies our us.
• 25:44
  [SPEAKER_02]: And I think a lot of us, like, a lot of us who have the freedom to choose a job are making something like a game like choice, right?
• 25:51
  [SPEAKER_02]: I think like I chose philosophy because the kind of activity in the goal of publishing papers is really satisfying to me.
• 25:58
  [SPEAKER_02]: And I think one of the end points of this is they're not totally different.
• 26:01
  [SPEAKER_02]: And if they were totally different, I couldn't give you the big finale, which is you should
• 26:09
  [SPEAKER_02]: Which is, one where you choose the rules in the scoring system that suits you.
• 26:15
  [SPEAKER_02]: But I think a lot of people don't do that.
• 26:16
  [SPEAKER_02]: I think the institutional context makes, I think about my students all the time.
• 26:19
  [SPEAKER_02]: They don't have a choice about their scoring system.
• 26:23
  [SPEAKER_02]: GPA is the scoring system.
• 26:25
  [SPEAKER_02]: And then once you've gotten that and you're in a job, you don't have a lot of freedom.
• 26:30
  [SPEAKER_02]: But the real thing that it happens is that there's a sentence
• 26:36
  [SPEAKER_02]: games have a certain degree of freedom just because they're disconnected.
• 26:39
  [SPEAKER_02]: Here's one thing that happens.
• 26:40
  [SPEAKER_02]: Okay, let me give you, here's a simplest contrast.
• 26:43
  [SPEAKER_02]: And then I think I need to give you this chunk of stuff about bureaucracies.
• 26:47
  [SPEAKER_02]: So here's one version of the difference.
• 26:49
  [SPEAKER_02]: Okay, let's say I'm playing Mario Odyssey.
• 26:54
  [SPEAKER_02]: I suck at Mario Odyssey.
• 26:55
  [SPEAKER_02]: I'm terrible at platformers.
• 26:57
  [SPEAKER_02]: And I put it on the easiest mode and just try to do the minimal to get through.
• 27:01
  [SPEAKER_02]: And my friend is super good.
• 27:03
  [SPEAKER_02]: They try to speed run it while hitting everything elegantly and we get...
• 27:08
  [SPEAKER_02]: points, but we've changed the constraints, we've changed our conception of what we're doing, and we need to figure out how do his speedrunning points compare to my, like, oh, I'm just trying to do this slowly and chilly and enjoy myself.
• 27:22
  [SPEAKER_02]: And the answer is, we actually don't need to cross compare those points.
• 27:27
  [SPEAKER_02]: So I have a freedom to reinterpret what I'm doing because we don't have that demand for cross comparison.
• 27:33
  [SPEAKER_02]: Now think about what's going on with GPA, right?
• 27:35
  [SPEAKER_02]: If a student
• 27:37
  [SPEAKER_02]: the degree to which the form in which GPA is intruding into their life, they can't really reinterpret it.
• 27:46
  [SPEAKER_02]: Okay.
• 27:47
  [SPEAKER_02]: I need to give you a chunk of context here, none of this will make sense until we do this.
• 27:52
  [SPEAKER_00]: Other half of the slab, which is something like the theory of bureaucracy.
• 27:56
  [SPEAKER_00]: That's good.
• 27:57
  [SPEAKER_00]: Yeah.
• 27:57
  [SPEAKER_00]: Okay.
• 27:58
  [SPEAKER_02]: Let's hear it.
• 28:00
  [SPEAKER_02]: So if you think about what scoring systems are, here's a theory of scoring systems.
• 28:07
  [SPEAKER_02]: So what a scoring system is something that makes explicit
• 28:12
  [SPEAKER_02]: how you're going to evaluate things so that we all will evaluate them in a similar way, okay?
• 28:18
  [SPEAKER_02]: So, here's something that it took me a while to understand.
• 28:22
  [SPEAKER_02]: You can have games and competition without a scoring system.
• 28:25
  [SPEAKER_02]: So, if we all are skateboarders and we're going to the skate park and we're all going to try to do the coolest trick.
• 28:32
  [SPEAKER_01]: Yeah.
• 28:33
  [SPEAKER_02]: We don't need to establish a scoring system and in fact each of us can come away with a different set of judgments about who did the coolest trick or even if that was actually no, there's not one there are like three very differently beautiful tricks are different in different ways that's fine nothing forces me to come to one judgment or forces us to come to one judgment sure a scoring system is something that we buy into that guarantees that we will all settle on the same official judgment okay okay so has that work on institutions
• 29:02
  [SPEAKER_02]: So in institutions scoring care systems, metrics get tied to data.
• 29:07
  [SPEAKER_02]: And this is the big crucial central question now.
• 29:11
  [SPEAKER_02]: What is data?
• 29:12
  [SPEAKER_02]: What is institutional data?
• 29:15
  [SPEAKER_02]: To understand what's distinctive about metrics and why they're weird to take on these values, you have to accept that to not everything is capturable as data.
• 29:23
  [SPEAKER_02]: You can get a hint of this in game.
• 29:25
  [SPEAKER_02]: So we don't even have to think what institutions think about this slide.
• 29:28
  [SPEAKER_02]: Again, it's not a clear-cut and dry distinction.
• 29:32
  [SPEAKER_02]: The history of skateboarding is super interesting because for a long time, skateboarding was dominated by an attempt at doing something very like stylistically rich.
• 29:41
  [SPEAKER_02]: That's the cool lines, flowing lines, beautiful lines, and then everything changed when skateboarding became an Olympic sport and we entered the ESPNX because then you needed to have a clear way to settle a winner.
• 29:54
  [SPEAKER_02]: You needed a scoring system.
• 29:56
  [SPEAKER_02]: and skateboarding changes about this, like the ESPN ex-games, people are measured on height and number of flips.
• 30:03
  [SPEAKER_02]: And so I think one quick and crucial way to put it is that skateboarding in a kind of natural pre-institutionalized version can focus on weird subtle beauties,
• 30:15
  [SPEAKER_02]: But when you have to institutionalize a method of counting together, suddenly that changes the target to something that's easy to measure and easy to agree upon in an institutional context.
• 30:28
  [SPEAKER_02]: So, I think you need to feel for this, but I was really helped to understand this stuff by a bunch of people who do work in a field called Science and Technology Studies, and there's a little subfield there about people that study quantification culture and politics and bureaucracy.
• 30:44
  [SPEAKER_02]: There's a book from Theodore Porter called Trusted Numbers.
• 30:47
  [SPEAKER_02]: He gives us really lovely theory of what bureaucratic numbers are.
• 30:51
  [SPEAKER_02]: He says that there are basically two kinds of ways of knowing things, qualitative and quantitative.
• 30:56
  [SPEAKER_02]: And they're each good, different things.
• 31:00
  [SPEAKER_02]: And one of the problems he's worried about is when people compulsively reach for the quantitative.
• 31:05
  [SPEAKER_02]: in every case, show a matter how appropriate it is.
• 31:07
  [SPEAKER_02]: So here's his account.
• 31:09
  [SPEAKER_02]: Qualitative information is rich and open-ended and nuanced.
• 31:14
  [SPEAKER_02]: It's dynamic.
• 31:15
  [SPEAKER_02]: You can change what you're talking about.
• 31:16
  [SPEAKER_02]: You can evaluate it in subtle ways.
• 31:19
  [SPEAKER_02]: You can evaluate multiple dimensions, but it travels badly between contexts.
• 31:24
  [SPEAKER_02]: because it requires a high amount of shared background knowledge to understand.
• 31:29
  [SPEAKER_02]: Sure.
• 31:29
  [SPEAKER_02]: Quantitative knowledge and institution works like this.
• 31:32
  [SPEAKER_02]: We pre-specify some kind of context-invariant kernel and we stabilize the meaning.
• 31:39
  [SPEAKER_02]: And in order to make it cross-context, we remove all the parts of it that require a lot
• 31:44
  [SPEAKER_02]: of high-context nuance and experience to understand.
• 31:48
  [SPEAKER_02]: And once we have that nugget, it can cross contact.
• 31:50
  [SPEAKER_02]: So an easy example for me as a teacher is qualitative information.
• 31:54
  [SPEAKER_02]: It's like long, qualitative feedback I give my students on their essays about originality and thoughtfulness and all that stuff.
• 32:01
  [SPEAKER_01]: Yeah, sure.
• 32:02
  [SPEAKER_02]: And quantitative is...
• 32:04
  [SPEAKER_02]: letter grades, which is a simple, stripped-down thing where the meaning is approximately fixed across a huge amount of society, and so everyone can collect into it, everyone can know roughly agree in the meaning, and we can aggregate easily.
• 32:18
  [SPEAKER_02]: So one of the core insights from Porter is that the reason that institutional numbers are powerful is precisely because they've been engineered to be de-contextualized.
• 32:30
  [SPEAKER_02]: because that's what makes them usable across a large scale institution of many people at many different backgrounds.
• 32:36
  [SPEAKER_01]: Sure.
• 32:36
  [SPEAKER_02]: There's a bunch more to say, but one fast way of putting it is that institutional metrics are tied to institutional data under this constraint, this like de-contextualization constraint.
• 32:50
  [SPEAKER_02]: And so that means that any kind of target,
• 32:53
  [SPEAKER_02]: that you're going to try to track in an institution via metric is constrained by this imperative of de-contextualization.
• 33:03
  [SPEAKER_02]: You can't target things.
• 33:04
  [SPEAKER_02]: Then only some people in the institution who have a lot of richness and subtly and sensitivity can see, you can only target the kind of things that everyone can see.
• 33:12
  [SPEAKER_02]: So there's this hyper strong accessibility constraint.
• 33:16
  [SPEAKER_00]: Does that make sense?
• 33:17
  [SPEAKER_00]: I think so.
• 33:18
  [SPEAKER_00]: I'm trying to also assign some things about what the upshot is.
• 33:20
  [SPEAKER_00]: So is the upshot something like, okay, so look.
• 33:23
  [SPEAKER_00]: Especially in larger institutions, they like quantitative metrics.
• 33:27
  [SPEAKER_00]: They want to measure things and they don't want to do, so that's the nature of their evaluation.
• 33:32
  [SPEAKER_00]: It was primarily if not solely quantitative.
• 33:35
  [SPEAKER_00]: And there's various reasons for that.
• 33:37
  [SPEAKER_00]: It's useful in a variety of ways, blah, blah, blah, blah.
• 33:40
  [SPEAKER_00]: Okay, I get that.
• 33:41
  [SPEAKER_00]: And then if I try to relate this to games and joy, and you should try to, in terms of enjoyment, and you should try to make work into a game where
• 33:50
  [SPEAKER_00]: Maybe it's not the fact that your work is scored, but it's the way in which it scored.
• 33:55
  [SPEAKER_00]: It's scored in a kind of coarse-grained way that is not interested in how you feel about things.
• 34:02
  [SPEAKER_00]: Because probably, this is just me, I don't know if I'm reading into this or just saying, I don't think how, because those metrics, I take it and say, work typically are measuring specifically your output.
• 34:15
  [SPEAKER_00]: So you win when you produce a certain output.
• 34:18
  [SPEAKER_00]: without regard to how much you enjoy the activity, whereas that doesn't seem to be the case with scoring games.
• 34:26
  [SPEAKER_00]: Yes, it measures or it's paying attention to the outcome or the output or whatever, but it doesn't assume that success just is.
• 34:39
  [SPEAKER_00]: plausibly the rules of the game are partly and formed by the kinds of activities that people enjoy doing right if you create the rules of a game Where there's an outcome to be one, but everyone hates playing that game.
• 34:50
  [SPEAKER_00]: That's a pretty shitty game But if everyone hates playing in the game in the manufacturing facility Don't care.
• 34:56
  [SPEAKER_00]: We're not that doesn't mean that we're not gonna play the game those are the game because we want the output and so is it something like that?
• 35:00
  [SPEAKER_00]: That's the scoring system in a bureaucracy or in a workplace like that
• 35:05
  [SPEAKER_00]: Those were those that score and those were rules.
• 35:09
  [SPEAKER_00]: I don't like you can't say the word off sudden.
• 35:10
  [SPEAKER_00]: Those were created in a context that was apathetic to intrinsic enjoyment and that's not the case with games.
• 35:17
  [SPEAKER_02]: Yeah, there's there's two levels of explanation.
• 35:19
  [SPEAKER_02]: This is so the first explanation is that some games can target subtle weird rich things.
• 35:26
  [SPEAKER_02]: Yeah, that aren't mechanistic.
• 35:28
  [SPEAKER_02]: So with skateboarding, like style.
• 35:30
  [SPEAKER_02]: You can just target beautiful style.
• 35:32
  [SPEAKER_02]: Yeah, yeah.
• 35:32
  [SPEAKER_02]: I play table top role playing games.
• 35:34
  [SPEAKER_02]: Some of the games I played just give people a point for telling a good story, okay?
• 35:38
  [SPEAKER_02]: You experienced points if you did a cool dramatic thing.
• 35:41
  [SPEAKER_02]: So you can just directly target these rich subtle things if you're not shy.
• 35:46
  [SPEAKER_02]: Yeah, the institutional setting.
• 35:48
  [SPEAKER_02]: Yeah, that's level one.
• 35:48
  [SPEAKER_02]: But two, we know a lot of games do target mechanistic, very simple targets, right?
• 35:55
  [SPEAKER_02]: Get the ball in the net.
• 35:56
  [SPEAKER_02]: Sure.
• 35:57
  [SPEAKER_02]: Right.
• 35:57
  [SPEAKER_02]: Get to the top of the mountain.
• 35:59
  [SPEAKER_02]: Yeah.
• 35:59
  [SPEAKER_02]: In that case, we get the second
• 36:01
  [SPEAKER_02]: stage, which is what's really crucial is that exactly what you said.
• 36:06
  [SPEAKER_02]: A, that games are designed, we can push around the scoring system.
• 36:13
  [SPEAKER_02]: Because the scoring system in games is disattached to external world outcomes, then we can push it and change it.
• 36:21
  [SPEAKER_02]: We can push it and change it to the point,
• 36:23
  [SPEAKER_02]: to optimize for our richest of experience.
• 36:26
  [SPEAKER_02]: And then the players can shift between different systems to find the one they want.
• 36:30
  [SPEAKER_02]: And what's really key here is the kind of plasticity of scoring in games.
• 36:34
  [SPEAKER_02]: If I teach game designers sometimes, and when you're playing something, you can just be like, oh, what if you didn't score, what if instead of scoring people's writing quality?
• 36:44
  [SPEAKER_02]: Sorry, you're making some kind of drawing game.
• 36:46
  [SPEAKER_02]: What if you instead of scoring their picture quality, you scored their speed, or what if you scored how many people guess it?
• 36:51
  [SPEAKER_02]: or what if you scored, how many people knew you are fake me it?
• 36:54
  [SPEAKER_02]: Because that score is distattached to anything that matters to the world.
• 36:58
  [SPEAKER_02]: You can just shove it around and totally delightful.
• 37:01
  [SPEAKER_00]: Yeah, yeah, yeah, yeah.
• 37:02
  [SPEAKER_02]: With institutional metrics, you don't A, the thing that points at has to be mechanistically clear.
• 37:09
  [SPEAKER_02]: And then B, you don't have the capacity to do this quick and easy shoving around.
• 37:15
  [SPEAKER_02]: And one way to put it is that
• 37:17
  [SPEAKER_02]: single human designers playing a game can step back and be like, uh, this scoring system isn't working for you.
• 37:22
  [SPEAKER_02]: It doesn't feel right.
• 37:23
  [SPEAKER_02]: Yeah, right, institutions in so far as the way they value has to be expressed in metrics, in so far as that's how the justification works in an institution, don't have that standpoint to back off and say, oh, this scoring system isn't working all they have is the scoring system.
• 37:40
  [SPEAKER_02]: So this isn't, I'm not saying this absolutely.
• 37:41
  [SPEAKER_02]: So if you work are the CEO of your company and you have total control
• 37:47
  [SPEAKER_02]: and you can shove them around in any way you want to create employees as satisfaction.
• 37:51
  [SPEAKER_02]: You can be like a game designer.
• 37:53
  [SPEAKER_02]: Yeah, that's not how large scale institutes.
• 37:54
  [SPEAKER_00]: Yeah, of course.
• 37:55
  [SPEAKER_00]: No.
• 37:55
  [SPEAKER_00]: Yeah, that's a good point to make the blind referee who's reviewing your paper, which I guess they're going to break up something pedantic like that.
• 38:00
  [SPEAKER_00]: What about the CEO?
• 38:01
  [SPEAKER_00]: Okay, shut up.
• 38:02
  [SPEAKER_02]: So this is actually really important because I'm not saying that every company and every bureaucracy is inevitably going to be terrible.
• 38:10
  [SPEAKER_02]: In fact, one of the outcomes of this is, if you have control and sometimes I'm talking to, if I get to talk to a university administrator, what I'm trying to say is you do have control of your metrics.
• 38:19
  [SPEAKER_02]: Yes, they're hand-think about it, like a computer designer.
• 38:22
  [SPEAKER_02]: But the problem is that isn't typically how bureaucracies work because bureaucracies in so far
• 38:30
  [SPEAKER_02]: Insofar as the metrics are how they evaluate things, insofar as that's how they figure out what's important, then that's the buck stops there.
• 38:39
  [SPEAKER_02]: Like there's no further standpoint to modify the scoring system.
• 38:42
  [SPEAKER_02]: And they're limited.
• 38:44
  [SPEAKER_00]: So let me ask you two, I think related questions.
• 38:47
  [SPEAKER_00]: One is,
• 38:49
  [SPEAKER_00]: I'm going to put your things slightly different than you tell me if it's a misrepresentation somehow.
• 38:54
  [SPEAKER_00]: The supposed to put on like a contene hat, and I say, if it seems like what you're saying is that metrics in a bureaucracy are created in a way that treats the people that they're measuring as mirror means to an end, the outcome, the output, whatever, whereas metrics
• 39:12
  [SPEAKER_00]: design in a game setting are treated with respect for the person and sense of what they would find delightful, as you said.
• 39:19
  [SPEAKER_00]: So is it a matter of something like, yeah, how the evaluation as it works treats the person that's being evaluated as a mere tool to bring about some output, some outcome versus as a being who intrinsically enjoys some activities, not others, blah, blah, blah, blah.
• 39:35
  [SPEAKER_00]: You're shaking your head, no.
• 39:36
  [SPEAKER_00]: So good.
• 39:36
  [SPEAKER_02]: I mean, that's too optimistic.
• 39:38
  [SPEAKER_02]: Okay,
• 39:39
  [SPEAKER_02]: Okay, so, if that were true, then good-hearted, it blurscale would be your actually, it loved people, it wouldn't be in this problem.
• 39:47
  [SPEAKER_02]: I don't think that's true.
• 39:48
  [SPEAKER_02]: My worry is that this, so, some of the key studies that I was really interested in here.
• 39:54
  [SPEAKER_02]: So, Sally Engelmarie has this incredible book, called the Directions of Quantification.
• 39:58
  [SPEAKER_02]: And it was looking at the UN attempt to quantify
• 40:02
  [SPEAKER_02]: growth development and human rights and in particular to quantify well-being.
• 40:07
  [SPEAKER_02]: So if you think, let's assume for the second, it's pretty plausible that you and this is a good hard institution that wants people to be better.
• 40:14
  [SPEAKER_02]: It's trying to maximize well-being.
• 40:16
  [SPEAKER_02]: The problem is that well-being for this institution is going to be constrained by what can be counted in this de-contextualizing, de-metricizing way.
• 40:24
  [SPEAKER_02]: I mean, one of my worries is, yeah, this is a problem for
• 40:30
  [SPEAKER_02]: corporate overlords.
• 40:32
  [SPEAKER_00]: Sure, sure.
• 40:32
  [SPEAKER_02]: I think the real, the thing, the really pessimistic worry for me is that the best-hearted health policy and the best-hearted people interested in well-being can be caught by the same problem if they're going to be guided by, by worries, this is about a limitation on data in general, not
• 40:53
  [SPEAKER_02]: The problem here is not about selfishness or treating people as it means to an end, but it's just a problem of the day-to-based way of looking at the world that there's certain kinds of things that data has trouble picking up on, which are highly contextual, highly variable things that require high degree of sensitivity to judge, and that even the best part of the person that treated people as an end, that the kinds of ends that people can target.
• 41:18
  [SPEAKER_02]: I mean, okay, let me give you some easy examples.
• 41:21
  [SPEAKER_02]: I was really, okay, well, let's get me canceled.
• 41:24
  [SPEAKER_02]: I was really worried, really, the COVID epidemic, that I was worried the way that public health strategies it saved people's lives, that also had enormous costs in happiness, culture,
• 41:37
  [SPEAKER_02]: community, that the argument in favor of saving lives would always win.
• 41:42
  [SPEAKER_02]: That's the kind of thing that's easy to gather.
• 41:44
  [SPEAKER_02]: Yeah, yeah.
• 41:45
  [SPEAKER_02]: So there's a moment in my book, one of my, you might like, this is, I think, two geeky for a lot of audiences, but you might be happy about this.
• 41:51
  [SPEAKER_02]: I was writing about how, when you're, when you're doing health policy about something, like saturated fats, and you're weighing on the one hand, the fact that saturated fats increase heart attack rates.
• 42:02
  [SPEAKER_02]: On the other hand, you're weighing like,
• 42:04
  [SPEAKER_02]: The deliciousness, how would it be?
• 42:06
  [SPEAKER_02]: Yeah, yeah, yeah, yeah, yeah, yeah.
• 42:08
  [SPEAKER_02]: The deliciousness of a good breed is really hard to quantify mechanically and to tend to lose the fight.
• 42:13
  [SPEAKER_02]: And when people want to say is, but we can't, we can't, how do we waste such an effable things as the goodness of a good breed?
• 42:21
  [SPEAKER_02]: What I want to say is, what's more?
• 42:23
  [SPEAKER_02]: Fable, what's more real than a good cheese?
• 42:26
  [SPEAKER_02]: What people actually mean is the goodest of cheese that's hard to abstract into something that's institutionally countable.
• 42:33
  [SPEAKER_00]: And so you can't eat any of my work is the same scale, something on the other side, yeah.
• 42:38
  [SPEAKER_02]: Yeah, so my worries are a sense of institutional accountability, is seized our sense of reality, such that we somehow think that the deliciousest of cheese is not real, because it's highly variable between people,
• 42:53
  [SPEAKER_02]: Many times people say something isn't real, but what they mean is there's not a mechanical procedure by which different people can count it reliably.
• 43:01
  [SPEAKER_00]: Yeah, yeah, so one thing that sort of relates to this is that like I was looking at my LinkedIn feed or whatever, and someone said, it doesn't matter what the decision was about.
• 43:08
  [SPEAKER_00]: He said, when it comes to such a decision, here's the algorithm that I use.
• 43:12
  [SPEAKER_00]: And then I pointed out one, if this, then though, and I thought, who thinks via algorithm?
• 43:16
  [SPEAKER_00]: What a weird way to conceive of your own decision making.
• 43:20
  [SPEAKER_00]: You should think of it as,
• 43:22
  [SPEAKER_00]: weighing various pros and cons, perhaps a reason for and against, and there's the weightiness of those reasons, but it's not mechanical like that.
• 43:30
  [SPEAKER_00]: Yeah.
• 43:31
  [SPEAKER_02]: Yeah.
• 43:32
  [SPEAKER_02]: Okay.
• 43:32
  [SPEAKER_02]: This is, I get to tell you about my favorite thing in the world that way.
• 43:36
  [SPEAKER_02]: It might become your favorite thing in the world.
• 43:37
  [SPEAKER_02]: I was trying to understand exactly this, this is exactly right.
• 43:40
  [SPEAKER_02]: One thing I want to say is, I often have conversations like this where I'm like, this is good and people are like, how do you know it's good?
• 43:45
  [SPEAKER_02]: I'm like, it's delicious, but give me the mechanical procedure to determine for sure if it's delicious.
• 43:52
  [SPEAKER_02]: What I think is this mindset.
• 43:55
  [SPEAKER_02]: is a bureaucratic mindset that's invaded many of our soul, but there's a really good story about what exactly it isn't where it comes from.
• 44:02
  [SPEAKER_02]: So I was really stuck on this point for a while.
• 44:03
  [SPEAKER_02]: This field, there was something going on really important that I couldn't put my finger at it on right in this space and I read Lorraine Daston's book rules, which is the best book.
• 44:12
  [SPEAKER_02]: So learning Dyson is an intellectual historian.
• 44:15
  [SPEAKER_02]: Her early work is about how the concept of objectivity is very over time, and this new book rules is about how the concept of a rule has changed over time, and she thinks the old conception of a rule that was dominant for most of human history is a rule as what she calls a printable, which is a general statement that admits of many exceptions where you need judgment and discernment.
• 44:37
  [SPEAKER_02]: Okay.
• 44:37
  [SPEAKER_02]: To know where the exceptions are.
• 44:38
  [SPEAKER_01]: Like a guideline.
• 44:39
  [SPEAKER_02]: Yeah.
• 44:40
  [SPEAKER_02]: So guys.
• 44:41
  [SPEAKER_02]: Typical example for me is like, show, don't tell, right?
• 44:45
  [SPEAKER_02]: Generally good principle.
• 44:46
  [SPEAKER_02]: There's a reason behind it.
• 44:47
  [SPEAKER_02]: You can break it all the time.
• 44:48
  [SPEAKER_02]: Sure.
• 44:48
  [SPEAKER_02]: If you understand the principle.
• 44:49
  [SPEAKER_02]: Yeah.
• 44:50
  [SPEAKER_02]: But you can't give a mechanical list of how what the exceptions are.
• 44:54
  [SPEAKER_02]: Yeah.
• 44:54
  [SPEAKER_02]: So she says that there's a new thing.
• 44:56
  [SPEAKER_02]: She calls it an algorithmic rule.
• 44:57
  [SPEAKER_02]: Identical and mechanical rule.
• 44:59
  [SPEAKER_02]: She says that it's designed to be followed unthakingly and absolutely.
• 45:03
  [SPEAKER_02]: Yeah.
• 45:04
  [SPEAKER_02]: And then she says, and this is the part of the blue line line.
• 45:06
  [SPEAKER_02]: She says, people think that these eros are on calculating machines.
• 45:09
  [SPEAKER_02]: They actually rose a hundred years earlier as an attempt to cheapen labor.
• 45:13
  [SPEAKER_02]: Yes, by creating clear rules that anyone without expertise or training could follow.
• 45:20
  [SPEAKER_02]: So I think the underlying feature, the kind of underlying logic of bureaucracies is actually worker fungibility.
• 45:29
  [SPEAKER_02]: Like, a, one of the core features of bureaucratic logic is you want to be able to hire and fire people at will and slot in someone new.
• 45:37
  [SPEAKER_02]: And you don't need long sensitivity, long experience.
• 45:40
  [SPEAKER_02]: What you just need is, does this person have a high school to be, or does it, do they have a BA then I can give me this ruleable?
• 45:45
  [SPEAKER_01]: Yeah, yeah.
• 45:46
  [SPEAKER_02]: And they'll perform exactly the same.
• 45:47
  [SPEAKER_01]: Yeah, yeah, yeah.
• 45:48
  [SPEAKER_02]: So what algorithmity here is one way to put it is the more mechanical something is, the more you can get any worker to be put in the same slot.
• 45:57
  [SPEAKER_02]: Yeah, I'm getting the exactly the consistent result.
• 45:59
  [SPEAKER_00]: Yeah, it's so funny that you're saying all this because I literally read something like this, maybe a month ago was reading a book called Team of Teams and it talked with a history of business management and I forgot the guy's name like Murphy or Tyler or something like that.
• 46:11
  [SPEAKER_00]: who's like, who created the science of management, which was exactly what you said, which is what's going from artisans, and you said, no, I could make this more efficient if I create rules that everyone follows, no one exercises judgment, they just do this, then they do this, then they do that, and that's how we maximize efficiency and maximize their by maximized profits, et cetera.
• 46:28
  [SPEAKER_00]: So here you're 100%, and, right, there's, it squeezes out the value of judgment.
• 46:34
  [SPEAKER_00]: Well, we probably call it practical judgment, just something along those lines.
• 46:37
  [SPEAKER_02]: Actually, I forgot to introduce one of the key terms for the books.
• 46:40
  [SPEAKER_02]: The other key concepts is driving play is value capture.
• 46:43
  [SPEAKER_02]: So value captures what happens when you have rich and subtle values or you're developing them.
• 46:48
  [SPEAKER_02]: And you get parked in the institution that feed you simplified, typically quantified versions and the simplified versions take over.
• 46:54
  [SPEAKER_02]: And I think value capture is just incentives.
• 46:56
  [SPEAKER_02]: It's no, you care about citations.
• 46:58
  [SPEAKER_02]: You care about page views.
• 47:00
  [SPEAKER_02]: So one way to put the problem of value capture is that you're internalizing
• 47:05
  [SPEAKER_02]: the demand for mechanicity.
• 47:08
  [SPEAKER_02]: You're an internalizing, you're only letting yourself be ruled by values that I've been designed to be usable by fungible workers, which means they cannot trigger off anything particular to you.
• 47:19
  [SPEAKER_00]: And this is why social media, one reason by social media sucks so bad.
• 47:23
  [SPEAKER_00]: Because people just do things that they think they're supposed to do to maximize whatever likes or reactions or comments without any sense of sometimes I think I'm not going to post that because it would be braggadotious and thereby undignified and that's just a thing I think that occurs to most people.
• 47:39
  [SPEAKER_00]: They're bragging so much on social media the only one that I really use as LinkedIn because my business but.
• 47:45
  [SPEAKER_00]: They're shameless, it's same, and I was just like, people have told me, oh, you should post more about this or that.
• 47:51
  [SPEAKER_00]: So they know that you're unlike a note, that's undignified.
• 47:53
  [SPEAKER_00]: I won't do it.
• 47:55
  [SPEAKER_00]: But they just know, you're trying to just make the error go up.
• 47:57
  [SPEAKER_00]: So that's no, I'm not.
• 47:59
  [SPEAKER_00]: That's a dumb way to live.
• 48:01
  [SPEAKER_02]: Yeah, this is just another case where, so one of the key, one of the places I started thinking about this was thinking about how Twitter changed community patterns.
• 48:09
  [SPEAKER_02]: Another place where you can think about the difference between what's easy to mechanically capture and what's important,
• 48:16
  [SPEAKER_02]: Twitter registers a like, which is a simple binary.
• 48:19
  [SPEAKER_02]: Yeah.
• 48:20
  [SPEAKER_02]: It does not capture information about whether someone was moved.
• 48:23
  [SPEAKER_02]: It does not capture deep transformation.
• 48:25
  [SPEAKER_02]: It captures a simple binary in this first moment of exposure.
• 48:28
  [SPEAKER_00]: Yeah, it's like a gripe approval.
• 48:31
  [UNKNOWN]: Right.
• 48:31
  [SPEAKER_02]: Exactly.
• 48:32
  [SPEAKER_02]: They're a bunch of approval.
• 48:33
  [SPEAKER_02]: And then aggregate.
• 48:34
  [SPEAKER_02]: It aggregates that because it's easy to capture and so easy to aggregate.
• 48:38
  [SPEAKER_02]: Yeah.
• 48:39
  [SPEAKER_02]: So let's let's a little vision of I think everything that metrics are doing to us.
• 48:43
  [SPEAKER_02]: Yeah, which is in so far as you're motivated by them, you're putting it in your center.
• 48:51
  [SPEAKER_02]: Values that have to pass the test of mechanical hyper accessible hyper-fungible usability.
• 48:57
  [SPEAKER_02]: Yeah.
• 48:57
  [SPEAKER_02]: Which means they cannot trigger off of anything.
• 49:01
  [SPEAKER_02]: delicate, sensitive, particular to you, born of your rich experience of the world.
• 49:06
  [SPEAKER_02]: They just have to trigger off of things that everyone can instantly see.
• 49:10
  [SPEAKER_00]: Yeah.
• 49:11
  [SPEAKER_00]: Let me ask you one final question, I was to wrap up.
• 49:14
  [SPEAKER_00]: I want to say a little bit more about this sort of choosing
• 49:17
  [SPEAKER_00]: Choosing but the work or the choosing to turn work into a game or something.
• 49:22
  [SPEAKER_00]: I know you're not saying something like, you should choose what you're passionate about.
• 49:26
  [SPEAKER_00]: This has become the sort of common trope.
• 49:29
  [SPEAKER_00]: So for the audience I'd say, may, can you disentangle the difference between, hey, you should choose the kind of work that you're passionate about from your sort of thesis about, choose the work that becomes a game or is a game or, et cetera, et cetera.
• 49:42
  [SPEAKER_02]: There are two ways to put it.
• 49:48
  [SPEAKER_02]: I'm saying, that's what I'm saying in a complicated way.
• 49:50
  [SPEAKER_02]: And what I'm saying is in particular don't let your passion to be redefined by what's easy to count or easy to express.
• 49:59
  [SPEAKER_02]: Listen, it's, of course, choosing your passion about, and then a lot of people's passions translate into something become captured and transformed into something that I'm passionate about.
• 50:10
  [SPEAKER_02]: Journalism and connection.
• 50:12
  [SPEAKER_02]: So now I care about page views or follows.
• 50:15
  [SPEAKER_02]: Yeah, yeah, yeah, yeah.
• 50:17
  [SPEAKER_02]: The worry for me in this case is a lot of the times that even that phrase gets captured because your passions themselves get retranslated by metrics.
• 50:27
  [SPEAKER_02]: And the thing I'm really trying to push for is to listen for the voice of the thing that is hard to quantify,
• 50:36
  [SPEAKER_02]: that is like the richer version of a passion.
• 50:39
  [SPEAKER_02]: So that's a simple thing.
• 50:40
  [SPEAKER_02]: Yeah.
• 50:40
  [SPEAKER_02]: The more complicated thing is I don't think Choose your passion is great advice when you don't have control over the social structure.
• 50:46
  [SPEAKER_02]: Right, in a sense, this is a game design issue and we're not all game designers.
• 50:50
  [SPEAKER_02]: A lot of us are forced to play a vast,
• 50:54
  [SPEAKER_02]: horribly designed that we have no control over, right?
• 50:58
  [SPEAKER_02]: So, I think a lot of work life for a lot of people is, you can choose either shitty monopoly or extra shitty monopoly, choose your passion, yeah, yeah, yeah, yeah, yeah, yeah, yeah, oh, crappy, yeah, and we can give you, we can explain reasons why they're crappy.
• 51:12
  [SPEAKER_02]: Part of the sorrow, the sorrow,
• 51:14
  [SPEAKER_02]: of thinking about this for me is that there's not an easy answer.
• 51:17
  [SPEAKER_02]: Choose your passion.
• 51:18
  [SPEAKER_02]: A lot of the times, this is a story about why large Shelby orocracies have systematically both gained power and ignore everything beautiful and weird and subtle.
• 51:27
  [SPEAKER_02]: But then they have all the power.
• 51:29
  [SPEAKER_02]: So it's not, oh, choose your passion.
• 51:30
  [SPEAKER_02]: Yeah.
• 51:30
  [SPEAKER_02]: Yes, crap at the end of the book, this is a little argument, maybe this will appeal to you the argument is this is I call the value collapse argument.
• 51:36
  [SPEAKER_02]: It just looks like this so imagine that there's some simple metric about what matters.
• 51:42
  [SPEAKER_02]: So in my space and education it's like graduation right imagine there's some simple metric and there are two populations there's one population of people that know that's important but also care about other things and then there's another population that's fully value captured they are all the way in the back.
• 51:58
  [SPEAKER_02]: They're going to be more efficient at hitting that metric.
• 52:01
  [SPEAKER_02]: And insofar as that metric is used to confer social power, into far as people get promotions and power based on hitting the metric, then our institution will systematically pick for people who are willing to sacrifice everything of excellence important in favor of the simple metric.
• 52:17
  [SPEAKER_00]: Yeah.
• 52:18
  [SPEAKER_02]: And if they, if such people tend to accumulate social power, they'll tend to make institutions or more metrics oriented.
• 52:26
  [SPEAKER_02]: And so there's this horrible feedback loop.
• 52:30
  [SPEAKER_02]: Based on the fact that the metrics are basically engineered, easy justification and engineered, easy communication.
• 52:36
  [SPEAKER_02]: Yeah.
• 52:36
  [SPEAKER_02]: So it's hard for me to say, choose your passion when there's this story about how the power structure of the world inevitably tend to focus on and give us systems design that are oriented not towards our passions.
• 52:53
  [SPEAKER_00]: That's a great optimistic note to end on, I think, thank you so much for doing this, this is super interesting.
• 53:07
  [SPEAKER_00]: Great stuff.
• 53:08
  [SPEAKER_00]: Ever go read the score.
• 53:09
  [SPEAKER_00]: Thank you so much, Ren.
• 53:12
  [SPEAKER_00]: Thanks everyone for listening.
• 53:14
  [SPEAKER_00]: I hope you enjoyed the episode.
• 53:15
  [SPEAKER_00]: And I want to thank a few people who make this all possible.
• 53:18
  [SPEAKER_00]: Delano Alexander, he's my producer.
• 53:19
  [SPEAKER_00]: He does such an amazing job on everything that he touches.
• 53:22
  [SPEAKER_00]: Garrett Lane is the sound engineer.
• 53:24
  [SPEAKER_00]: That's honestly, he's just, he's too good for me.
• 53:26
  [SPEAKER_00]: Authamia Jairam has joined us recently.
• 53:28
  [SPEAKER_00]: He's helping us find the best guests to bring on to this podcast.
• 53:32
  [SPEAKER_00]: And Ashley Wasselenko helps us organize all this.
• 53:34
  [SPEAKER_00]: Get people on the schedule.
• 53:36
  [SPEAKER_00]: Get people's bios, headshots, and really keeps the train on the track.
• 53:39
  [SPEAKER_00]: So thanks guys, really appreciate all the hard work you do.