Latest / Masters in Business / Joshua Miller Discusses the Hot Hand Phenomenon (Podcast)

Transcript

Joshua Miller Discusses the Hot Hand Phenomenon (Podcast) · Masters in Business

• 0:02
  Speaker 1: This is Masters in Business with Barry Ridholts on Bloomberg Radio.
• 0:10
  Speaker 1: This week on the podcast, I have a fascinating guest,
• 0:14
  Speaker 1: And if you're at all interested in statistical analysis of sports, UH,
• 0:20
  Speaker 1: behavioral finance data analysis, understanding um streakiness, understanding the money
• 0:27
  Speaker 1: hole problem, and then extrapolating that towards things like, UH,
• 0:32
  Speaker 1: the hot hand in basketball, you're gonna find this to
• 0:35
  Speaker 1: be absolutely fascinating. UH. Joshua Benjamin Miller comes from California,
• 0:40
  Speaker 1: where he basically racked up all the degrees he could
• 0:44
  Speaker 1: at some of the u C schools before getting his
• 0:47
  Speaker 1: PhD at in economics UH in Minnesota. Josh and his
• 0:51
  Speaker 1: co author have taken apart some of the more interesting
• 0:55
  Speaker 1: statistical assumptions made UH in the original hand study with
• 1:01
  Speaker 1: Tom Gilovich and Amos Tversky UH, and they found something
• 1:05
  Speaker 1: really unusual by looking at the data from a slightly
• 1:10
  Speaker 1: different perspective. And I approached their paper with tremendous amount
• 1:14
  Speaker 1: of skepticism. I thought the randomness of the hot hands
• 1:19
  Speaker 1: was a fairly well proven study that Tversky and Gilovich did,
• 1:25
  Speaker 1: But when you look at the data and you look
• 1:26
  Speaker 1: at how they analyzed it, it's hard not to reach
• 1:30
  Speaker 1: the conclusion that there is some sort of a hot hand.
• 1:33
  Speaker 1: It's quite sophisticated mathematics, but Josh does a very nice
• 1:39
  Speaker 1: job reducing it to some very easily understandable UM. Probability,
• 1:45
  Speaker 1: we don't we don't know math is required. Um, you
• 1:49
  Speaker 1: just have to know the difference between a head or
• 1:51
  Speaker 1: a tail when you're flipping a coin. If you're at
• 1:53
  Speaker 1: all interested in anything probability, sports related statistical, you're gonna
• 1:58
  Speaker 1: find this to be a fascinating, lee wonky and tremendously
• 2:02
  Speaker 1: interesting conversation. So, with no further ado, my conversation with
• 2:07
  Speaker 1: the economists and statistician Josh Miller. My special guest today
• 2:17
  Speaker 1: is Joshua Benjamin Miller UH. He is the co author,
• 2:21
  Speaker 1: along with Adam Sanjorro, of a fascinating paper that puts
• 2:27
  Speaker 1: challenge to the myth of the myth of the hot hands. Uh.
• 2:31
  Speaker 1: He comes to us with a BA in economics and
• 2:34
  Speaker 1: an m a UH in mathewmatical statistics from UC Santa Barbara.
• 2:38
  Speaker 1: He has his PhD from University of Minnesota, and he
• 2:42
  Speaker 1: is currently a professor in the economics department at the
• 2:46
  Speaker 1: University of Alacante. In Spain, where he focuses his research
• 2:51
  Speaker 1: on behavioral economics, judgment and decision making, game theory, and
• 2:57
  Speaker 1: statistical and experimental methods. Josh Miller, Welcome to Bloomberg. Thanks
• 3:02
  Speaker 1: for having me, Barry, So a little background. We kind
• 3:06
  Speaker 1: of met after I interviewed Thomas Gilovich, who I was
• 3:10
  Speaker 1: mostly interested in due to all of his work on
• 3:13
  Speaker 1: behavioral finance, but he also co authored a fascinating paper,
• 3:18
  Speaker 1: uh that basically pointed out the hot hands. You co
• 3:22
  Speaker 1: authored that with Amos Tversky, by the way, that the
• 3:24
  Speaker 1: hot hand was really a myth and it was just
• 3:27
  Speaker 1: we were all being fooled by randomness. How did that
• 3:30
  Speaker 1: paper come to your attention? And what what fascinated you
• 3:34
  Speaker 1: buy it? So that paper came to the attention of
• 3:37
  Speaker 1: my co author Adam who's also at University of Vallecante.
• 3:40
  Speaker 1: Pretty Much everyone who takes a behavioral economics class and
• 3:43
  Speaker 1: even earlier gets exposed to that paper. It's like it's
• 3:46
  Speaker 1: one of the you know, the prime examples of a
• 3:48
  Speaker 1: bias because it's such apparently power. It's part of the
• 3:51
  Speaker 1: canon of oh, look how easily we're all fooled exactly,
• 3:55
  Speaker 1: and in the in the beginning of any kind of
• 3:57
  Speaker 1: behavior economics class, you have to show the real world
• 4:00
  Speaker 1: implications first to kind of motivate students. And here is
• 4:02
  Speaker 1: this um one that professionals are faul victim too, and
• 4:07
  Speaker 1: they're so resistant to it. I mean they were shown
• 4:10
  Speaker 1: that this hot I mean, we haven't to find hot
• 4:11
  Speaker 1: hand yet, but you know, the hot hand is this
• 4:13
  Speaker 1: idea that you're in the zone. You know that success
• 4:15
  Speaker 1: breeds success. And if you look at basketball players, UM
• 4:19
  Speaker 1: and coaches, they all believe in this thing. And so
• 4:21
  Speaker 1: when they discovered that there was no pattern there and
• 4:25
  Speaker 1: they came and revealed revealed the output of their research,
• 4:29
  Speaker 1: the professionals were it was difficult to convince them. Oh,
• 4:32
  Speaker 1: there was tremendous pushback. There's a famous quote you that's
• 4:36
  Speaker 1: been referenced was red or back up at the Boston Celtics.
• 4:40
  Speaker 1: I don't I don't care what this professor says. So
• 4:43
  Speaker 1: they do a study. Who cares? Right? Yeah? So, I
• 4:45
  Speaker 1: mean the stubbornness that that came out of the practitioners
• 4:48
  Speaker 1: was was really dramatic because typically you can convince someone
• 4:51
  Speaker 1: that is motivated to get things right if you can
• 4:53
  Speaker 1: demonstrate that they'll benefit from it, Um, and they just
• 4:57
  Speaker 1: discounted it. And so there's this famous quote from Amis Diversky,
• 5:00
  Speaker 1: after all the stubbornness that they encountered repeatedly, of people
• 5:03
  Speaker 1: just not even looking at the evidence they were showing them,
• 5:05
  Speaker 1: Um that he said, I've been in thousand arguments, one
• 5:08
  Speaker 1: them all, but convinced no one. And he was He
• 5:13
  Speaker 1: was very famous for being um, not only quite brilliant,
• 5:17
  Speaker 1: but a little hardheaded and a little aggressive when it
• 5:19
  Speaker 1: came to debating people. According at least according right, at
• 5:24
  Speaker 1: least according to Michael Lewis's book The Undoing Project. Between
• 5:28
  Speaker 1: Konomen and Seversky, they were two very distinct personality types.
• 5:32
  Speaker 1: So so let's get back to you before we were
• 5:34
  Speaker 1: gonna spend a lot of time on the hot hands. Um,
• 5:36
  Speaker 1: you're not what I would think of as a traditional economist.
• 5:41
  Speaker 1: What what sort of work do you focus on? Both
• 5:43
  Speaker 1: my ca author and I focused on individual decision making, right,
• 5:46
  Speaker 1: so we're looking at is it individual decision making within
• 5:49
  Speaker 1: a group, within an institution, or just as a lone wolf.
• 5:53
  Speaker 1: So so there you know, there are the psychological factors
• 5:56
  Speaker 1: like UM, my co author works on search and attention
• 6:00
  Speaker 1: and things like this UM, but there are also UM
• 6:03
  Speaker 1: factors of the institution, the design, like how information is
• 6:06
  Speaker 1: presented to you UM, and these things, while it may
• 6:09
  Speaker 1: be important to an individual, they also bubble up in
• 6:12
  Speaker 1: terms of how it affects, you know, decision making in groups,
• 6:15
  Speaker 1: and how it affects financial markets, and so in the
• 6:17
  Speaker 1: end it does impact UM policy and economic outcomes. It
• 6:22
  Speaker 1: has real world effects. In other words, these aren't just
• 6:25
  Speaker 1: ivory tower abstract discussions. There's real world application for how
• 6:30
  Speaker 1: decisions are making and how information is presented, so that
• 6:34
  Speaker 1: that's really quite interesting. UM. When you and you mentioned
• 6:37
  Speaker 1: one of your research areas is behavioral finance. Has all
• 6:41
  Speaker 1: the low hanging fruit in this space been picked or
• 6:44
  Speaker 1: is there still lots and lots of things to be discovered.
• 6:47
  Speaker 1: There's still lots of fruit, whether it's low hanging. I
• 6:51
  Speaker 1: think you always have to work for it, right right, UM,
• 6:53
  Speaker 1: So I think the way you get the fruit, you
• 6:57
  Speaker 1: have to think a lot about how to measure things,
• 6:59
  Speaker 1: how a theoretical grounding and what you're trying to get at,
• 7:03
  Speaker 1: And you can't just rely on existing data and existing
• 7:06
  Speaker 1: things that have been counted. You have to go out
• 7:08
  Speaker 1: and measure things yourself. A bit um and do some
• 7:11
  Speaker 1: work to collect that data. UM and so you know what,
• 7:15
  Speaker 1: Like so a lot of the modern work you'll see
• 7:17
  Speaker 1: is going beyond just the choices that people make, like
• 7:19
  Speaker 1: when you're paying them to make decisions and looking at
• 7:21
  Speaker 1: their choices, and you can get a lot more about
• 7:26
  Speaker 1: You learn a lot more about what people want and
• 7:27
  Speaker 1: what they believe by looking at other things like reaction times,
• 7:31
  Speaker 1: how they search for things, how you know what they're
• 7:33
  Speaker 1: paying attention to, um And in other words, you're not
• 7:37
  Speaker 1: just bringing in a bunch of undergrads, sticking them in
• 7:39
  Speaker 1: a room, giving them twenty bucks for the night, and
• 7:41
  Speaker 1: saying we're gonna put you through a series of things.
• 7:44
  Speaker 1: You're looking at very You're looking at a very different
• 7:46
  Speaker 1: data set that's measuring very different things. Yeah, I mean
• 7:50
  Speaker 1: you can improve even with the undergrads. But I think
• 7:52
  Speaker 1: a lot of the innovative work goes and collects unique
• 7:55
  Speaker 1: data from unique subjects. Like I have a friend alex Emis,
• 7:58
  Speaker 1: I just present this very interesting it's on in the
• 8:01
  Speaker 1: top topic of finance. We went and looked at institutional
• 8:03
  Speaker 1: investors tons of data. These are people with big positions um,
• 8:08
  Speaker 1: and found that there they're actually quite skilled at buying um,
• 8:13
  Speaker 1: buying stocks, but they aren't so skilled at selling them
• 8:15
  Speaker 1: UM and it seems to be distinct skills. It's very
• 8:19
  Speaker 1: distinct skills because it's easy to buy, that's the easy part.
• 8:22
  Speaker 1: Sellings where the money gets made, those are not um
• 8:26
  Speaker 1: equal level of difficulty things. I'm absolutely not surprised to
• 8:30
  Speaker 1: hear the selling demonstrates less skill than buying. Is that
• 8:35
  Speaker 1: basically what you know? So the finding is that they
• 8:38
  Speaker 1: were unloaded extreme winners um too quickly, um before they
• 8:44
  Speaker 1: really exploited the you know, information advantage that they had.
• 8:47
  Speaker 1: So they made a good job choosing it, but they
• 8:49
  Speaker 1: sold it too soon, a classic mistake. Let's talk a
• 8:52
  Speaker 1: little bit about the original hot hand paper, which, as
• 8:59
  Speaker 1: we discussed earlier, became cannon in the world of behavioral finance. Uh,
• 9:05
  Speaker 1: when did you first start to get an inkling that
• 9:09
  Speaker 1: the original thesis might not have been all it seemed
• 9:14
  Speaker 1: to be. So the original inkling was that people sometimes overreact.
• 9:18
  Speaker 1: It's that it's a myth, the thing doesn't exist. Why
• 9:21
  Speaker 1: does that generate such a strong intuitive pushback from people?
• 9:25
  Speaker 1: I mean, I have my thesis, I'm curious as to yours.
• 9:28
  Speaker 1: I think everyone has some experience in their own athletic
• 9:32
  Speaker 1: performance where they have moments where they're particularly locked in,
• 9:36
  Speaker 1: and then they realized that outside of athletic performance, you
• 9:40
  Speaker 1: just have these moments where you're in the you're in
• 9:42
  Speaker 1: the zone, that's the best word for it. Um. You're
• 9:45
  Speaker 1: just you're firing on all cylinders, and you would expect
• 9:48
  Speaker 1: that you would see that um somehow in basketball data
• 9:51
  Speaker 1: as well. So so my personal experience. I used to
• 9:54
  Speaker 1: play hoops as a kid, but as I've gotten older,
• 9:57
  Speaker 1: I've become a tennis player, and I know from personal
• 10:00
  Speaker 1: experience it takes a good twenty minutes for me to
• 10:03
  Speaker 1: calibrate my forehand so that I am consistently hitting the
• 10:08
  Speaker 1: bowl more or less towards where I want it, more
• 10:12
  Speaker 1: or less with the right amount of spin, more or
• 10:14
  Speaker 1: less with the right height. But it's not something that
• 10:17
  Speaker 1: I could just grab a racket and swing and oh
• 10:19
  Speaker 1: there it is. It takes a while, too too fast,
• 10:22
  Speaker 1: too much whip, keep it loosening your wrist, bringing around,
• 10:25
  Speaker 1: make sure you're dropping the head like I'm running through
• 10:28
  Speaker 1: a series of steps in my head. Hey you're too close.
• 10:30
  Speaker 1: Watch your footwork well, one after another and I am
• 10:34
  Speaker 1: now good enough to know I suck. I'm in that
• 10:36
  Speaker 1: Dunning Krueger drop where Oh you know, I used to
• 10:40
  Speaker 1: think I was good. Now I realize I'm really I'm
• 10:43
  Speaker 1: good enough to know how good I actually am not. Um,
• 10:46
  Speaker 1: But it takes a while to calibrate that. I imagine
• 10:49
  Speaker 1: a basketball player in the midst of a game has
• 10:52
  Speaker 1: to go through some sort of fine tuning of their shooting.
• 10:56
  Speaker 1: You can warm up all you want, um, when you're
• 10:59
  Speaker 1: just shooting by yourself before the game, but when people
• 11:02
  Speaker 1: are on you and you're running, it has to be
• 11:03
  Speaker 1: a very very different set of circumstances. Or am I
• 11:07
  Speaker 1: overstating this? Well, I mean that's that's the strongest intuition
• 11:11
  Speaker 1: is based on this calibration thing. I mean there's probably
• 11:14
  Speaker 1: other elements. Will not get to that later. Um, but
• 11:16
  Speaker 1: if you look at um, yeah, I mean, if you're
• 11:18
  Speaker 1: sitting on the bench for ten minutes and then you
• 11:20
  Speaker 1: come off, that's very different. I mean in the NFL
• 11:23
  Speaker 1: you see field goal kickers warming up go on. You
• 11:26
  Speaker 1: don't see that so much in the NBA. They don't
• 11:27
  Speaker 1: have like an extra hoop on this, that's right, So
• 11:30
  Speaker 1: I'd imagine, yeah, that's an important element there. What else
• 11:32
  Speaker 1: is so intuitively attractive about the idea of the hot hand?
• 11:37
  Speaker 1: Is it simply just the zone? Is it the adrenaline
• 11:41
  Speaker 1: and the endorphins. Why do we think that, hey, suddenly
• 11:45
  Speaker 1: I'm on I'm on a streak. Why do we believe
• 11:46
  Speaker 1: that streak is going to continue? And I'm not talking
• 11:49
  Speaker 1: about blackjack or roulette or games of chance. These are
• 11:53
  Speaker 1: really games of skill played at the highest level. So
• 11:57
  Speaker 1: why we believe it? Um? I would imagine sometimes when
• 12:00
  Speaker 1: we believe it, it's not really there. And so there
• 12:02
  Speaker 1: is this feeling, you know. You know, part part of
• 12:05
  Speaker 1: the feeling is the feedback, right you You see that
• 12:07
  Speaker 1: your successful gives you some confidence, you know. So it's
• 12:10
  Speaker 1: not always simply this zone that emerges. Um. Sometimes you
• 12:14
  Speaker 1: get a few successes in a row and it gives
• 12:16
  Speaker 1: you more confidence in your training. You don't overthink it,
• 12:20
  Speaker 1: and you return and trust your training. So you're essentially
• 12:23
  Speaker 1: unconscious and you bring that whereas if you maybe miss
• 12:25
  Speaker 1: a funeral, you lose your confidence. You start making adjustments,
• 12:29
  Speaker 1: and if you're making adjustments, you're not going to have
• 12:31
  Speaker 1: much consistency. So let's go back to the original research.
• 12:35
  Speaker 1: Tom Gilovich one of the co authors said about the
• 12:38
  Speaker 1: work that you and Adam did, this is unlike a
• 12:43
  Speaker 1: lot of stuff that's come down the pike, since this
• 12:47
  Speaker 1: is truly interesting. How encouraging it was that from one
• 12:51
  Speaker 1: of the original authors who ostensibly disproved the hot hand.
• 12:56
  Speaker 1: I mean, it's always nice when somebody appreciates your work,
• 12:59
  Speaker 1: especially in one of Tom Gilovich's stature at the time
• 13:02
  Speaker 1: that he said that our our paper, while it had
• 13:04
  Speaker 1: gone through the public peri review process, it hadn't gone
• 13:07
  Speaker 1: through the formal one. And so just last week our
• 13:10
  Speaker 1: paper was finally published its online, not in the print
• 13:12
  Speaker 1: edition yet in Econometrica, which is you know, top journal
• 13:15
  Speaker 1: and economics. There's the top five. They're kind of all equal, um,
• 13:19
  Speaker 1: And you know, so now it's been kind of formally
• 13:22
  Speaker 1: taken in, so I think Tom Gilvitch might have a
• 13:23
  Speaker 1: different opinion, um, now that it's gone through this process.
• 13:26
  Speaker 1: So so the paper is ready to be published or
• 13:29
  Speaker 1: was just published. Yeah, so the November issue of Econometrica, Um,
• 13:33
  Speaker 1: it came out and that's got to be very exciting. Oh,
• 13:36
  Speaker 1: very exciting. Yeah, it's something. So what's the takeaway from
• 13:39
  Speaker 1: the original research, What was it that was wrong in
• 13:43
  Speaker 1: the structure of the original myth of the hot hands paper. Right,
• 13:48
  Speaker 1: So um, the original hot hand paper. They're interested in
• 13:53
  Speaker 1: seeing if people do better after recent success then after
• 13:56
  Speaker 1: recent failure. Um, that was the most important measure, Like,
• 14:00
  Speaker 1: is your probability of success increase when you've hit in
• 14:02
  Speaker 1: a few funeral versus if you've missed a funeral? And well,
• 14:06
  Speaker 1: we don't know what someone's probability is. It seems like
• 14:08
  Speaker 1: our best guests would be would just look at the
• 14:10
  Speaker 1: percentage of time they make it, right, And so they
• 14:13
  Speaker 1: just look at all the events when you've had a
• 14:15
  Speaker 1: streak of recent successes and all the events when you
• 14:18
  Speaker 1: have a streaking recent failures, and just see what the
• 14:20
  Speaker 1: change in your shooting percentages between those two conditions. And
• 14:24
  Speaker 1: you know, that's very natural and very intuitive to expect
• 14:27
  Speaker 1: that would be your best guess. And they do that
• 14:29
  Speaker 1: and they don't find any difference. Um. And so that's
• 14:33
  Speaker 1: how the problem was set up. So before we get
• 14:36
  Speaker 1: to your solution, the the immediate pushback is, Hey, after
• 14:40
  Speaker 1: a shooter gets on a bit of a streak, the
• 14:43
  Speaker 1: defense collapses on them. They become they're forced to either
• 14:47
  Speaker 1: pass the ball and more or take more difficult shots.
• 14:51
  Speaker 1: At the time in there was no way to account
• 14:55
  Speaker 1: for that difference. However, in the intervening years, every shot
• 15:00
  Speaker 1: gets marked. You you describe this in one of your
• 15:04
  Speaker 1: um publications recently, explain the degree of difficulty that is
• 15:08
  Speaker 1: now tracked on every single basketball shot that's taken. Right, So,
• 15:14
  Speaker 1: there there's a it's a new company now I don't
• 15:16
  Speaker 1: remember the company, but sport View was the first company
• 15:18
  Speaker 1: that did this, where they have optical tracking of the
• 15:25
  Speaker 1: I mean, the precision isn't super high, but it's it
• 15:27
  Speaker 1: gets in the general area, and so you can control
• 15:30
  Speaker 1: for a lot more factors than you could say, or
• 15:33
  Speaker 1: they had the seventies sixers and they're just looking at
• 15:34
  Speaker 1: the play by playout and so you know, even in
• 15:37
  Speaker 1: that data, what they would find is, yes, they'd have
• 15:39
  Speaker 1: this evidence of the defense adjusting to what they believed
• 15:42
  Speaker 1: to be a hoth hand, making it more difficult for
• 15:44
  Speaker 1: the player. But the player stats to shoot from time
• 15:47
  Speaker 1: to time to keep the defense honest. And so the
• 15:49
  Speaker 1: important thing isn't so much as the player doing better
• 15:51
  Speaker 1: in the context of the game, but as it helped
• 15:53
  Speaker 1: the teammates if they're hot, because then it opens upthing
• 15:55
  Speaker 1: for their teammates. Makes sense. Yeah, So the the innovations
• 15:58
  Speaker 1: that have happened, I think the first in vation actually
• 16:00
  Speaker 1: was Justin Rao, who's u the head economists at Home Away.
• 16:04
  Speaker 1: He was the first one to actually come out and
• 16:06
  Speaker 1: and measure how many defenders are around the player and
• 16:09
  Speaker 1: try to control from these things in a different way
• 16:11
  Speaker 1: by using the videos and show that yes, there's a
• 16:13
  Speaker 1: lot of evidence that there's this defense factor, and if
• 16:16
  Speaker 1: you just control for a few of these things, the
• 16:18
  Speaker 1: effects that they had found in the previous study went away.
• 16:21
  Speaker 1: So in other words, what looks like it's random is
• 16:24
  Speaker 1: you're shooting the same percentage but with a whole lot
• 16:27
  Speaker 1: more defensive activity on you. Therefore, it's a continuation of
• 16:32
  Speaker 1: the of the streak. Yes, so he didn't necessarily find
• 16:34
  Speaker 1: evidence of the streak there because he controlled for a
• 16:37
  Speaker 1: subset of factors. But as you add more more controls,
• 16:40
  Speaker 1: it looks like there might be some evidence there. But
• 16:42
  Speaker 1: these are very difficult things to measure in the context
• 16:44
  Speaker 1: of the game. The original study had this critical test
• 16:47
  Speaker 1: and it's been repeated with other teams. Um where they
• 16:50
  Speaker 1: take them and they pay them to shoot the basketball.
• 16:52
  Speaker 1: So in other words, you're you're not playing during a
• 16:55
  Speaker 1: live game, you're just doing foul shooting with three point
• 16:57
  Speaker 1: shooting or whatever exactly. Or you'll to the NBA three
• 17:00
  Speaker 1: point shooting contest and and and in those studies you
• 17:03
  Speaker 1: can get rid of the defense and get a little
• 17:05
  Speaker 1: more zero in on your question. A bit more. Let's
• 17:08
  Speaker 1: talk a little bit about the surprising math of coin flips.
• 17:13
  Speaker 1: My best guests and my understanding of statistics has always been,
• 17:17
  Speaker 1: if you take a true coin and flip it, the
• 17:20
  Speaker 1: odds of a head of a tail is fifty fifty.
• 17:23
  Speaker 1: This is regardless of what came before it. Coins have
• 17:26
  Speaker 1: no memory. But you found something surprising in the data set.
• 17:32
  Speaker 1: After you flip a coin a hundred times, and you
• 17:36
  Speaker 1: were to pick a specific series, the odds are somewhat different.
• 17:41
  Speaker 1: Explain that. Yeah, So, my author Adamson, and I, after
• 17:45
  Speaker 1: having watched the NBA three point shooting contests, we had
• 17:47
  Speaker 1: a particular player, Craig Hodges, who was obviously hot, and
• 17:50
  Speaker 1: we went and used the original analysis on his data
• 17:53
  Speaker 1: and it said that he wasn't and that was puzzling,
• 17:56
  Speaker 1: and so we had to go and see, well, we
• 17:58
  Speaker 1: don't really know how Craig Hodges generated his shots. It's
• 18:03
  Speaker 1: kind of a black box. But let's create an environment
• 18:06
  Speaker 1: where we have the ground truth when we know what's happening,
• 18:08
  Speaker 1: and so coin flips is a world like this. So
• 18:11
  Speaker 1: you can actually go and flip a coin many many times,
• 18:13
  Speaker 1: or do it on a computer and see what do
• 18:16
  Speaker 1: you get if you analyze we're interested, is the probability
• 18:19
  Speaker 1: of heads after a few heads different than the probablity
• 18:21
  Speaker 1: of heads after a few tails? We know that's the same.
• 18:24
  Speaker 1: That's we have the ground truth. But now let's go
• 18:26
  Speaker 1: out and generate that data and make our best guests
• 18:30
  Speaker 1: from that data. What's our best guests is the percentage
• 18:33
  Speaker 1: of heads that you get after a few heads in
• 18:35
  Speaker 1: a row the same as the percentage of heads you
• 18:37
  Speaker 1: get after a few tails in a row. And analyzing
• 18:40
  Speaker 1: it in the way they analyzed it, we found that no,
• 18:42
  Speaker 1: it's different. The percentage of tails after a few heads
• 18:45
  Speaker 1: in a row is higher, which which is so counterintuitive
• 18:50
  Speaker 1: because perspectively, so understand before people lose their mind and
• 18:55
  Speaker 1: start sending emails, what we're not talking about is looking
• 18:59
  Speaker 1: forward in a live situation, no matter what the previous
• 19:03
  Speaker 1: with with a true coin. You could have a thousand
• 19:06
  Speaker 1: heads in a row. Highly improbable, but not mathematically impossible.
• 19:11
  Speaker 1: The odds on that next flip are still gonna be
• 19:15
  Speaker 1: That's not what we're saying. We're saying, flip a coin
• 19:17
  Speaker 1: a hundred times, look at the data set, and then
• 19:20
  Speaker 1: go back and randomly pick any head in that order
• 19:24
  Speaker 1: or any tail in the order. What are the odds
• 19:26
  Speaker 1: that the next flip is ahead of a tail? And
• 19:30
  Speaker 1: it turns out that's not so? Explain that because it's
• 19:35
  Speaker 1: it's a complete It blows people's minds because you've been
• 19:39
  Speaker 1: told over and over again, hey, coins have no memory.
• 19:42
  Speaker 1: But that's not what this is. This is an existing
• 19:45
  Speaker 1: data set. When we randomly pull any of those flips,
• 19:49
  Speaker 1: what are the probabilities as to the outcome in the
• 19:52
  Speaker 1: next flip after it's already been done. So, so how
• 19:55
  Speaker 1: do you end up with instead of so the complete
• 19:59
  Speaker 1: exper name would take some time, but we can kind
• 20:02
  Speaker 1: of get it an intuition. Um, if you flip a
• 20:05
  Speaker 1: coin a hundred times, there's gonna be a certain number
• 20:06
  Speaker 1: of heads and tails there when when you're done about guarantee,
• 20:12
  Speaker 1: but it's gonna be some number. Now, if you just
• 20:15
  Speaker 1: choose any flip your best guess, just choose a flip
• 20:18
  Speaker 1: your best guess. Is if I choose flip forty two,
• 20:21
  Speaker 1: for example, my best guess or heads, that's my you know,
• 20:28
  Speaker 1: And so that's different though then if I choose flip
• 20:32
  Speaker 1: forty two because flip forty one is the heads, So
• 20:36
  Speaker 1: if I choose one of the flips where the previous
• 20:38
  Speaker 1: flip is the heads, or just choose a flip that's
• 20:42
  Speaker 1: ahead and see what the next flip is in the
• 20:43
  Speaker 1: same way of looking at it. Now there's something else
• 20:47
  Speaker 1: because the flip you chose because the previous flip is ahead,
• 20:51
  Speaker 1: is using information about the outcomes of adjacent flips, and
• 20:56
  Speaker 1: that information kind of gets contained within your flip, and
• 20:59
  Speaker 1: that's this. This gets a little complicated, but one way
• 21:02
  Speaker 1: to think about it is you've taken a head's away
• 21:04
  Speaker 1: from the finite number of heads that you have and
• 21:06
  Speaker 1: you can't see it again. You've reduced that data set,
• 21:08
  Speaker 1: and the remaining tails now should be slightly than it shifts.
• 21:13
  Speaker 1: And there's another element that uses the kind of space
• 21:15
  Speaker 1: away there arrange that makes it a bit stronger. But
• 21:18
  Speaker 1: that that that's this smells to me slightly like the
• 21:23
  Speaker 1: Monty Hall problem. Is there any element in the Monty
• 21:26
  Speaker 1: Hall problem? You go from choosing one in three to
• 21:30
  Speaker 1: one and two. So suddenly what was the thirty three
• 21:33
  Speaker 1: percent chance becomes a fifty fifty Why not make the
• 21:36
  Speaker 1: switch that That is a little counterintuitive, but once you
• 21:40
  Speaker 1: see the statistics, it you can't unsee it. It always
• 21:43
  Speaker 1: you always should make the change. There is a is
• 21:47
  Speaker 1: there a tiny element of this and that more than tiny.
• 21:50
  Speaker 1: So my co author and Adamson Hot and I also
• 21:52
  Speaker 1: wrote another paper connecting this to the Monty Hall problem
• 21:56
  Speaker 1: and explaining it via this principle from Bridge, which is
• 21:59
  Speaker 1: the stable restricted choice, which is essentially the intuition of
• 22:03
  Speaker 1: Bay's rule, and so the way to think about it
• 22:06
  Speaker 1: in the Monty Hall problem, you of these three doors, right,
• 22:08
  Speaker 1: so you're on this game show. There's three doors. Well,
• 22:10
  Speaker 1: let's make this our problem exactly the same as money,
• 22:13
  Speaker 1: except it's a hundred doors, not necessarily three doors, so
• 22:16
  Speaker 1: it becomes much harder. We can make it three doors.
• 22:19
  Speaker 1: So your games show, you're on a game show, and
• 22:23
  Speaker 1: usually that you have this car and two goats, and
• 22:24
  Speaker 1: you gotta find the car, and there's a car behind
• 22:26
  Speaker 1: one of the doors. You gotta guess. Well, let's get
• 22:28
  Speaker 1: rid of the goats and cars. Now, let's just flip
• 22:30
  Speaker 1: a coin behind each door. So eat behind the shorts
• 22:33
  Speaker 1: fifty fifty. But you're the contestant. The host knows what
• 22:35
  Speaker 1: the outcome of the flips are. You don't you want
• 22:37
  Speaker 1: to guess, Hey, where's it? Where's the heads? Let's say
• 22:41
  Speaker 1: you want to find the heads um, so you guess
• 22:44
  Speaker 1: you know door three. Now, if you guess door three,
• 22:47
  Speaker 1: let's say the host looks behind the door. You didn't
• 22:49
  Speaker 1: guess door one and two, and he's going to reveal
• 22:52
  Speaker 1: the heads if you can't. So let's say the host
• 22:55
  Speaker 1: opens door one and shows you ahead. Do you want
• 22:57
  Speaker 1: to switch or do you want to stay? If you're
• 23:00
  Speaker 1: looking for the heads, you want to stay. If you're
• 23:04
  Speaker 1: looking for the tails, you want to switch. Now, the
• 23:07
  Speaker 1: intuition is not going to be clear immediately, but if
• 23:11
  Speaker 1: you think about now, the host looking at door one
• 23:14
  Speaker 1: and two used information about both doors to determine which
• 23:18
  Speaker 1: door to open up to you. Now, if both doors
• 23:21
  Speaker 1: were heads, the host could have opened door too. But
• 23:25
  Speaker 1: if it was heads tails, the door host had to
• 23:28
  Speaker 1: open door one. Because the host is gonna show your
• 23:29
  Speaker 1: heads if you can't, right, he doesn't want to show
• 23:31
  Speaker 1: you a tail because that's what you're looking to avoid.
• 23:34
  Speaker 1: That's the goat. Yea. So the world we don't know
• 23:36
  Speaker 1: which world we're in where the first is heads and
• 23:38
  Speaker 1: the second is tails. Of the first his heads and
• 23:40
  Speaker 1: the you know second, his heads. But the world where
• 23:44
  Speaker 1: it's heads tails is the world where the host is
• 23:47
  Speaker 1: more restricted. The host has to open door one, and
• 23:51
  Speaker 1: so the world that so that you should avoid door
• 23:53
  Speaker 1: two in those circumstances because it's a higher probability of
• 23:56
  Speaker 1: if you're if you're hunting for the heads you should have,
• 23:58
  Speaker 1: you should avoid door too because tails is more likely
• 24:02
  Speaker 1: because in that world of heads tails, the host had
• 24:05
  Speaker 1: to open door one and it's yeah, so it's a
• 24:08
  Speaker 1: higher probability. So that door three, even though it's a
• 24:12
  Speaker 1: coin that's flipped independent of the other two, when you're
• 24:15
  Speaker 1: dealing with that data set, you're better off with three
• 24:18
  Speaker 1: because of the circumstances that lead the host to pick
• 24:21
  Speaker 1: one and not pick two. That makes some rational degree
• 24:26
  Speaker 1: of sense. Once you get the monty whole aspect of this,
• 24:29
  Speaker 1: it makes a whole lot more sense. It's just it
• 24:31
  Speaker 1: just seems, uh, it's quite fascinating. We were discussing um
• 24:36
  Speaker 1: the coin flip issue and the hot hand scenario. Let's
• 24:40
  Speaker 1: circle back to that hot hand and the original research.
• 24:43
  Speaker 1: The original research said that if there's a streak of
• 24:47
  Speaker 1: three hits in basketball or three misses in basketball, the
• 24:52
  Speaker 1: odds of the next shot going in or not is
• 24:56
  Speaker 1: whatever the shooters historical shooting percentage hges, which sort of
• 25:02
  Speaker 1: seems that there's no hot hand. But that presumes after
• 25:06
  Speaker 1: streak that their next shot should be dead center in
• 25:11
  Speaker 1: their percentage. You found out it should be worse than that.
• 25:15
  Speaker 1: Explain exactly. So that's the counterintuitive thing. If you go
• 25:19
  Speaker 1: out and you watch a player shoot a basketball and
• 25:21
  Speaker 1: you look at their shooting percentage after a streak of
• 25:23
  Speaker 1: hits and compare it to their shooting percentage after a
• 25:26
  Speaker 1: streak of misses, and you find that it's the same
• 25:30
  Speaker 1: and not. The intuitive thing is to say, oh, they're
• 25:32
  Speaker 1: just they have the same rate, but actually you would
• 25:35
  Speaker 1: expect them to do worse. Explain that, because that's the
• 25:38
  Speaker 1: most fascinating part of it. Someone is on a shooting streak,
• 25:41
  Speaker 1: we take a data set of a whole run of shots.
• 25:45
  Speaker 1: What do you find after the streak, and why is that?
• 25:48
  Speaker 1: So you said, you find their percentage actually goes down
• 25:52
  Speaker 1: after a streak. In the world where there's no hot hand,
• 25:55
  Speaker 1: where they're a consistent shooter, their percentage will go down
• 25:58
  Speaker 1: after a streak in the data. Not in reality, their
• 26:01
  Speaker 1: probability is always the same. But we don't observe the probability.
• 26:04
  Speaker 1: We calculate the percentage, and that's where the biases come in.
• 26:07
  Speaker 1: And so the original authors found that the shooting percentage
• 26:10
  Speaker 1: was around the same, and that's correct. We go and
• 26:12
  Speaker 1: we check and they were right. They report they did
• 26:14
  Speaker 1: all the analysis in that sense, the calculations correctly. But
• 26:18
  Speaker 1: the mistake is understanding the benchmark. You have to go
• 26:21
  Speaker 1: out and say, okay, now let's look at the world
• 26:24
  Speaker 1: where we know, um, we can control it. So on
• 26:27
  Speaker 1: a computer you can say, we can generate coin flips,
• 26:29
  Speaker 1: so we can make a player that has no hot hand,
• 26:31
  Speaker 1: and then look at how that player does when we
• 26:33
  Speaker 1: analyze the data and we realize, oh, they should do
• 26:35
  Speaker 1: worse after a few in a row. So once you
• 26:36
  Speaker 1: adjust for that bias, you find out that actually, if
• 26:41
  Speaker 1: they're doing the same, that's indicative that they're doing about
• 26:45
  Speaker 1: ten percentage points are more better after hitting a funer
• 26:48
  Speaker 1: row than missing in a few funeroe and that's huge.
• 26:51
  Speaker 1: That's like the difference between the median and the best
• 26:53
  Speaker 1: NBA three points, thereby confirming the hot hands. So I
• 26:57
  Speaker 1: have to challenge the data set because it again, everything
• 27:02
  Speaker 1: about this, each step along the way, is so counterintuitive.
• 27:05
  Speaker 1: So why would we expect a shooter who's on a streak,
• 27:09
  Speaker 1: who's in the zone, who has the hot hand, whatever
• 27:13
  Speaker 1: we want to call it, Why would we expect his
• 27:15
  Speaker 1: shooting percentage to be lower after they hit several shots
• 27:21
  Speaker 1: in a row. Why would we expect it to be
• 27:24
  Speaker 1: lower for a real human or for for anybody, for
• 27:27
  Speaker 1: a professional, for a real human. When you look at
• 27:29
  Speaker 1: a data set of here's here's all the NBA streak
• 27:33
  Speaker 1: shooters or all the NBA shooters, what does the data
• 27:36
  Speaker 1: show after a streak they're shooting percentage actually becomes So
• 27:41
  Speaker 1: if you're talking about live action games, we have those
• 27:45
  Speaker 1: issues that we spoke about. The defense will adjust and
• 27:48
  Speaker 1: so that becomes a little bit more complished. So so
• 27:50
  Speaker 1: let's talk about three point contests. So if the hot
• 27:54
  Speaker 1: hand didn't exist in a world like that, we would
• 27:59
  Speaker 1: expect players to shoot worse after making a few in
• 28:02
  Speaker 1: a row in the data, so simply just mean reversion,
• 28:05
  Speaker 1: is that all. It's not mean resursion. It's the same
• 28:08
  Speaker 1: thing we talked about with the coin flips. Right. And
• 28:10
  Speaker 1: so as a researcher, you're taking the data after it's
• 28:13
  Speaker 1: already been generated, and you're picking through it, and you're
• 28:16
  Speaker 1: looking only at the events that you're interested in. Right,
• 28:17
  Speaker 1: You're looking at their probability of success given recent success.
• 28:20
  Speaker 1: You're just picking out those events when they had recent success.
• 28:23
  Speaker 1: Let's say where they just made three in a row.
• 28:25
  Speaker 1: So you're changing the data set so now there's three less.
• 28:29
  Speaker 1: So if you someone shoots three in a row, when
• 28:31
  Speaker 1: we're looking at the data set, let's say they've shot
• 28:34
  Speaker 1: twenty shots, and after three in a row, how they
• 28:36
  Speaker 1: do well, guess what, You've pulled three hits out of
• 28:40
  Speaker 1: the set, meaning there's a disproportionate number of mrs left.
• 28:44
  Speaker 1: That's part of the bias. And there's this other element
• 28:46
  Speaker 1: that didn't quite get into is that you can you
• 28:49
  Speaker 1: have this essentially a stopping rule, so as you collect
• 28:52
  Speaker 1: the data, the moment they miss it, you're not interested
• 28:55
  Speaker 1: anymore in looking because you're gonna wait for a streak
• 28:57
  Speaker 1: of hits again, so you've kind of you're biased towards
• 29:00
  Speaker 1: stopping at a miss, so you might get a miss
• 29:02
  Speaker 1: right away right then everything you collected in those events
• 29:07
  Speaker 1: of their shots or misses because you just collected one
• 29:09
  Speaker 1: shot and they're all misses, and so you're you can
• 29:11
  Speaker 1: biasing towards collecting misses in a side that that that's
• 29:15
  Speaker 1: quite that's quite fascinating. So what other areas like this
• 29:20
  Speaker 1: are you studying, because it's really it's really quite quite
• 29:24
  Speaker 1: fascinating stuff. Are there other sport myths that you're looking
• 29:28
  Speaker 1: at that have a probabilistic element that's very counterintuitive or
• 29:33
  Speaker 1: is this pretty much the biggest one out there? Um,
• 29:37
  Speaker 1: this is the biggest one that we're studying. A lot
• 29:39
  Speaker 1: of what you're doing is statistical and probability work at
• 29:44
  Speaker 1: a level that the average sports fan is really not
• 29:46
  Speaker 1: familiar with. Forget the live game. When you explain relative
• 29:50
  Speaker 1: to three point shooting contest, it's really not so much
• 29:55
  Speaker 1: about the streakiness of the shooter, but the mathematics of
• 29:59
  Speaker 1: the data set it. And I think that is really counterintuitive,
• 30:03
  Speaker 1: but it doesn't seem anyone's been able to disprove what
• 30:07
  Speaker 1: you and your co author found. So there have been
• 30:09
  Speaker 1: a lot of challenges to that original study, a lot
• 30:12
  Speaker 1: of challenge and legitimate challenges. You know, there are issues
• 30:15
  Speaker 1: with UM what they call statistical power. Right, so we
• 30:18
  Speaker 1: have we have a friend and colleague, Daniel Stone, who
• 30:21
  Speaker 1: made this nice point that you have this thing called measurement.
• 30:23
  Speaker 1: Are we want to know how do you do after
• 30:25
  Speaker 1: hitting a few in a row? Um, that's what we
• 30:28
  Speaker 1: actually look at. But what we're really interested in is
• 30:31
  Speaker 1: how will you do when you're hot? So hitting a
• 30:34
  Speaker 1: few in a row, you're not always hot, and so
• 30:37
  Speaker 1: you can underestimate how hot someone is if you use
• 30:42
  Speaker 1: only the data that you can observe, which is zeros
• 30:44
  Speaker 1: and one. So the you know, the econometrician, the statistician
• 30:47
  Speaker 1: has kind of a weak measure of that. So you know,
• 30:50
  Speaker 1: these this kind of evidence is um just the mathematical evidence.
• 30:54
  Speaker 1: Do you ever do interviews of players? Do you ever
• 30:57
  Speaker 1: say to them, Hey, were you in the zone? How
• 31:00
  Speaker 1: did you feel? How? How how do you find that
• 31:03
  Speaker 1: data set? So that data UM, the original study looked
• 31:08
  Speaker 1: at data like that. They spoke to the seventies sixers
• 31:11
  Speaker 1: and they asked them kind of qualitative questions. Do you
• 31:14
  Speaker 1: get in the zone and you feel hot? And they
• 31:18
  Speaker 1: all do right, Um, but it's it's hard to work
• 31:20
  Speaker 1: with that. That's just looking at whether they believe in
• 31:23
  Speaker 1: it or not, but then getting a sense of do
• 31:26
  Speaker 1: they believe in it too much or not? That get
• 31:28
  Speaker 1: that gets a bit harder because you have to be
• 31:30
  Speaker 1: able to somehow measure you know, they have to decide
• 31:33
  Speaker 1: when are they hot? You know, so you really need
• 31:35
  Speaker 1: a lot more cooperation from say like a coach or
• 31:37
  Speaker 1: player to kind of sit there and maybe watch the
• 31:38
  Speaker 1: games with you or something like that. That would be
• 31:41
  Speaker 1: maybe a a better way of testing their you know,
• 31:44
  Speaker 1: their beliefs. So so when um Traverseking Gilvitch's original study
• 31:50
  Speaker 1: came out, I'm forgetting the third person in g VT.
• 31:56
  Speaker 1: When when that study came out, there was a tremendous
• 31:59
  Speaker 1: amount of push back from coaches amount around the league.
• 32:02
  Speaker 1: We mentioned ridd are back. Your study comes out and
• 32:06
  Speaker 1: you basically say, no, you professional coaches, You were right.
• 32:11
  Speaker 1: There is a hot hand, there is a streak. What
• 32:13
  Speaker 1: sort of feedback have you gotten from players and coaches
• 32:16
  Speaker 1: about your research? Well, we're not entirely sure whether players
• 32:22
  Speaker 1: and coaches were ever frowsled by the original study, so,
• 32:26
  Speaker 1: you know, validating their beliefs for them. It's so, yeah,
• 32:30
  Speaker 1: we kind of never believe that result to begin with.
• 32:33
  Speaker 1: So so we haven't gone and sought the opinion of
• 32:36
  Speaker 1: you know, players and coaches because it's it's not so
• 32:39
  Speaker 1: clear how far that original conclusion reached into that world. Um,
• 32:43
  Speaker 1: while it did, especially you can see announcers mentioning it. Um.
• 32:47
  Speaker 1: But yeah, yeah, so so when you what about some
• 32:50
  Speaker 1: of the outlawer players. If you look at a Michael
• 32:52
  Speaker 1: Jordan's um or a Steph Carry, guys who literally become
• 32:59
  Speaker 1: just conscious and what Reggie Miller is another one, and
• 33:03
  Speaker 1: the most improbable shots on a consistent basis start to
• 33:08
  Speaker 1: drop when when you look at players like that, do
• 33:11
  Speaker 1: different players seem to have a different set of streakiness,
• 33:15
  Speaker 1: a different hot hand? Can you can you calibrate how
• 33:19
  Speaker 1: much of a hot hand different players have? So using
• 33:23
  Speaker 1: game data that's a bit more of a challenge. So
• 33:26
  Speaker 1: my couth or Adam and I looked at Spanish semipro players.
• 33:29
  Speaker 1: We could collect a lot more data and we had
• 33:30
  Speaker 1: more of their cooperation, and there seemed to be a
• 33:33
  Speaker 1: clear difference with players. I mean, there's the obvious one
• 33:35
  Speaker 1: is that you know, centers and forwards, people that don't
• 33:37
  Speaker 1: shoot that often. It's hard for them to get on
• 33:40
  Speaker 1: a roll because you have to be consistent and they're
• 33:42
  Speaker 1: kind of not that consistent when they don't touch the ball.
• 33:44
  Speaker 1: All that and all that, right, and so you know,
• 33:48
  Speaker 1: those are the people you'd expect maybe you know, they
• 33:50
  Speaker 1: can't really sustain a streak. And that's what we find.
• 33:53
  Speaker 1: You know, so there's some players that can and some
• 33:54
  Speaker 1: players that seem like they can't. Um. If we go
• 33:57
  Speaker 1: to real NBA players, you know, that's a bit of
• 33:59
  Speaker 1: a challe And so we have looked at the three
• 34:01
  Speaker 1: point shooting contest and we have a paper on that. Um.
• 34:04
  Speaker 1: The issue with the three point shooting contest is a
• 34:06
  Speaker 1: lot of the players don't have much more than say
• 34:08
  Speaker 1: a hundred shots total in the contest. You know, may
• 34:11
  Speaker 1: some have a few more. You have a Craig Hodges
• 34:13
  Speaker 1: who has over five hundred in our data, and we
• 34:14
  Speaker 1: find evidence there. Um. But what we can say is
• 34:18
  Speaker 1: that among all the three point shooting contest contestants, there
• 34:22
  Speaker 1: were way more that did better after a few in
• 34:25
  Speaker 1: a row than making a few row the missing a
• 34:28
  Speaker 1: funeral than you'd expect, but you don't really know which
• 34:31
  Speaker 1: of them are really hot. You just know there's more
• 34:33
  Speaker 1: of them than you expect, but you need more data
• 34:35
  Speaker 1: to be really confident when you pick out an individual.
• 34:38
  Speaker 1: So at this point in the state of research on
• 34:41
  Speaker 1: the hot hand, do you have any doubt that the
• 34:43
  Speaker 1: hot hand exists? I don't have any doubt that the
• 34:47
  Speaker 1: hot hand exists. What you mean by the hot hand
• 34:50
  Speaker 1: is where the doubts come in, because there's many different
• 34:53
  Speaker 1: mechanisms that can lead to evidence in the data that
• 34:56
  Speaker 1: your success after recent success is higher than you know
• 34:59
  Speaker 1: that it's higher than after recent So so the confidence factor,
• 35:04
  Speaker 1: the endorphin factor, the further pressure that the other team
• 35:08
  Speaker 1: is placing. All those things they add up. You ask
• 35:12
  Speaker 1: a player, they're gonna say, yeah, of course you get hot.
• 35:14
  Speaker 1: But when you ask the stat statistician, the data supports
• 35:18
  Speaker 1: it as well. Right, quite fascinating. We have been speaking
• 35:22
  Speaker 1: to Joshua Miller. He is an economics professor and researcher
• 35:26
  Speaker 1: at the University of Alacante in Spain. If you enjoy
• 35:31
  Speaker 1: this conversation, well be sure and come back and check
• 35:34
  Speaker 1: out our podcast extras where we keep the tape rolling
• 35:37
  Speaker 1: and we continue discussing all things statistical, sports and behavior.
• 35:43
  Speaker 1: You can find that at iTunes, overcast, at your Bloomberg
• 35:47
  Speaker 1: dot com, wherever your finer podcasts are sold. We love
• 35:52
  Speaker 1: your comments, feedback and suggestions right to us at m
• 35:56
  Speaker 1: IB podcast at Bloomberg dot net. You can follow me
• 36:00
  Speaker 1: on Twitter at rid Halts, or check out my daily
• 36:02
  Speaker 1: column at Bloomberg dot com slash Opinion. I'm Barry Ridhults.
• 36:07
  Speaker 1: You're listening to Masters in Business on Bloomberg Road. Welcome
• 36:25
  Speaker 1: to the podcast. So, Josh, I have to tell you
• 36:28
  Speaker 1: I was very much a skeptic. Um a little background.
• 36:33
  Speaker 1: So first, I'm a fan of Gilovich for a long
• 36:36
  Speaker 1: time when I you could take noise. We don't know
• 36:39
  Speaker 1: when I started in this business a hundred years ago
• 36:41
  Speaker 1: as a trader. It was before the bad old days
• 36:44
  Speaker 1: of behavioral economics had made its way to Wall Street,
• 36:48
  Speaker 1: and I found a book by Gilovich, How We Know
• 36:51
  Speaker 1: What Isn't? So it was the first mass book, more
• 36:57
  Speaker 1: more popular book, not that it was all that popular,
• 37:00
  Speaker 1: but it was the first book for a popular audience
• 37:04
  Speaker 1: that had an enormous behavioral finance component. To it, so
• 37:07
  Speaker 1: I found him absolutely intriguing. He led me down the
• 37:11
  Speaker 1: rabbit hole of behavioral finance, and it's been an enormous
• 37:16
  Speaker 1: influence on my um professional career because very often, when
• 37:21
  Speaker 1: I couldn't figure out what the hell is going on
• 37:23
  Speaker 1: according to what the head trader was saying, behavioral finance
• 37:28
  Speaker 1: gave a much better answer. And the same is true
• 37:30
  Speaker 1: when you're looking at markets, or the economy or what
• 37:33
  Speaker 1: people get wrong. So my bias was to say, Traversky Gilvitch,
• 37:38
  Speaker 1: these are two legends. Of course they're right. But I
• 37:41
  Speaker 1: have to tell you this, having gotten through as much
• 37:44
  Speaker 1: of your paper as I could until the formulas started
• 37:47
  Speaker 1: to show up. It's a compelling argument that when we
• 37:51
  Speaker 1: look at the data set, players on a streak from
• 37:56
  Speaker 1: within that data set should have a lower shooting percentage
• 38:01
  Speaker 1: following three in a row. Then you would intuitively inspect, expect,
• 38:05
  Speaker 1: and when they don't shoot worse, It in and of
• 38:09
  Speaker 1: itself is evidence of the hot hand. It's such an
• 38:13
  Speaker 1: eloquent and unexpected way to do the analysis of the
• 38:18
  Speaker 1: hot hand. I have to ask, how did you guys
• 38:21
  Speaker 1: come upon that? I mean, I would never I'm not
• 38:24
  Speaker 1: a statistician but I would never have thought, because so
• 38:27
  Speaker 1: much of it is so intuitive, I would not have thought, hey,
• 38:31
  Speaker 1: let's look at what the expected shot is, because with
• 38:35
  Speaker 1: coins it should be fifty fifty. Why would you expect
• 38:38
  Speaker 1: it to be anything less following three in a row?
• 38:42
  Speaker 1: How did you sort of work your way towards that research?
• 38:46
  Speaker 1: So you know, both my cauthor, Adamson ho Ho and
• 38:49
  Speaker 1: I we didn't see any problem in that respect um
• 38:54
  Speaker 1: with the original papers. So we didn't say, oh, they're
• 38:56
  Speaker 1: clearly making a mistake here. No one did, you know,
• 38:59
  Speaker 1: since we discuss for this thing, we've gone and we've
• 39:01
  Speaker 1: asked statisticians, people that are very good. They look at
• 39:03
  Speaker 1: the that test and they say, oh, you know, maybe
• 39:05
  Speaker 1: it's underpowered, or they might have some little quibbles, but
• 39:08
  Speaker 1: they don't have any expectation that you would shoot worse
• 39:11
  Speaker 1: after funeral. In order to do that, you actually have
• 39:14
  Speaker 1: to go out and simulate or go sit down and
• 39:16
  Speaker 1: really calculate, and so it doesn't strike you in any way.
• 39:19
  Speaker 1: So we discovered it. You know, it was a bit
• 39:22
  Speaker 1: of a stroke of luck. Um. We were looking at
• 39:25
  Speaker 1: the NBA three point contest data. We had to analyze
• 39:27
  Speaker 1: it very quickly. Using a method different than the way
• 39:30
  Speaker 1: we used it. So we just used the method of
• 39:32
  Speaker 1: the original study, which was much quicker to run. So
• 39:35
  Speaker 1: we ran that and we found this player who we
• 39:36
  Speaker 1: knew was hot and have mentioned that earlier, Craig Hodges,
• 39:39
  Speaker 1: and he shot no better after making a few in
• 39:42
  Speaker 1: a row, and that just didn't make sense. Was it
• 39:44
  Speaker 1: was that a brute force quit down and dirty? So
• 39:48
  Speaker 1: so you moved to something a little more um sophisticated,
• 39:53
  Speaker 1: what's the better word for this. So so the sophistication
• 39:55
  Speaker 1: came later. So we we you know, we just took
• 39:57
  Speaker 1: the test that used in the original study and that
• 40:00
  Speaker 1: measure and it wasn't showing anything, and we that didn't
• 40:02
  Speaker 1: agree with our perception of what we saw in those
• 40:04
  Speaker 1: videos and and and and some of the elementary things
• 40:08
  Speaker 1: he did, like he hit nineteen in a row at
• 40:09
  Speaker 1: one point, never missed more than five, and he was
• 40:11
  Speaker 1: around fifty percent shooter, which would be you'd never expect
• 40:14
  Speaker 1: from a point and so, okay, nineteen a row is
• 40:17
  Speaker 1: astonished as this instance. Yeah, it's incredible. So then we
• 40:20
  Speaker 1: went and we said, well, what if he were a coin?
• 40:23
  Speaker 1: What if Craig Hodges was a coin? So let's just
• 40:25
  Speaker 1: generate his shots as if he was a coin where
• 40:27
  Speaker 1: he's really and we repeat this, like, imagine we did
• 40:29
  Speaker 1: this many many times, and look what we'd expect from all.
• 40:32
  Speaker 1: You know, if we run this many times and we see, oh,
• 40:34
  Speaker 1: you'd actually shoot worse after making a funeral, and that
• 40:37
  Speaker 1: seems very count We were struck. This doesn't seem like
• 40:41
  Speaker 1: it's right. But this is what the analysis is giving us.
• 40:44
  Speaker 1: We have to understand this. This is what the data
• 40:46
  Speaker 1: is saying. Two things we've discussed. One is after you
• 40:52
  Speaker 1: have a streak of six in a row and you
• 40:54
  Speaker 1: have a finite number of shots, well now there are
• 40:57
  Speaker 1: six less heads in the in groups, so therefore there's
• 41:01
  Speaker 1: a higher probability of tails after that. That makes perfect sense, right,
• 41:05
  Speaker 1: because you're just changing the remaining data set by what
• 41:08
  Speaker 1: you're looking at given a fixed data, given a fixed
• 41:11
  Speaker 1: number of coins, fixed number of shots um. And then
• 41:14
  Speaker 1: of course mean reversion assumes after a long streak of
• 41:18
  Speaker 1: heads you should start to see a streak of tails,
• 41:21
  Speaker 1: which like Gambler's fallacy a little bit. So let's let's
• 41:23
  Speaker 1: go into that. Explain that. Yes, so I mean the
• 41:25
  Speaker 1: gambler's fallacy is this idea that comes out of the casino,
• 41:29
  Speaker 1: and it's been known for hundreds of years that if
• 41:32
  Speaker 1: you see say five six blacks in a road or
• 41:35
  Speaker 1: that table, it feels like that that the red must
• 41:38
  Speaker 1: be more likely, right, um, And so people get drawn
• 41:42
  Speaker 1: into this and they start betting more. Maybe, but it's
• 41:46
  Speaker 1: still fort and the green. Yeah, but it's almost fifty
• 41:51
  Speaker 1: fifty regardless, right, So in reality the probabilities haven't changed.
• 41:55
  Speaker 1: But but when you but when you look at a
• 41:58
  Speaker 1: when you look at a fixed data set that you
• 42:00
  Speaker 1: expect to be fifty fifty, not not perspectively at the
• 42:04
  Speaker 1: roulette table in real time. But we know that, hey,
• 42:07
  Speaker 1: there's a hundred coin flips, we're gonna assume half of
• 42:11
  Speaker 1: them are tails and half our heads. After you've had
• 42:16
  Speaker 1: a wrong long streak of heads, the assumption is that
• 42:20
  Speaker 1: out of that full data set, there should be more
• 42:22
  Speaker 1: tails coming up. I'm I'm in real time, it's truly
• 42:25
  Speaker 1: the gambler's fallacy. But when you're looking retrospectively with the
• 42:29
  Speaker 1: data set, it's basically just a variation of, Hey, you've
• 42:33
  Speaker 1: already exhausted a lot of heads, therefore there are more
• 42:35
  Speaker 1: tails out there. Yes, exactly and as we mentioned before,
• 42:39
  Speaker 1: there's a little bit there's an extra wrinkle on top.
• 42:42
  Speaker 1: You know that it determines on how you know, how
• 42:44
  Speaker 1: are these streaks ordered. So like when you when you
• 42:46
  Speaker 1: pick a pick up a shot, because the previous three
• 42:50
  Speaker 1: were heads, the shot you pick up, these are either
• 42:54
  Speaker 1: heads or tails, but it's much more likely to be
• 42:55
  Speaker 1: tails one because of the heads that were removed too,
• 42:58
  Speaker 1: because if it were a tail, you've interrupted the streak
• 43:01
  Speaker 1: and you you can't begin until you have to wait
• 43:04
  Speaker 1: until you begin. So so there's you're pulling a big
• 43:07
  Speaker 1: chunk of the possible selections out, so all the streaks
• 43:11
  Speaker 1: come out they're all heads, so you're not picking that one.
• 43:14
  Speaker 1: And and plus the total number of heads that you've used,
• 43:17
  Speaker 1: so what's left becomes just from a data set group,
• 43:22
  Speaker 1: what's less have become a not probability, which is which
• 43:29
  Speaker 1: is fast. So you guys are doing this research, at
• 43:32
  Speaker 1: what point do you say, holy cow, this is really
• 43:35
  Speaker 1: a fascinating discovery, Like it's it's not just a tiny chance.
• 43:40
  Speaker 1: Ten percent is a huge number in this sort of
• 43:44
  Speaker 1: data series. When did you guys look at each other
• 43:46
  Speaker 1: and say, hey, this is something really important. We knew
• 43:50
  Speaker 1: it was a big deal of the moment we saw it.
• 43:51
  Speaker 1: Really we were on the phone where you didn't set
• 43:54
  Speaker 1: yourself this has to be wrong. Ten P. How did
• 43:56
  Speaker 1: nobody pick this up? In thirty years? Nobody has seen this?
• 44:00
  Speaker 1: So we we had this is two years after we
• 44:03
  Speaker 1: had begun the project. Well maybe not that long, but
• 44:05
  Speaker 1: almost two years, and we had read every paper in
• 44:08
  Speaker 1: the literature, so we knew no one had had had
• 44:10
  Speaker 1: nobody had seen this, no, no one had said so
• 44:12
  Speaker 1: we knew it was a big deal for that literature.
• 44:14
  Speaker 1: So the only question we had is how new. Well,
• 44:18
  Speaker 1: I mean, we'd run you know, we knew the you
• 44:20
  Speaker 1: know we can trust the computer, right, And of course
• 44:22
  Speaker 1: you have to make sure you didn't make an error
• 44:24
  Speaker 1: in your code. You have to sit down and do
• 44:25
  Speaker 1: the simple example to make sure you didn't do a
• 44:27
  Speaker 1: calculation there. So once we did that with okay, this
• 44:29
  Speaker 1: is clearly a true thing. Now the only question is
• 44:32
  Speaker 1: did anyone know this about coin flips before? Is this
• 44:35
  Speaker 1: a new discovery about coin flips? And yes, there's some
• 44:39
  Speaker 1: mathematical things that are somewhat related, but no, it was
• 44:41
  Speaker 1: even new in that dimension, So we knew we had
• 44:42
  Speaker 1: something really big and that was exciting because you have
• 44:45
  Speaker 1: this moment where you're the only person in the world
• 44:48
  Speaker 1: that knows something. It's kind of it's an exciting moment.
• 44:50
  Speaker 1: I feel that way every day I wake up and
• 44:52
  Speaker 1: I have that sensation, so I can appreciate you probably
• 44:55
  Speaker 1: not as solidly based as as yours, at least that's
• 44:58
  Speaker 1: what my wife wee. So that's amazing. You guys come
• 45:02
  Speaker 1: up with this incredible breakthrough. Nobody has has found this.
• 45:07
  Speaker 1: It's been decades and it's it's been widely accepted. It's
• 45:12
  Speaker 1: become part of the cannon. But it's classic confirmation bias,
• 45:17
  Speaker 1: which is so um reflexive and meta. There is a
• 45:21
  Speaker 1: study that says people are fooled by randomness and think
• 45:24
  Speaker 1: there are streaks, which turns out perhaps to be confirmation
• 45:29
  Speaker 1: biased by behaviorists who are warning people against being fooled
• 45:33
  Speaker 1: by randomness and seeing what they want to see. It's
• 45:36
  Speaker 1: got a little bit of Mandel brought reflectiveness built into it.
• 45:40
  Speaker 1: It's it's quite amazing. Yes, you know, in a sense
• 45:43
  Speaker 1: that mistake proves kind of the spirit of the general
• 45:47
  Speaker 1: point about misinterpreting randomness. Even the best of us, the
• 45:50
  Speaker 1: best researchers there are out there still make these mistakes
• 45:55
  Speaker 1: due to randomness, and while saying others are making the mistake,
• 45:59
  Speaker 1: you're making the mistake even within. So they accidentally proved
• 46:04
  Speaker 1: their point, which is it's very easy to be fooled
• 46:08
  Speaker 1: by a random data set into thinking there's a broader
• 46:11
  Speaker 1: conclusion there, until subsequent research discovers that, hey, this isn't
• 46:18
  Speaker 1: quite as random as you think it is. There's a
• 46:20
  Speaker 1: ten gap between true randomness and the remaining data set.
• 46:25
  Speaker 1: That that's quite that's quite fascinating. So you guys look
• 46:29
  Speaker 1: at each other and say, hey, we're onto something real.
• 46:32
  Speaker 1: How did it progress from there? What year was this?
• 46:34
  Speaker 1: This was? This was February. Found this and we knew
• 46:40
  Speaker 1: so we knew it was important. So, um, we presented
• 46:43
  Speaker 1: our work and when you see the eyes light up,
• 46:46
  Speaker 1: you realize it's even bigger than you thought it was.
• 46:48
  Speaker 1: And then you realize, hey, wait a minute, we don't
• 46:50
  Speaker 1: have the paper yet, and now other people know about it.
• 46:54
  Speaker 1: Who did you present it to originally? Um? So at
• 46:58
  Speaker 1: Oxford University. That was the first review, right, and you
• 47:01
  Speaker 1: know you see the eyes light up in the room. Um,
• 47:04
  Speaker 1: are you genuinely concerned at that moment? Oh? Someone's gonna
• 47:07
  Speaker 1: try and beat us to publication, and so we put
• 47:10
  Speaker 1: everything aside and we just we we went to the grind.
• 47:13
  Speaker 1: We within two months we had the paper and no
• 47:16
  Speaker 1: one was going to catch you. At that point you
• 47:17
  Speaker 1: would enough of the head start and you were the
• 47:19
  Speaker 1: original people who found this. So two months later the
• 47:23
  Speaker 1: preliminary papers come out. We put online. You put you
• 47:26
  Speaker 1: posted online n b R and everywhere else or wherever
• 47:29
  Speaker 1: you know, just get that time stamp right where wherever
• 47:32
  Speaker 1: finer white papers are are sold. Um. And so that's
• 47:37
  Speaker 1: what April of the paper went online June. What's the
• 47:43
  Speaker 1: response to that? Um? The response was big. So statistician
• 47:51
  Speaker 1: at Columbia University, Andrew Gelman, who has this blog and
• 47:54
  Speaker 1: everybody's heard of Andrew Galman. Well, or let me rephrase that,
• 47:56
  Speaker 1: anybody who's interested in statistics knows who Gellman It Columbia
• 48:00
  Speaker 1: is fair, fair statement. He's at the crossroads of pretty
• 48:04
  Speaker 1: much all the social sciences, sciences when it comes to
• 48:07
  Speaker 1: data and statistics, right, and so getting attention from Andrew
• 48:11
  Speaker 1: Gelman huge, It's huge, and that that was you know,
• 48:15
  Speaker 1: high fives all around. Yeah, but it's also scary when
• 48:18
  Speaker 1: you get attention from Andrew Gelman, because if you made
• 48:20
  Speaker 1: a mistake, it's open peer review season. They're getting in
• 48:24
  Speaker 1: there in the comments. He'll get you, you know, like
• 48:26
  Speaker 1: they're just having fun. They love talking about data and
• 48:29
  Speaker 1: and they're not gonna worry about how you feel about
• 48:31
  Speaker 1: it because they're just interested in the main points, like
• 48:34
  Speaker 1: what did the statistics say? And you're you know, you're
• 48:36
  Speaker 1: sitting there sweat and ball. It's hoping you got you
• 48:38
  Speaker 1: didn't make a mistake somewhere. It's at that level. This
• 48:42
  Speaker 1: isn't you know, Twitter fights and ad homin Hum attacks.
• 48:45
  Speaker 1: It's hey, let's get into the math. Let's see if
• 48:48
  Speaker 1: they did they're crunching their numbers correctly. Let's see if
• 48:51
  Speaker 1: we can find an error in their modeling. What what
• 48:53
  Speaker 1: did Gelman discover? So Gelman went and did the work himself,
• 48:59
  Speaker 1: and he found what he found agreed with what we found,
• 49:02
  Speaker 1: and so he said, hey, guess what. There is a
• 49:04
  Speaker 1: hot hand that was his post and then it's kind
• 49:07
  Speaker 1: of snowballed from there. That's it. So then there's a
• 49:09
  Speaker 1: Wall Street Journal piece on it, and then there was
• 49:12
  Speaker 1: an ESPN or a sports illustrator was one of the
• 49:15
  Speaker 1: sports are there, yeah, there there was ridiculous, like we're like, okay,
• 49:19
  Speaker 1: when's the fifteen minutes gonna end? But I guess that,
• 49:22
  Speaker 1: you know, the news world is so kind of balkanized
• 49:25
  Speaker 1: by this point that like its not from subject to subject,
• 49:29
  Speaker 1: it just kept rotating. And I saw something, and you
• 49:31
  Speaker 1: guys published another a number of fair I have to say,
• 49:36
  Speaker 1: you're published popular stuff. I think you undersold the math
• 49:42
  Speaker 1: on this because it's not that you dumbed it down,
• 49:46
  Speaker 1: it's that you were so circumspect. And so maybe modest
• 49:52
  Speaker 1: is the right word. Like if I'm a different person
• 49:56
  Speaker 1: than you, I would have written written something that said, dude,
• 50:00
  Speaker 1: just listen up the whole no hot hand things. Let
• 50:03
  Speaker 1: us show you why that's not true. Here's the math.
• 50:06
  Speaker 1: It's ten. It's a giant impact, and here's why. Like
• 50:11
  Speaker 1: I thought, you guys were very circumspect in your what
• 50:15
  Speaker 1: was it the conversation or the yeah, the Australian conversation. Yes,
• 50:18
  Speaker 1: that was like a fairly modest discussion. You know. I
• 50:22
  Speaker 1: would have been like, hey, pay attention to this. We're
• 50:25
  Speaker 1: changing an understanding of sports streaking nous, this is a
• 50:29
  Speaker 1: big deal. What other applications are there? Of of the
• 50:34
  Speaker 1: finding of both the flips of coins and the streak
• 50:39
  Speaker 1: nous of shooters. Where else can this be applied? Are
• 50:43
  Speaker 1: there other uses of this mathematical? Why should call its
• 50:47
  Speaker 1: statistical observation? Yes? So the bias that that we found
• 50:52
  Speaker 1: has uh and it can it can manifest itself in
• 50:55
  Speaker 1: many areas. So it's not just about time, right, So
• 50:58
  Speaker 1: we're looking at like how how you did recently? Does
• 51:00
  Speaker 1: that affect how you do in the future or how
• 51:02
  Speaker 1: you do next? Right? If we found some biases there,
• 51:04
  Speaker 1: But it's not it's not about time. It's essentially about
• 51:07
  Speaker 1: space because you're looking at data and we represent time
• 51:10
  Speaker 1: with space because we have period one, period two, period three,
• 51:13
  Speaker 1: they're all next to each other, and so you have
• 51:15
  Speaker 1: this kind of one dimensional spatial thing continuing the line.
• 51:19
  Speaker 1: But it can go in either directions. So it's not
• 51:21
  Speaker 1: you know, time's arrow that's determining it. Right. If I
• 51:24
  Speaker 1: hit three in a row, the chance that the previous
• 51:26
  Speaker 1: one that just preceded that streak as it heads is
• 51:29
  Speaker 1: actually lower two um for the exact same reason, which
• 51:33
  Speaker 1: means that the actual streaking nous of the player isn't
• 51:36
  Speaker 1: relevant to the prior one, even though we would expect
• 51:40
  Speaker 1: it to be relevant to the subsequent one. It's all
• 51:43
  Speaker 1: the same statistical data set prior less, less heads in
• 51:48
  Speaker 1: the remaining pool, etcetera. So you can extend this beyond
• 51:51
  Speaker 1: time and talk about space. Right, So if you're interested
• 51:54
  Speaker 1: in you know, if I'm surrounded by you know, red people,
• 51:58
  Speaker 1: am I more likely to be blue? You might go
• 52:01
  Speaker 1: and hey, let's look at the data set. And this
• 52:03
  Speaker 1: is the ping pun bulls in the vase statistical problem. Yeah,
• 52:07
  Speaker 1: so you know, the people study segregation and clustering, you know,
• 52:10
  Speaker 1: and where people live and things like this, and and see,
• 52:12
  Speaker 1: you might go into data set and use this intuitive
• 52:14
  Speaker 1: measure like let's see if I'm more likely to be
• 52:16
  Speaker 1: blue if I'm surrounded by reds. Um, you have the
• 52:19
  Speaker 1: same issue here now, you know, if I were a blue,
• 52:22
  Speaker 1: I've kind of excluded other possibilities of being surrounded by reds.
• 52:26
  Speaker 1: Wherever that blue is what it actually makes blue more
• 52:29
  Speaker 1: likely for some of the same reasons why we have
• 52:32
  Speaker 1: this bias, you know, when we're talking about time, and
• 52:34
  Speaker 1: so there are potentially many other areas where bias is
• 52:38
  Speaker 1: similar to this could could manifest themselves. It might be
• 52:40
  Speaker 1: stumbling it so I remember a couple of years ago
• 52:43
  Speaker 1: the cancer clusters around power lines, um, which a lot
• 52:48
  Speaker 1: of statisticians came out and said, well, no, this is
• 52:52
  Speaker 1: just you know, the heads and tails problem. Again, you
• 52:55
  Speaker 1: have a lot all these non clusters around other power lines.
• 52:58
  Speaker 1: So if it's a cause element, why is it causing
• 53:01
  Speaker 1: it here but not a half mile down the same
• 53:03
  Speaker 1: power line. It's just a random aggregation of data, and
• 53:07
  Speaker 1: you're seeing something that it's of course you're gonna get
• 53:10
  Speaker 1: ten heads in a row if you flip a coin
• 53:12
  Speaker 1: a million times. That's all you're seeing. Do you have
• 53:16
  Speaker 1: an application to those sort of of cognitive issues, um,
• 53:20
  Speaker 1: So we haven't found the specific application, to be honest,
• 53:23
  Speaker 1: we haven't scoured that literature. UM. You know, we have
• 53:26
  Speaker 1: found papers that have measures of clustering, like how likely
• 53:29
  Speaker 1: am I going to live next to someone of who's
• 53:31
  Speaker 1: like me, you know versus not like me depending on
• 53:34
  Speaker 1: who's around? Um. And there are some measures that are
• 53:37
  Speaker 1: biased for a similar reason. Um that we have this
• 53:39
  Speaker 1: bias now in the councilor cluster one. UM. You know
• 53:42
  Speaker 1: that's that's a little bit different. UM. And it's and
• 53:44
  Speaker 1: it's because of you know what you say is it's
• 53:46
  Speaker 1: kind of blade of grass fallacy that you know. You know,
• 53:48
  Speaker 1: there's lots of blade of grass. You know, you shoot
• 53:51
  Speaker 1: a speck of water and it hits the blade of grass,
• 53:53
  Speaker 1: the blade of grass. Oh look, I'm so lucky it
• 53:55
  Speaker 1: was coming for me. What had to hit some blade
• 53:58
  Speaker 1: of grace? Right now, someone's gonna win the lot. Someone's
• 54:00
  Speaker 1: got to win the lot everyone. You know, the chances
• 54:01
  Speaker 1: that somebody wins the lottery is super high. Not so much. Yeah,
• 54:07
  Speaker 1: so that's interesting. So before I get to my favorite questions,
• 54:12
  Speaker 1: I asked all my guests, I have to ask you
• 54:14
  Speaker 1: what else do you guys um working on? What are
• 54:19
  Speaker 1: the research is coming from the minds that brought you
• 54:23
  Speaker 1: um proof that the hot hands exists? Well, you know,
• 54:26
  Speaker 1: so in our world, it's very tempting to move on
• 54:29
  Speaker 1: to the next one before you know, finishing what you started, right,
• 54:33
  Speaker 1: so you have not exhausted everything out of this one
• 54:37
  Speaker 1: one piece of day, right, So we have we have
• 54:39
  Speaker 1: a lot of kind of uh you know, I used
• 54:42
  Speaker 1: to dot, he used to cross. But you know a
• 54:43
  Speaker 1: little bit more than that. You know, you want to
• 54:45
  Speaker 1: you want to finish and get the message out, but
• 54:47
  Speaker 1: also share, you know, the other insights that we have,
• 54:50
  Speaker 1: because you know, they come out of the same you know,
• 54:52
  Speaker 1: so this is our you know, say, the main insight,
• 54:54
  Speaker 1: but there are other very subtle and interesting insights that
• 54:57
  Speaker 1: we have because you know, when you master something, yeah,
• 54:59
  Speaker 1: and you come back after working on something for a while,
• 55:02
  Speaker 1: there's a lot to share. So tell us some what
• 55:04
  Speaker 1: what other insights can be derived from from the hot
• 55:08
  Speaker 1: hand papers. Yeah, so there's another result in in that study,
• 55:11
  Speaker 1: um um Gilvich introversity study that Gilvich mentioned in the
• 55:15
  Speaker 1: book that you talked about earlier, which is okay, so
• 55:19
  Speaker 1: they kind of got that people, uh you know, that
• 55:21
  Speaker 1: they measured hot hand in a certain way, and they realize, well,
• 55:24
  Speaker 1: maybe we're not capturing everything that means about hot hand,
• 55:27
  Speaker 1: and maybe some players are are seeing something that we,
• 55:30
  Speaker 1: the statisticians, the econometricians you know, aren't measuring. And they
• 55:34
  Speaker 1: went and had people predict and bet on outcomes and
• 55:36
  Speaker 1: they found that they their bets don't really correlate with
• 55:40
  Speaker 1: the outcomes, and so that's kind of evidence. Well, okay, well,
• 55:43
  Speaker 1: even if we're not measuring everything, look if the players
• 55:45
  Speaker 1: are seeing something you think they would bet successfully, and
• 55:48
  Speaker 1: that you could also take that as evidence that now
• 55:50
  Speaker 1: that there is a hot hand there, well, it's at
• 55:53
  Speaker 1: least it's evidence that there's somehow not using exploiting it
• 55:55
  Speaker 1: in a profitable way. But there was actually a mistake
• 55:59
  Speaker 1: as well in in that in that analysis, which is,
• 56:02
  Speaker 1: even if someone were perfect at detecting the hot hand,
• 56:05
  Speaker 1: they knew you know, you can imagine Ann and Bob.
• 56:07
  Speaker 1: Bob's a shooter and his you know, predictor she's observing Bob.
• 56:10
  Speaker 1: She knows when Bob's hot, and whenever Bob's hot, she's
• 56:12
  Speaker 1: going to predict that Bob's going to make the shot. Now,
• 56:15
  Speaker 1: you would expect if she's good, then that good, then
• 56:18
  Speaker 1: her bets, her predictions are going to correlate really well,
• 56:21
  Speaker 1: um with Bob's outcome. But actually you wouldn't expect that.
• 56:24
  Speaker 1: And that's that's another counterintuitive thing is that while she's
• 56:27
  Speaker 1: perfect at detecting his state, the outcome of the state
• 56:30
  Speaker 1: is noisy. You're just getting one draw from Bob's earns.
• 56:32
  Speaker 1: So even if Bob's moving from a seventy to eight
• 56:36
  Speaker 1: probability shooter, if you only take one draw from that urn,
• 56:40
  Speaker 1: you're not getting a very good signal on Bob's state
• 56:43
  Speaker 1: you need a lot. Yeah, and so even if you're
• 56:45
  Speaker 1: getting many predictions from Ann and Bob, you're still only
• 56:47
  Speaker 1: getting you know, one draw on each one. And so
• 56:52
  Speaker 1: the evidence that they have there was actually enough evidence
• 56:54
  Speaker 1: that's consistent with and being very good at detecting it
• 56:58
  Speaker 1: and actually with you, Rhianna. Is the data you find
• 57:01
  Speaker 1: that Bob shoots seven around seven percentage points better when
• 57:05
  Speaker 1: and predicts that he's gonna make it, you miss it.
• 57:09
  Speaker 1: So in their data they have real people that are
• 57:11
  Speaker 1: paid basketball players are betting on each other's shots. And
• 57:14
  Speaker 1: that's the evidence that we find that's quite interesting. That
• 57:18
  Speaker 1: sounds that betting on the outcome of a shot sounds
• 57:23
  Speaker 1: very much like fund managers selecting stocks for a portfolio.
• 57:28
  Speaker 1: Have you applied any of the hot hands? Two? How
• 57:33
  Speaker 1: do fund managers do when they're on a hot streak
• 57:36
  Speaker 1: or a cold streak? And there's a ton of mean
• 57:39
  Speaker 1: reversion in that data series, right, So we haven't gone
• 57:42
  Speaker 1: and analyzed and the mechanisms for being hot in the
• 57:45
  Speaker 1: financial world are going to be quite different than in
• 57:47
  Speaker 1: the basketball world. Right. So, you know, one way of thinking,
• 57:50
  Speaker 1: you look for sec indictments, you look for no I'm
• 57:52
  Speaker 1: just kidding, um, for sure. It becomes so affected by
• 57:57
  Speaker 1: such large macro things it's hard to give credit or
• 58:01
  Speaker 1: or not or your You know, your model of the
• 58:04
  Speaker 1: world happens to be uniquely fit the current situation, and
• 58:07
  Speaker 1: you recognize that, but you know that may or may
• 58:09
  Speaker 1: not be temperary, and that's going to know that that
• 58:11
  Speaker 1: would probably expire, but that's a very different mechanism, and
• 58:14
  Speaker 1: say how it would emergency a basketball game? So I
• 58:16
  Speaker 1: interrupted you, what what else did you do you see
• 58:18
  Speaker 1: as an application of this elsewhere, um, an application of our,
• 58:23
  Speaker 1: of your of what you've discovered to to the world
• 58:27
  Speaker 1: of finance, to the world of finance. So the immediate
• 58:30
  Speaker 1: applications maybe not so much. But if you think about
• 58:35
  Speaker 1: people picking stocks, let's say not so much investing, but
• 58:38
  Speaker 1: someone wants to prove that they're good, um at predicting
• 58:41
  Speaker 1: when a stock is going to go up or down.
• 58:43
  Speaker 1: You have to pay attention not too how often they're right,
• 58:47
  Speaker 1: but how much money they make when they're right and
• 58:50
  Speaker 1: when they're wrong. Because it's very easy to gain these things.
• 58:53
  Speaker 1: So if I were to say, let's say it's fifty fifty,
• 58:56
  Speaker 1: I want to prove that I'm good at predicting coin flips.
• 58:59
  Speaker 1: And so if every month you know a coin is flipped,
• 59:01
  Speaker 1: each day stock goes up or down. But I only
• 59:04
  Speaker 1: bet when there's three heads in a row. When the
• 59:07
  Speaker 1: stock goes up three times in a row, and I
• 59:09
  Speaker 1: bet it's gonna go down. Right in any given month,
• 59:13
  Speaker 1: I'm gonna be right more often than I'm wrong, And
• 59:16
  Speaker 1: so I can game you know if you don't, if you,
• 59:19
  Speaker 1: if you, if you brack it to the month level,
• 59:22
  Speaker 1: I'm gonna be more often right in certain months, and
• 59:24
  Speaker 1: it's gonna look like I'm doing well. But the thing
• 59:26
  Speaker 1: you haven't paid attention to is how often I was
• 59:29
  Speaker 1: right and half when I was wrong in the months
• 59:31
  Speaker 1: that I did poorly. And so if I'm always betting
• 59:34
  Speaker 1: that it's gonna go down when there's a few ups
• 59:36
  Speaker 1: in a row, there's gonna be those months when it
• 59:38
  Speaker 1: keeps going down, right, But you know there's gonna be
• 59:41
  Speaker 1: a few of those months, and there's gonna be many
• 59:43
  Speaker 1: months when I did well, but I didn't predict very
• 59:45
  Speaker 1: many times, and so you're not controlling for how often
• 59:47
  Speaker 1: I predicted, and so it looks like like I do
• 59:49
  Speaker 1: really well. But if I were to bet I wouldn't
• 59:52
  Speaker 1: be making any money because I'd be losing a lot
• 59:54
  Speaker 1: of money in the months where I didn't predict that well,
• 59:56
  Speaker 1: only winning a little bit of money in the months
• 59:58
  Speaker 1: that I predicted well. The interesting thing is if you
• 1:00:01
  Speaker 1: talk to active traders UM who have been successful, they're
• 1:00:06
  Speaker 1: not aiming for fifty fifty. They're aiming for those opportunities
• 1:00:09
  Speaker 1: where a trade becomes a winner and they don't sell
• 1:00:12
  Speaker 1: too early. So it's not your banning average, it's how
• 1:00:16
  Speaker 1: far the ball goes when you actually hit it. That's right,
• 1:00:19
  Speaker 1: Meaning you could have a winning trading record, but in
• 1:00:23
  Speaker 1: terms of percentage of winning trades, but in terms of
• 1:00:26
  Speaker 1: dollars one and lost, those twenty more than make up
• 1:00:29
  Speaker 1: for the remaining eighty. And I always find a lot
• 1:00:31
  Speaker 1: of new traders don't understand that. They think they're hitting
• 1:00:34
  Speaker 1: for percentage, but they're not. They're heading for distance. To
• 1:00:37
  Speaker 1: bring in a different sports metaphor anything else you want
• 1:00:41
  Speaker 1: to share about UM, the research, or what you guys
• 1:00:44
  Speaker 1: might have coming out in the near future, you and
• 1:00:46
  Speaker 1: your co author UM. Off the top of my head, no,
• 1:00:50
  Speaker 1: I think we The one thing that we will say
• 1:00:52
  Speaker 1: is actually there's one thing, So it's not simply that
• 1:00:56
  Speaker 1: we found that the original analysis was vowed in the
• 1:00:59
  Speaker 1: original conclusions were invalid. You know, if you go back
• 1:01:02
  Speaker 1: and you reanalyze that data, you find that players shoot
• 1:01:06
  Speaker 1: a lot better. But it's not simply in that data set.
• 1:01:08
  Speaker 1: So we've gone and collected many other data sets that
• 1:01:11
  Speaker 1: replicated their original study got the same conclusion because they
• 1:01:14
  Speaker 1: had the same bias that the original study had. And
• 1:01:16
  Speaker 1: so when you go back and you fix that, you
• 1:01:18
  Speaker 1: find evidence everywhere, and and and that's we have a
• 1:01:22
  Speaker 1: paper that we're finishing that's showing how robust our conclusions were.
• 1:01:27
  Speaker 1: So I know that there are all sorts of interesting
• 1:01:31
  Speaker 1: UM awards from mathematical and statistical research. Are you looking
• 1:01:35
  Speaker 1: at applying for any of these? How how does that work?
• 1:01:38
  Speaker 1: Are you can you self nominate? Does the institution have
• 1:01:41
  Speaker 1: to nominate you? How do you say? How how does
• 1:01:44
  Speaker 1: that process work? Have you guys thought about this? It's
• 1:01:46
  Speaker 1: not something we thought about. Oh well, let me plant
• 1:01:49
  Speaker 1: that seed. And if this is significant enough, you should
• 1:01:52
  Speaker 1: uh apply for either a grant or a mathematical award,
• 1:01:56
  Speaker 1: although most of these you have to be nominated by
• 1:01:59
  Speaker 1: other of people. But how hard is that to have
• 1:02:02
  Speaker 1: your department share and nominate you. That's that's easy enough.
• 1:02:05
  Speaker 1: I have to ask you. I didn't ask this earlier.
• 1:02:08
  Speaker 1: You grew up in California, you went to Santa Barbara.
• 1:02:10
  Speaker 1: How did you end up in Spain? Oh? So I
• 1:02:15
  Speaker 1: think you remember two thousand and eight, two thousand nine
• 1:02:17
  Speaker 1: a little bit. So that academic jar market was an
• 1:02:21
  Speaker 1: interesting one. I was going on the market in late
• 1:02:23
  Speaker 1: two thousand and eight, two thousand nine, and so a
• 1:02:27
  Speaker 1: lot of academic appointments, non appointments, advertisers for positions were
• 1:02:32
  Speaker 1: disappearing because of you know, the crisis. You would think
• 1:02:35
  Speaker 1: academia is with large endowments and what have you, someone
• 1:02:39
  Speaker 1: insulated from the vagaries of the stock market and even
• 1:02:44
  Speaker 1: the broader economy. But apparently not yea. And so my
• 1:02:48
  Speaker 1: advisor came to me and said, look, this year, everyone's
• 1:02:52
  Speaker 1: applying everywhere, so you need to apply to Europe, even
• 1:02:54
  Speaker 1: if you weren't even think, you weren't thinking about it.
• 1:02:56
  Speaker 1: So at that time I applied everywhere and it was
• 1:02:58
  Speaker 1: great because it opened my mind mine to the great
• 1:03:00
  Speaker 1: opportunities that are there. So right, so I moved to
• 1:03:03
  Speaker 1: Italy in two thousand nine. That was my first stop
• 1:03:05
  Speaker 1: Where were you in Italy? Bacona University in Milan? That
• 1:03:09
  Speaker 1: there are worse places in the world to ride out
• 1:03:11
  Speaker 1: a recession. Yeah, no, it was a good It was
• 1:03:13
  Speaker 1: a good time, I can imagine. And then from Milan,
• 1:03:15
  Speaker 1: how do you end up in in Spain? So I
• 1:03:18
  Speaker 1: wanted to join my co author and finish our work.
• 1:03:20
  Speaker 1: Is that where he was located? Yeah, and he still is.
• 1:03:23
  Speaker 1: So we're both the University of the Content on the
• 1:03:25
  Speaker 1: lovely Mediterranean ceo of there there again, parts of that
• 1:03:30
  Speaker 1: whole whole Millranian coast is just spectacular, isn't it. So
• 1:03:36
  Speaker 1: you don't miss California too much? I get back a
• 1:03:39
  Speaker 1: couple of times a year. Quite quite interesting. All right,
• 1:03:41
  Speaker 1: let's jump to our favorite questions. I can't believe you
• 1:03:44
  Speaker 1: guys never thought of saying, hey, maybe we should apply
• 1:03:46
  Speaker 1: just for some of these grants and some of these
• 1:03:49
  Speaker 1: award rates. We think of applying to grants, But the
• 1:03:51
  Speaker 1: award thing, I don't know. Yeah, all right. I always
• 1:03:56
  Speaker 1: thought academics had to do stuff like that in order
• 1:03:59
  Speaker 1: to maintain their academic standing. Grants and I apply for money,
• 1:04:06
  Speaker 1: that's for sure, But the the the recognition, um, it's
• 1:04:11
  Speaker 1: a good idea. Yeah, not a bad idea. Let me
• 1:04:13
  Speaker 1: let me when you give your acceptance speech. Yeah, definitely,
• 1:04:17
  Speaker 1: all right. So, um, let's jump into a favorite questions,
• 1:04:21
  Speaker 1: which I'll modify slightly because, um, most people, I think
• 1:04:27
  Speaker 1: I don't know you personally. So let me ask the question,
• 1:04:30
  Speaker 1: what's the most important thing, um, that your friends and
• 1:04:33
  Speaker 1: family don't know about you? Friends and families? So you
• 1:04:39
  Speaker 1: had mentioned this question earlier, and I was going to say,
• 1:04:41
  Speaker 1: the most important thing we've actually already revealed, which is
• 1:04:45
  Speaker 1: which is it's what most people didn't know we haven't
• 1:04:49
  Speaker 1: shared this much? Is that this might be in this research, um,
• 1:04:52
  Speaker 1: the first joint eureka moment, right, Usually someone discovers something.
• 1:04:57
  Speaker 1: You when they say simultaneous discovery, someone discover something at
• 1:05:01
  Speaker 1: one point in time and somebody at another, but no
• 1:05:02
  Speaker 1: one knew about it. Did The electric light bulb as
• 1:05:05
  Speaker 1: a classical radio is another classic example. So, but my
• 1:05:09
  Speaker 1: cauthor and I were in this, you know, we were
• 1:05:10
  Speaker 1: in the same room, we were on the phone at
• 1:05:12
  Speaker 1: the same time, and we both had the you know,
• 1:05:14
  Speaker 1: the gold is there and then how often does that happen? So?
• 1:05:19
  Speaker 1: Who were some of your mentors in your early career?
• 1:05:23
  Speaker 1: So in my early career, um, I would say, and
• 1:05:27
  Speaker 1: my author would say the same. You know, my my advisor,
• 1:05:30
  Speaker 1: Alto ruster KINI is a professor at University of Minnesota,
• 1:05:33
  Speaker 1: and he's just you know, he's a neuroscientist, he's a mathematician,
• 1:05:35
  Speaker 1: he's an economist. He's all about the science, right, And
• 1:05:38
  Speaker 1: I'm thinking a renaissance person. Oh, he's a renaissance person.
• 1:05:41
  Speaker 1: And you know when you see that, um, and you
• 1:05:44
  Speaker 1: see someone that's just you know, zeroed in on that
• 1:05:46
  Speaker 1: and you see how they work, you kind of you
• 1:05:47
  Speaker 1: get you kind of absorbed what they do through osmosis.
• 1:05:50
  Speaker 1: And my CA author with the same to say the
• 1:05:51
  Speaker 1: same thing about his advisor, Vince Crawford at Oxford University
• 1:05:55
  Speaker 1: and a very deep guy, very brilliant, you know, both
• 1:05:57
  Speaker 1: very brilliant. Um. And and those are formative years when
• 1:06:02
  Speaker 1: you're in grad school for for sure. For sure. UM,
• 1:06:05
  Speaker 1: So what other behaviorists and statisticians influenced your approach um
• 1:06:12
  Speaker 1: to thinking about the mathiness of things like shooting streaks?
• 1:06:18
  Speaker 1: So that I would say the statisticians that have influenced
• 1:06:21
  Speaker 1: me out there is one, right, there's Andrew Galman reading
• 1:06:24
  Speaker 1: his blog has been eye opening for so many people.
• 1:06:27
  Speaker 1: I mean, he just introduced how to think about data
• 1:06:32
  Speaker 1: in a way that most people don't get in their
• 1:06:35
  Speaker 1: formal training because he's dealing with real practical examples all
• 1:06:39
  Speaker 1: the time. Um, and so I would say he's been
• 1:06:41
  Speaker 1: one of the biggest influences interesting about the what about
• 1:06:46
  Speaker 1: on the behavioral side? On the behavioral side, there's just
• 1:06:50
  Speaker 1: so many great you know. I mean, there there was
• 1:06:52
  Speaker 1: this vanguard of the people that the folks that came
• 1:06:54
  Speaker 1: in in the eighties that really had to fight through,
• 1:06:57
  Speaker 1: you know, the review process of all the skepticism towards
• 1:07:00
  Speaker 1: you know, why a psychology you know, relevant to economics,
• 1:07:03
  Speaker 1: Why are these other social science disciplines? What do they
• 1:07:06
  Speaker 1: have to say about? People really had to fight a
• 1:07:08
  Speaker 1: lot of skepticism. So give us some names. I'm putting
• 1:07:11
  Speaker 1: it on the spot, Okay. So the people that had
• 1:07:13
  Speaker 1: to fight through that, I mean that's like, you know,
• 1:07:14
  Speaker 1: Amos Tversky and Daniel Kaneman were very influential, but they
• 1:07:18
  Speaker 1: know they were within psych psychologist they were fine. So
• 1:07:21
  Speaker 1: the people that had to deal, you know, with this
• 1:07:23
  Speaker 1: kind of pushback. I mean you say, like you know,
• 1:07:25
  Speaker 1: Richard Taylor, you know, as much as you know he
• 1:07:28
  Speaker 1: he's been a bit skeptical of our work, but you
• 1:07:31
  Speaker 1: have to respect, you know, and and you know, both
• 1:07:34
  Speaker 1: his the insights he has into human behavior UM, and
• 1:07:38
  Speaker 1: also just what he had to fight through get get
• 1:07:41
  Speaker 1: listened to UM. So he was a guest, and my
• 1:07:44
  Speaker 1: favorite quote from him was UM. Early on he decided
• 1:07:48
  Speaker 1: he would never convince his peers. So he thought, I'm
• 1:07:51
  Speaker 1: going to bypass them and just try and convince the
• 1:07:53
  Speaker 1: grad students, and we'll just wait it out. After enough funerals,
• 1:07:57
  Speaker 1: we will have one. And and it's really turned out
• 1:08:00
  Speaker 1: to be quite true. If you if you are influencing
• 1:08:04
  Speaker 1: the next generation, that's more and more impactful than what
• 1:08:08
  Speaker 1: Tversky said, winning all these arguments and convincing nobody. Uh,
• 1:08:13
  Speaker 1: it turned out to be very clever. Anybody else you
• 1:08:15
  Speaker 1: want to mention from that group or oh no, I mean,
• 1:08:19
  Speaker 1: I mean I wouldn't want to single out any person,
• 1:08:22
  Speaker 1: you know, you just you look at the people that
• 1:08:24
  Speaker 1: really kind of did a lot of the fighting that
• 1:08:26
  Speaker 1: you're kind of push push the ideas through. But you know,
• 1:08:28
  Speaker 1: in terms of the ideal dal level, there's so many right,
• 1:08:31
  Speaker 1: you know, so even in the you know, when we
• 1:08:33
  Speaker 1: were presenting our work. You know, you have somebody to say,
• 1:08:35
  Speaker 1: like Colin camer cal Tech. You know, he actually just
• 1:08:39
  Speaker 1: met him at a conference and he's a he's really
• 1:08:42
  Speaker 1: a fascinating dude. Oh yes, And you know there's a
• 1:08:44
  Speaker 1: lot of similarities you know that I recognize in him.
• 1:08:46
  Speaker 1: That's kind of similar to the advisor that Altarosta Kidia
• 1:08:49
  Speaker 1: University of Minnesota. You know, just this, You know that
• 1:08:52
  Speaker 1: he's all about the science and really, I mean he
• 1:08:54
  Speaker 1: came to one of our talks and he brought up
• 1:08:57
  Speaker 1: Footnotes seventy two, which was the weakest point, the point
• 1:09:01
  Speaker 1: he really wanted, and he found it. He found it.
• 1:09:03
  Speaker 1: You know, we had a nice discussion about it. He
• 1:09:06
  Speaker 1: saw our perspective after we had to talk. It's like, Wow,
• 1:09:09
  Speaker 1: he's really taking this seriously, and it's just it's it's
• 1:09:12
  Speaker 1: nice to see that that. That's got to be so delightful.
• 1:09:15
  Speaker 1: I believe we have him tied up for the spring
• 1:09:18
  Speaker 1: as a guest. Um. Yeah, he's really I love the
• 1:09:21
  Speaker 1: work he does with virtual reality and showing people in
• 1:09:26
  Speaker 1: incredible detail what they're gonna look like when they're older,
• 1:09:29
  Speaker 1: and it affects their decision making. Dramatically in terms of planning,
• 1:09:33
  Speaker 1: not just like a computer generated picture that's been aged,
• 1:09:37
  Speaker 1: but when you have this immersive VR experience of here's
• 1:09:41
  Speaker 1: your life when you're eighty, it leads to all sorts
• 1:09:44
  Speaker 1: of amazing changes when you're forty. It it's quite astonishing.
• 1:09:48
  Speaker 1: I'm gonna to do that. So I'm glad, glad you
• 1:09:50
  Speaker 1: brought that up. Um, so I'm gonna put down Gellman
• 1:09:53
  Speaker 1: as one of those people who influenced your approach to
• 1:09:56
  Speaker 1: UH statistics. Let's talk about books. What are some of
• 1:09:58
  Speaker 1: your favorite books? So I'll pick out a book. Um,
• 1:10:02
  Speaker 1: you know, we could pick out a lot of nonfiction
• 1:10:04
  Speaker 1: books and a lot of books like that. It hits
• 1:10:07
  Speaker 1: you at the right time. And then if I tell
• 1:10:09
  Speaker 1: you that book, I might you know, if I were
• 1:10:11
  Speaker 1: to look at it now, it might feel trivial obvious
• 1:10:13
  Speaker 1: things like this. You never know if the book is
• 1:10:15
  Speaker 1: targeted for the right person. So I'll bring up a
• 1:10:17
  Speaker 1: book that both my cauthor and I were a lot
• 1:10:19
  Speaker 1: of were very much influenced by. But it's literature. So
• 1:10:22
  Speaker 1: there's this book called The Alexandria Quartet by Lawrence Durrow.
• 1:10:25
  Speaker 1: We both read it in our university days, and it's
• 1:10:28
  Speaker 1: kind of like The Blind Men and the Elephant, but
• 1:10:30
  Speaker 1: for human relationships, and it has this really novel. Idea's
• 1:10:33
  Speaker 1: four books. The first three books are about three different
• 1:10:36
  Speaker 1: perspectives on a serious relationships and events that happened in
• 1:10:40
  Speaker 1: a particular time in Egypt before World War two, um,
• 1:10:44
  Speaker 1: and from different people's perspectives. And so that's the space
• 1:10:48
  Speaker 1: kind of. So it was inspired a bit by Einstein's relativity.
• 1:10:51
  Speaker 1: So there's like three different perspectives on space and then
• 1:10:53
  Speaker 1: they go forward in time and do the reflection back
• 1:10:57
  Speaker 1: on on those relationships and it gives you, you know,
• 1:11:00
  Speaker 1: this kind of you know, humility, kind of see like
• 1:11:03
  Speaker 1: how how small your perspective is, how much missing information
• 1:11:06
  Speaker 1: you have about what's happening, And it's it's it's it's
• 1:11:09
  Speaker 1: a it's a nice read, at least it was. And
• 1:11:10
  Speaker 1: when I was in my twenties, that sounds quite fascinating
• 1:11:13
  Speaker 1: that this question is what people ask me more about
• 1:11:18
  Speaker 1: than any other question because they want to get a
• 1:11:22
  Speaker 1: book recommendation from somebody who's accomplished something done, something interesting,
• 1:11:27
  Speaker 1: has some experience. And when someone says, oh, and by
• 1:11:30
  Speaker 1: the way, this book is worth reading, it's the greatest
• 1:11:33
  Speaker 1: endorsement anybody can ever get. So I'm going to press
• 1:11:36
  Speaker 1: you and say, give us one or two more books,
• 1:11:39
  Speaker 1: even if you think they may have been very time
• 1:11:41
  Speaker 1: specific to you. So Duncan Watts has this book called
• 1:11:45
  Speaker 1: Everything Is Obvious. Beautiful book. It's so interesting. It's all
• 1:11:50
  Speaker 1: about client side bias and how you see things after
• 1:11:54
  Speaker 1: the fact. It's it's you're the first person who's brought
• 1:11:57
  Speaker 1: that book up and I find it. I love the
• 1:11:59
  Speaker 1: cover with the wheel. I think that's a triangle instead
• 1:12:03
  Speaker 1: of a circle. It's um, it's really a very fascinating book.
• 1:12:06
  Speaker 1: It's you know, it's as you have this curse of knowledge,
• 1:12:08
  Speaker 1: right once you know something, it's obvious to you and
• 1:12:10
  Speaker 1: you can't imagine how not obvious it would be to
• 1:12:13
  Speaker 1: someone else. And in the references in that book, I
• 1:12:17
  Speaker 1: mean he you know, he's an academic, so he's really
• 1:12:21
  Speaker 1: given you, you know, the road map, and like if
• 1:12:23
  Speaker 1: you want to go beyond that book, all the references
• 1:12:25
  Speaker 1: are there in that book. It's great. Um. And then
• 1:12:28
  Speaker 1: I would say another book that's like Along the Lines,
• 1:12:31
  Speaker 1: only because in the last year I read it. Um.
• 1:12:34
  Speaker 1: You know. Super Forecasting by Philip another prior guest, delightful. Yeah,
• 1:12:39
  Speaker 1: I mean it gives you, you know, humility to kind
• 1:12:41
  Speaker 1: of realize how how you know, if you want to
• 1:12:44
  Speaker 1: start projecting three years, five years and you're wasting you're
• 1:12:48
  Speaker 1: wasting your time. But you know there's a lot more
• 1:12:49
  Speaker 1: than that, right, just a disciplined approach. Right, So it's
• 1:12:52
  Speaker 1: not simply you know and one one mistake you can make,
• 1:12:54
  Speaker 1: because you know mistake I made, say around the financial
• 1:12:57
  Speaker 1: crisis time, I really was convinced City Bank would be
• 1:13:00
  Speaker 1: bailed out, they wouldn't let City Bank completely crash. Well,
• 1:13:03
  Speaker 1: you were, you were not wrong. They did bail it. Eventually,
• 1:13:06
  Speaker 1: it was two hours at the time, but still they
• 1:13:09
  Speaker 1: were bailed out. Had you said the same about leaven Brothers,
• 1:13:12
  Speaker 1: that would have been a different situation. What's most fascinating
• 1:13:17
  Speaker 1: about Tetlock talk about recognizing your own issue. Tetlock's original book,
• 1:13:25
  Speaker 1: If You Go Back, was on expert political judgment and
• 1:13:29
  Speaker 1: that nobody is good at forecast now, and he then
• 1:13:34
  Speaker 1: over time led to what what led from him going
• 1:13:39
  Speaker 1: from oh, we're really bad as a species at forecasting too,
• 1:13:43
  Speaker 1: But a handful of people have do a number of
• 1:13:45
  Speaker 1: things that make them better at it, and therefore that's
• 1:13:48
  Speaker 1: how you end up with super forecasters. That's a fascinating
• 1:13:51
  Speaker 1: arc over I don't know, third twenty years separating the
• 1:13:55
  Speaker 1: two books. Yes, and and the thing I really got
• 1:13:58
  Speaker 1: from that book is not getting fixed at it on
• 1:14:00
  Speaker 1: your your one insight and putting all your cards on that.
• 1:14:03
  Speaker 1: You know. So, so you know, these these super forecasters
• 1:14:07
  Speaker 1: are right in the long run. You know they're using
• 1:14:09
  Speaker 1: the law of large numbers. It's not that they're saying, oh,
• 1:14:11
  Speaker 1: I have this one idea, I'm gonna fixate city Bank
• 1:14:14
  Speaker 1: has to be bailed out. No, no, take all your
• 1:14:16
  Speaker 1: ideas and spread your bets across all your ideas, just
• 1:14:19
  Speaker 1: like the forecasters, and you'll do well eventually. But don't
• 1:14:22
  Speaker 1: get fixated on that one. And I think that's a
• 1:14:24
  Speaker 1: nice feature, but quite fascinating any of the books before
• 1:14:28
  Speaker 1: we move on, I think that's good. Those three, those
• 1:14:30
  Speaker 1: are three three good ones. Um, so what are you
• 1:14:33
  Speaker 1: excited about right now? What what do you jazzed about
• 1:14:35
  Speaker 1: in the world of academic research. Well, and I think
• 1:14:38
  Speaker 1: mastery is addictive, right, So there's a story. There's a
• 1:14:41
  Speaker 1: lot of drive by research out there, and you know,
• 1:14:44
  Speaker 1: we're all a little guilty of it because you know
• 1:14:47
  Speaker 1: there's this pressure to publish, and when you've gone and
• 1:14:51
  Speaker 1: you've really dug into something, you've really mastered something, and
• 1:14:54
  Speaker 1: just mind all as much of the gold as you have,
• 1:14:56
  Speaker 1: but you also have this feeling of mastery. It just
• 1:14:58
  Speaker 1: feels so great and wanting to do that again. Right, So,
• 1:15:02
  Speaker 1: so the exciting thing is to take that understanding of
• 1:15:06
  Speaker 1: how good it feels, how fun it is to master something,
• 1:15:09
  Speaker 1: and take it to the next subject. Well, of course,
• 1:15:11
  Speaker 1: still finishing what you started. So that's that's the exciting things.
• 1:15:15
  Speaker 1: What's next really really interesting? There have been all sorts
• 1:15:19
  Speaker 1: of criticisms of the lack of reproducibility in academic research.
• 1:15:25
  Speaker 1: What changes are you looking forward to into. Do you
• 1:15:28
  Speaker 1: think that increased big data and AI is ever going
• 1:15:34
  Speaker 1: to help us with this reproducibility problem we're running into
• 1:15:39
  Speaker 1: in academic research and in other research. Why aren't we
• 1:15:43
  Speaker 1: seeing academic research being replicated and and even corporate research
• 1:15:47
  Speaker 1: being replicated. I think there's a lot of cherry picking
• 1:15:50
  Speaker 1: that happens. Um. So when you go out and you
• 1:15:54
  Speaker 1: analyze something, and you measure ten different things, and you
• 1:15:56
  Speaker 1: just pick out the things that worked, Um, you're you're
• 1:16:00
  Speaker 1: not acknowledging what didn't work, and so you're gonna you
• 1:16:02
  Speaker 1: have this kind of you know, winner's curse in a
• 1:16:04
  Speaker 1: sense where you know I won the that's sailors of
• 1:16:10
  Speaker 1: his early books. Yeah, we don't have time to explain
• 1:16:12
  Speaker 1: that I realized. But but um so, let me re
• 1:16:15
  Speaker 1: ask that question. So what are you looking forward to?
• 1:16:18
  Speaker 1: What changes do you think you're gonna affect um your
• 1:16:22
  Speaker 1: your world, the changes that are gonna affect the academic world.
• 1:16:26
  Speaker 1: Um so, I think that the important change that's going
• 1:16:28
  Speaker 1: to make this better. They're gonna gonna fix this, fix
• 1:16:31
  Speaker 1: maybe not, but make it better. Is the idea of preregistration,
• 1:16:35
  Speaker 1: meaning what you pre register what you're going to analyze.
• 1:16:38
  Speaker 1: You register your predictions, and so you know your hands
• 1:16:43
  Speaker 1: are tied. So therefore you're going to go out and
• 1:16:45
  Speaker 1: actually research what you're claiming as opposed to, oh, look
• 1:16:48
  Speaker 1: at this anomaly, let's talk about that, even though it
• 1:16:51
  Speaker 1: could be random or cherry pick or what have you,
• 1:16:54
  Speaker 1: and still valuing even whatever your conclusion is, don't take
• 1:16:58
  Speaker 1: that as truth when we have to make sure it
• 1:17:01
  Speaker 1: also replicates, because even then you may have what if
• 1:17:04
  Speaker 1: you don't find it you decide not to write it up, Right,
• 1:17:06
  Speaker 1: isn't that a big issue that people don't publish on
• 1:17:10
  Speaker 1: um negative find things because there is value to say, Hey,
• 1:17:13
  Speaker 1: we analyze this, couldn't find anything. It's a huge issue
• 1:17:15
  Speaker 1: because then you get this kind of implicit cherry picking
• 1:17:18
  Speaker 1: and that like, I don't want to spend my time
• 1:17:19
  Speaker 1: writing up this paper because it's not a big finding. Well,
• 1:17:22
  Speaker 1: then no one's seeing that. So then the papers we
• 1:17:24
  Speaker 1: see are the ones that are kind of implicitly selected,
• 1:17:28
  Speaker 1: and so you have the same kind of degrees of
• 1:17:30
  Speaker 1: freedom that's happening. But it's like socially in what do
• 1:17:32
  Speaker 1: you call survivorship bias about things that don't work out?
• 1:17:36
  Speaker 1: So I guess it is just straight up survivorship bias, right,
• 1:17:39
  Speaker 1: In other words, what's public's negative shelter? You know, the
• 1:17:44
  Speaker 1: research ideas that don't make it to the publication stage
• 1:17:47
  Speaker 1: have died, and so the publication ones are kind of
• 1:17:50
  Speaker 1: the ones that are kind of randomly better but not
• 1:17:52
  Speaker 1: necessarily truly better. Interesting tell us about the time you
• 1:17:56
  Speaker 1: failed and what you learned from the experience. So one failure,
• 1:18:01
  Speaker 1: it was a success in a failure. So a friend
• 1:18:03
  Speaker 1: of mine, Patrick Flannagan from graduate school. We said, up
• 1:18:05
  Speaker 1: this garage band hedge fund we called it. We were
• 1:18:07
  Speaker 1: loaning money on the internet, and we thought we had
• 1:18:09
  Speaker 1: this great idea and it was I mean, it weathered
• 1:18:11
  Speaker 1: the crisis. We didn't lose money. We got like five percent.
• 1:18:14
  Speaker 1: We weren't. It wasn't big money. It's maybe a hundred
• 1:18:16
  Speaker 1: thousand or something. We're students, but this is peer to
• 1:18:18
  Speaker 1: peer lending. That But um, the thing we didn't anticipate was,
• 1:18:23
  Speaker 1: you know, the legal uncertainty of the of the enterprise.
• 1:18:25
  Speaker 1: And we weren't lawyers, and we just oh we we
• 1:18:27
  Speaker 1: we modeled. We we had a real confidence in our model.
• 1:18:30
  Speaker 1: We had our automated bidding algorithm. It was like it
• 1:18:32
  Speaker 1: was great, we're doing well. But the thing we didn't
• 1:18:35
  Speaker 1: get is that the SEC could potentially crack down on
• 1:18:39
  Speaker 1: on this and ruin our business model because then they
• 1:18:41
  Speaker 1: changed the rules completely that made it impractical. So we
• 1:18:44
  Speaker 1: just we just left. Because instead of having a direct
• 1:18:48
  Speaker 1: a connection to the person that you're loaning, and it
• 1:18:50
  Speaker 1: was now mediated through the company, and now you had
• 1:18:53
  Speaker 1: to somehow pricing the risk of the company itself rather
• 1:18:56
  Speaker 1: than a loan. So um and and so interest. Yeah,
• 1:19:00
  Speaker 1: quite interesting. What do you do for fun when you're
• 1:19:03
  Speaker 1: not crunching numbers? Well, I mean this is pretty fun
• 1:19:05
  Speaker 1: right here, I am in New York. You know, you
• 1:19:08
  Speaker 1: get to travel around, you get to meet with your
• 1:19:10
  Speaker 1: co authors, finish your papers, and nice locations, and you know,
• 1:19:14
  Speaker 1: meet interesting people. Um, I would say just seeing family,
• 1:19:18
  Speaker 1: because I get to travel so much. I get to
• 1:19:20
  Speaker 1: see my family and friends in different cities, and that's
• 1:19:22
  Speaker 1: a great thing. What sort of advice would you give
• 1:19:24
  Speaker 1: to a millennial or recent college grad who was interested
• 1:19:27
  Speaker 1: in a career either in behavioral finance or stat statistics
• 1:19:31
  Speaker 1: or um, any of the sort of work that you
• 1:19:34
  Speaker 1: you do, economics, whatever. Don't be in too impatient to
• 1:19:38
  Speaker 1: have life figured out. Um, it's not too late. I've
• 1:19:42
  Speaker 1: seen people in their twenties and their thirties go back change,
• 1:19:47
  Speaker 1: go back to school. You know, maybe they have to
• 1:19:49
  Speaker 1: start a little lower rank school than they wanted to
• 1:19:52
  Speaker 1: begin with. But you can get funding at those schools,
• 1:19:54
  Speaker 1: and if you work hard, you can transfer, you can
• 1:19:56
  Speaker 1: apply to another school, you can move up. And a
• 1:19:59
  Speaker 1: lot of people get this kind of false notion that, oh,
• 1:20:01
  Speaker 1: if I didn't if I wasn't a serious student, in
• 1:20:03
  Speaker 1: high schoore in college that I'm too far behind. It's like, no,
• 1:20:08
  Speaker 1: if you're really motivated and you're you know, you're you're capable,
• 1:20:10
  Speaker 1: and there's plenty of people that are. You can catch up.
• 1:20:12
  Speaker 1: You just have to be patient, take a few years off.
• 1:20:14
  Speaker 1: It's never too late to get serious. Yeah, never too
• 1:20:17
  Speaker 1: late to get serious. And our final question, what do
• 1:20:19
  Speaker 1: you know about the world of statistics and data analytics today?
• 1:20:23
  Speaker 1: You wish you knew a decade or so ago. Fake
• 1:20:27
  Speaker 1: data simulation basically creating you want to know. You can't
• 1:20:32
  Speaker 1: just go out and analyze data and show that however
• 1:20:36
  Speaker 1: I analyze it, I still get the same result. No,
• 1:20:38
  Speaker 1: you have to sit down and generate fake data. So
• 1:20:42
  Speaker 1: what if the world looked like this, how would my
• 1:20:45
  Speaker 1: analysis behave? What if the world look like that? How
• 1:20:47
  Speaker 1: the analysis behaves? You have to do this hard work
• 1:20:50
  Speaker 1: of like building models of the world and then seeing
• 1:20:53
  Speaker 1: what does your analytical approach tell you under those different
• 1:20:57
  Speaker 1: kind of assumptions about the model of the world. And
• 1:20:59
  Speaker 1: to do that you need fake data. And that when
• 1:21:02
  Speaker 1: you say fake data, I think of that as a
• 1:21:04
  Speaker 1: counterfactual or how do you contextual? I mean, I guess
• 1:21:07
  Speaker 1: everything is a counter factual because all models are wrong, right, Um,
• 1:21:10
  Speaker 1: But but but some are useful, but some are useful,
• 1:21:14
  Speaker 1: And you want to know how how you know under
• 1:21:18
  Speaker 1: under different assumptions. If the world you know looked differently
• 1:21:22
  Speaker 1: than you think it looks, is your analysis still gonna
• 1:21:26
  Speaker 1: say something meaningful or not? Um And you need to
• 1:21:28
  Speaker 1: actually go out and check that, And a lot of
• 1:21:30
  Speaker 1: people don't do that. And so that's what happened in
• 1:21:32
  Speaker 1: this hot hand example, right, I mean, what would this
• 1:21:35
  Speaker 1: analysis give you if there were no hot hand? Quite
• 1:21:38
  Speaker 1: quite interesting. We have been speaking to Josh Miller of
• 1:21:42
  Speaker 1: the University of Alcante. If you enjoy this conversation, we'll
• 1:21:46
  Speaker 1: be sure and look up an inch or down an
• 1:21:47
  Speaker 1: inch on Apple iTunes, where you can see the past
• 1:21:51
  Speaker 1: two and fifty or so previous conversations we've had. We
• 1:21:56
  Speaker 1: love your comments, feedback and suggestions right to us at
• 1:22:00
  Speaker 1: m IB podcast at Bloomberg dot net. I would be
• 1:22:04
  Speaker 1: remiss if I did not thank the crack staff that
• 1:22:06
  Speaker 1: helps put together these conversations each week. Medina Parwana is
• 1:22:10
  Speaker 1: our producer, Michael bat Nick is my head of research.
• 1:22:14
  Speaker 1: Taylor Riggs is our booker. Slash producer. Attica val Bron
• 1:22:18
  Speaker 1: is our project manager. Tim Harrow is our audio engineer.
• 1:22:22
  Speaker 1: I'm Barry Ritolts. You've been listening to Masters in Business
• 1:22:25
  Speaker 1: on Bloomberg Radio.