Latest / What's Your Problem? / How the Nerds Conquered the NBA

Transcript

How the Nerds Conquered the NBA · What's Your Problem?

0:15
Speaker 1: Pushkin. About a decade ago, special cameras in the rafters
0:24
Speaker 1: of NBA arenas started following all the players around the
0:27
Speaker 1: court and tracking the ball everywhere it went. It was
0:30
Speaker 1: this huge new trove of data, but nobody really knew
0:34
Speaker 1: what to do with it. This is a very familiar
0:37
Speaker 1: problem in the modern world, tons of data that just
0:40
Speaker 1: kind of sits there. Then a few computer scientists came
0:44
Speaker 1: along and had an idea for how to solve the problem.
0:48
Speaker 1: They thought they knew how to take all that data
0:50
Speaker 1: and turn it into profoundly useful information, and so they
0:55
Speaker 1: started a company called Second Spectrum to see if their
0:58
Speaker 1: idea would work. I'm Jacob Goldstein, and this is What's
1:05
Speaker 1: Your Problem, the show where entrepreneurs and engineers talk about
1:07
Speaker 1: how they're going to change the world once they solve
1:10
Speaker 1: a few problems. My guest today is Regeev Mahiswaran. He's
1:14
Speaker 1: the co founder and president of Second Spectrum. The company
1:17
Speaker 1: started out turning video of NBA games into information that
1:21
Speaker 1: coaches could use. Today, they work with every NBA team
1:24
Speaker 1: and with Major League Soccer teams in the US and
1:27
Speaker 1: with Premier League teams in the UK. Regiev's problem is
1:31
Speaker 1: this how do you teach a computer to understand sports?
1:38
Speaker 1: Regiev and his co founders started Second Spectrum back in
1:41
Speaker 1: twenty thirteen. Last year, the company was acquired by a
1:44
Speaker 1: sports analytics company called Genius for two hundred million dollars.
1:49
Speaker 1: My conversation with Regieve focused on his work with the NBA.
1:52
Speaker 1: That's where Second Spectrum has been working the longest. But
1:55
Speaker 1: he started out talking about this bigger idea, how do
1:58
Speaker 1: you turn data, really any kind of data into something useful.
2:02
Speaker 1: So I think this happens everywhere. There's a lot of
2:05
Speaker 1: companies that collect data, faster data, more data. But what
2:09
Speaker 1: generally happens is the big piles of data. I feel
2:11
Speaker 1: like they're like grain. They just if people don't know
2:13
Speaker 1: what to do with them, so they just stick them
2:14
Speaker 1: in the closet and it's like, yep, I got piles
2:16
Speaker 1: of grain in there, and it keeps piling up. And
2:19
Speaker 1: I think what we said was, you know, we brought
2:21
Speaker 1: a bunch of machine learning, a bunch of other stuff
2:23
Speaker 1: to and said, you know, the data, this massive amount
2:25
Speaker 1: of coordinate information. No one can work with it. Coaches
2:28
Speaker 1: can't work with it. The leagues that have a tough
2:30
Speaker 1: time working with the media has a tough time worker
2:32
Speaker 1: that we are going to sort of grind that grain
2:34
Speaker 1: into story elements that people can actually use. So we
2:38
Speaker 1: turn it into oh, that's a pass, or that's a shot,
2:41
Speaker 1: or that's a pick and roll, or that's a between
2:43
Speaker 1: the lines pass, or that's a blitz, and so we
2:45
Speaker 1: start turning them into words by which people can tell stories.
2:48
Speaker 1: So I would say, like the grain is not useful,
2:50
Speaker 1: you don't want to make like bread and donuts. Is
2:52
Speaker 1: there some particular example you can give me from the
2:56
Speaker 1: early days of the company of basically of a problem
3:00
Speaker 1: you solved, of a thing you set out to do.
3:01
Speaker 1: Maybe it was hard to do, maybe it didn't work
3:03
Speaker 1: the way you thought it would, but in the end
3:04
Speaker 1: you made some useful thing for coaches, teams, whatever. There
3:09
Speaker 1: are two things that we did that sort of a
3:10
Speaker 1: big we're a big game changer, right. So one was
3:13
Speaker 1: the idea that people were taking shots and you couldn't
3:17
Speaker 1: tell if they were a good shot or a bad shot.
3:19
Speaker 1: You would use some rules. It's like, oh, if you
3:20
Speaker 1: took it here, you know, if people weren't guarding you
3:23
Speaker 1: too closely, it was good. But you know, if you
3:25
Speaker 1: dribbled a lot, and you jumped in somebody's face. It
3:27
Speaker 1: was bad. I think we were able to use math
3:29
Speaker 1: and say, oh, that shot should go in forty two
3:32
Speaker 1: percent of the time. That shot goes in forty nine
3:34
Speaker 1: percent of the time for an average player, it goes
3:36
Speaker 1: in for this player. So we were able to quantify
3:38
Speaker 1: the quality of a shot, which was basically the core
3:41
Speaker 1: thing in basketball is you want to get high quality
3:43
Speaker 1: shots and prevent high quality shots, and there was no
3:46
Speaker 1: way of measuring it. And we basically said, like, no, no,
3:48
Speaker 1: we know exactly what the quality of the shot is,
3:51
Speaker 1: and then we also know which players add value beyond
3:53
Speaker 1: that shot. So players like Steph Curry and Kevin Durant
3:57
Speaker 1: can take a shot that it might be a forty
3:59
Speaker 1: five for an average player, and for them it goes
4:00
Speaker 1: in fifty five. And that was a big deal. That
4:03
Speaker 1: was the first big thing, just to step through that
4:06
Speaker 1: a little bit more slowly. So before you came along,
4:10
Speaker 1: what did people know? What did coaches know about sort
4:13
Speaker 1: of percentage probabilities that a shot will go in? They
4:17
Speaker 1: would just track it much more coarsely, so they would say, oh,
4:21
Speaker 1: from the corner three, or you would shoot this, and
4:24
Speaker 1: from above the break you would shoot this, or in
4:26
Speaker 1: the lane you would shoot this. But they didn't know
4:28
Speaker 1: things like, hey, there's a really tall defender running straight
4:32
Speaker 1: at you, or you're moving sideways. You know, and these
4:34
Speaker 1: things also matter important. Yeah, and then the raw data
4:38
Speaker 1: allowed us to know all that, and then what we
4:40
Speaker 1: did was start to build models that basically predicted the
4:44
Speaker 1: likelihood of various shots going in. So one of the
4:46
Speaker 1: things you did is coming up with a better metric
4:49
Speaker 1: for shot quality that has more inputs. The other thing
4:53
Speaker 1: you did, is it right, was around the pick and roll.
4:56
Speaker 1: So I think there are all these events in basketball,
4:58
Speaker 1: and you can get pretty esoteric. There's a pick and roll,
5:01
Speaker 1: but there's many types of pick and rolls. There's many
5:03
Speaker 1: types of ways to defend pick and rolls. Just to
5:06
Speaker 1: be clear, let me let me just say we won't
5:07
Speaker 1: go into detail, but pick and roll it's just a play.
5:10
Speaker 1: It's an offensive. It's a play. Yeah, it's somewhat complicated.
5:13
Speaker 1: It involves two people on offense and two people on defense.
5:16
Speaker 1: Let's just say that's that's all you need to know. Okay,
5:18
Speaker 1: So what did you do with the pick and roll?
5:21
Speaker 1: So I think the first thing we did was we
5:23
Speaker 1: had a machine be able to identify it by staring
5:26
Speaker 1: at the data. So before us, it was basically humans
5:30
Speaker 1: would watch the video and just start counting how many
5:34
Speaker 1: happened and counting what the type of defense was, and
5:36
Speaker 1: basically they would just collate all that information. But I
5:39
Speaker 1: think the big thing that we discovered is when machines
5:42
Speaker 1: did it. The big example was there was a year
5:45
Speaker 1: where the human collection said Chris Paul led the league
5:48
Speaker 1: running eight hundred pick and rolls, and we said, well,
5:51
Speaker 1: Chris Paul led the league running four thousand pick and rolls,
5:55
Speaker 1: and one of these is not right. But then we
5:58
Speaker 1: could just say like, well, here's our four thousand, and
6:01
Speaker 1: then you're saying, oh, okay, you're missing sort of you know,
6:03
Speaker 1: eighty percent of them. And so I think then I
6:06
Speaker 1: think once we did that, we could automate a lot
6:08
Speaker 1: more complicated things, like having the machine understand all the
6:12
Speaker 1: defenses that you could play against various types of pick
6:14
Speaker 1: and rolls, and then once we solve those, we sort
6:17
Speaker 1: of built this engine that could generate words accurately, and
6:20
Speaker 1: then we started growing teams and each one of them
6:22
Speaker 1: would say, hey, can you do these words can you
6:25
Speaker 1: do these words? Can you do these words? Like say,
6:26
Speaker 1: some of the words they were asking for. So they're
6:28
Speaker 1: sort of like you know, flares and loops and zippers
6:31
Speaker 1: and jams and trails and whips and you know veer backs,
6:34
Speaker 1: and I mean the sort these things are basically what
6:38
Speaker 1: plays their their defenses, their moves. That's right. So there's
6:41
Speaker 1: like sort of you know, the particular kind of screen
6:44
Speaker 1: that they used to get Steph Curry free off in
6:47
Speaker 1: the side versus the name of a defense that the
6:50
Speaker 1: person running after him might choose to play while he's
6:53
Speaker 1: doing that thing, right, so to avoid that screen. Yeah,
6:57
Speaker 1: and when they say learn a work, they really mean
6:59
Speaker 1: can you teach your machine or get your machine to
7:03
Speaker 1: learn to recognize this play this thing? Because they want
7:08
Speaker 1: to know every time they do this two questions. Every
7:12
Speaker 1: time they do this and we do this, how efficient
7:14
Speaker 1: is it? And or every time they do this and
7:16
Speaker 1: we do this, show me every time that that a
7:19
Speaker 1: video of every time that happened over the last several years.
7:22
Speaker 1: Those are the questions they tend to ask. Okay, and
7:24
Speaker 1: so it had it has a bunch of quirks, you know,
7:26
Speaker 1: working with it, But but it was not easy. But
7:28
Speaker 1: we've done it, and now we've created sort of five
7:31
Speaker 1: hundred of these words that you know, coaches and you
7:34
Speaker 1: know players use on a daily basis. And so I
7:39
Speaker 1: can think of two parts of this that would have
7:41
Speaker 1: been sort of hard problems to solve, one being the
7:45
Speaker 1: kind of nathy computer part, like building the machine, and
7:49
Speaker 1: the other part being the getting people to believe you
7:52
Speaker 1: and use your insu So tell me a hard thing?
7:59
Speaker 1: Well from each right, So what's a hard thing from
8:01
Speaker 1: the from the MATTHI computer part? I think the mat
8:05
Speaker 1: the mat side is just that you don't have a
8:07
Speaker 1: lot of data. So it's for example, a lot of
8:08
Speaker 1: people who do machine learning will say like, oh, give
8:11
Speaker 1: me millions of examples of a thing and I will
8:13
Speaker 1: learn it. Where well, there aren't millions of pick and
8:16
Speaker 1: rolls that you're going to get this sort of how
8:17
Speaker 1: do you solve that? Right? Small sample size? Classic problem,
8:20
Speaker 1: small sample size, So you know, how do you solve it? Well,
8:24
Speaker 1: there's some mathematical chicanery that we invented. The answer to
8:27
Speaker 1: be very clever, like we're really good at bath. Is
8:30
Speaker 1: that the unsatisig outs? Yeah, let me let me ask you.
8:33
Speaker 1: This was there some like things you tried that didn't work.
8:36
Speaker 1: It was just like one equation didn't work and then
8:38
Speaker 1: another did. I mean, yeah, yeah, yeah, I know. I
8:40
Speaker 1: think we've we we you know, I think we we
8:42
Speaker 1: we said over the course of our life, we've used
8:45
Speaker 1: everything in the AI textbooks to try and solve the problem,
8:47
Speaker 1: and they've evolved over years to try and get them
8:49
Speaker 1: get to be better. Um. But you know, I think
8:52
Speaker 1: that that's part of the adventure of sort of trying
8:55
Speaker 1: to figure that stuff out. It wasn't like we used
8:57
Speaker 1: we used method X and the answer popped out, and
9:00
Speaker 1: so we've been constantly evolving the ability to do But
9:02
Speaker 1: I think I think part of the I mean, whatever
9:05
Speaker 1: the secret is, you have to leverage the structure of
9:07
Speaker 1: the problem. You have to leverage the fact that you
9:08
Speaker 1: know something about the fact that it's a bunch of
9:10
Speaker 1: people playing a particular sport. When you don't have enough
9:13
Speaker 1: when you don't have enough data, you have to leverage
9:15
Speaker 1: structure in some way. Yeah, so you're you're able to
9:19
Speaker 1: because it's such a constrained environment, because you know the rules,
9:22
Speaker 1: because you know the things you wanted to find. You
9:25
Speaker 1: can sort of say this is a pick and roll
9:27
Speaker 1: this is a pick and roll. This is a pick
9:28
Speaker 1: and roll. This is not a pick and roll. This
9:30
Speaker 1: is not a pick and roll that kind of thing. Yeah, good,
9:33
Speaker 1: So okay, So that's the math, sort of technical side.
9:36
Speaker 1: Then there's the getting people to believe you and you know,
9:39
Speaker 1: buy what you're selling ultimately, both metaphorically and literally. Tell
9:43
Speaker 1: me about that side, like, were there any interesting I
9:47
Speaker 1: don't know, people who didn't want to buy it, you know,
9:50
Speaker 1: little stories from that side. Almost every coach that we've
9:53
Speaker 1: talked to couldn't believe that we did what we believed.
9:55
Speaker 1: So the story is generally involved just sitting there in
10:01
Speaker 1: front of these coaches. You go up a couple of levels.
10:03
Speaker 1: So I think that you know, this happened with two coaches.
10:07
Speaker 1: It's a very similar story. They were very well known coaches,
10:11
Speaker 1: you know, long resumes, have been around the NBA for
10:14
Speaker 1: a long time, won lots of championships. You know, it
10:17
Speaker 1: took several meetings to even get an audience with them.
10:20
Speaker 1: You would go through sort of layer one, than layer two,
10:22
Speaker 1: then layer three, and then you would get to them.
10:24
Speaker 1: It's like knitting the Pope or something exactly, and then
10:27
Speaker 1: you would sort of get to them and there they were.
10:29
Speaker 1: They would just basically say it like, you have no
10:32
Speaker 1: idea what I intend for the players to do. There's
10:34
Speaker 1: no way this machine can understand what what you need.
10:37
Speaker 1: And we would say try us, and we would say, okay,
10:41
Speaker 1: show me all the times that the play started with
10:43
Speaker 1: the pick and roll and there was three passes and
10:45
Speaker 1: I took a shot in the corners like okay, here
10:47
Speaker 1: it is like, show me all the times there was
10:48
Speaker 1: a screen. I mean like you have a laptop and
10:51
Speaker 1: you have your software running or something. Yep, that's right,
10:54
Speaker 1: and they would we would have tables and charts and
10:56
Speaker 1: they would we would project onto a screen and they
10:58
Speaker 1: would basically just give us the quiz, the sort of
11:01
Speaker 1: the the They would put us through the ringer for
11:03
Speaker 1: for like two hours, so it's like a it's basically
11:06
Speaker 1: a impromptu grilling from the coach and make your machine
11:10
Speaker 1: dance for me and show me the correct answers. And
11:13
Speaker 1: any time you come up with an answer that I
11:14
Speaker 1: don't agree with, you know, I wouldn't think you're an idiot.
11:18
Speaker 1: But we held up. In fact, sometimes these coaches would say, wait,
11:20
Speaker 1: show me this list, and if these two guys aren't
11:22
Speaker 1: number one, and number two, I think your thing is wrong,
11:25
Speaker 1: and then we would do it, and like thank goodness,
11:27
Speaker 1: the right players would end up number one and number
11:29
Speaker 1: two and and then normally after sort of two hours
11:32
Speaker 1: of grilling, they would be pretty good and they would
11:35
Speaker 1: just leave the room, and then we knew we would
11:37
Speaker 1: have a contract. And is the output like the thing
11:45
Speaker 1: that actually you see on the screen when you're running
11:48
Speaker 1: your software. Is it lists? Is it little videos where
11:51
Speaker 1: all the players are dots? Is it both of those things?
11:53
Speaker 1: It's everything. So we have sort of ranking stables that
11:56
Speaker 1: can answer who's the best at X, Y and Z.
11:58
Speaker 1: We have sort of a variety of visualizations that show
12:02
Speaker 1: breakdowns of various actions. You can always click on anything
12:05
Speaker 1: and show all the video of every moment you know
12:08
Speaker 1: anything that you asked it. So we've built lots of
12:11
Speaker 1: visualizations and data formats to make it easy for various
12:14
Speaker 1: people in the organizations to use them. There are instances
12:17
Speaker 1: in the world in different domains where like a data
12:20
Speaker 1: driven approach suggests doing one thing, but that thing is
12:23
Speaker 1: contrary to conventional wisdom, right, But like in football, it
12:26
Speaker 1: seems pretty clear that the data suggests coaches should go
12:30
Speaker 1: for it on fourth down more often than they do,
12:32
Speaker 1: and it seems like as the result, coaches have started
12:35
Speaker 1: going for it more on fourth down. But there is
12:37
Speaker 1: this thing where if a coach goes for it on
12:38
Speaker 1: fourth down and doesn't make it, the team doesn't get
12:41
Speaker 1: the first down, then the coach gets pillaried. Right. So
12:45
Speaker 1: even though the data says it makes sense to go
12:48
Speaker 1: for it in a kind of a personal incentives way,
12:51
Speaker 1: it might not make sense right in a social way,
12:54
Speaker 1: which is real and important, it might not make sense.
12:57
Speaker 1: Is there a basketball version of that? Is there a
12:59
Speaker 1: version of that you have observed where your data suggests
13:02
Speaker 1: that coaches should be doing something, but they're reluctant to
13:04
Speaker 1: do it because it's contrary to sort of conventional wisdom. No,
13:08
Speaker 1: I think there's a lot more nuance and what we do.
13:10
Speaker 1: I think there's the public basically is aware that they say, oh,
13:14
Speaker 1: people should take more threes. But I think the reality
13:17
Speaker 1: is much more complex than that, because I think it's
13:19
Speaker 1: not about just simply take a three. What you want
13:22
Speaker 1: to do is get the best possible shot for yourself.
13:24
Speaker 1: Sometimes that's a three, sometimes that's something else. Different kinds
13:27
Speaker 1: of threes are not the same, and your talent is
13:29
Speaker 1: not the same. In fact, if you look at all
13:30
Speaker 1: the you know that you know, I'm just gonna sort
13:33
Speaker 1: of Lebron James and Kevin Durant and Steph Curry and
13:35
Speaker 1: Janis and Yo Kitchen, all these players, and you know
13:38
Speaker 1: they are very different players, and teams are built around
13:41
Speaker 1: these players, and what you want to do is optimize
13:44
Speaker 1: the team built around these very unique players. And so
13:46
Speaker 1: a lot of it is in the details of how
13:48
Speaker 1: you structure your offense and your defense to sort of
13:51
Speaker 1: use them to get your team the best possible shot.
13:54
Speaker 1: So a lot of it is that these coaches do
13:55
Speaker 1: a lot of work on basically on the micro level
13:58
Speaker 1: to sort of create these plays. That it's not just
14:00
Speaker 1: sort of walk up the court and take a three.
14:03
Speaker 1: They do a lot of work to try and put
14:05
Speaker 1: their position their players in positions to have a lot
14:08
Speaker 1: of good options to get the best possible shot. So
14:10
Speaker 1: I think that it's it's really a lot more nuanced
14:13
Speaker 1: in the actual execution of it. Fans would be really
14:21
Speaker 1: surprised at the level of sophistication entirely across the league
14:27
Speaker 1: and many leagues in sports, especially the top ones. There's
14:29
Speaker 1: there's a degree of sophistication and how they use data
14:32
Speaker 1: and video that I think would surprise almost everyone. You
14:37
Speaker 1: think that the sort of play calling side of coaching
14:41
Speaker 1: will become more and more delegated to the machine. I mean,
14:43
Speaker 1: I could see coaches as essentially psychologists, being persistently useful.
14:49
Speaker 1: But do you see a time when the sort of
14:51
Speaker 1: core strategic decision making will just be be done better
14:55
Speaker 1: by a machine than by coach. I don't. I don't
14:58
Speaker 1: think so. Our thesis has always been we build iron
15:01
Speaker 1: Man suits and and different people will want iron you know,
15:05
Speaker 1: different iron Man suits, and you will want because you
15:07
Speaker 1: want you need somebody in the middle of the iron
15:09
Speaker 1: Man suit directing it, using everything they know about the world.
15:12
Speaker 1: But you want to be a lot more powerful. And
15:14
Speaker 1: I think that that's the model we've always used, is
15:17
Speaker 1: we build Iron Man suits for everybody, and then different
15:19
Speaker 1: coaches and different assistant coaches are going to use them,
15:21
Speaker 1: you know, to be a lot more powerful. That's what
15:23
Speaker 1: we want to do. We want to build Iron Man
15:25
Speaker 1: suits for everybody. Does basketball look different? Is it played
15:30
Speaker 1: differently because of your work? I certainly know that there
15:35
Speaker 1: have been examples where you know, in a couple of
15:37
Speaker 1: NBA Finals strategies were changed because of the data that
15:41
Speaker 1: we provided. You know, close ones are where one team
15:45
Speaker 1: basically discovered a pick and roll defense strategy that was
15:48
Speaker 1: effective and employed that and it to you know, very
15:51
Speaker 1: good effect. Another one it was one of these sort
15:53
Speaker 1: of off ball screen defense strategies where they found the
15:57
Speaker 1: data show that there were particular strategies because that would
16:00
Speaker 1: shave off a couple of points of efficiency from the
16:02
Speaker 1: other team, and they employed that very heavily and they
16:04
Speaker 1: won a very close series. So I think, can you
16:07
Speaker 1: say what teams it was? I should Okay, I get
16:10
Speaker 1: people looking at they could probably figure it out. The
16:14
Speaker 1: point that I want to realize, like there are a
16:15
Speaker 1: lot of people who come out and think like, oh,
16:16
Speaker 1: these these teams are sort of the geniuses and these
16:19
Speaker 1: teams are the Luddites, and that's not the case. Like
16:22
Speaker 1: almost every team is. Like people would be surprised that
16:25
Speaker 1: at the minimum, that's fine. I mean, everybody in the
16:27
Speaker 1: NBA is every player in the NBA is good at
16:30
Speaker 1: playing basketball, but some are better than others. And so similarly,
16:33
Speaker 1: you might imagine that every sort of quant team in
16:36
Speaker 1: the NBA is good at at being you know, doing
16:39
Speaker 1: quantitative analysis, but some are probably better than others. So
16:42
Speaker 1: I would say, you're right, there are some who are
16:43
Speaker 1: better than others, but the minimum level is a lot
16:45
Speaker 1: higher than when everyone would sure, yeah, yeah, And can
16:48
Speaker 1: you say who's better than others? I cannot. I I
16:53
Speaker 1: this is our lifeblood of being the reason that all
16:56
Speaker 1: these coaches trust us and is that we don't. We
16:59
Speaker 1: don't leak. So I'm sorry. It would be fun one day,
17:02
Speaker 1: One day, I'll call all the stories. It's not okay,
17:04
Speaker 1: I'm sorry. Regiev is not going to name names today Alas,
17:10
Speaker 1: but he is going to talk about a big, interesting
17:13
Speaker 1: problem that second spectrum is close to solving but has
17:16
Speaker 1: not quite nailed yet. That's in a minute. Now back
17:27
Speaker 1: to the show. I want to talk about things you
17:30
Speaker 1: haven't figured out yet, like what are the next problems
17:33
Speaker 1: you're trying to solve, Like what is the frontier? So
17:36
Speaker 1: I think the frontier where we're trying to basically basically
17:38
Speaker 1: get full body pose of the human, not just a
17:40
Speaker 1: little dot. We're trying to get their entire skeleton. So
17:43
Speaker 1: the basic sort of product you have, the basic thing
17:46
Speaker 1: you do turns each player into a dot, just like
17:48
Speaker 1: in the classic like x's and o's diagram or whatever.
17:51
Speaker 1: And so you can watch little dots moving around and
17:52
Speaker 1: you can see how far the dots are from the other.
17:55
Speaker 1: But there's a lot going on that you don't see
17:57
Speaker 1: in the dot, right you might want to know, like
17:59
Speaker 1: what are the things that are important that you don't
18:01
Speaker 1: see in the dot? That's exactly right. So the challenge
18:04
Speaker 1: we're working out right now is turning the dot into
18:06
Speaker 1: a human skeleton and then having that skeleton and generate
18:10
Speaker 1: data in you know, one hundred to two hundred milliseconds.
18:12
Speaker 1: So that's the challenge that we're working on right now,
18:14
Speaker 1: and we've actually made a lot of progress on that,
18:16
Speaker 1: and that's right now. And so what's an example of
18:19
Speaker 1: why that would be useful? What are some of the
18:21
Speaker 1: ways you could learn from seeing each player as a
18:24
Speaker 1: whole body rather than as a dot. So I think
18:28
Speaker 1: there are a lot of things you can do. There's
18:29
Speaker 1: one is, obviously, you know, with health and fitness, where
18:32
Speaker 1: you can figure out did they land on one foot,
18:34
Speaker 1: did they land on two feet? You know, did they
18:36
Speaker 1: you know they take off from this port of that foot.
18:38
Speaker 1: So there's a lot of health and fitness stuff that
18:40
Speaker 1: you can figure out health and fitness, meaning learning to
18:44
Speaker 1: reduce the risk of injury, learning that certain kinds of
18:47
Speaker 1: moves or subtle distinctions in moves make a player more
18:52
Speaker 1: or less likely to be injured. That's right, And so
18:54
Speaker 1: that's one class of things we can do. Another class
18:57
Speaker 1: is sort of just understanding moves better. So if a
18:59
Speaker 1: player made, you know, dribble moves between their legs and
19:03
Speaker 1: crossovers and step backs and did all all kinds of
19:06
Speaker 1: things with their hands and feet to get open, now
19:08
Speaker 1: we can categorize those things. There's that, there's health, there's strategy,
19:12
Speaker 1: there's media, there's officiating. There's you can help with officiating
19:15
Speaker 1: if you start knowing where people are. So there's a
19:17
Speaker 1: lot of things that you can do once you understand
19:19
Speaker 1: more of the human skeleton. And let me ask you this,
19:22
Speaker 1: why is that hard? It sounds hard, but like tell
19:25
Speaker 1: me about that being hard. So basically, you have a
19:28
Speaker 1: bunch of cameras in a stadium and it sees these
19:30
Speaker 1: sort of blotches of that you tell them it's a
19:34
Speaker 1: human and and you're to say like, oh, that's a knee,
19:36
Speaker 1: and that's an ankle, and that's a that's a waste,
19:39
Speaker 1: and so it's just sort of you're you're teaching a
19:41
Speaker 1: machine to see a human from scratch, and it's not
19:44
Speaker 1: that it's it's it's you know, there's been a lot
19:46
Speaker 1: of progress in computer vision over the years that as
19:48
Speaker 1: making this making this problem, you know, easier and easier.
19:51
Speaker 1: A lot of it is also just doing it extremely
19:53
Speaker 1: quickly like you want to. You want this machine to
19:55
Speaker 1: be able to do it and on the order one
19:56
Speaker 1: hundred two hundred milliseconds, to be able to enable a
19:58
Speaker 1: lot of things to happen, and happen in real time
20:00
Speaker 1: in sports one hundreds or two hundred milliseconds, but you
20:03
Speaker 1: might want it to be only delayed from reality by
20:06
Speaker 1: a tenth of a second. Let me ask a dumb question,
20:10
Speaker 1: why can't you just let it take its time to
20:12
Speaker 1: figure it out. You're not trying to do it in
20:14
Speaker 1: the middle of the game, right, or are you trying
20:15
Speaker 1: to do it in the middle. So, because I think
20:17
Speaker 1: sports is one of these interesting this is part of
20:19
Speaker 1: the reason why sports machines. Understanding of sports is so hard.
20:22
Speaker 1: In sports, it's not like you want to analyze a
20:24
Speaker 1: video and then know what happened a long time later
20:27
Speaker 1: that might be the case for coaches, but for a
20:28
Speaker 1: lot of media applications or refereeing applications, you want to
20:31
Speaker 1: be able to do it basically instantaneously. Yeah, right, And
20:35
Speaker 1: so it's almost like the self driving car, like you
20:37
Speaker 1: need it to be able to do its thing very
20:39
Speaker 1: very quickly to be able to react to it. You know,
20:42
Speaker 1: is self driving car research helpful to you? Are you
20:44
Speaker 1: borrowing from that? Well? Yes, because I think that's generally
20:47
Speaker 1: pushing the field of computer computer vision forward because it's
20:51
Speaker 1: basically the same problem. You need to be identify people
20:54
Speaker 1: and what they're doing very very quickly, So one is
20:56
Speaker 1: to sort of avoid running into them, the other one
20:58
Speaker 1: is sort of figuring what's going on on a sporting event.
21:01
Speaker 1: And so I think the general trend of computer vision
21:05
Speaker 1: research moving to solve these problems quickly is helpful. Have
21:10
Speaker 1: you figured out anything yet about injury, Like you have
21:13
Speaker 1: any useful injury insights? I mean I've always been, you know,
21:16
Speaker 1: to be honest, wary of injury because it's such a
21:18
Speaker 1: it's it's a hard thing to do. But I think
21:20
Speaker 1: that well we have why wary of studying it, wary
21:23
Speaker 1: of trying to to be honest that the teams use
21:27
Speaker 1: the data we give them in that way, even though
21:29
Speaker 1: I don't want it to happen, because because they just
21:31
Speaker 1: it's just it's any information is better than no information.
21:34
Speaker 1: And so I've always been, you know, the whole nine years,
21:37
Speaker 1: I've said, we'll give you the information, but I'd be
21:39
Speaker 1: I'd be careful. But you know, it's such a such
21:41
Speaker 1: an important thing about keeping athletes. You know, why are
21:44
Speaker 1: you so wary of trying to figure out injury risk?
21:47
Speaker 1: I just I don't know that at this point. I
21:50
Speaker 1: think that I've seen stuff that is is uh, super
21:54
Speaker 1: predictive that the scientists that I would sort of laid
21:56
Speaker 1: down on. I mean, I know there are a bunch
21:57
Speaker 1: of other companies who have so because it's such a
22:00
Speaker 1: hard problem, because you don't feel like you can you
22:02
Speaker 1: can reliably predict it. Yet, Yeah, what is it about
22:05
Speaker 1: injury risk that makes it so much harder to understand
22:09
Speaker 1: than sort of things you know that are more kind
22:11
Speaker 1: of within the game itself. I mean, I just think
22:13
Speaker 1: that you know, it's it's it's almost a sample sized thing.
22:16
Speaker 1: You know, people get injured in all sorts of you know,
22:20
Speaker 1: unique ways, you know, not super often, right, and so
22:23
Speaker 1: you know, I said, there aren't that many pick and rolls,
22:25
Speaker 1: but they're way more pick and rolls than there are
22:26
Speaker 1: you know, yeah, whatever ACL tears or whatever these and
22:30
Speaker 1: presumably injury is more complex. Yeah, it's a more complex problem,
22:33
Speaker 1: and it happens way less often, that's right, which makes
22:36
Speaker 1: it harder on both sides. That's right. I want to
22:38
Speaker 1: talk a little bit about the applications or potential applications
22:42
Speaker 1: of your work beyond basketball, the extensions of your work
22:47
Speaker 1: beyond beyond sports. Um. I mean sports and games more
22:52
Speaker 1: generally seem interesting as a sort of testing ground. Obviously
22:58
Speaker 1: they're useful in and of themselves, but also as a
23:00
Speaker 1: testing ground for I guess for AI in particular. Right,
23:04
Speaker 1: I mean if you think of of chests famously and
23:07
Speaker 1: then go, you know, you had deep mind figure out
23:09
Speaker 1: chess and then go and then they solve the protein
23:11
Speaker 1: folding problem, this profound problem in in biochemistry basically, right, Um,
23:18
Speaker 1: tell me about that. I mean, well, why are games
23:25
Speaker 1: and sports a good place to start? Yeah? So I
23:29
Speaker 1: think that, you know, I think one of the things
23:30
Speaker 1: that we're doing in sports is the general problem of
23:33
Speaker 1: human activity recognition. Right, So that's really what we're doing.
23:36
Speaker 1: But people do a bunch of stuff in a space,
23:37
Speaker 1: and we want to figure out what they're doing. And
23:39
Speaker 1: you know, we didn't actually put you know, sports in
23:41
Speaker 1: the name of the company because we didn't we thought
23:43
Speaker 1: we might go beyond sports, but it did not. There
23:45
Speaker 1: were so much stuff to do in sports we sort
23:47
Speaker 1: have stayed here. Sports is interesting because it's just it
23:50
Speaker 1: has the ability to create a lot of data capture
23:54
Speaker 1: and the activities are are sort of bounded and well known,
23:58
Speaker 1: so you can just have this, you know, this intense
24:00
Speaker 1: capture of this rectangle, various rectangles based on sports, and
24:05
Speaker 1: then it's very clear what some of the activity recognition
24:08
Speaker 1: is and so it's a it's a great place to
24:10
Speaker 1: sort of start with human activity recognition. So what is
24:14
Speaker 1: what is the like obvious adjacent thing in the world
24:19
Speaker 1: where what you do might be useful. I think that
24:22
Speaker 1: if you look around the world, this can be applied everywhere.
24:26
Speaker 1: So for example, you know, in your house, right if
24:29
Speaker 1: you have cameras in your house, and you'll be able
24:31
Speaker 1: to figure it out like oh, you know, someone has
24:32
Speaker 1: fell down or some you know, the kids are fighting
24:35
Speaker 1: right my house. It feels a little creepy to me,
24:38
Speaker 1: like I don't particularly want it in my house. I
24:40
Speaker 1: will say that's fine, Yeah I'm not I'm not necessarily
24:43
Speaker 1: against it, but like it doesn't feel creepy at all
24:45
Speaker 1: in the NBA to know that. So a lot of
24:50
Speaker 1: the other things are sort of you know, I've seen
24:52
Speaker 1: there are other companies that basically watch how people move
24:55
Speaker 1: around stores so you can say, oh, this is where
24:57
Speaker 1: we should put food, or this is where people are congregating,
25:00
Speaker 1: or this is where we get blockages, or this is
25:03
Speaker 1: the travel patterns inside a store. So I know a
25:05
Speaker 1: bunch of companies doing that. So I know that's sort
25:08
Speaker 1: of security and stores. There's a bunch of companies out there.
25:12
Speaker 1: You know. We were approached by people who are like
25:13
Speaker 1: to go to concert venues and say, okay, can you
25:15
Speaker 1: can you put a bunch of cameras and figure out
25:17
Speaker 1: how people move in and out of concerts and figure
25:20
Speaker 1: out where the bottleneck exactly, how does the foot traffic flow.
25:23
Speaker 1: So we would have approached by lots of different industries
25:25
Speaker 1: to say, can you apply what you're doing to us?
25:28
Speaker 1: And what have you said? When those people have approached you,
25:30
Speaker 1: I always like I would like to get to it.
25:32
Speaker 1: But I think the sports has kept us so busy
25:34
Speaker 1: over the years that we've just had plenty of work
25:37
Speaker 1: to do in sports. In a minute, the lightning round
25:41
Speaker 1: with lots of questions about basketball and a little bit
25:44
Speaker 1: about soccer. That's the end of the ads. Now we're
25:52
Speaker 1: going back to the show. Um, still a lightning round,
25:58
Speaker 1: Lightning Round. Who is your favorite NBA player of all time?
26:02
Speaker 1: Larry Bird? What does the data say about Larry Bird
26:05
Speaker 1: as a player? He was very good? That lines up.
26:11
Speaker 1: So it seems like still, at least to some extent,
26:14
Speaker 1: there is there is this cultural divide in the NBA
26:18
Speaker 1: and in other sports between the data people and the
26:20
Speaker 1: sort of old school sports people. And I'm curious, what
26:24
Speaker 1: do the data people not get about the sort of
26:27
Speaker 1: traditional sports people. I think that what happened was, you know,
26:32
Speaker 1: the traditional sport sports people spoken words and the data
26:36
Speaker 1: people spoken numbers, and I think that that's what needed
26:40
Speaker 1: to bridge, Like basically both people had to speak words
26:42
Speaker 1: in numbers, and I think that's happened so that you
26:44
Speaker 1: think that divide is done now it no longer exists. Yeah,
26:48
Speaker 1: Is it still fun for you to watch basketball? Or
26:49
Speaker 1: does it just feel like work. Oh no, it's fun.
26:53
Speaker 1: It's fun. According to the data, who is the greatest
26:55
Speaker 1: basketball player of all time? We're unable to do that
26:59
Speaker 1: because the data capture only goes back five years. Who's
27:02
Speaker 1: the greatest player of the last five years? The last
27:04
Speaker 1: five years? Steph Carry? Who's the greatest soccer player the
27:09
Speaker 1: last five years? MESSI? Can you compare Steph Curry and
27:12
Speaker 1: Messy in some quantitative way? Yeah? I think that all
27:16
Speaker 1: the great players have some number where they perform something
27:20
Speaker 1: far above expectation. I mean, can you is it a
27:25
Speaker 1: dumb question to say who's better Steph Curry or MESSI?
27:28
Speaker 1: It's not, but it's not a question that coaches tend
27:31
Speaker 1: to worry about. Well. Sure, but but if it's not
27:34
Speaker 1: a dumb question, I'm going to ask you who's better
27:36
Speaker 1: Steph Curry or Messy? The reason is that we haven't
27:39
Speaker 1: built it. So it's almost every question requires building a
27:41
Speaker 1: set of tools to answer them. We could, but as
27:44
Speaker 1: no one has asked to build those tools do. I
27:45
Speaker 1: think what you would want to do is say you
27:47
Speaker 1: could answer that question, but I'd have to pay you
27:49
Speaker 1: to do it. Basically, because I think, like I think,
27:51
Speaker 1: there is a way to say it because like how
27:53
Speaker 1: much of it? Basically, these questions you're asking is how
27:55
Speaker 1: much was an outlier? Was this particular person compared to everyone? Yeah?
28:00
Speaker 1: How much value did he add? Would be a clumsy
28:02
Speaker 1: way to say it. Yeah, yeah, And I think that
28:04
Speaker 1: there are definitely ways to answer that. That's sort of
28:06
Speaker 1: not that is not where we have spent our time.
28:09
Speaker 1: But I don't think it's unanswerable. I think that there
28:10
Speaker 1: are interesting ways you can go about answering that question.
28:13
Speaker 1: This one goes to sort of data versus kind of
28:15
Speaker 1: public acclaim among NBA players, Like based on the data
28:21
Speaker 1: versus what people think in general. Who's the most underrated
28:26
Speaker 1: player right now? Say Chris Paul. I mean he's not
28:31
Speaker 1: really underrated, but he's not quite at the pantheon. You
28:33
Speaker 1: can be highly rated but still underrated. Underrated does not
28:36
Speaker 1: mean low rated. It means not rated high enough. That's right.
28:40
Speaker 1: Who do you think the most overrated player? Oh, that's
28:43
Speaker 1: a good question. It's it's it's tough. I don't know.
28:46
Speaker 1: I'm saying no. One just jumps to mind because I think, like, well, also,
28:49
Speaker 1: you get it. You could get in trouble, right, you're
28:51
Speaker 1: gonna you're gonna call out one are your clients stars? Well,
28:54
Speaker 1: there's there's two questions when it's like, hey, I thought
28:55
Speaker 1: of someone, but I don't know who to say. I
28:57
Speaker 1: don't want to say it, but right now I can't
28:58
Speaker 1: actually think it because because I I it's a lot
29:01
Speaker 1: of people. People are much better at rating nowadays. I
29:05
Speaker 1: think that's that's a lot of what has changed over
29:07
Speaker 1: the years is that because there's so many more numbers
29:09
Speaker 1: like the error bar, and ratings have gotten a lot narrower.
29:14
Speaker 1: So the people have gotten more, players have gotten more
29:16
Speaker 1: appropriately rated because fans are savvier about exactly. So what
29:20
Speaker 1: has happened in the place is big centers are not
29:23
Speaker 1: as valuable in basketball. But you actually see that, like
29:27
Speaker 1: they're not rated as highly, you know, and so a
29:28
Speaker 1: lot of the a lot of big centers who would
29:30
Speaker 1: have had you know, massive contracts ten plus years ago
29:33
Speaker 1: aren't getting those now. But that's because you know, the
29:35
Speaker 1: ratings have adapted to value them less. The data shows
29:38
Speaker 1: that big centers aren't as valuable as people thought. That's right.
29:41
Speaker 1: You think you're going to work at second spectrum for forever,
29:44
Speaker 1: for as long as you're working. It's our baby, it's
29:46
Speaker 1: been our the baby of many, many people. I mean,
29:48
Speaker 1: babies grow up. I think that there are problems I
29:52
Speaker 1: want to solve, and sure I would love to be
29:54
Speaker 1: able to solve all those problems, and then once they
29:55
Speaker 1: I'll say what will I do next? But I think
29:58
Speaker 1: there's certainly plenty of problems to be solved. Rejieve mehs
30:04
Speaker 1: Run is the co founder and president of Second Spectrum.
30:08
Speaker 1: Today's show was produced by Edith Russolo, engineered by Amanda
30:12
Speaker 1: kay Wong, and edited by Robert Smith. I'm Jacob Goldstein,
30:16
Speaker 1: and we'll be back next week with another episode of
30:18
Speaker 1: What's Your Problem