Latest / Unicorn Football Talk / Defensive Metrics (NFL)
Transcript
- Emily: Hi Hillary, we're back for another deep dive episode. Hilary: Hey Emily, good to see you. Hear you again What what's on tap? Emily: So one thing we've started doing is I'm gonna take a topic. Hillary's gonna take a topic. So this is this is my week. And I started with one topic, but I ended up with a different one. So I started with defense wins championships. Is that true? It's a saying that people say all the time, right? you have to have heard it. Hilary: Ye of of course. Emily: that's what I started with. And I found out it was an extraordinarily complex topic that might require a PhD thesis and deep analytics skills and knowledge. And I think that would be fun, but I don't have the time for that. So I pivoted because while I was looking into that. I learned a lot about different statistics that they're using nowadays to measure the quality of a defense. but first, I want to do a shout out. There is a really great YouTube video that I will link in the show notes on this topic about whether defenses win championships. And there's this guy, and I don't know if I'm pronouncing his name, Michael McKelvy. he did a deep dive on this, and he basically said, no, defenses don't win championships, which Hilary: Really? Emily: which surprised me because I think we all feel that to be true. I'm gonna talk a little bit about it when I talk about the stats and how historically teams have matched up with the stats, but that's not the focus for today. before we start, if I gave you the option, Hillary, of having your team, the Seattle Seahawks, have a really, really great offense or a really, really great defense, which one would you pick? Hilary: Do I have to choose one or the other? Emily: Yeah, I mean I know you want both, but like let's just say like Hilary: I yeah. Emily: I don't like where do you lean on you love really great defense or you love really great offense? Hilary: think I'm on the record stating that I'm a defensive gal. I enjoy the defensive side of the game and the the challenge of trying to shut down an a an aggressive or really accurate offense. So I guess I'm gonna pick defense. What the heck? Emily: Same. Me too. I definitely pick defense. I love interceptions, pass breakups, sacks, tackles for loss. And I feel like if you have a really bad defense, it's gonna be a miserable day watching your team get scored on a lot. Just ask Dallas fans from last season about that. Their offense was scoring so much, but their defense was allowing so many points. So you just kind of hope your offense can, squeak out enough. but I would pick top-tier defense. But Hilary: Okay. Emily: I should also say that presumes I don't have the worst quarterback in the league. So, best defense and you know, some of my favorite teams that I've watched have had the best defenses. So yeah, I'm on record as well. We're officially on the record. Hilary: Okay, do special teams count as defense? Emily: No, special teams is special teams. We should do a deep dive on special teams. Hilary: Right, next up. Emily: Next step. all right. if I said a good defense, how do you determine what is a good defense? If you ask the NFL, official NFL site, they tend to look at total yards allowed per game for ranking. they also look at total points allowed per game, sacks, takeaways, those kinds of things. But is that a good metric, do you think? I'm kinda leading you. Hilary: You are kind of leading me. I think it all kind of goes together and it's whatever you a good defense is adaptable and gives you whatever you need on that given game day based on who you're playing against. Emily: Yeah, just raw yards doesn't take into account a lot of other things. like, what about garbage time? when there's a blowout and then your defense is just allowing the offense to kind of march down the field. Does that mean they're a bad defense? No, they had shut them out and now they're just playing a little differently. Or weather or the strength of opponent. so if you just look at raw yards, it is interesting to look at that, but it's not. Good enough. it's been an interesting last 25 years in statistics. A lot of work has been done to create new statistics to help measure the quality of and not just defense, offense as well. But I'm I'm focusing on defense here. so two new stats have evolved. And they're called EPA and D V O A. I think I'd casually heard about them as I'm listening to sports radio, but I knew nothing about them. Do you know anything about those two? Hilary: I've heard the terms, but I couldn't define them if you asked me to. Emily: Yeah, hopefully after this podcast you can. That's my goal. But I will say this is a really dense topic. And I spent a lot of time reading these descriptions over and over and over. So it may take time to seep in. I'm curious whether people that gamble or hardcore fantasy football, if they're aware of these stats, because I think they're used heavily for predicting game outcomes. So maybe some casual fans are aware of them, but because I don't gamble or do fantasy football, I had never heard about these. All right, let's start with DVOA. what is it? Let's see if I can if I can do this, Hillary. It is Hilary: Yeah. Emily: a metric that breaks down every single NFL play and compares a team's success on that play to every other team's success in a play like that. So if you're looking at a very specific situation, like Hilary: Okay, we're Emily: fourth and inches. It's fourth and inches on your own 20 with two minutes left, or fourth and inches on your opponent's 20 with two minutes left. It takes into account various factors like exact amount of distance you need to gain, your field position, the current score, the quarter, time remaining. It values plays like a four yard gain on third and three is highly valuable versus a four yard gain on third and ten. You didn't meet the objective. So if you go back to total yards, four yards is four yards, right? But it's not Hilary: Right. Emily: four yards is not four yards. The situation Hilary: Okay. Emily: matters. so it it'll look at the value of a play and then it'll It'll compare that to other teams and how they did in the exact same situation. so that's what kind of makes it interesting. Hilary: Well so is that across one season or across history? Emily: Across one season. So this is all season Hilary: Okay. Emily: based. Now you can't really compare against other teams in the first few weeks. So that part of the metric kicks in at week five. They have ways of doing analysis the first four weeks. And then starting week five, they factor in the opponent adjustment, if that makes sense. Hilary: Yeah, that makes sense. Emily: So the metric is given as a percentage. If you have a 0% DVOA, it means you are a perfectly average team. You are right in line with everybody else. If you have a DVOA of 10%, then you're 10% better on offense than the average team. so it's actually a an offensive and it's a defensive statistic. So if you're on the positive side, it means your offense is producing above average. And if you're on the negative side, it means your defense is doing a great job at keeping the offense from moving the ball, you want as a defense to have a high negative percentage, and on the offense, you want to be high. Hilary: Okay. Emily: Just take my word for it. Hilary: I'm having visuals of elementary school and the big number line across the front of the classroom and it's like, are you on the plus side of zero? Are you on the minus side of zero? Emily: I mean, it's this is a hard topic to grasp, so I'm gonna just try to keep it as simple as possible. The article I read on DVOA on the site is like twenty pages long. there's more to it than what I'm saying. But another example is if a quarterback specifically has a DVOA of negative twenty percent, they are twenty percent worse than the average quarterback. So when you're on offense you want positives when you're on defense you want negatives. Hilary: Okay, got it. Emily: so where did it come from? It's kind of a cool story. There's somebody named Aaron Schatz, and it's it's spelled S-C-H-A-T-Z. I asked AI how to pronounce it. So hopefully I'm saying it correctly. in 2002, he was just a fan. He's not a data scientist or a mathematician by trade. He's just a fan that was frustrated by the standard NFL rankings That labeled teams with high yardage totals as the best, ignoring all the stuff we talked about. So he started collecting information and he created this new metric on his own. He officially launched his own site called Football Outsiders in 2003. And the name of the metric, I said DVOA, it's Defense Adjusted Value over Average. in the early years, getting the data was an immense obstacle because the NFL didn't provide like public play-by-play information. So he just had a small circle of volunteers that had to manually type in stuff into Microsoft Excel. but now it's automated. Hilary: Wow. Emily: So that's good. And then eventually people picked up on it and Fox Sports began publishing power rankings with D V OA in two thousand five. and then ESPN started working with him starting in two thousand eight. It became an exclusive centerpiece of ESPN. I should point out for years, the DVOA database only went back to nineteen eighty-nine based on the data that they could pull. But over two decades, Schatz and his team painstakingly logged historical play by play data. And they eventually retrofitted this metric all the way back to nineteen fifty, which is pretty impressive. Hilary: That's amazing. Emily: in 2003, Football Outsiders abruptly shut down, And the historical DVOA database was at risk of being lost entirely. But he moved his operation over to FTN Fantasy, which is now the official home of DVOA. So this is a site where you can pay for detailed metrics to help you with betting and fantasy football. when he arrived there, he he continues to evolve his model. So he's on version 8.0 and 8.1. So he's continuously tweaking his algorithm to come up with a better and better way of truly measuring the quality of a an offense or defense. the tweaks he recently made actually did. overhaul the historical rankings, which I'll get to in a moment. Hilary: I'm just sitting here with my head spinning a little bit thinking about the volumes of data that they're collecting across a l season to come up with these numbers. It's astounding. Emily: It's astounding. I don't think it's shocking that this the one that I'm gonna tell you about next was created in the last twenty five years. I mean, this is when compute and data and storage became, more, cheaper, Hilary: User Emily: There's, cloud storage. There's just more and more ability to do stats and number crunching. So, not super surprising that this didn't exist back in 1950. So before I move on to tell you about the second metric, second and final metric, I went to the FTN site. You don't have to pay for all information. And I was able to pull up historically how DVOA ranks the best defenses of all time. So I was interested in that. I've seen articles written about this. I have a couple in my head that I think of as the best of all time. And are they on the list? so I'm just gonna rattle off the top 10, and then I've got a couple of observations, top 10 was interesting of all time based on DVOA. The 1991 Philadelphia Eagles, the 1969 Minnesota Vikings, the 2002 Tampa Bay Bucks, the 1980s 86 Bears, the 1985 Bears, the 2018 Bears, the 1976 Steelers, the 2008 Baltimore Ravens, the 1962 Detroit Lions, and then rounding out the top 10 is the 2008 Pittsburgh Steelers. So I have a few Hilary: So Emily: reactions. I don't know if they're the same reactions as yours. I don't know if you want to go first. Hilary: Well, I'm just really fascinated that by those metrics, the most recent top ten contender is fifteen plus years old. Nothing nothing Emily: Yeah. Hilary: current. Emily: it is interesting, isn't it? for me, there's two things that jumped out. One, I know you got into football a little bit later than me, but you probably have heard everyone talk about the 85 Bears. The 85 Bears is arguably the best of all time, maybe more arguably now that this list came out. And what was interesting to me is that the 86 Bears were ranked higher than the 85 Bears. The 85 Bears won the Super Bowl. but the 86 Bears played a much tougher regular season schedule, and they played better in the Hilary: Hell. Emily: regular season. Hilary: Interesting. And the Bears were on the Emily: So that's why they Hilary: list three of the top ten. Emily: Yeah, I don't remember the Hilary: Did I hear that right? Emily: yeah, the 2018 Chicago Bears, I don't remember them. They did not win a Super Bowl. so the other thing that bopped out at me, the other two teams that people often talk about as being the top of all time were not in the list. The 2000 Ravens and the 2013 Seahawks, were not in the top 10. the other thing I'll call out is originally I'd started this as do defense wins championships. Well, only three Of the statistically best D VOA defenses in the history won. So Tampa Bay in two thousand two, the nineteen eighty five Bears and the two thousand eight Steelers won. Hilary: Okay, I would have expected the Steelers to be on the list more often also because I think of them as a team with kind of that legendary aura about always having a defense. Emily: Yeah, at least they made the list. The Seahawks aren't on the list, which disappointed me. Hilary: Yet. Emily: I can't really speak to the nineteen sixty-nine Vikings or the nineteen sixty-two Hilary: Ha ha ha. Emily: Lions, but super interesting. I mean, defenses were different, offenses were different back then. So I think it's kind of an interesting way to measure. This tries to normalize for that because you're comparing. Hilary: Sure. Shorter season back then too, right? Less games. Emily: Shorter season. And you're really comparing your success that season to other teams' success in that season. So I think it is really a cool way to measure. but is it the best of all time? I don't know. I feel like we'll be debating that forever. before I move on to the next metric, I wanted to call out that the 2025 Seahawks did have the best D VOA last year and they won the Super Bowl. All right. So I know this is a meaty topic I'm going to tell you about one other stat that people are using heavily today. It is called EPA, stands for expected points added. Have you heard of this one? Sounds like. Hilary: This one I've heard of, yes. Emily: Okay. This one's a little different. It's not a percentage. And it's a metric that tries to answer the question how much did each specific play increase or decrease the chances of scoring on the drive? So again, you're breaking down play by play and not summing up total yards. and so it's kind of an interesting way to approach it. And how does it do that? It does that by saying at the beginning of the play. how many expected points would you have based on where you are in the field and what down in distance are you at versus after the play? How did you change it? Did you make it go up or down? So I'll give you an example. Before the ball is snapped, there's a data model. That uses decades of historical NFL play-by-play data to assign a single point value to a current situation. This is called expected points. So like being at your own 20-yard line might be worth 0.5 expected points. All right. Versus, yeah, Hilary: Odds you're gonna score points are not good. Emily: you're they're not good on your 20, but But you would expect to score on the drive. That's what the point five is. It's not a negative. a first down is vastly more valuable than a third down. Having one yard to go is more valuable than having 15 yards to go. So it's a complex formula that I did not study in detail. and then after the play happens, they take what your EP was before and your EP was after. So I thought I could do an example. I think that might help. Hilary: I think that would help. Emily: Okay, You're in the midfield, okay? It's Hilary: Yep. Emily: third down, two yards to go. Before the snap, they would assign a 1.5 expected points on the drive number plus 1.5. then we're gonna have two different outcomes. Let's just say the offense gains three yards, gets a first down. And it's now first and ten in your opponent's territory. You've just bumped it up the expected points to 2.3 for the next play. So now it takes 2.3, it subtracts 1.5, and that play had a net gain of point eight P EPA. Hilary: Got it. Emily: All right, so that's a successful play. You have a positive EPA. You did something that increased the expected points on your drive. Now, what if you didn't convert the first down? So again, you had 1.5 expected points prior to the play, and instead you're stopped short. It's now fourth and one. You're gonna punt and now your expected points drops to negative 0.2, meaning it's more likely that the other team will score at this point. So it would take negative 0.2 and subtract the 1.5 and your EPA is now negative 1.7. So it's not good. You are not looking good. Hilary: Okay, okay. Could you turn the ball over? Emily: You turn the ball over. that was not a successful play. You had a positive going in. You're now turning the ball over. it's more likely that the other team will score. Not good. Hilary: Okay. Okay. And who's tracking this? The teams are tracking all of this across their playbook too, to have values assigned to all of their plays, which are more successful than others? Emily: Well, let's talk about that. Let's talk about where it was created and who's actually keeping track of this. I will say to answer your specific question first, teams do subscribe to information and have this information, but they also have their own stats people but yeah, EPA is a little bit different. DVOA is this stat. Owned by this company, basically. EPA is a little bit different. So it was kind of has actually a really interesting history. the idea started around 1970 by a quarterback named Virgil Carter, and he played for multiple teams, And he was getting a master's degree at Northwestern, my alma mater, and he was frustrated by coaches considering raw yardage, as we've talked about. So he worked with his professor and published a paper on this idea of expected points. And this was just a paper that was published and it was mostly ignored until about nineteen eighty-eight, when there was a book which I would love to read sometime called The Hidden Game of Football. which was written, it's kind of like the the analytics book on football. they dusted that concept off. And then again, somebody named Brian Burke around 2010 started using compute power that we talked about, which is more available, Hilary: Right, more available. Emily: to access data. And he created a website and popularized the modern EPA stat. in 2015, E ESPN hired Burke and started including EPA into their QBR metric, which is its quarterback rating metric. and today the NFL embraces this metric. The league has started to integrate EPA into its next gen stats platform. but yes, this is a much more recent stat. what is interesting about this stat is that the data to calculate it is available on an open source platform. If you have coding skills, you can access this data yourself. Hilary: I think I'll leave that to somebody else. Emily: Yeah. There's no need to. We've got other things to do with our day. they only go back, kinda sorta only go back to nineteen ninety nine, but they've had some success getting some data pre nineteen ninety nine. It's just not all of the data. the I think that they have gone back to do some kind of analysis on pre nineteen ninety nine, but the best data goes back to nineteen ninety nine. before we wrap up, what are the best defenses historically from an EPA perspective? Are they the same as D VOA? Hilary: I'm gonna say no. Emily: They're not. They're not. Which just makes the whole thing more confusing. People are trying to create new stats, so shouldn't they come to the same conclusion? no, they don't. So I'm gonna rattle off the top 10 and then I will comment. the nineteen ninety-one Eagles, the two thousand two Bucks, the twenty thirteen Seahawks, the two thousand Ravens, the two thousand twenty-five Houston Texans. the 2025 Seattle Seahawks, the 2019 New England Patriots, the 2008 Pittsburgh Steelers, the 2015 Denver Broncos, and the last one is the 2021 Buffalo Bills. Hilary: So that shifted everything current was the only overlap team I heard there the Bucks in two thousand two? Emily: There are three, Hilary: Or one of the one of the very few? Three. Emily: there are three overlaps. The nineteen ninety-one Eagles, interestingly, is number one on both. So Hilary: Mm. Emily: I feel like I need to get more familiar with that. They did not actually go to the playoffs because of like a quarterback meltdown or injury. I'm not sure. But they had a nickname called the Gangrene defense. So it might be fun to look into them sometime. So they must be pretty good if they ranked high on both. Hilary: Right. Emily: the the two thousand two bucks and the two thousand eight Steelers, those are the three that are on both. Hilary: Okay. Emily: why is that? I talked a lot with AI to try to understand why they're so different. I'm definitely not an expert, so this will be very surface level. But there's some pretty big differences between the two. DVOA does more to suppress or completely disregard things like garbage time. It factors in components like weather. and EPA does it. it's just a number that's assigned and it's it factors in some things, but weather and location not not a thing. People generally think that DVOA is better for evaluating overall team strength and predicting future success because it contextualizes the environment, opponent quality, garbage time. But sports analysts still use the EPA Neither of the models consider rest edge, which is something you and I talked about on a previous episode. Hilary: We did. And I think that's a big factor, especially in the seventeen game season now and how many teams you play coming off of their bye week. Those kinds of things. Emily: Yeah. I think if I had to take a guess, I would say that maybe the guy that manages the D V O A might want to incorporate that in in the future. maybe I'll email him. Maybe he'll respond. I don't know. Hilary: Ha ha ha. Emily: so I asked AI to give me a quick high level, let's just see what it had to say. DVOA is best for season rankings. The main advantage is that just for opponent's strength. Its biggest flaw is a complex black box formula. EPA per play is best for in-game quality. its main advantage is pure situational context and ignores the biggest flaws, ignores if the opponent is elite or poor. So my understanding is that people use both. And I learned a lot. I'm going to start looking for that stat. Hilary: Yeah, there's a lot more to learn and dig deeper there because it's nuanced when which particular methodology is more effective or accurate for the situation, right? Emily: Yeah, And I had a lot more that I had collected for this episode, but it's so complex I only want to speak about things that I'm a little bit more confident in. but you could probably take a a college course on it in a semester, probably on both stats. Hilary: Okay, but gut feel. Does defense win championships? That was the original question you set out to answer. Have you got an answer? Emily: You can't my answer is not in a vacuum. No. I don't think an elite defense can win without something of an offense. What about you? Hilary: Okay. Ooh, you're putting me on the spot. Emily: Whatever you say isn't wrong because nobody like there is no true consensus. It's just a debate. Hilary: It's just a debate. I'm still gonna say that a really superior defense will get you further than only a superior offense with no defense. Emily: Yeah, case in point, the Cowboys from last season. Really great Hilary: Correct. Emily: offense. Really, yeah. I agree. I that's why we'd rather have the great defense if we come full circle back to the beginning. Hilary: Well, thanks for the deep dive, Emily. I feel like I learned a lot more about DVOA and EPA. Did I just say I'm right? Emily: You did. Hilary: And I will take that into account when I'm doing some stats diving for my fantasy football team. Emily: All right. Well, thanks for listening. Thanks for your attention and great questions, Hilary. We'll see you next time. Hilary: Yep. See you next time on Unicorn Football Talk.