Introduction

Special teams and its influence on field position is the most underrated component of the football game. American football is a game of inches – every yard matters, and the ability to pin opponents deep, fight for extra yards on returns, and finish surefire tackles can make the difference in any given game.

For this project, we will be attempting to model punt return yards. We hope to develop an accurate model that can be used to evaluate punt returners, gunners, and teams as a whole and draw additional insights on the punt play as a whole. The model will be used to predict the punt return yards (PRY) of any given return. The prediction will be referred to as expected punt return yards (xPRY) and we’ll call the difference between the two values punt return yards added (PRYA).

Methodology

The data includes returned punts from 2018 through 2020 excluding penalties. Many different variables were tracked over a total of 1877 plays. The coordinates of each player relative to the returner, their speed, orientation, direction of motion, Euclidean distance from returner, etc.

I also experimented with Voronoi features, such as the area of the returner’s Voronoi region and the x-value of the leftmost vertex of the returner’s Voronoi region (after standardizing play direction). I also calculated the proportion of the returner’s Voronoi region’s perimeter that bordered a defender’s Voronoi region and the area of return unit players’ Voronoi regions that bordered the returner’s region. I tested indicators such as whether the line between a defender and the returner went through a blocker’s Voronoi region.

After feature selection, a parsimonious XGBoost model was trained using only variables representing how far each defender is from the returner, the x-value of the leftmost vertex of the returner’s Voronoi region, the area of the region, and the area of offensive players’ Voronoi regions that bordered the returner’s region.

I trained the model on data from the 2018 and 2019 season so that analysis could be done on the 2020 season.

Returner Evaluation

The most obvious application of xPRY is to evaluate individual punt returners based on their ability to get more yards than expected. We can predict the outcome of each returned punt in 2020 that didn’t have a flag thrown and compare the yardage to the actual output for each returner. The top ten returners in total PRYA are shown below.

First-team All-Pro returner Gunner Olszewski finished with the most punt return yards added in the 2020 NFL season while second-team selection Jakeem Grant came in third. Saints returner Deonte Harris had another strong season after a first-team All-Pro campaign in 2019.

Diontae Spencer had the most valuable return of the season in terms of PRYA, as he was expected to gain just 4.4 yards on an 83-yard punt return touchdown against the Panthers on December 13th, 2020.

Spencer was immediately met by two gunners on a play in which most returners probably would’ve called for a fair catch. Instead, he escaped the initial pressure and found an opening through the left side of the field, gaining 78.6 more yards than expected.

We can also rank the players who performed the worst relative to xPRY.

The average punt return for these players gained less yards than the model predicted. It’s interesting that many of these players continued to be the primary punt returner for their teams despite the poor performances. Perhaps the numbers don’t tell the whole picture. After all, Pharoh Cooper was an All-Pro as both a kick returner and a punt returner in 2017. Has he regressed so much? Here’s his worst punt return this season based on xPRY.

On this October 11th return against the Falcons, Cooper received the punt at the 15-yard line and was then predicted to gain 7.8 yards. Instead, Cooper lost three yards.

It may seem strange that Cooper is being dinged (-10.8 PRYA) for a return where he was met so quickly. It should be mentioned that Cooper’s expected gain on this play of 7.8 yards is below average. The average xPRY for 2020 punt returns is 8.9 yards, so the model wasn’t expecting Cooper to do anything crazy.

A possible reason for the prediction not being lower, though, is the apparently opening on Cooper’s left side – the gunner at the top of the field was not in position to stop Cooper from breaking away to that side. Instead, he hesitates and tries to break to the opposite sideline which may have worked if he was able to get past the first defender, but he was in strong position which forced Cooper to turn back around. By that point, it was too late.

A common theme from negative returns is hesitation or “trying to do too much.” For example, Christian Kirk has multiple plays where he could surge forward for a mediocre but respectable gain, but he tries to run laterally towards the sideline looking for the big play even when the opportunity is not there. These are the habits that NFL coaches may want to weed out as field position is too important to throw away yards.

Gunner Evaluation

The task of evaluating gunners is more complex than it was for returners. When we evaluated individual returners, the main metric of interest was PRYA. Similar to a running back’s rushing yards over expectation, the task is simple – get more yards than expected. In the case of punt coverage, there are arguably two important and separate skills: limiting a returner’s xPRY and then after the catch, limiting their return to less than their xPRY.

If a gunner is consistently forcing low xPRY numbers, they’re probably speeding down the field and limiting the returner’s space. That’s an important part of their role – tackling/slowing down the returner is another.

In the interest of brevity, I’ll simply rank the best gunner duos this season based on cumulative PRYA.

The best duo appears to be J.T. Gray and Justin Hardee for the Saints. Through 12 punts that were returned, the two gunners held the returnman to a low average of just 5.92 expected return yards. They put themselves in position to succeed and then capitalized, actually holding those 12 returners to just a two yard average. Overall, the unit is credited with saving 47 yards. This type of holistic evaluation based on multiple metrics allows us to quantify different types of skills that are all related to a gunner’s duties.

Case Study: xPRY-Driven Evaluation of Washington Football Team Gunners

Our statistical analysis of punt return tracking data can be a beneficial complement to the work of NFL coaches and scouts. In a brief case study of the Washington Football Team’s gunners in the 2020 season, we’ll study the production of their two most frequent gunner combinations: Troy Apke & Danny Johnson and Cam Sims & Danny Johnson.

Each respective gunner duo defended against eight punts that were returned in the 2020 season. The actual return yards allowed are plotted on the y-axis along with the expected return yards allowed based on the model.

It seems that the two gunner pairs gave up roughly the same amount of predicted yards. The difference comes in the end result of the play – not a single return covered by Sims/Johnson gained more yards than expected, while three such punts did for Apke/Johnson (points above the expectation curve).

The observed difference in average allowed PRYA is 4.57, so the Apke/Johnson duo allowed 4.57 more yards over expectation on average than the Sims/Johnson duo. The sample size is quite small, though. We can estimate the probability of observing this difference due to random chance through a two-sided permutation test. The permutation test gives a probability of ~2.38% of observing a difference greater than or equal to 4.57 or less than or equal to -4.57.

Of course, conclusions can’t be drawn on these numbers alone. Many variables are not accounted for, such as the skill of the returner, other teammates on the field, etc. These results could give the Washington Football Team coaching staff a reason to deeper analyze the performance of the two gunner combinations.

For instance, they could go on to study the sixteen plays in question. Which players were actually credited with the tackles? They would find that Apke was not credited with any tackles on the eight returned punts in which he played the gunner role with Danny Johnson. Meanwhile, Sims was credited with three tackles that held the returner to less than the yards they were expected to gain.

Of course, this initial analysis is just one piece of the puzzle. Watching the film of the sixteen returns would reveal that one of the ‘big plays’ given up by the Apke/Johnson duo occurred when Apke dove at the returner to slow them down enough for Deshazor Everett to clean-up the tackle. However, Everett stumbled in space and the returner gained an extra seven yards.

Also, PRYA only considers punts that are actually returned. It’s certainly possible that Apke’s 4.34 40-yard dash speed allows him to get down the field and force fair catches more often than Sims. We can quantify this in a vanilla way by crediting the ‘forced fair catch’ to the player on the kicking team closest to the returner at the time of the fair catch. We find that since 2018, Sims forced a fair catch on six punts out of 32 (0.188) versus Apke’s eight out of 73 (0.110). Furthermore, Sims’ average distance from the returner at the time of the fair catch is 3.8 yards versus 5.8 yards for Apke. While this topic can be looked into further, these are the type of numbers that can serve as a reference point for NFL coaches and scouts.

Team Evaluation

In addition to its value in player evaluation, xPRY can be used to evaluate the punt coverage and punt return ability of NFL teams.

We can calculate the average expected yards given up on punt returns by each team along with their actual yards allowed.

We found that the difference in predicted and actual punt return yards allowed by NFL teams is correlated with their punt DVOA, or Defense-adjusted Value Over Average (from Football Outsiders) where a higher DVOA represents a better punt unit. Teams who give up more yards than they were expected to also tend to have a worse punt DVOA. The opposite is also true – teams who limit returners to less yards than expected have a better punt DVOA on average.

The Chargers had the worst punt DVOA in the league by a wide margin (-37.8, ahead MIN at –13.7) and these numbers partially explain why that is. The Chargers gave up the second-most xPRY in the league on average (10.8) while also allowing the second-most average punt return yards (16.8). Not a good combination.

Approximately 32% of the variation in punt DVOA is explained by a team’s average PRYA allowed. This is particularly noteworthy because xPRY does not include the many punts that are never returned while DVOA obviously does. Thus, an xPRY approach to team evaluation can provide valuable insight into a specific component of a team’s special teams performance.

The same approach can be taken to evaluate a team’s punt returning ability.

We found that a team’s return yards over expectation are once again correlated with DVOA. In this case, average PRYA explained 66% of the variance in punt return DVOA. Once again, xPRY-based analysis is linked with DVOA, the gold standard of team evaluation metrics available to the public.

The post PRYA: Modeling NFL Punt Returns for Player Evaluation appeared first on The Spax.

Time for a hypothetical you’ve surely heard a thousand times. You have the number-one pick in an NFL fantasy draft. You’re certainly going to select a quarterback. But which one? Which quarterback would you want to build your team around? Sometimes the hypothetical is open-ended: every player in the history of the league is available to you at their 22-year-old form. That’s a pretty fun one. Or maybe it’s a closed question: your only choices are 22-year-old Aaron Rodgers and 22-year-old Tom Brady. The person presenting that hypothetical is probably trying to prove a point. A bit less fun.

Let’s meet in the middle. Imagine that right now, during the 2021 NFL offseason, a fantasy draft is announced. The draft pool will consist of every player currently signed to a team. Every player will receive an equal value contract of length four years with the obvious possibility of extending the deal in the future. So, current contracts are irrelevant. But their age isn’t. This ranking is not meant to rank each quarterback based on their performance in 2020. It’s not meant to be a projection of their performance in 2021. It’s simply a ranking of their current value as a player disregarding contracts. Does it mean anything? Not really. But hypotheticals can be fun, so let’s try it.

Quick recap of the rules:

Also, each player’s age is in parentheses. Got it? Let’s begin.

40. Joe Flacco (36.3)

39. Ben Roethlisberger (39.2)

38. Ryan Fitzpatrick (38.4)

37. Taysom Hill (30.7)

36. Taylor Heinicke (28.1)

The bottom of the barrel. Well, an arbitrary selected barrel. I could’ve extended this to fifty players by including the Dwayne Haskinses and Kendall Hintons of the world. This hypothetical could also be framed as including every currently living human being which really wouldn’t shake up this top-40 list very much, but it’d bump up these guys to the 100th percentile in a pool of eight billion people. Depends on the barrel, I guess.

Three quarterbacks in the twilight of their career round up the bottom three in this barrel. Fitzpatrick has actually played some great football recently, but there’s still zero long-term upside and not that much in the short-term. Who knows how much time he has left? That question is also relevant to Roethlisberger and Flacco who have simply not been impressive at all over the past two seasons. Nowhere to go but down.

If Taysom was 24-years-old, I’d actually be somewhat hopeful. He’s incredibly athletic and I honestly think he’s shown flashes in the limited time he’s gotten at quarterback for the Saints. But he’s not 24-years-old. He’s on the wrong side of 30, has very minimal meaningful experience at the position, and a pretty frightening injury history. Putting him at #37 was arguably generous.

The attention Taylor Heinicke got for his playoff performance was completely warranted. I mean, most people never heard of him and he came out and looked good against a very good Buccaneers defense. But that’s basically all he’s shown in his short career. At least he’s actually a quarterback, though.

35. Nick Foles (32.3)

34. Andy Dalton (33.5)

33. Cam Newton (32.0)

32. Mason Rudolph (25.8)

31. Marcus Mariota (28.5)

I hope the Bears are able to properly address the quarterback position in the NFL Draft tomorrow because man, neither Dalton nor Foles are exactly inspiring at this point in their careers. Both of them together is just sad. Sad enough for me to take Cam Newton over them, despite him having gone through more than a career’s worth of injuries clearly impacting his throwing motion and scrambling ability.

Mason Rudolph has gotten a solid amount of playtime in his first two years as a pro whenever Roethlisberger has been injured. And while he hasn’t looked great, he hasn’t exactly looked terrible either. He’ll be a 26-year-old at the start of the season, so hopefully he’ll be able to progress even further.

Mariota’s five year stretch with the Titans was disappointing, to say the least. But he was drafted with the second overall pick in the 2015 NFL Draft for a reason. And he looked pretty good in his one game played in 2020 against the Chargers. Yeah, not a lot to go on here, but I’d confidently take him over everyone below him on this list.

30. Jacoby Brissett (28.4)

29. Drew Lock (24.5)

28. Mitch Trubisky (26.7)

27. Daniel Jones (23.9)

26. Sam Darnold (23.9)

Jacoby Brissett has carved out a solid start to his career for a third round guy. He posted the 20th best ANY/A in the NFL in 2019, his last full season as a starter. Not great, but certainly not the worst.

As an early second rounder, Drew Lock has had higher expectations than Brissett but hasn’t impressed. He posted a 5.31 ANY/A through 13 games in 2020, the seventh-worst mark in the league and hasn’t sparked any sort of reason for optimism in his play. Definitely not enough to put him ahead of similarly underperforming high first round picks like Trubisky, Jones, and Darnold who at least have higher ceilings. I think the latter two are capable of making a leap in their 24-year-old season. Trubisky isn’t quite as young as the others but he has definitely shown more flashes than them. I think the most likely outcome is that one of these guys is able to become a good starter and it’s hard to pick which one that’ll be, especially considering how poor the talent and coaching was around a guy like Darnold.

25. Carson Wentz (28.3)

24. Gardner Minshew (25.0)

23. Matt Ryan (36.0)

22. Teddy Bridgewater (28.5)

21. Tom Brady (43.8)

You know those underperforming young guys in the last group of players? Specifically Sam Darnold, Daniel Jones, and Mitch Trubisky? Well, Wentz was worse than any of them in 2020. Significantly so. His ANY/A was the worst in the NFL, as was his EPA+CPOE composite. It’s fair to say that he was the worst quarterback in the NFL. And he’s older than everyone in that group of players as well. So, why’s he ahead of them? Simple: Wentz was better than them from 2017 to 2019, especially in 2017 when he had an MVP level season. But it’s extremely concerning that he has performed worse every year since then. People blame coaching, the offensive line, his weapons, etc, but Wentz just sucked in 2020. Like, he was awful. I’m not sure how that happens to a young quarterback so quickly after they were an MVP caliber player, but he’s clearly got it in him. Worth a flier.

Minshew has had an amazing career start for a sixth round pick. Over his first two seasons, he’s performed quite well given the expectations. I think his ceiling is lower than the five players he’s ranked above, but hey, ceilings are pretty overrated. How many first round quarterbacks come close to their supposed potential as a player? Not many. It certainly matters, or else Minshew would be even higher, but his draft position and playstyle aren’t enough for me to write him off.

Teddy’s a weird quarterback to rank. He’s the most average player I’ve ever seen. I adamantly believe he’s not a franchise quarterback, nor will he ever be. His ceiling is his floor. But he’s not bad. And his game should age well. At the end of the day, I don’t think he’ll ever really be contributing much to a contender team. But his floor is definitely higher than the guys behind him.

And then there’s Tom Brady. He might’ve been the hardest player to rank in this list. He’s one of the greatest quarterbacks to play the game and his career boasts arguably the most impressive longevity in the history of American pro sports. But he’s going to be 44-years-old during the 2021 regular season. Yeah. Forty-four. Surely he’s gonna fall off that cliff at some point? Right? Realistically, I’ll go on record saying he has a maximum of two years left to play at a high level. I’d take two years of contending over the guys below him who have a great shot at giving you none.

I put Ryan right behind Brady because although he should have more time left, Brady will likely be a good bit better in his two years left than Ryan will in his four. Arbitrary numbers, I know, but I think they’re fair estimates. And possibly even generous to Ryan. Will he be a solid quarterback at 40? History isn’t on his side. Brady’s the exception, not the rule.

20. Tua Tagovailoa (23.2)

19. Jalen Hurts (22.7)

18. Jimmy Garoppolo (29.5)

17. Jared Goff (26.6)

16. Jameis Winston (27.7)

What were considered Tua’s strengths coming out of college? It wasn’t his arm talent. The hope (or expectation, really) was that Tua’s other skills (ability to read a defense, smart decision-making) would be enough for him to be a good quarterback at the next level. It has not. It’s only been his rookie season, but this guy was hailed as the most pro-ready quarterback. That was his appeal. People recognized that his ceiling wasn’t the highest but he was supposed to come into the league better than he did. He was not good. People blame the talent, but Fitzpatrick was far better with the same personnel. Of course, Tagovailoa’s only 23-year-old and quarterbacks typically take a leap in their second season. I won’t entirely write him off yet. But I’m not optimistic.

Garoppolo and Goff were both “decent but not great” quarterbacks who helped¹ to lead their team to Super Bowl appearances. They are both certainly good enough to be starting quarterbacks and they both have room for improvement. I’d say Garoppolo is clearly better right now but Goff may have more potential. Granted, a lot of Goff’s shortcomings stem from his struggle to read defenses and make good decisions with the ball, both skills that are not easy to develop.

Winston’s got all of the talent in the world. He showed that with Florida State and the Buccaneers. Then again, he also showed a lot of bad. No team in the modern era can endure their quarterback throwing thirty picks in a season. I don’t care how many passing yards or touchdowns they have. However, I think Winston has the potential to be far more efficient. The Buccaneer’s offense was not at all friendly to a quarterback with suspect decision-making like Winston. In fact, Arians’ scheme was probably one of the worst Winston could’ve played under. I’d bet on a career resurgence from him.

15. Matthew Stafford (33.2)

14. Baker Mayfield (26.1)

13. Aaron Rodgers (37.4)

12. Ryan Tannehill (32.8)

11. Kirk Cousins (32.7)

I find this to be an interesting group of players. First, we’ve got the three solid players in their early thirties. Stafford, Tannehill, and Cousins have been playing good football over the past two years. In fact, the latter two have played great football in 2019 and 2020. People credit a lot of Tannehill’s success to Derrick Henry even though he’s equally efficient without him on the field. He’s just a good quarterback. His ANY/A led the league in 2019 and came in at fourth in 2020. These numbers don’t tell the story — he’s not a top five guy. But he’s genuinely really good, and he should have plenty of time left in his career.

Statistically, Kirk’s a step behind. His ANY/A this season came in at tenth and it was the seventh best mark in the league in 2019. But I’d say that he is easily in a tougher spot. The Vikings’ pass protection is nonexistent. They’ve been near the bottom of the league in pass blocking grade in back-to-back seasons. We all saw just how much pass protection matters in the Super Bowl. The league’s best quarterback was reduced to a non threat. What Kirk has been doing with that lack of offensive line talent is extremely impressive. He should also have a lot of good football left in him, similar to Tannehill.

Stafford has been a very underrated quarterback for a while now and has carried an absolutely inept franchise in the Detroit Lions. He had a great season in 2019, but suffered a decent drop-off in 2020 as his ANY/A fell from fifth best in the league to 13th. That’s not bad, of course. And I’d certainly say the Rams are contenders with him at the helm of the offense. But I don’t think he’s played as well as Kirk or Tannehill over the past two years.

You might expect Baker to be a bit higher here. After all, he’s a former first overall draft pick who had quite a good season leading the Cleveland Browns to their first playoff appearance in years. I just don’t think Baker’s quite as good as the stats suggest. He had one of the best offensive supporting casts in the NFL and he wasn’t exactly carrying them. He was also awful in his sophomore year in 2019. I’m not terribly high on him.

Remember when I said Brady might’ve been the hardest player to rank? Well, I wasn’t lying. Aaron Rodger is up there too, though. We all know how good he is — he’s the reigning league MVP! We all know that Rodgers is one of the best quarterbacks to ever play the game. However, he’s on the wrong side of thirty. His incredible 2020 season came after a multi-season mid-thirties drought full of underperforming from the former Super Bowl MVP.

How do we know Rodgers won’t go back to that in the future? Well, we don’t. But consider this: what sparked the late career resurgences in Peyton Manning and Drew Brees? An adjustment in playstyles. Their deep passing decreased and they relied more on passes that travel as far past the line of scrimmage.

Now let’s look at the case of Aaron Rodgers. In 2016, Rodgers’ pass traveled an average of 9.0 air yards per attempt. Skipping 2017 due to injury, his average pass attempt traveled 8.9 air yards in 2018. In 2019, this average dropped to 8.8 per attempt. Very consistent. And pretty high — he chucked it plenty. What about 2020? In Rodgers’ latest MVP season, his average pass traveled just 8.0 air yards. With that shift in pass distribution, Rodgers’ completion percentage over expectation jumped to a league best 6.8%.

What’s the point of all of this? Well, I think Rodgers is beginning to shift his playstyle just as Manning and Brees did before him. And I think this will give him at least two more years of great quarterback play. It could very well be longer. And given how good those two years will likely be, you could argue that he should be even higher in the ranking. Given the risk, though, I think his spot is fair.

10. Derek Carr (30.1)

9. Russell Wilson (32.4)

8. Dak Prescott (27.8)

7. Joe Burrow (24.4)

6. Kyler Murray (23.7)

Consider Carr and Wilson to be an extension of the Cousins, Tannehill, and Stafford group of good quarterbacks in their early thirties. Carr’s enjoyed consecutive impressive seasons with the Raiders, finishing ninth in ANY/A in both 2019 and 2020. His EPA+CPOE composite in 2019 and 2020 came in at sixth and ninth respectively. Yeah, he’s quietly been playing very well. And he just turned thirty — there’s plenty of time for him to improve and maintain a solid peak.

I used to hype up Russell Wilson as arguably the best quarterback in the league, but I’ve been disappointed by some of his performances. After a fantastic start to the 2020 season prompting early MVP narratives to form, Wilson absolutely tanked. He threw seven interceptions and lost three fumbles in a four game span. Yeah, ten turnovers. Not great. More than a four game span though, Wilson has simply never been able to play great football for a sustained period of time. There are excuses that can be made, like the Seahawks’ outdated offense really not providing much help. On the other hand, Wilson deserves plenty of blame too. His pass protection isn’t the best, but I’ve seen plenty of him causing sacks by holding onto the ball for far too long, a concerning problem for a 32-year-old. I’m also not super convinced that his playstyle will be able to age well. I’m still putting him into the top-10 because he’s certainly very good, but there are reasons I can’t justify putting him ahead of some of these young guys with great potential.

After a legendary rookie season in 2016, Dak Prescott wasn’t able to demonstrate flashes of the same level of play until 2019. He finished ninth in the league in EPA+CPOE after two down years. He looked to be on track for another good season in 2020 through Week Four before a gruesome ankle injury sidelined him for the rest of the year. One could argue that Prescott should be higher on this list if you are to believe that his past twenty-one games played are an indicator of his future play. However, there are some valid reasons for concern. For one, Dak has never been a consistent passer. He’s also not a true dual-threat like some of the guys ahead of him (and a devastating ankle injury may not help in that department).

And it’s not like he’s really been elite over the past two seasons. His EPA+CPOE since 2019 comes in at 11th in the league. He’s been trending up, but he’s most definitely not at that top tier. And of course, he had more offensive help in his rookie season with a stellar prime, Ezekiel Elliott, Cole Beasley and Dez Bryant all playing well. However, his potential is still fantastic and one could definitely argue to put him ahead of Burrow given that Dak could just now be entering his prime. Next season could be a great indicator.

After all, Burrow’s rookie season stats were not the best. That’s standard for rookie quarterbacks. But they were pretty good for his situation. In fact, perhaps better than expected despite being a first overall pick. Despite playing with swiss cheese for an offensive line, Burrow put up the 17th best EPA+CPOE in the NFL (right behind Herbert). Many fans feared that he would be a one-hit wonder, referencing his draft stock being essentially entirely based on his historic Heisman campaign with LSU in 2019. However, he certainly passed the eye test and I don’t think it’d be unreasonable to take him with the first overall pick again in a 2020 redraft.

Kyler Murray clearly took a leap in his sophomore season and showed flashes of future greatness. His passing ability developed and his scrambling is still absolutely elite. And he’s still one of the youngest quarterbacks in the league. He obviously has some clear flaws — his decision-making isn’t great and he locks onto reads. After finishing 12th in EPA+CPOE in 2020 as a 23-year-old, Murray has clearly demonstrated his low floor. His rushing ability is already game-changing, as he led all quarterbacks in passing yards (819) and came in second for passing touchdowns (11) and first downs (52). As his passing continues to progress, Murray should become one of the league’s most valuable players.

5. Justin Herbert (23.1)

Guess who the second youngest quarterback on this list is? Justin Herbert. Guess who just had an all-time great rookie season? Justin Herbert. Herbert surpassed all reasonable expectations and made his mark as one of the league’s top young quarterbacks to keep an eye on. He’s the clear quarterback of the future for the Los Angeles Chargers.

It’s a bit risky to put a rookie this high in the rankings. Remember the last notable time a quarterback had a fantastic rookie season? Dak Prescott in 2016. Unfortunately, Dak has not become just as great as some expected after his performance in his rookie year. The same could hold true for Herbert, especially considering Dak was even better in 2016 than Herbert in 2020. And Dak would probably have been just as high if this ranking took place in 2016 (and he’s just one spot down — he’s still quite a good player). But given everything Herbert dealt with, including shoddy pass protection and coaching, I’m quite confident in him.

4. Josh Allen (25.0)

Every young quarterback with a bad start to their career in the near future will draw references to Josh Allen.

“What about Josh Allen?” “Did you forget about Josh Allen?” “He could be the next Josh Allen!”

Allen entered the NFL Draft as a clear project. He had talent. A ton of it. His arm strength may be the best in the league. But he didn’t have a great college career — his 56% collegiate completion percentage was quite concerning. He was viewed as a clear boom-or-bust candidate.

Boom. That was the right answer. After an awful rookie season and a way better but not great sophomore season, Allen put his name into the hat of elite quarterbacks with a monster 2020 season in which he led the Bills to a 13-3 record and an AFC Championship berth. Allen was extraordinary throughout the season, posting the third-best EPA+CPOE composite in the league and finishing second in MVP voting behind Aaron Rodgers.

When you watch Allen, his ability to get out of the pocket, drop dimes on the run, and deliver absolute darts across the field stands out. He showed his great passing ability in 2020, with the fifth most passing yards in the league and the fifth highest ANY/A. However, the most underrated part of his game is his rushing ability.

Allen has scored at least eight times on the ground in all three of his seasons in the league. He contributes an average of 36 yards per game with his legs, and his ability to buy time and escape the pocket adds countless more through the air.

And just like the other guys at the top of the list, Allen is super young. This might only be the smart for him.

3. Lamar Jackson (24.3)

After an uneventful rookie season, Lamar Jackson shocked the league with an all-time great sophomore leap. Jackson led the ravens to 13-2 record in his fifteen starts by boasting an 8.19 ANY/A, fourth best in the league, while adding a ridiculous 1206 yards with his legs. Jackson already led the league with 36 passing touchdowns and added another seven on the ground. He’s one of the most game-breaking scramblers we’ve ever seen. All things considered, his heroic efforts as a 22-year-old earned him unanimous Most Valuable Player honors.

The 2020 season wasn’t as nice to Lamar. He put up the 17th highest ANY/A in the league and the 13th best EPA+CPOE composite. Not a bad season by any means, but certainly a disappointing one. No one expected Jackson to sustain a nine percent passing touchdown rate, but his steep drop-off prompted concerns of a one-off performance.

So, what happened? Why has Jackson been such an inconsistent passer? There are a few problems. In games like the 2020 postseason loss to the Bills, Jackson clearly struggles under pressure. He also lacks a top pass-catcher. The acquisition of Stefon Diggs brought out the best in Allen and Lamar could similarly really use an upgrade in the receiving corps. And the passing offense is simply unimaginative and doesn’t help Lamar at all. This article from The Draft Network does a great job at diving into the problems in the Ravens’ passing offense. In short, I don’t think their struggles are a major fault of Jackson’s. I certainly believe that he has the talent to be a top-flight quarterback in this league.

2. Deshaun Watson (25.6)

Deshaun Watson’s stock has been high for years. His EPA+CPOE composite from 2017 to 2019, his first three seasons as a pro, was the eighth highest in the league. That’s incredible on its own. In 2020, he took it to the next level with his best year to date. His ANY/A was the third best in the league while his EPA+CPOE came in at fifth. What stands out is that Watson was doing this with by far the worst supporting cast he’s played with in either college or the NFL. I went into this topic in depth in a recent article but the summary is that when JJ Watt told Watson, “We wasted one of your years. I mean, we should have 11 wins,” he was telling the truth. Historically speaking, NFL teams with a quarterback who performed like Watson did in 2020 win eleven games on average. The 2020 Texans won 4. They were awful. No defense, no rushing offense, poor pass blocking. I’m not convinced they would’ve won a game if not for Watson.

1. Pat Mahomes (25.6)

Was there ever any doubt? Really, did you expect anyone else? Mahomes might be the most valuable 25-year-old in the history of football. Through his first three seasons as a pro, Mahomes has been the best quarterback in the league despite not hitting his 26th birthday yet. He has an MVP under his belt along with two Super Bowl appearances and one Super Bowl win. His earliest playoff exit is an AFC Championship loss in his first season as a starter in a game in which he didn’t touch the ball in overtime. Come on.

A big part of the reason that the three guys right behind Mahomes on this list hold their spot is because of one really great season. Lamar had the MVP year, Allen had his 2020 MVP runner-up season, and Watson proved to be a superstar by carrying a bottom of the barrel team in 2020 as well. Mahomes has dominated the league for all three years that he’s been a starter. The gap between him and the next most valuable quarterback is massive.

And he’s not just playing his role on a dominant team centered around different players. Wilson’s early career dominance was on a Seahawks team that was led by the Legion of Boom. Similarly, Brady was a game manager at the start of his career when the Patriots’ defense led the team to Super Bowl wins.

Not Mahomes. He is the reason that the Chiefs are contenders year in and year out. It’s like watching LeBron James or Michael Jordan in pads.

The post NFL 2021 Quarterback Value Rankings appeared first on The Spax.

It has recently been reported that Houston Texans quarterback Deshaun Watson requested a trade from the team. Needless to say, this is a pretty big deal — it’s not everyday that a 25-year-old elite quarterback is available for trade. In fact, I can’t think of an example of it happening at all. It’s unprecedented.

However, it makes more sense once you realize that the circumstances of Watson’s 2020 season were also unique in NFL history.

In his fourth year as a pro, Watson enjoyed his best single-season performance by a wide margin. He led the league with 4,823 passing yards, completed 70.2% of his passes, threw a career-high 33 touchdowns and a career-low 7 interceptions. His 8.22 ANY/A in 2020 was also the third-highest in the league, only behind MVP candidates Aaron Rodgers and Patrick Mahomes. His PFF grade was tied for the 2nd highest in the league this year. All things considered, one could easily argue that Watson was a top-3 quarterback in 2020. At worse, he was among the five best passers in the NFL.

Yet the Texans lost 12 games.

One could legitimately argue that a quarterback has never carried a team as hard as Deshaun Watson did in the 2020 regular season.

We can illustrate the huge gap between the 2020 Texans’ win percentage and the win percentage you’d expect based on Watson’s performance by computing a linear regression between win percentage and era-adjusted adjusted net yards per pass attempt (ANY/A+) for single-season passing performances since 1970.

The line of best fit clearly illustrates the moderate positive correlation between era-adjusted passing efficiency and team win percentage. If a team’s quarterback plays well, they’ll probably win games. Otherwise, they probably won’t. Also, league average ANY/A in any given season corresponds with an ANY/A+ of exactly 100. Thus, the best fit line hits 100 ANY/A+ at a win percentage of around 50% — average qb efficiency typically results in an average team. Makes sense.

However, a few points clearly don’t follow the trend. Notice the gold dot, representing Archie Manning’s 1980 season with the New Orleans Saints. Archie’s 5.39 ANY/A was the 9th best in the league — nothing particularly impressive, but certainly above average. Yet he finished with a 1-15 record as starter. Based on this linear regression, a quarterback with Archie’s 108 ANY/A+ in 1980 would be expected to have a 57.3% win percentage, or approximately nine wins in a 16 game season. In other words, the 1980 Saints won eight less games than expected based on the efficiency of their starting quarterback. That’s the largest negative difference in this dataset of 768 single-season performances.

Now, let’s take a look at the subject of this article. Based on this regression, a team whose quarterback had Watson’s era-adjusted efficiency in 2020 would be expected to win approximately 11 games. The Buccaneers won 11 games this regular season and they will play in Super Bowl LV seven days from now. Simply put, an eleven win team tends to be a pretty good team. Yet the 2020 Texans won just four games. This difference of negative seven wins is the second largest negative residual in the dataset behind only Archie Manning’s 1980 season.

Here’s another way to understand the extent of Watson’s carryjob in 2020. His ANY/A+ in 2020 was 125 (approximately 1.67 standard deviations above league average). No team in NFL history whose quarterback had an ANY/A+ of 110 or higher has ever had a win percentage as low as the Texans’ 20% win rate in 2020. And 125 is far better than 110 — it’s essentially the gap between Deshaun Watson and Baker Mayfield (11-5) or Phillip Rivers (11-4) this season. Not exactly a trivial difference. An ANY/A+ of 110 means the quarterback’s efficiency is not even a full standard deviation above league average. Yet no team with a quarterback in that tier in NFL history has ever performed worse than the 2020 Texans, who had a quarterback far better than 2020 Baker Mayfield or 2020 Phillip Rivers.

Of course, simply comparing era-adjusted passing efficiency doesn’t tell us the whole story. What if a quarterback is padding their efficiency down by 21 late in the fourth quarter? That doesn’t mean their team is letting them down. Thus, let’s eliminate garbage time stats and take a look at passing efficiency in a different way.

We will use a metric called EPA+CPOE, which is a composite metric consisting of Expected Points Added (EPA) and Completion Percentage Over Expectation (CPOE). It is the most predictive metric for future efficiency that we currently have access to.

In the 2020 season, Watson finished 3rd in EPA+CPOE behind Aaron Rodgers and Josh Allen. The top five was rounded out by Patrick Mahomes and Russell Wilson. Pretty much exactly where you’d expect to see him.

Let’s remove plays in which the quarterback’s team had a win probability of less than five percent or greater than ninety-five percent. Watson’s EPA+CPOE index remains 3rd in the league. If we increase the threshold to a win probability between 10% and 90%, Watson is still 3rd.

So, why did the Texans fail to win more games? Well, their team defensive DVOA ranked 30th in the league. Their rush offensive DVOA ranked dead last in the NFL this season. Their special teams’ DVOA ranked a below average 20th in the league, which helped contribute to the Texans having the 2nd worst average starting field position in the league. Oh, and the team’s pass block win rate ranked 19th in the NFL.

Someone might argue that while Watson’s stats are great on paper, he doesn’t contribute to winning football and he’s just an “empty stats” player. Given that Watson went 11-5 in 2018 and 10-5 in 2019 as a starter (with a playoff win) and won the National Championship with Clemson, it’s fair to say that he’s certainly capable of leading a winning team. The rest of the squad just didn’t pull their weight in 2020.

So when JJ Watt told Deshaun Watson, “We wasted one of your years. I mean, we should have 11 wins,” he was telling the truth. It’s hard to blame Watson for wanting out.

The post The Wasted Excellence of Deshaun Watson appeared first on The Spax.

Almost one year ago, I posted this article analyzing the efficiency of the shotgun formation in the NFL. I found that the frequency of the shotgun formation has sharply increased over the past ten years, along with the efficiency of passing outside of the shotgun formation.

The theory I posed to explain the discrepancy between passing efficient in and outside of shotgun formation was that defenses are more likely to expect a passing play when the quarterback is lined up 5-7 yards away from the center. After all, 79.5% of offensive plays from the shotgun formation since 1999 have resulted in passes. It would make sense for the defense to prepare for the quarterback to drop back and pass.

If this theory were to be true, it would also make sense for rushing to be more efficient in the shotgun formation because the defense is expecting a pass. And it is: since 1999, designed run plays gain an average of 4.71 yards per carry from the shotgun and just 4.05 yards otherwise. Running plays from the shotgun formation add approximately -0.044 expected points, versus -0.088 expected points in other formations. Running isn’t incredibly efficient in general, but it is more efficient from the shotgun.

Well, for most players. Adrian Peterson is widely regarded as one of the best rushers in league history. His 2012 MVP campaign is arguably the all-time greatest single-season performance from a running back. However, one of the biggest criticisms of his on-field performance is his ineffectiveness in the shotgun formation. Through Peterson’s career, he has gained an average of 4.71 yards on rushes when the offense isn’t lined up in shotgun formation. When opposing defense are more likely to anticipate a pass, Peterson’s rushing efficiency actually plummets to a mere 3.85 yards per carry.

Is Adrian Peterson the most extreme example of a running back who under performs in the shotgun formation? And which rushers are at their best when the quarterback isn’t lined up under center? To find out, I determined each running back’s average Expected Points Added (EPA) on carries out of the shotgun formation along with their separate EPA on carries from under center. The data goes back as far as 1999, but due to running in the shotgun formation not being prominent prior to the past decade, most of the eligible players are active players.

The straight line through the center of the graph is the line y = x. If a player is directly on that line, they are equally as efficient on shotgun runs as they are on runs from under the center. Alvin Kamara is the only real example of this. Kamara and Jamal Charles are the only two players to average a positive EPA on both types of runs.

The players to the right of the straight line are more efficient shotgun runners. Marlon Mack is by far the best runner in the shotgun formation since 1999. The top three is rounded out by Kareem Hunt and Alvin Kamara.

On the left, you see the players that are better at runs from under the center than from the shotgun formation. As expected, Adrian Peterson is among those names. But an even greater disparity is seen with Todd Gurley, something that I was not personally aware of.

The bottom left of the graph is obviously where no running back wants to be — awful efficiency on both types of runs. It’s amazing to think that Trent Richardson was the third overall selection in the 2012 NFL Draft. What a legendary bust. He’s the worst shotgun runner since 1999 and one of the worst runners from under center, but not the worst. That titles belongs to Carlos Hyde.

Also, notice that large clump of unlabeled points directly to the right of the line. This group demonstrates the fact that the straight line is not the line of best-fit because, as previously discussed, shotgun runs tend to be more efficient. So, more players are better at shotgun runs than under center runs than vice versa.

As the NFL shifts to becoming more shotgun-oriented, running backs like Adrian Peterson are becoming less valuable. Players like Alvin Kamara or Christian McCaffrey are the prototype of the modern day back. Someone who can run from different formations and be of value on passing downs is exactly what NFL teams need. Not the one-trick ponies of the past.

The post The NFL’s Most and Least Efficient Shotgun Running Backs appeared first on The Spax.

One of the most commonly cited metrics for evaluating quarterback performance is completion percentage: the number of completions divided by the number of attempts. The blatant problem with completion percentage is that it does not account for the fact that not every pass attempt is equally difficult or valuable.

Imagine a situation in which it’s 3rd down and the offense has 20 yards to go. The quarterback elects to throw a checkdown at the line of scrimmage to their running back, who gets tackled for a meager gain of two yards. The offense is forced to punt the football away. The quarterback contributed little¹ to their team’s chances of winning with that play and the pass was one that any NFL quarterback could make. Yet that pass boosted their completion percentage.

The solution would be to determine that the probability of that completion being made is quite high, so it isn’t particularly impressive that the quarterback completed the pass. And if the completion probability for every pass attempt could be estimated, we could also estimate any given quarterback’s expected completion percentage and compare it to their actual completion percentage for a better representation of their accuracy.

This is not a novel idea, of course. The NFL’s Next Gen Stats hosts a metric called “Expected Completion Percentage” on their site that takes into account variables such as passer speed, time to throw, target separation, and pass rusher separation. This data is not available to the public, unfortunately, so the point of this project is to create a model for completion probability using the limited data that is freely available.

The first task is to actually get the data. I used a play-by-play data set that consists of every regular season play dating back to 2009.

So, what features should be used for the model? The first answer that comes to mind is the distance of the pass attempt, or the number of air yards. Passes that travel a further distance tend to be completed at a lower rate. I plotted this relationship below — note that the size of each point represents the total number of attempts that traveled a certain number of air yards.

Passes don’t typically travel further than 20 yards from the line of scrimmage. The most common distance for a pass attempt is just five air yards. The mean pass distance is approximately 8.43 air yards, while the median distance is just six air yards.

Our model is gonna need more features than just distance, though. What about the down of the play? I calculated expected completion percentage (a preliminary version of it, not what will be the final product) using just the number of air yards. Then I compared this with the actual completion percentage on every down to see if there’s a difference:

If the down of a play did not have an impact on completion probability, one would expect the difference between expected and actual completion percentage to be the same for every down. But this is clearly not the case. Completion percentage on 3rd and 4th downs is far lower than expectation based on the number of air yards alone. Why? My guess is that defenses expect a pass on those downs so are better equipped to stop it. Since 2009, quarterbacks have dropped back on 46.5% of 1st down plays versus a whopping 74.7% rate on 3rd down.

That leads us to another potential feature for the model: yards to go. The defense probably isn’t going to set up the same way on a 3rd-and-1 as they would for a 3rd-and-15. Let’s compare the expected (still based solely on a regression of air yards) and actual completion percentages on the number of yards to go for each down.

The error (difference between expected and actual completion percentage) on 1st and 2nd down is consistently low for basically yards to go values.² However, there is clearly a noticeable level of variance in the error on 3rd down, suggesting that the number of yards to go is indeed a factor that impacts completion probability.

What other factors impact the passing game? It becomes easier to defend a passing attack as the field gets shorter, i.e. as the offense moves closer to the opposing end zone. When you’re only 20 yards away from scoring a touchdown, there isn’t as much field to work with. Your receivers can’t space the field as much, so to speak. A graph of the expected and actual completion percentages at every spot on the field clearly demonstrates this phenomenon.

A model based on just air yards is pretty accurate until the offense is approximately 25 yards away from scoring. It plummets in goal-to-go situations, where the average completion percentage is just 50.1%.

There are plenty of other features that I ended up putting in the model that I won’t analyze in as much detail. Among these are the score differential (the difference in score impacts the probability of a pass play being called), whether the play is in shotgun formation, whether the play is in no-huddle formation, and the pass location (left, middle, or right), whether the play occurred outdoors or in a dome, the wind speed, etc.

Anyway, let’s get to the results.

The line that runs through the graph is not the line of best fit. It is what I like to call the “expectation curve.” A quarterback that falls on that line has a completion percentage equal to their expected completion percentage — perfectly average accuracy. Anything above the line is above average accuracy, and anything below the line is below average accuracy. Also, the size of each point is representative of the number of passes they attempted. Only quarterbacks with at least 900 passes³ thrown since 2009 were included in the graph.

The most accurate quarterback of all-time is unsurprisingly head and shoulders above his peers. Yes, even after including a feature in the model that identifies whether a pass attempt occurred outdoors or in a dome. While Brees has certainly benefited from playing in a dome for most of his career, it does not discredit his unprecedented precision as a passer.

Deshaun Watson has the second-highest completion percentage over expectation, but his sample size is obviously quite small, as shown by the size of his point. It is also rather sad that the data only goes as far back as 2009, because I’m curious to see how Peyton Manning would stack up with the rest if more of his peak seasons with the Indianapolis Colts were included (2004 especially).

If people were asked to guess who the far left point represented, I think Jameis Winston would be a popular answer. You don’t put up a TD-INT ratio of 33-30 in a single-season without being the absolute definition of a gunslinger. Credit to him for maintaining a completion percentage above the low expected rate, though.

Also, Mahomes and Rodgers share basically identical expected and actual completion percentages. It lends even more credence to the comparisons frequently made between the two passers.

In general, the points below the line appear to be smaller than the points above it. After all, the Blaine Gabberts of the world aren’t good enough to keep a starting job for very long.

I also calculated single-season completion percentage over expectation (CPOE) and found that Brees’ 2018 season was the best since 2009 in terms of accuracy. While his expected completion percentage of 63.6% was high, his actual completion percentage of 74.1% blew it out of the water. Russell Wilson (2019) and Tony Romo (2014) followed him up with stellar seasons of their own that didn’t warrant a Most Valuable Player award.

One last thing: I think it’s a shame that this data only goes as far back as 2009. Therefore, I created a separate model for completion probability that dates back to 2006. The caveat is that it includes far less variables and is less accurate, but it’s a bit better than just ignoring three years of football. Fortunately, the most important feature by far (which is confirmed by the feature weights for our first model), air yards, is available dating back to 2006.

All of this data is available under the site’s navigation menu. I have labeled this model that dates back to 2006 as a “basic” version of completion probability, while the model that dates back to 2009 is “advanced” due to its additional features.

In case you’re lazy, the links below will direct you to the pages with the full data:

Cumulative Advanced CPOE Data (Since 2009)

Single-Season Advanced CPOE Data (Since 2009)

Cumulative Basic CPOE Data (Since 2006)

Single-Season Basic CPOE Data (Since 2006)

Here’s a sneak peek: Drew Brees owns three of the top four most accurate single-season passing performances since 2006.⁴ And four of the top-six. And five of the top-seven. Once again, the most accurate quarterback of all-time. But you don’t need a model to see that.

The post Estimating Completion Probability in the NFL appeared first on The Spax.

What stats do you typically use to evaluate punt returners? Probably yards per return and total return touchdowns, right? Nothing that actually tells you anything about the value of the return specialist.

In an earlier article, I examined this same problem on the other side of the punt return: the statistical evaluation of NFL punters is typically quite shallow. I ended up using the Expected Points metric to quantify punting value. In this article, I’ll attempt to use the same methodology to evaluate punt returners.

The main idea here is that we can determine the number of points the offense is expected to score on any given drive based on their starting field position.

Let’s say a returner catches a punt at their own 20-yard-line (so 80 yards from the end zone). If the offense started their drive at that spot, their Expected Points would be right around 0.473. However, the returner managed to shake off a couple of defenders and gain ten yards on the play. Now, the offense would be expected to score approximately 1.29 points on average. The return specialist is therefore credited with adding 0.817 Expected Points.

Here’s a scatter plot for the total number of return yards and the corresponding Expected Points Added for punt returns over the past 21 years. That’s a lot of dots — 35,578 to be exact.

I color-coded the points so you can clearly discern the three types of returns. Everything here is exactly what you’d expect. Losing a fumble is always a negative play, but losing a fumble that’s returned for a touchdown is an extremely negative play. Gaining positive yards on a punt return is always a positive play (if you don’t fumble), while losing yards is always a negative play. Groundbreaking stuff, I know.

It’s not obvious from the scatter plot, but the most common event by far is a return for zero yards in which the returner doesn’t lose a fumble, which always equates to a gain of zero Expected Points. You also may notice that I said there were 35,578 dots on the scatter plot even though there have been less than 24,000 punt returns since 1999. Well, you probably didn’t notice that. But it’s true. It turns out the cause of this massive discrepancy is the fact that I’m using a data set which counts fair catches as returns. The NFL does not officially record fair catches as returns.

This raises an important question: should I include fair catches in this analysis?

The point of this article is to get an idea of how much value the punt returner position carries and which punt returners are the most and least valuable. I don’t think it makes sense to ignore fair catches when they do have an impact on the game. Let’s say a player lines up to return a punt on ten different plays. On nine of these occasions, the player raises their hand for a fair catch. On one occasion, they return the punt for a touchdown. I think it would be disingenuous to say that the player adds approximately seven points to his team per punt return. Those fair catches still happened, and ignoring them just arbitrarily inflates the value of punt returners. I think it would only serve to skew the results. I could be wrong, though. The NFL chooses to disregard fair catches, so I’m sure they have a good reason for it. If you’d like to reproduce the work here while excluding fair catches from the data, here is the data set I used.

Anyway, let’s get back on track. I could proceed by aggregating the EPA (Expected Points Added) and TPA (Total Points Added) for each punt returner based on the data shown in the scatter plot. However, this wouldn’t tell us much about the punt returners. Take a look at the distribution of the data:

Both the distribution curves for return yards and EPA are right-skewed. The average play in the data set is a 6.04 yard return adding 0.348 Expected Points. Now we can calculate the number of points each returner added above average in order to determine their true value.

Here are the full results:

The most valuable punt returner from 1999 to 2019 was Devin Hester, who is regarded by many as the greatest return specialist in NFL history. No surprise there. Among players with at least 100 returns, only Adam “Pacman” Jones has Hester beat in terms of efficiency (Average Expected Points Added, or aEPA). A few other players average more yards per return than Hester, but his remarkable touchdown rate gives him the edge in value.

Plotting the results clearly shows the strong correlation between yards per return and average EPA:

The coefficient of determination suggests that approximately 80.9% of the variance in aEPA can be predicted from the independent variable, yards per punt return. It’s not surprising that this is the most significant predictor of value. After all, the job of a punt returner is to give the offense the best possible field position.

Anyway, what do the results tell us about the value of a punt returner? I found that the best single-season by a punt returner came in 2007 when Hester gained 13.6 yards per return and ran back four touchdowns. In total, Hester added 26.7 total points above average over the course of the 16 game season. For reference, Peyton Manning added 160.6 expected points above average in 2013. In terms of EPA, Hester therefore carried 16.6% of Manning’s value. The highest paid quarterback in the NFL is currently being paid $35M/yr. Therefore, we may expect a returner of Hester’s caliber to be worth $5.8M/yr.

This is obviously a very rough estimate, but I think it demonstrates that a good return specialist is worth the money. Of course, finding a returner at the caliber of Devin Hester is virtually impossible. But if you can find somebody even close to that level, they’re worth holding onto.

The post Assessing the Value of NFL Punt Returners appeared first on The Spax.

One could argue that the most underrated position in the NFL is the punter. Nobody really cares about the guys who single-handedly control the opposing team’s field position, which is obviously directly linked with scoring. Intuitively, it seems like a very important job. But it feels like fans and analysts don’t really put much focus at all on punting despite its inherent value. If I asked somebody what statistics comes to mind when you think of evaluating punters, they’d probably struggle to come up with a single statistic at all.

Well, I already trained a fairly accurate field goal model for an improved method of evaluating placekickers in the NFL. Why not stick with the special teams theme?

The goal of a punter is to start the opposing offense’s drive as far away from your end zone as possible. It’s simple — the team that scores more points wins the game, and it is easier to score points if you’re closer to the end zone. So, the punter wants to make sure the opposing offense is far away from the punting team’s end zone so that their chance of scoring points decreases.

One way people evaluate punters is with the ‘punts inside the twenty’ statistic. After all, punters aim to pin the opposing offense as close to their own end zone as possible without the punt actually going into the end zone for a touchback. Best case scenario: your punter gets the ball to rest at the one-yard line. So, it makes sense to see which punters are actually able to pin opposing offenses inside their own 20-yard line, right?

Here’s the issue: it’s far easier to pin the opposing offense inside the twenty if your offense is getting to mid-field. It’s incredibly difficult to really give the other team trouble if you’re forced to punt deep within your team’s own territory because of your offense’s inability to drive down the field. My idea is to account for this with a punting metric that takes into account the position in which a punter is in. I was pretty excited for this idea and thought it might be unique because I hadn’t ever heard of it. Of course, I’d be remiss if I didn’t mention that Harvard beat me to it over five years ago. So, this isn’t an original idea. Think of this as a five year refresher, I suppose.

Let’s go through the process of crafting this metric with some visualizations. First, I plotted the average offensive starting field position after punts from every yard-line over the past 11 years. There are 25,100 total punts in this dataset.

To understand the axes, just imagine a number line from 0-100. The punting team is punting towards point 0. In other words, the x-axis values are how far the punting team is from scoring a touchdown when they’re choosing to punt. The y-axis is essentially the x-axis minus the punt distance plus the return. The net average yardage for a punt from a team’s own one-yard line (at x=99) is about 40.65 yards. This remains rather constant until the punting team is around mid-field. Before this point, the punter is just trying to boot the ball as deep as possible — they don’t have to worry about punting it too far because … well, they’re just not physically capable of punting it that far.

I should note that the post-punt field position includes the yardage gained from the punt return. There’s an argument to be made that return yardage shouldn’t be included because punters would be punished if their team is bad at special teams coverage. However, I believe that punters should also be rewarded for punts that are not returnable. On the other side, it’s easier to return punts when the punter is backed up — which, again, is out of the punter’s control. There’s valid arguments for both sides.

Anyway, we can quantify the value of field position using Brian Burke’s Expected Points metric.

If a team starts their drive far from the opposing end zone, their EP is far lower than if they start their drive close to the opposing end zone. Of course.

The offense’s average starting distance to the end zone can be plotted with the punting team’s field position, and the offense’s EP can be plotted with their average starting distance to the end zone. So let’s put everything together and plot the opposing offense’s expected points scored by the punting team’s field position.

If an offense is 99-yards away from scoring when they’re punting, the opposing offense is going to score, on average, three points on their next drive. As the punting team gets closer to the opposing end zone, this number naturally decreases.

I’ll provide an example to understand where the idea for this metric is heading. It’s 4th-and-10 and the offense has the ball at their own 25-yard line (x=75 on above graph). The punting unit is brought onto the unit. Given historical averages, the offense is expected to receive the ball at their own 30-yard line. They’ll be 70 yards away from the end zone, which puts their Expected Points at around 1.30. That’s why the fitted line on the previous graph goes through (75, 1.30).

The hypothetical punt turns out to be better than expected. Ten yards better than expected, to be exact. The opposing offense receives the ball at their own 20-yard line. Now they’re 80 yards away from the opposing end zone. Their Expected Points on the drive drops to 0.47. The average EP for the opposing offense given the punter’s initial position was 1.30. The actual EP for the opposing offense ended up being 0.47. So, the punter is credited with saving 0.83 points on the play.

I calculated how many Expected Points all 25,100 punts in the dataset saved. With this information, I could determine every punter’s APS (Average Points Saved) and TPS (Total Points Saved), which are similar to POE and PAA from my previous article on placekicking. APS and POE are measures of efficiency, while TPS and PAA take volume into account.

You can find a full table with the cumulative data and single-season data using the site menu. For now, though, here’s a table of the cumulative punting stats I calculated for punters from 2009 to 2019.

I mentioned earlier that the idea behind this idea has been done before by the Harvard Sports Analysis Collective. While true, the results are not the same. The article from HSAC mentioned that Morstead added 11.03 points over just under five seasons, which is “equivalent to 2.32 points over average each season.” This figure was used to imply that the punter position is not particularly valuable and they might be overpaid.

My conclusions were a bit different.

I found that Johnny Hekker has added a whopping 18.44 points per season so far in his career. What does this mean? Well, HSAC’s article also mentions that Peyton Manning added 160.6 expected points above average in 2013, when he was the best QB in the NFL. Hekker’s average season point contribution is equal to 11.5% of Manning’s 2013 season point contribution. Johnny Hekker is paid $3.75M/yr. The highest paid quarterback in the NFL currently is paid $35M/yr. Hekker is paid 10.7% of that. I think that’s pretty fair.

But that doesn’t even tell the whole story. I used Hekker’s season-by-season average, while the value for Manning is from arguably his best season. In Hekker’s best season (2016), he put up a TPS of 45.93. Using the same logic I used above, his contributions were worth approximately $10M.

Therefore, I disagree with the article’s following claim: “the marginal benefit a team could expect from signing a top punter is not worth nearly the current market cost.” A more thorough analysis could be done, but I think the punter position is more valuable than HSAC’s article suggested.

The post A Better Way to Evaluate Punters appeared first on The Spax.

A little less than a year ago, I posted this article describing a metric called Expected Field Goal Percentage (xFG%). It only took one variable into account: distance. And it did so in the most elementary way possible, simply placing a field goal attempt into different categories (0-19 yards, 20-29 yards, 30-39 yards, 40-49 yards, 50+ yards). While it was certainly an improvement over traditional field goal percentage, there was a lot of room for improvement. Now, I’ve made that improvement.

Using a dataset consisting of 10,879 field goal attempts over the past 11 years, I sought to create a more complex model for field goal accuracy. The new model is composed of the following features:

And the label for the model is obviously whether or not the field goal attempt was successful. A snippet of the dataset looks like this:

The first three columns are just for identification purposes. The fourth column is the label of the model, and the rest are the features. The features are the inputs that are used to predict the label.

I trained an XGBoost machine learning model on a portion of the dataset (the training data) and then tested the model on the remaining data (the testing data). I calculated the accuracy score, Brier score, and AUC score in order to evaluate the model.

Accuracy: 0.8455882352941176
Brier: 0.11384989954855385
AUC: 0.7814352078200157

Of course, this isn’t the first attempt to model NFL field goal kicking. I used these values to compare the accuracy of my model to past models. This research paper by Pasteur and Cunningham-Rhoads created a model with a Brier score of 0.1226. A lower Brier score is better (because it is a loss function), so my model seemed to grade better in that regard. In this paper by Osborne and Levine, a model with an AUC of 0.7646. A higher AUC is better, so my model edged out their’s according to that evaluation. Finally, in this article from Jacob Long, a model with a Brier score of 0.1160 is achieved, which is worse than mine, as with the model’s AUC of 0.7811.

All things considered, I think most of these models are fairly similar in accuracy. The differences do not appear to be significant, but I’m satisfied with the fact that it’s even close at all.

With the new and improved xFG% model, I calculated every player’s cumulative xFG% and compared it with their FG% on kicks from 2009 to 2019. Here’s the full cumulative data. I also went ahead and applied the model to single-season performances, which you can check out here.

Here’s a plot with some of the noteworthy points (only players with at least 100 FGA were plotted).

That straight line is the expectation curve. A player on that line is perfectly average — their actual FG% is equal to their xFG%. Being above the line means a player outperforms expectation, while being below the line means they perform worse than expected. I think this visualization does a good job at demonstrating the value of this model. If the only way you evaluated players was with their actual field goal percentage, you would likely conclude that Shayne Graham was a better kicker than Sebastian Janikowski. Once you account for other variables, though, like the fact that an average Janikowski field goal attempt was over four yards longer than one from Graham (40.0 vs 35.6), you would see that Janikowski was actually considerably more effective.

There are a lot of external factors that impact a field goal. These are factors that are neglected when we just look at basic FG% to evaluate placekickers. New advanced metrics are constantly created in the sporting world in order to add more context to the evaluation of specific players. This is one of them. It’s a massively improved version of a very basic metric I created one year ago. And there’s still so many variables that are unaccounted for: wind direction relative to the direction of the kick, wind speed at the actual time of the kick, type of turf, etc. Maybe in 2021 we’ll be able to improve upon this version of xFG%.

The post Modeling the NFL Field Goal appeared first on The Spax.

It’s a scenario we’ve seen time and time again. With less than 10 seconds left in regulation, the team with possession lines up for what could be a game-winning field goal. The outcome of the game comes down to a single kick. The opposing coach can do nothing but stand on the sideline and pray that the kick is either blocked or the kicker simply misses. Well, they actually have one other option: calling a timeout. This tactic, known as ‘icing the kicker,’ is meant to mentally disrupt the opposing kicker and hopefully cause him to make a mistake. It’s a long shot, but it’s really all you can do at that point. The question: does it work?

I’ll conduct this experiment using nothing but Python. Everything done in this article will be reproducible using just the code presented here. If you just want an answer to the question and aren’t interested in the code, click here to jump to the conclusion.

First, I obviously need data to analyze. Here’s some simple code that can be used to create a Pandas DataFrame containing play-by-play logs for every regular season and postseason game prior to the current 2019 postseason. Credit to the nflscrapR team for compiling this data.

### preliminary imports
import matplotlib.pyplot as plt
from scipy import stats
import pandas as pd
import numpy as np

df_list = []

### get regular season logs
for yr in [2009,2010,2011,2012,2013,2014,2015,2016,2017,2018,2019]:
    url = 'https://raw.githubusercontent.com/ryurko/nflscrapR-data/master/play_by_play_data/regular_season/reg_pbp_' + str(yr) + '.csv'
    df = pd.read_csv(url,index_col=0,parse_dates=[0])
    df_list.append(df)

### get postseason logs
for yr in [2009,2010,2011,2012,2013,2014,2015,2016,2017,2018]:
    url = 'https://raw.githubusercontent.com/ryurko/nflscrapR-data/master/play_by_play_data/post_season/post_pbp_' + str(yr) + '.csv'
    df = pd.read_csv(url,index_col=0,parse_dates=[0])
    df_list.append(df)  
    
### combine
full_pbp = pd.concat(df_list).reset_index(drop=True)

Now we have a dataset full_pbp with data for every NFL play since 2009. Let’s see just how large the dataset is.

full_pbp.shape

(519520, 255)

Over half a million rows consisting of 255 columns. That’s a lot. It would be a good idea to create a second DataFrame with just the columns we’ll actually work with.

pbp = full_pbp[['home_team','posteam','score_differential','qtr','kick_distance','field_goal_result','timeout','timeout_team']]

I created a separate DataFrame instead of just editing the original one because if I made a mistake while editing it (like realizing I want more columns than the ones I selected), I would have to waste a lot of time reloading the original DataFrame.

The timeout column is a binary indicator (0 or 1) for whether there was a timeout called. However, there is a separate row for every timeout — it doesn’t tell us whether there was a timeout called before a given play.

I want the 1 in the timeout column to be on the same row as the field goal play, so I’ll shift the entire timeout column down a single row. Same with the timeout_team column.

pbp['timeout'] = pbp.timeout.shift(1)
pbp['timeout_team'] = pbp.timeout_team.shift(1)

Now that we’ve identified plays in which timeouts were called prior to the snap, we can drop all of the rows in pbp that don’t represent field goal attempts. The field_goal_result column contains a value of NaN if there was not a field goal attempt on a given play, so we can just keep the rows that don’t contain NaN in field_goal_result.

pbp = pbp[~pbp.field_goal_result.isna()].reset_index(drop=True)

And now let’s check the size of our trimmed dataset:

pbp.shape

(11281, 8)

That’s much better than 519,520 rows and 255 columns.

Now that we have all of the data we need, we should make sure the data is in the correct form. Let’s take a look at what a snippet of pbp actually looks like.

pbp.head()

It’s nice that our data tells us specifically whether a kick is missed because it was blocked. However, this isn’t relevant to our analysis. All we care about is whether the field goal is good or no good. So, let’s convert this column into binary form.

pbp.field_goal_result.replace(['missed','blocked','made'], [0,0,1], inplace=True)

This code replaces each of the three possibilities for the field_goal_result column with numeric values. A 1 represents a successful field goal, while a value of 0 represents a missed field goal.

We also need a column which serves as a binary indicator for whether or not the kicker was iced on a given field goal. We’ll define an icing as a field goal in which a timeout was called prior to the snap by the opposing team.

ind = np.where((pbp.timeout == 1) & (pbp.timeout_team != pbp.posteam))[0]
pbp['iced'] = 0
for i in ind:
    pbp.iced[i] = 1

The first line of code outputs an array ind with the indices for every row in which the conditions are met for an iced kicker. Then, the column iced is initialized and filled with 0. The for loop is used to replace that 0 with a 1 at the previously identified indices.

Great. Now, we also need to create a column that tells us whether or not a given kick is a high-pressure field goal. After all, kickers are typically iced on these kicks. Maybe field goal percentages drop on clutch kicks, and kickers also just happen to be iced more often on clutch kicks. If we don’t identify clutch kicks, we won’t know if the trend is due to the kicker being iced or just because it’s a high-pressure kick, which are kicks in which kickers are more likely to be iced.

ind = np.where(((pbp.score_differential >= -3) & (pbp.score_differential <= 0)) & ((pbp.qtr / 4) >= 1))[0]
pbp['clutch'] = 0
for i in ind:
    pbp.clutch[i] = 1

The same exact technique that was used for the iced column was used to create this clutch column. I defined a high-pressure field goal as a field goal in the 4th quarter or overtime to either tie the game or go-ahead.

One more potential confounding variable to take care of: home-field advantage. We’ll just create a column that will serve as a binary indicator for whether the home team are the ones kicking.

def home(hometeam,posteam):
    if hometeam == posteam:
        return 1
    else:
        return 0
pbp['home'] = np.vectorize(home)(pbp.home_team, pbp.posteam)

I used a bit of a different technique this time because for some reason, the np.where method took ages to complete this task, while np.vectorize was instantaneous.

Now, let’s remove the columns we used to create the iced and clutch columns because they are no longer needed. I’ll also rename a couple of columns just so the names are shorter.

pbp = pbp[['kick_distance','field_goal_result','iced','clutch','home']]
pbp.columns = ['distance','made','iced','clutch','home']

While we’re at it, let’s take a peek at what a snippet of pbp looks like now.

pbp.head()

Excellent! All of our data is now numeric. Well, almost. there’s a NaN value for distance in eight of the 11,281 rows of pbp. I’ll just cut my losses and drop those rows.

pbp = pbp.dropna()

Now, it’s time to actually try and answer our initial question: does icing the kicker work?

Let’s start off by taking an elementary look at a conditional mean DataFrame grouped by the iced column.

pbp.groupby('iced').mean()

Since 2009, kickers hit 6.18% less of their field goal attempts when the opposing team called a timeout prior to the snap. So we’ve solved the question, right? Not quite. This data also tells us that the average ‘iced’ field goal is also over 3 yards longer and the 18.79% more likely to be a high-stakes kick. We can’t draw any conclusions without accounting for these variables.

I already proved in my previous article that kickers are less accurate at greater distances (shocking development, I know):

This very basic model can be used to calculate a probability of success for a field goal attempt based on distance. Here’s how we can create it from scratch:

means = pd.DataFrame(pbp.groupby('distance')['made'].mean()).reset_index(drop=False)
x = means.distance
y = means.made

trend = np.poly1d(np.polyfit(x,y,3))
pbp['xfg_pct'] = trendpoly(pbp['distance'])

First, the means DataFrame is created, which consists of two columns: distance and made. The made simply has the average field goal percentage for every distance. These two variables can be used to fit a polynomial model to the data and use it to create a prob column for the probability of a given field goal to be successful. So, this is what pbp looks like now:

pbp.head()

Great. Now, I think we can analyze the effect of icing the kicker by accounting for the two lurking variables: the distance of a field goal and whether it’s a high-pressure situation. I’m not including home-field advantage because the previous conditional mean DataFrame showed that there is not a significant difference in home-field advantage when a kicker is or isn’t iced (unsurprisingly). I’ll just drop that column now, right before I create another conditional mean DataFrame after our adjustments.

pbp = pbp[['distance','made','iced','clutch','prob']]
pbp[pbp.clutch == 1].groupby('iced').mean()

In the second line, we’re once again creating a conditional mean DataFrame grouped by the iced column. However, this time we’re only looking at plays where the value in the clutch column is 1. Our polynomial model is accounting for distance, and now we can account for high-pressure situations by only looking at clutch field goals. Let’s take a look at the output.

On field goals to tie the game or go-ahead in the 4th quarter or overtime, a kicker’s probability of success is 5.29% lower than expected based on distance if a timeout is called prior to the snap. Otherwise, the probability of a made field goal is just 0.29% lower than expected. It appears that kickers don’t really become less accurate if the stakes are high — they become less accurate if they’re iced.

Is this statistically significant? A t-test can answer that for us.

d1 = pbp[(pbp.clutch == 1) & (pbp.iced == 0)].made
d2 = pbp[(pbp.clutch == 1) & (pbp.iced == 1)].made
p_val = stats.ttest_ind(d1,d2)[1]
print('p-value: ' + str(p_val))

p-value: 0.017371021957504926

The p-value is less than 0.05, so we can reject the null hypothesis that icing the kicker does not have an impact on the success rate of field goals.

Keep calling those timeouts, NFL coaches. Just hope you time it correctly and you don’t end up giving the kicker a free practice swing.

The post Icing the Kicker: A Python Analysis appeared first on The Spax.