Introduction
Special teams and its influence on field position is the most underrated component of the football game. American football is a game of inches – every yard matters, and the ability to pin opponents deep, fight for extra yards on returns, and finish surefire tackles can make the difference in any given game.
For this project, we will be attempting to model punt return yards. We hope to develop an accurate model that can be used to evaluate punt returners, gunners, and teams as a whole and draw additional insights on the punt play as a whole. The model will be used to predict the punt return yards (PRY) of any given return. The prediction will be referred to as expected punt return yards (xPRY) and we’ll call the difference between the two values punt return yards added (PRYA).
Methodology
The data includes returned punts from 2018 through 2020 excluding penalties. Many different variables were tracked over a total of 1877 plays. The coordinates of each player relative to the returner, their speed, orientation, direction of motion, Euclidean distance from returner, etc.
I also experimented with Voronoi features, such as the area of the returner’s Voronoi region and the x-value of the leftmost vertex of the returner’s Voronoi region (after standardizing play direction). I also calculated the proportion of the returner’s Voronoi region’s perimeter that bordered a defender’s Voronoi region and the area of return unit players’ Voronoi regions that bordered the returner’s region. I tested indicators such as whether the line between a defender and the returner went through a blocker’s Voronoi region.
After feature selection, a parsimonious XGBoost model was trained using only variables representing how far each defender is from the returner, the x-value of the leftmost vertex of the returner’s Voronoi region, the area of the region, and the area of offensive players’ Voronoi regions that bordered the returner’s region.
I trained the model on data from the 2018 and 2019 season so that analysis could be done on the 2020 season.
Returner Evaluation
The most obvious application of xPRY is to evaluate individual punt returners based on their ability to get more yards than expected. We can predict the outcome of each returned punt in 2020 that didn’t have a flag thrown and compare the yardage to the actual output for each returner. The top ten returners in total PRYA are shown below.
First-team All-Pro returner Gunner Olszewski finished with the most punt return yards added in the 2020 NFL season while second-team selection Jakeem Grant came in third. Saints returner Deonte Harris had another strong season after a first-team All-Pro campaign in 2019.
Diontae Spencer had the most valuable return of the season in terms of PRYA, as he was expected to gain just 4.4 yards on an 83-yard punt return touchdown against the Panthers on December 13th, 2020.
Spencer was immediately met by two gunners on a play in which most returners probably would’ve called for a fair catch. Instead, he escaped the initial pressure and found an opening through the left side of the field, gaining 78.6 more yards than expected.
We can also rank the players who performed the worst relative to xPRY.
The average punt return for these players gained less yards than the model predicted. It’s interesting that many of these players continued to be the primary punt returner for their teams despite the poor performances. Perhaps the numbers don’t tell the whole picture. After all, Pharoh Cooper was an All-Pro as both a kick returner and a punt returner in 2017. Has he regressed so much? Here’s his worst punt return this season based on xPRY.
On this October 11th return against the Falcons, Cooper received the punt at the 15-yard line and was then predicted to gain 7.8 yards. Instead, Cooper lost three yards.
It may seem strange that Cooper is being dinged (-10.8 PRYA) for a return where he was met so quickly. It should be mentioned that Cooper’s expected gain on this play of 7.8 yards is below average. The average xPRY for 2020 punt returns is 8.9 yards, so the model wasn’t expecting Cooper to do anything crazy.
A possible reason for the prediction not being lower, though, is the apparently opening on Cooper’s left side – the gunner at the top of the field was not in position to stop Cooper from breaking away to that side. Instead, he hesitates and tries to break to the opposite sideline which may have worked if he was able to get past the first defender, but he was in strong position which forced Cooper to turn back around. By that point, it was too late.
A common theme from negative returns is hesitation or “trying to do too much.” For example, Christian Kirk has multiple plays where he could surge forward for a mediocre but respectable gain, but he tries to run laterally towards the sideline looking for the big play even when the opportunity is not there. These are the habits that NFL coaches may want to weed out as field position is too important to throw away yards.
Gunner Evaluation
The task of evaluating gunners is more complex than it was for returners. When we evaluated individual returners, the main metric of interest was PRYA. Similar to a running back’s rushing yards over expectation, the task is simple – get more yards than expected. In the case of punt coverage, there are arguably two important and separate skills: limiting a returner’s xPRY and then after the catch, limiting their return to less than their xPRY.
If a gunner is consistently forcing low xPRY numbers, they’re probably speeding down the field and limiting the returner’s space. That’s an important part of their role – tackling/slowing down the returner is another.
In the interest of brevity, I’ll simply rank the best gunner duos this season based on cumulative PRYA.
The best duo appears to be J.T. Gray and Justin Hardee for the Saints. Through 12 punts that were returned, the two gunners held the returnman to a low average of just 5.92 expected return yards. They put themselves in position to succeed and then capitalized, actually holding those 12 returners to just a two yard average. Overall, the unit is credited with saving 47 yards. This type of holistic evaluation based on multiple metrics allows us to quantify different types of skills that are all related to a gunner’s duties.
Case Study: xPRY-Driven Evaluation of Washington Football Team Gunners
Our statistical analysis of punt return tracking data can be a beneficial complement to the work of NFL coaches and scouts. In a brief case study of the Washington Football Team’s gunners in the 2020 season, we’ll study the production of their two most frequent gunner combinations: Troy Apke & Danny Johnson and Cam Sims & Danny Johnson.
Each respective gunner duo defended against eight punts that were returned in the 2020 season. The actual return yards allowed are plotted on the y-axis along with the expected return yards allowed based on the model.
It seems that the two gunner pairs gave up roughly the same amount of predicted yards. The difference comes in the end result of the play – not a single return covered by Sims/Johnson gained more yards than expected, while three such punts did for Apke/Johnson (points above the expectation curve).
The observed difference in average allowed PRYA is 4.57, so the Apke/Johnson duo allowed 4.57 more yards over expectation on average than the Sims/Johnson duo. The sample size is quite small, though. We can estimate the probability of observing this difference due to random chance through a two-sided permutation test. The permutation test gives a probability of ~2.38% of observing a difference greater than or equal to 4.57 or less than or equal to -4.57.
Of course, conclusions can’t be drawn on these numbers alone. Many variables are not accounted for, such as the skill of the returner, other teammates on the field, etc. These results could give the Washington Football Team coaching staff a reason to deeper analyze the performance of the two gunner combinations.
For instance, they could go on to study the sixteen plays in question. Which players were actually credited with the tackles? They would find that Apke was not credited with any tackles on the eight returned punts in which he played the gunner role with Danny Johnson. Meanwhile, Sims was credited with three tackles that held the returner to less than the yards they were expected to gain.
Of course, this initial analysis is just one piece of the puzzle. Watching the film of the sixteen returns would reveal that one of the ‘big plays’ given up by the Apke/Johnson duo occurred when Apke dove at the returner to slow them down enough for Deshazor Everett to clean-up the tackle. However, Everett stumbled in space and the returner gained an extra seven yards.
Also, PRYA only considers punts that are actually returned. It’s certainly possible that Apke’s 4.34 40-yard dash speed allows him to get down the field and force fair catches more often than Sims. We can quantify this in a vanilla way by crediting the ‘forced fair catch’ to the player on the kicking team closest to the returner at the time of the fair catch. We find that since 2018, Sims forced a fair catch on six punts out of 32 (0.188) versus Apke’s eight out of 73 (0.110). Furthermore, Sims’ average distance from the returner at the time of the fair catch is 3.8 yards versus 5.8 yards for Apke. While this topic can be looked into further, these are the type of numbers that can serve as a reference point for NFL coaches and scouts.
Team Evaluation
In addition to its value in player evaluation, xPRY can be used to evaluate the punt coverage and punt return ability of NFL teams.
We can calculate the average expected yards given up on punt returns by each team along with their actual yards allowed.
We found that the difference in predicted and actual punt return yards allowed by NFL teams is correlated with their punt DVOA, or Defense-adjusted Value Over Average (from Football Outsiders) where a higher DVOA represents a better punt unit. Teams who give up more yards than they were expected to also tend to have a worse punt DVOA. The opposite is also true – teams who limit returners to less yards than expected have a better punt DVOA on average.
The Chargers had the worst punt DVOA in the league by a wide margin (-37.8, ahead MIN at –13.7) and these numbers partially explain why that is. The Chargers gave up the second-most xPRY in the league on average (10.8) while also allowing the second-most average punt return yards (16.8). Not a good combination.
Approximately 32% of the variation in punt DVOA is explained by a team’s average PRYA allowed. This is particularly noteworthy because xPRY does not include the many punts that are never returned while DVOA obviously does. Thus, an xPRY approach to team evaluation can provide valuable insight into a specific component of a team’s special teams performance.
The same approach can be taken to evaluate a team’s punt returning ability.
We found that a team’s return yards over expectation are once again correlated with DVOA. In this case, average PRYA explained 66% of the variance in punt return DVOA. Once again, xPRY-based analysis is linked with DVOA, the gold standard of team evaluation metrics available to the public.