Estimated Value: Isolating Quarterback Performance

Shanna Lockwood / USA TODAY Sports

There are plenty of various metrics used to analyze quarterback performance. Unfortunately, all of these figures can be easily swayed by exterior factors.

For instance, many passer efficiency statistics evaluate a QB’s performance on a per-play basis. But what if certain quarterbacks take on a greater workload than others? For example, Seahawks quarterback Russell Wilson contributed 7.28 adjusted net yards per pass attempt this season. Meanwhile, Packers quarterback Aaron Rodgers gained 6.96 adjusted net yards per pass attempt. ANY/A is widely considered the best non-propriety passing efficiency metric, so this should give decent insight into the performance of these two passers. However, it doesn’t take into account the fact that Wilson dropped back on approximately 54% of his team’s offensive snaps (the league average is 62%). Meanwhile, Rodgers dropped back on a whopping 70% of the Packers’ offensive snaps, the most in the NFL. Teams expected the Packers to pass more often and they did — Rodgers’ job was more difficult than Wilson’s. ANY/A doesn’t factor this into the equation.

Certain quarterbacks are also fortunate to have better targets to pass to. The best way to account for this is to consider a quarterback’s air yards. Air yards are simply calculated as passing yards minus yards after the catch. It doesn’t give the quarterback credit for what a wide receiver does with the ball after the pass was completed. The teams with the most yards after the catch happen to be the teams with the best receiving corps. Unsurprisingly, the Kansas City Chiefs, Pittsburgh Steelers, Los Angeles Chargers, and Los Angeles Rams are at the top. Teams with fewer weapons are at the bottom: the Washington Redskins, Baltimore Ravens, and Arizona Cardinals. This makes sense: better receivers get more open and will have more space to run after completing a catch. A quarterback shouldn’t be credited with 80 yards for completing a wide-open slant route that goes the distance. Most passing metrics do give quarterbacks credit for these yards after the catch, though.

Finally, the strength of the defenses a quarterback faces. You may think that any variance in schedule difficulty is minimized over a 16-game season, but that’s not the case. NFC South quarterbacks were fortunate to play against all of the poor secondaries in the division, while AFC North passers had to play multiples games against the stout Bears and Vikings defenses.

With this in mind, I wanted to take the best non-propriety quarterback metric at our disposal (ANY/A) and adjust it so that it isolates a quarterback’s performance from the team’s performance as well as possible.

The Process

This is the existing formula for ANY/A:

adjusted net yards per passing attempt: (pass yards + 20*(pass TD) – 45*(interceptions thrown) – sack yards) / (passing attempts + sacks)

First, I added rushing statistics to the equation. It’s not a unique idea, but I think it’s important, especially in today’s game where there are more mobile quarterbacks than ever before. After all, we’re trying to quantify a quarterback’s overall performance, not just their passing ability.

Next, I adjusted the coefficients to more accurately match the results of studies which have been done to calculate the yardage value of touchdowns and interceptions. Chase Stuart determined that a passing touchdown is worth 19.3 yards while Brian Burke found that throwing an interception is worth a loss of 60 yards. Burke’s conclusion was only for first downs, though. As the down increases, the value of the interception decreases. Therefore, the value for an interception should be closer to 50 yards. We’ll go with 19.3 yards for a touchdown and -50 yards for an interception.

I also accounted for the number of first downs, worth 9 yards, to the equation because they’re obviously a valuable play which quarterbacks should be rewarded for. We’ll have to go back and change the value of touchdowns to 10.3 yards because touchdowns are counted as first downs.

Now, we deal with yards after the catch. As I mentioned earlier, a team’s YAC numbers typically reflect the strength of their receiving corps. The dilemma we face is figuring out how to account for this. Removing all of a quarterback’s yards after the catch might be misleading because a quarterback may give his receiver a better chance at getting yards after the catch. The route some people have gone is removing 50% of a QB’s air yards. This is just as arbitrary, though. Instead, I decided to use the NFL’s Next Gen Stats to solve this problem. The NFL tracks a nifty metric called Expected Yards After Catch (xYAC). To quote their glossary, this value represents “the expected yards after [the] catch, based on numerous factors using tracking data such as how open the receiver is, how fast they’re traveling, how many defenders/blockers are in space, etc.” If a receiver gains the expected yards after the catch, it is not as if they are boosting their quarterback. If a receiver gains more yards after the catch than expectation, though, then the quarterback is benefiting from the receiver’s above-average abilities. Therefore, we can subtract the yards a quarterback gains from any YAC that is above expectation because it is the result of stellar receiver play, not any boosted quarterback performance. The issue with this is that the numbers are not exact. Next Gen Stats gives us rounded values based on particular receivers, not quarterbacks. For quarterbacks who played a full 16 game season, this doesn’t matter. However, the method does not accurately judge quarterbacks who played any less than that, especially players from the same team like Winston and Fitzpatrick. Hopefully, this will be solved with more specific data available to us in the future.

Great. We accounted for a team’s wide receiver corps. As most people would expect, this adjustment makes Drew Brees skyrocket up the standings to #1 with 7.8 … estimated yards per attempt? We haven’t reached our final product so I don’t have a name for this. Anyway, Mahomes sits at second place with 7.33 estimated yards per attempt. This isn’t surprising — Brees was held down by a mediocre receiving corps outside of Michael Thomas while Mahomes probably had the best offensive supporting cast in the NFL.

We aren’t done yet, though. After accounting for the ability of a quarterback’s targets in the passing game, I accounted for the strength of a quarterback’s running game. I used the aforementioned data on the percentage of offensive snaps in which a quarterback drops back to either pass, run, or get sacked. I thought this was useful for two main reasons. First, a team that runs the ball more probably has a better run game, which lightens the load for a quarterback. Also, a team would focus more on stopping the run game if a quarterback doesn’t pass the ball as much, so when the quarterback does drop back, it will be easier for him to succeed because the opposition would have been warier of a run. The league-average quarterback play percentage is 62.4%. We penalized quarterbacks if their workload was below average and rewarded players if their workload was above average.

You can probably guess what happened now. Drew Brees (7.40) falls to 3rd because his quarterback play percentage of 56.1% was the fourth-lowest in the league. He’s very close behind Patrick Mahomes (7.47) and Matt Ryan (7.41), though. For now, that is.

Finally, we have to consider a quarterback’s strength of schedule. This should be obvious — certain quarterbacks play better defenses, especially due to the divisional schedule. Matt Ryan and Drew Brees played some horrible NFC South secondaries while Aaron Rodgers had to play the tough Bears and Vikings defenses … twice. I determined the average estimated yards per attempt a team allows and compared this to the league-average. Aaron Rodgers played teams which allowed less estimated yards per attempt than league-average, while Ryan and Brees didn’t. Therefore, we rewarded Rodgers and penalized Ryan and Brees. Simple. Some quarterbacks, like Patrick Mahomes, played relatively average schedules, so their numbers experienced very little change.

And we’re done.

Oh, one more thing. I also went through and removed garbage time stats. Playstyles change radically when you’re ahead or behind by three possessions late in the game. This impacts the numbers without really telling us anything valuable.

Alright, now we’re done.

In summary, we took the ANY/A and made the following adjustments:

(1) added quarterback rushing stats

(2) removed garbage time stats

(3) changed touchdown value to 19.3 yards

(4) changed interception value to 50 yards

(5) added first downs

(6) accounted for strength of wide receiver corps (removed receiver yards after the catch above expectation from a quarterback’s passing yards)

(7) accounted for strength of running game (penalized or rewarded quarterbacks if they dropped back on a lower or higher percentage of offensive snaps than league average)

(8) factored in strength of schedule (relative strength of the defenses which a quarterback faced)

Let’s see the results.

The Results

Before you get outraged over the final ratings, make sure you understand the meaning of this metric. Yeah, there might be a few weird surprises. This really shouldn’t be that surprising, though. Think about it; this statistic isn’t built to show which quarterbacks are the best and worst. It shows how well individual quarterbacks played after considering the strength of their supporting cast.

Patrick Mahomes6667.444957
Deshaun Watson6667.374908
Drew Brees5377.243889
Matt Ryan6837.114854
Aaron Rodgers6897.064865
Andrew Luck7027.064955
Jameis Winston4547.053202
Carson Wentz4667.033275
Ryan Fitzpatrick2786.971937
Jared Goff6376.794322
Russell Wilson5456.673637
Philip Rivers5586.573669
Kirk Cousins6906.514494
Mitchell Trubisky5266.513426
Joe Flacco4146.132536
Cam Newton6015.883534
Ben Roethlisberger7305.754198
Baker Mayfield5505.733150
Alex Smith3915.682221
Tom Brady6145.673482
Marcus Mariota4565.552530
Andy Dalton4025.542229
Dak Prescott6575.473592
Josh Allen4375.432373
Blake Bortles4925.062490
Derek Carr6285.053168
Eli Manning6385.023200
Sam Darnold4884.792338
Matthew Stafford6204.742940
Case Keenum6464.733052
Ryan Tannehill3414.631580
Nick Mullens3094.081261
Josh Rosen4613.321529

Quarterback Plays are the number of times a quarterback dropped back and either attempted a pass, run the ball themselves, or get sacked. It’s included so that you can potentially disregard smaller sample sizes.

EV/A stands for Estimated Value Per Attempt. It represents the number of yards a quarterback contributed per drop back.

Their total yards contributed are shown as EV, or Estimated Value.

What type of conclusions can be drawn from this data? Well, we can identify quarterbacks who didn’t garner much recognition this season who actually performed well considering their supporting casts. Deshaun Watson and Matt Ryan are obvious examples.

It should be noted that a quarterback being low on this list does not mean they are not a good quarterback. Was Russell Wilson the 11th best quarterback in the NFL this year? I don’t think so. He was punished because of how often the Seahawks ran the ball, but that isn’t his fault. It does need to be accounted for because it makes his numbers less impressive.

Perhaps we can think of it like this: a player who ranks high in EV/P is very unlikely to be a poor quarterback, but a lower ranking does not imply that a player is a poor quarterback. It’s difficult to draw conclusions on a quarterback performing well with a great supporting cast, but it’s easier to praise a quarterback performing well with a poor supporting cast, especially once garbage time stats are omitted.

Of course, that doesn’t mean that all the quarterbacks at the bottom had great supporting casts and all the quarterbacks at the top had poor supporting casts. But if a player experiences a huge boost in ranking compared to ANY/A, then their supporting cast probably isn’t great. It’s also difficult to quantify the strength of a supporting cast. I’m fairly satisfied with my objective method, but it’s not perfect. YAC Above Expectation probably isn’t the best way to quantify the strength of receiving corps.

I believe that the best use for this metric would be to apply it to college quarterbacks in an attempt to predict their success in the NFL. Unfortunately, college football statistics for air yards are not publicly available — they may not even exist, as air yards are a relatively new innovation.

In the future, I will calculate the Estimated Value for quarterbacks in the 2016-17 and 2017-18 seasons. I also plan on delving deeper to analyze Estimated Value’s predictability and stability. Subscribe to the newsletter on the sidebar so you don’t miss it. Between now and then, I might make adjustments to the metric based on any comments written by readers. What do you think?

Notify of
Inline Feedbacks
View all comments