Why Passer Rating Is Broken

Jeremy Brevard / USA TODAY Sports

Background

In the early 1970s, NFL commissioner Pete Rozelle was faced with a challenge. The league needed a new way to crown a passing leader.

The rushing leader was the player with the most rushing yards. The receiving leader was the player with the most receptions. Quantifying passing efficiency was a different struggle.

In the 1930s, the passing crown went to the quarterback with the most passing yards. That would be Ben Roethlisberger this year. Nobody thinks he was the best quarterback this season. The NFL subsequently decided to switch to completion percentage as the standard after six years of using passing yardage. Kirk Cousins and Chad Pennington are both in the top-3 for all-time completion percentage and the statistic itself has zero correlation with winning. Probably not the best idea.

The league kept switching their criteria over the next thirty years until a system was derived where quarterbacks were graded relative to their peers’ performance in four different passing categories. The issue was that a quarterback’s score could not be calculated until all other quarterbacks had played in a given week. Because of this glaring flaw, Rozelle assigned the league’s statistical committee to the job of fixing it. The committee called up Don Smith, a Pro Football Hall of Fame executive, for help.

“They asked if I had any ideas about rating passers. I did,” said Smith in 2001, after his system was heavily criticized. “I’m the guilty guy.”

The Problems

Passer rating includes four components: completion percentage (completions / attempts), yards per attempt, touchdown percentage (touchdowns / attempts), and interception percentage (interceptions / attempts).

Smith determined the league average totals (in 1970) for these four factors. He then devised a conversion formula which would give a player between 0 and 2.375 points for each component. Imagine that the formula above is written as (A + B + C + D)*(100/6). A completion percentage of 70% would, therefore, result in an A value of 2. That’s exceptional — a value of 2 for all four components would yield a passer rating of 133.3. The maximum rating (2.375 points in all four categories) results in the maximum rating: 158.3.

If a quarterback were to score 1 point (league average) in each category, their passer rating would be 66.7. For reference, rookie quarterback Josh Rosen had the worst passer rating in the NFL this year. His rating? 66.7.

Average in 1971 is rock-bottom in 2018.

For over thirty years, the game has become increasingly offensive oriented. Passing efficiency has skyrocketed; league averages for completion percentage, touchdown percentage, and yards per attempt are the highest since the NFL-AFL merger. Rule changes have made passing more viable due to defenses being allowed less physicality. A metric which is based around the game of football in 1970 is not very useful in 2018.

So, why not just create an era-adjusted passer rating? Well, it has more flaws than just being outdated.

For example, completions are effectively counted twice. Yards per attempt is a good statistic, but it already rewards players with higher completion percentages. If you complete a higher percentage of your passes, you’re probably going to be gaining more yards per attempt. Completion percentage just rewards quarterbacks for dinking and dunking even if their results are not actually beneficial for the team.

Do you want a quarterback who completed 35 passes of 35 attempts for 100 yards or a quarterback who completed 17 passes of the same number of attempts but for a total of 300 yards? I’ll take the latter. Passer rating instead rewards passers a bonus of 20 yards merely for completing a pass (with no yardage gain).

Another problem with passer rating is that certain components are too heavily weighted.

Interceptions are extremely unpredictable and are often the consequence of poor team play and not necessarily poor quarterback play. Passers are far more likely to throw an interception when their team is losing because they are more desperate to score points to close the lead. Some teams are trailing in games more often — naturally, those teams’ quarterbacks would have a higher chance of turning the ball over.

Then again, turnovers are very important and shouldn’t be entirely excluded. After all, passers still need to avoid them because they have a drastic impact on a team’s win probability. The issue is that according to passer rating, interceptions are worth approximately 100 yards. That’s crazy. The authors of the influential work The Hidden Game of Football found that a more accurate value for an interception would be 45 yards. Expert statistician Brian Burke calculated a modern value of 60 yards. Both of these estimates are far more reasonable than passer rating’s 100 yard weight.

Likewise, touchdowns are also overweighted. While passer rating gives quarterbacks an 80-yard bonus for just crossing the goal-line, statisticians have recently calculated a more reasonable value of 20 yards. Like Troy Aikman’s agent once said, should he be penalized this heavily for not throwing four-yard touchdown passes because he has Emmitt Smith behind him even if he made the pass that got the ball to the four?

Finally, sacks. Passer rating does not take into account how often a quarterback is tackled behind the line of scrimmage. A sack is very detrimental to a team’s chances of scoring points. A team’s Expected Points on a drive decreases approximately 1.7 points after a sack but only one-ninth of a point on incompletions. After all, sacks are essentially incompletions where a team also loses yards. Therefore, shouldn’t we penalize players for losing yards for their team?

One may argue that a quarterback being sacked is the fault of the offensive line. However, studies show that sack rate fluctuates very much when a team changes quarterbacks, but a quarterback’s sack rate remains consistent when he changes teams — even more consistent than touchdown percentage and interception percentage, stats which we showed are excessively weighted by passer rating.

Conclusion

Passer rating sufficed in 1970, but its flimsiness is extremely apparent in an age with more advanced statistics being used than ever before. While new ratings are emerging, like Pro Football Focus’ grades and ESPN’s Total Quarterback Rating, fans prefer trusting the traditional metric they’ve always used. That needs to change.

1
Comments

1 Comment threads
0 Thread replies
0 Followers
 
Most reacted comment
Hottest comment thread
1 Comment authors
  Subscribe  
newest oldest most voted
Notify of
Bruce Q.

You are really putting out some great content. I am so excited to see what else you have to write!