Using Individual Matchup Data to Evaluate Defense in the NBA

The NBA’s recent statistical revolution has had a profound impact on the way we view and play the game. However, a statistical approach to evaluating a player’s performance is still quite imperfect, especially on the defensive side of the ball. There are many different metrics out there which attempt to accurately portray a player’s aptitude on defense. Certain metrics use shooting data, others use box score numbers, while recent innovations are based around plus-minus figures from lineup data.

However, all of these metrics still fall short. The shooting data used for the DRAYMOND rating doesn’t account for the quality of a player’s opponent. It must be easier to contain Andre Roberson than LeBron James, right? And while basic box score stats are easy to derive and analyze, they don’t tell you much. Stephen Curry led the league in steals in 2016, but I don’t think anybody would argue that he’s an elite perimeter defender. Finally, many plus-minus metrics struggle with attribution and collinearity. Players who play a small role on a good team are often given too much credit because the team happens to be playing well even if they are not the most important cause of that success. This can be partially remedied by looking at long-term data (i.e. multi-year samples) but that isn’t always a convenient possibility.

Of course, while none of these metrics are anywhere close to flawless, they are certainly not useless. However, the lack of an extremely reliable all-in-one defensive stat left me wanting to test a relatively untouched idea just for the sake of experimentation. Could we use individual defensive matchup data to evaluate defensive performance?

The NBA made their defensive matchup data available beginning in 2018. The numbers used Second Spectrum’s player tracking system to determine which defender guarded each offensive player for the majority of each possession. For instance, we could compare James Harden’s performances while guarded by Stephen Curry versus Klay Thompson.

The basis for a defensive metric would be to determine the number of points a defender would have been expected to directly concede over the course of a season based on players they matched up against. In other words, holding Stephen Curry to 20 points over 100 possessions is weighted greater than containing Rajon Rondo to the same output because Curry would be expected to score more points than Rondo. The actual metric looks at more deeper numbers than just points and possessions (how many shots they took, how many of those shots were three-pointers, how many turnovers did they commit compared to their average turnover rate, etc), but the fundamental concept is the same.

Using a Python script to comb through every single matchup in the NBA over the past two years, we can determine how many points a defender saved compared to the expectation in each matchup. Finally, we average this number and scale it to 100 possessions in order to arrive at our result.

Listed below is each qualifying NBA player’s Defensive Points Saved (DPS) over the past two seasons. The number is an estimate of the number of points each player stopped from being scored by their offensive matchup per 100 possessions.

The first thing that pops out is Rudy Gobert at the top of the list by a significant margin. Considering Gobert has won the Defensive Player of the Year award both of the seasons which this data covers, it isn’t surprising to see him at the top. Aron Baynes may be higher than he would be if he had played as many possessions as some of his fellow players, but he is certainly known as an extremely good defensive big man. Finally, Jrue Holiday rounds out the top three with the highest DPS of any non-center. This isn’t surprising either. Holiday has deservedly been named to an All-Defensive team each of the past two years and he is routinely matched up against the opposing team’s best wing players. Among the players he defended most often over the past two seasons were James Harden, Kevin Durant, Paul George, Damian Lillard, Donovan Mitchell, and Jimmy Butler. Not bad.

At the bottom of the table, there are also not many shocking results. The defensive struggles of Antonio Blakeney, Malik Monk, Marquese Chriss, and Collin Sexton are not well-kept secrets. Meyers Leonard’s ranking as the worst defensive center in the NBA probably isn’t a surprise to any fans of the Portland Trail Blazers.

Jahlil Okafor’s two-year DPS is basically exactly zero due to a below-average 2018 season followed by an above-average 2019 season with the Pelicans. Some of the league’s elite scoring point guards (Stephen Curry, Kyrie Irving, Damian Lillard, and Russell Westbrook) are also hovering around average ratings, which make sense. None of them are game-changers on defense, but they also aren’t liabilities like they are sometimes portrayed to be.

Speaking of elite offensive players who are described as below-average defenders, James Harden comes in as the 8th best defensive point guard on the list with a DPS of 1.34. Considering how often people say he “doesn’t play defense,” you’d expect something lower. Instead, this metric backs up the claim that Harden’s defense has become underrated.

Another polarizing player on the defensive side of the ball is LeBron James. A glance at LeBron’s defensive FG% numbers would lead you to believe that he has become heavily underrated in that aspect of the game. However, those numbers may be inflated because of the players he tends to guard. The six players LeBron was matched up with for the most possessions last season were Bojan Bogdanovic, DeMarre Carroll, Dario Saric, Michael Kidd-Gilchrist, and Stanley Johnson. DPS takes this into consideration and gives him a more accurate evaluation as the 13th best defensive small forward in the league. Well, in the regular season, at least. We can’t count out playoff LeBron.

Although many of these results match the eye-test, some of them do not. Is JJ Redick actually a slightly above average defensive player? Is Darren Collison really as good of an individual defender as Draymond Green? And is Joel Embiid legitimately a worse defender than guards like Kemba Walker and TJ McConnell? To be clear, excessive use of the eye-test to judge new metrics isn’t a good idea because the unreliability of the eye-test is the exact reason we need metrics. However, some discrepancies deserve to be pointed out.

So, what’s the problem? Why is individual matchup data sometimes giving us results that don’t make sense? Let’s imagine the following scenario: a defender allows his matchup to blow by him, forcing another defender to provide help and pick him up, which leaves another offensive player open for a three-pointer. Well, the initial defender deserves blame for allowing his matchup to blow by him for what would have been an easy bucket if not for the help defense. So, the NBA’s matchup data notes that the offensive player was credited for an assist during the matchup. However, what if that player gets an assist when it isn’t the defender’s fault? What if the defender’s teammates are playing bad off-ball defense while the primary defender is playing good on-ball defense? In both situations, the result is the same: the offensive player assists a teammate for a bucket. However, the blame on defense switches completely. It’s not easy to capture this with basic matchup data because there’s far too much nuance in basketball.

In conclusion, DPS is like any other defensive metric in the sense that it’s nowhere near an extremely reliable all-in-one stat, but it also doesn’t appear to be entirely useless. Maybe with future improvements in player tracking systems, more reliable metrics can be used to tackle the problem of defensive evaluation in the NBA.