Increasingly there are media and fans who talk about KenPom.com. In theory, this is wonderful. We believe the site is an excellent resource and appreciate the continual work that Ken has been putting into it. A subscription is highly recommended.
Much of the site is a database of information that many others calculate themselves. “Basic” advanced statistics for college basketball are difficult enough for much of the media and fans to understand, but when you get to the unique proprietary analysis (i.e., KenPom rankings), even those with a grasp of the basics are often confused.
The example below is just one of many, but a good one and a reminder that KenPom is a predictive system. So when you see a person like Gary Parrish write, “the top-ranked Wildcats are 9-0 and in possession of an adjusted defensive efficiency rating of 82.6, which is a fancy way of saying they’re allowing 82.6 points per 100 possessions once the raw numbers are adjusted to take the competition into account,” it may not be accurate.
What Parrish apparently doesn’t understand is that KenPom does not simply adjust for the opponent faced and is not a ranking of how well a team has performed, adjusted for strength of competition (although it can be a good proxy).
Let’s get into our example.
2013-14 Defensive Efficiency – Minnesota and Iowa.
|Defensive Four Factors||Iowa||Minn|
|Off. Reb. %:||29.0||31.2|
Iowa’s defense was better than Minnesota’s in each of the four factors. Not some – all of them.
Iowa’s defensive efficiency (DE) in 2013-14 was 99.6 compared to Minnesota’s 103.5 – a significant difference of 3.9. However, KenPom’s adjusted defensive efficiency (Adj DE) had Iowa at 102.7, or 2.3 worse than Minnesota’s 100.4.
Iowa’s Adj DE was 3.1 higher (unfavorable) than its unadjusted DE. Minnesota’s Adj DE was favorable by 3.1. Thus, we’ve got a large 6.2 change when comparing these teams on an unadjusted vs. KenPom adjusted basis.
Iowa had the 64th best DE in 2013-14, but KenPom will show you an Adj DE of #120.
Strength of Competition; Game Location
Iowa’s opponents had an average adjusted offensive efficiency in 2013-14 of 107.6. Minnesota’s opponents were better, but not by much (108.8).
LNH calculated game-by-game Adj DE for each team as the KenPom and averaged them as the KenPom system does. The favorable adjustment for Minnesota was 3.0, which appears reasonable based on KenPom’s Adj DE for the Gophers.
However, for Iowa the favorable adjustment was even larger than it was for Minnesota, yet the Hawkeyes adjustment was unfavorable!
LNH then adjusted the game-by-game figures for location of the game. No significant changes resulted. The question remains – why is Iowa’s Adj DE taking such a large hit in the KenPom system?
Consider this high-level information:
- In the first 25 games of the year, Iowa’s average (per game, unweighted basis) defensive efficiency was 93.7
- In their final 8 (1-7 record), Iowa’s defensive allowed 118.1 points per 100 possessions
Now we get back to what so many fans and media struggle with: KenPom is a system designed to predict the future, not a reporting system designed to recap and rank what has actually occurred.
If you’re predicting how Iowa is going to play in a theoretical next game of 2013-14, you may very well find it reasonable to predict their defense is going to be much worse than their season average due to what has happened recently. It’s fine to use KenPom for this purpose (after you’ve considered reasons for the recent trend in defense and if believe it will continue).
Effective MOV Capping
As @totally_t_bomb reminded us, KenPom’s changes to the system in the fall of 2013 added what is effectively capping of certain scoring margins. So, Iowa’s Adj DE didn’t get the same props last year from KenPom when opened the season with an 82-39 win over UNC-Wilmington in a 72 possession game as it would have in the previous version of KenPom.
Similar to a weighting recent games more heavily, capping the margin of victory may make perfect sense for a predictive system.
If you’re recapping Iowa’s defensive efficiency for the full year and use only the KenPom Adj DE, you have made an error.
Again, the Iowa defensive efficiency example is just one of many. We’ve talked about others in the past, including January 2013’s Big Ten Efficiencies – Raw vs. Adjusted.
KenPom is a wonderful site, but when using adjusted efficiency information (and therefore KenPom team rankings) for anything more than a quick reference, consider taking a step back and asking, “why might these not be appropriate for my use?”
(By the way, Iowa’s DE this year is #12 in the nation; their Adj DE is #22)