Thanks for the details BBB, I hadn’t spent any time investigating the details of these ratings, because who has time…and because I’d decided that it wasn’t worth my time given they were clearly flawed.
Of course CD know what it does and doesn’t measure, but the choice to name it “AFL player ratings” and put it out there every week is deliberately misleading and detrimental to both their brand and the wider field of sports statistics.
A key component of any statistical analysis is the interpretation and presentation of results. CD could quite easily name these “ratings” as “offensive efficiency ratings” or some other name that provided additional detail and context to the reader; however, they have made a decision (likely commercially driven) to present them in the way they they do.
I think that they have it all wrong in CD world. (And AFL data world). The algorithms are what makes CD anything more than a data entry company. If they believe themselves to actually be an analytics company they should be providing better analysis for public consumption. (given that they are choosing to provide analysis for public consumption).
No one is asking them to publish their algorithms, and tbh I don’t think we even want their most highly valued metrics. But we want something more than the Herald sun level statistical analysis that we currently get.
I also think what we want (as a minimum) is historical raw data sets released for public research and analysis. I’m sure we’d all happily pay a fee if the price was right. It wouldn’t be as interesting as weekly data dumps, but I accept that we’re not getting it because the AFL have enforced an anti-conpetitive monopoly situation that they won’t want to break.
Imagine the interesting information that would come out (and fill summer news pages) if they dumped the season’s complete raw data package to the community on October 15.