Wednesday, September 24, 2008


Who Played Who

Preamble: this is Part III of what will hopefully be a comprehensive series; see Part I and Part II c/w more preamble. This installment attempts to follow directly from Part II. Questions, suggestions, and clarifications in the comments re: the content continue to be not just welcome, but desired.

The limitation of Timeonice is that because it's all about the details, you can't just click around for a couple of minutes and get a broad perspective on which NHL players are seeing the toughest and weakest competition.

That's from Lies and the Coaches Who Tell Them, and the logical question to follow is, Is anyone trying to quantify "quality of competition"? Or put another way, is there somewhere you can go to get this broad perspective on who coaches lean on to neutralize the opponent's badasses, and who they deploy in an effort to exploit the opponent's poorer players? Fortunately, Yes.

**First and foremost is, which makes what I think is a pretty solid attempt to numerically assess the quality of linemates and quality of competition for individual players (note: what quality is, exactly, is a discussion for a future post). The math is relatively simple to understand:
Now: there are clear limitations to this method, but it can still be useful without the need to toss common sense aside. Without getting into it in too much detail, the problem (such as it as) is that RATING is what is used to determine QC and QT, but obviously RATING is itself significantly influenced by QC and QT.

Taking a look at the list of NHL forwards (min. 50GP and 500mins EV TOI), ranked from best to worst by RATING, should drive that home. Parts of it, particularly near the top and bottom, look awfully... screwy. Is David Perron really the best forward in the NHL? No. Are Pahlsson, Moen, and Niedermayer 3 of the 25 worst forwards in the NHL? Uh, ever heard of Thornton or Kopitar jumping for joy that they get to go to Anaheim and exploit that bunch?

That said, there are a couple of good reasons why Desjardins' QC/QT are worth something.
  1. More often than not, RATING is a decent approximation of player quality. For every David Perron (or Sami Pahlsson) whose RATING is absurd (heavily influenced by the way they are used and/or luck), there are 2 or 3 or 5 players whose RATINGs are in the right ballpark. And since a player's QC/QT is dependent on the RATING of many, many players, then we should expect that his QC/QT is, on average, a better stat than RATING.
  2. There is a kernel of truth inside a lot of the RATING numbers that seem (or are) absurd. The EV offensive numbers for Briere and Marleau, for example, were appallingly bad this past season, and their teams really did have (much) worse results when they were on the ice. The Pahlsson trio rarely scores, so while it might easily be true to say that they are the toughest line in the league to score on, it is almost certainly false that they are the toughest line to outscore -- you dig?
The consensus about Desjardins' QC/QT around the hockey blogosphere, here in September 2008, seems to be this: it's pretty successful at ranking who, within a given team, is seeing the toughest competition and who is seeing the softest, but it is not (yet) useful as a raw number for comparing players between teams.

**The other major effort out there that looks to connect hard data to who is seeing the tough competition (and whether they are succeeding at it) has its own unique charms, and is again at Vic Ferrari's TimeonIce. It arises from some things both he and Tyler/mc79hockey worked on a couple of years ago.

Its premise is that there are certain players who we know are very good offensively and just generally "plus players". The idea, then, is to create a list of these players, and then check out how a particular roster or player is deployed against those players. See Vic's and Tyler's old posts for a better general explanation of the concept than I could provide.

So, let's say I'm mulling over the likelihood of the Oilers winning their first regular season division title since 1987, and one of the things I'm interested in is whether Horcoff & Hemsky will be able to outplay, or at least come out even against, the other top lines in the Western Conference. To see how they managed last season, I use this URL.

Now before you say "ah crap, that already looks too complicated just from the URL", click on the damn thing, because there is an excellent guide at the top of the page. The EDM refers to the team/players we want to look at. All the numbers (and commas) denote top players on other teams. The list of them is here, but it's even less complicated than it looks.

2607 denotes Keith Tkachuk: the 26 refers to the Blues, as St. Louis is 26th/30 in alphabetical order of NHL teams, and the 07 refers to Tkachuk, as in "#7 in your program, #1 in your hearts". So if you'd rather see Kariya there than KT, change the 07 to 09. (Or if you think they both suck, just delete the number altogether.) Important note: don't use more than one player per opposing team, because it will double-count events for which both players were on the ice (viz. the data for vs. both Sedins and vs. Daniel only).

The best thing about this method is that it's so much easier to understand. There will always be extreme exceptions, but seeing that Dustin Penner was on for a total of 300 shots for & against vs. "the stars" while Andrew Cogliano was on for 177 SF+SA vs. Thornton, Getzlaf, Iginla et al is awfully sound evidence that Penner was used a lot more often against tough competition than Cogliano was.

Also: while the "true meaning" and benefits of outshooting the opposition is a whole other long post, I think it's a lot easier to accept that SF/SA vs. star players is meaningful. It's one thing to argue that a player, or a team, creates relatively high quality shots (by design or otherwise), and as such, a straight shot count does them a disservice. But it's something else entirely -- i.e. wishful thinking -- to say the same thing about shifts vs. the best players in the conference.

Back at the first link, you'll see that Fernando Pisani was 76SF/106SA, but an even 9GF/9GA. I like Pisani's game as much as any non-Oiler fan, but that's lucky. If he is outshot vs. the stars by nearly 50% this season, then you can expect him to get outscored by that much, or more -- the best players tend to create higher quality chances and have above-average on-ice shooting percentages, which is of course a big reason why they're the best players.

Lines in the Sand: Even (or simply thereabouts) in SF minus SA is terrific, especially if the player is not really counted on to create offense. Careful not to overinterpret samples that are too small, though you'll see on the example that Stortini and Glencross come out well (as do Torres, and Pitkanen -- at least Zacman is still around!). Per the previous post, where the shifts start will have a definite effect on these numbers (presumably Stoll and Reasoner look worse than they would otherwise because they started a lot of these shifts with an own-zone faceoff).

Calgary's numbers are here: note that apart from changing EDM to CGY, the other change in the URL is to replace Iginla (712) with Hemsky (1283).


Good stuff Matt. I haven't been around long enough to know how to manipulate that little tool of Vic's; that's useful knowledge.

It's brand new (at least the current format) as far as I know, I stole it from his post about the Sedins. Nifty as hell, though.

I'm still mulling how best to use it to get the information I'm interested. If you're interested in the "coach's use" angle, the selection of players he/I showed is probably good, although then TOI would probably be a more consistent way to look at it than Shots & Goals.

If you're interested in the "results" angle though, I'm not sure how best to do it. I don't really like lumping Zed and Thornton in with Arnott and Getzlaf, let alone with Doan and Tkachuk. And picking a guy on some teams is hard. Take Dallas: do you pick Ribeiro (effectively he AND Morrow) because the Stars scored a lot more with those two guys on, or Modano, because he's harder to score ON, historically a terrific two-way forward, and his coach plays him (at EV) like he's their best forward?

What about using two groups of forwards, one for outscoring, the other for scoring period? I.E., Anaheim, use Getzlaf in Group I and Pahlsson in Group II. For the Oilers last season, Hemsky in I and, I dunno, maybe Torres in II.

Still doesn't seem like a great solution.


I know exactly what you mean. I suspect that MC chose his list from the "outshooters" on every squad, which aren't necessarily the best players. They might just have the gameplan builkt around them because they aren't complete players.

I chose my list from the guys that I've seen MacTavish build his game plan around. So Kopitar, Cammalleri, and Nagy might strike some as the King's best players, but Frolov is the guy the other team's coach worries about. Knowing that it's not just about how much you create or how much you surrender ... it's the difference between the two that matters to winning or losing, at least in the long run.

The Wild rank all players as A, B, C or D. Where A is difference-maker and D is replacement level. A guy like Dennis, who is switched on and watches way more non-Alberta games than the rest of us ... he could probably compile a list that wasn't far off of Lemaire's/Riseborough's.

And we could work it through like this ... and maybe we'd realize that some "B" guys really deserved to be "A" guys, given who they were playing against when we weren't watching, and vice versa.

So we'd revalue and run them again. Rinse and repeat again.

If we have Dennis counting scoring chances, and Slipper pushing down on the Staples EASY(TM) button for soft icetime, and the opposite for the other kind, based on tiredness of OPP, quality, and zone ... then we're getting somewhere.

We don't have the resources, but we can chip away. And 60% of Sabres fans, 70% of Penguins fans, 97% of Oiler fans and 99% of Flames fans will think we are nuts.

God bless them all.

Below the fold here, I can get a bit more arcane with the math stuff. I wanted to say...

Unless my memory of my considerable mathematical education is wrong (which it is, to some extent), there is a solution to the problem with the Desjardins numbers, that being that RATING is the both *the basis for* QC/QT and significantly *impacted by* QC/QT. It requires a smart person, an iterative method, and some computational power.

Also (or is it Alternatively), I was wondering if maybe most of the issue wouldn't be resolved by splitting QUALCOMP into offensive and defensive components. This might actually be quite interesting.

The final product would basically show whether Player X went against above or below-average defensive players, and how their scoring stacked up in that context, AND THEN, whether they went against above or below-average offensive players, and how their goal prevention stacked up in that context.

Apart from what you could learn just from perusing those numbers, I think you would then be that much closer to using QC/QT to create an actual (numerical) correction factor for scoring and goal prevention, and consequently have the ability to compare players who are used in different roles (or who play for different teams) with more confidence.

Or maybe I'm way the hell off. :)

And Vic, I like your Kings example. That's as good an illustration as I can think of for one of the main Communication Breakdowns that seems to be unbridgeable.

The Certain Segment: "These nerds have a bunch of numbers that show Frolov is a better player than Kopitar! Hahahah what a waste of time...."

Vic, as well as me and presumably many others bumping around these parts: "How in the *fuck* can you watch the Kings for more than about a period and not see that Frolov is their best forward."

Also (or is it Alternatively), I was wondering if maybe most of the issue wouldn't be resolved by splitting QUALCOMP into offensive and defensive components. This might actually be quite interesting.

Would you use PTS/60 as the basis for offensive production? I've been thinking about that quite a bit - if all of the players were rated offensively based on their PTS/60, and then we could look at Player X's GAON/60 vs. the average PTS/60 of his opposition.

Would you use PTS/60 as the basis for offensive production?

Maybe? My first instinct would be to use GFON/60, but EVPTS/60 might be better... I'd have to mull over some what-ifs.

It requires a smart person, an iterative method, and some computational power.

I seem to remember a commenter at MC suggesting that a few months ago, or something like how they rank chess so that matches between Doofus A and Doofus B don't count the same as matches between Grandmaster C and Grandmaster D.

Also (or is it Alternatively), I was wondering if maybe most of the issue wouldn't be resolved by splitting QUALCOMP into offensive and defensive components. This might actually be quite interesting.

I've always thought this, too. It's a big part of the reason I like to look at only GF/60 or GA/60 rather than the rating figure when comparing players.

I bet it'd be a cinch for Gabe to put together.

The other thing you could do is break it into home and away, as favourable matchups for coaches would be more likely to come in their own rink. No?

I never like to describe someone else's work as a cinch... as for the other thing, I don't think so at least. Home coach gets the matchups he wants, but relative to the road, those will necessarily be tougher for some of his players and easier for others.


Several years ago, in an interview with a Bay Area business magazine, Ron Wilson talked about how the Sharks ranked players around the league, guys that they might be interested in acquiring.

His explanation was overly verbose, and made it sound more complicated than it was. What they are measuring, in Desjardinsish shorthand, is EV-PTS/60 minus EV-GA-ON/60.

Now obviously you'd want to consider the context of their icetime when they put up those numbers, and you'd need to look at a few years of production to smooth out the lucky or unlucky years. And surely you'd scout the players, make sure they were in good health, consider their age, review a bunch of game film, etc. before pulling the trigger on a deal. Still, he claimed it was a key factor.

The thinking is simple enough, everyone gets an equal share of the blame for a goal-against, and only the point-getters get credit for offense.

Back on OilFans a few people used to churn out the 'Ron Wilson Numbers' every now and again. For forwards it generally worked out near enough the same ranking as EV+/- rate. But for the defencemen, who almost always had huge minus numbers, it really separated the point-getters from the others.

I speculated at the time that S.J must just have been using it for forwards. Clearly that's not the case, look at the type of defensemen they covet.

Staios was terrific at this for a long stretch, btw. Up there with a bunch of guys who earned big dough. Granted Steve brings nothing to the PP. Anyhow, I suspect that there really was substance to the persistent rumours of the Sharks being interested in Staios two summers ago.


Yes, of course that's right, and it's the holy grail of oddsmaking. It doesn't take much to figure out the probabilities of each team winning. Figuring out the impact of individual players on their teams is a different matter altogether.

Iterative methods, a recursive algorithm, that is the way to do it.

It may just be semantics, and may well be obvious, but Desjardins' EV+/-ON/60 minus EV+/-OFF/60 is precisely the same thing as saying EV+/- rate corrected for every team being completely average.

So when Chris Snow comments on The Wild putting different weights on different Quality of Competition metrics, and then segues into talking about the difference between the quality of the East and West conferences ... well he either didn't really understand the question, or more likely, he understands the first step in correcting Desjardins' QualComp. And I have no idea what other metrics he's talking about, as far as I know, Gabe is the only guy who publishes something like this.


You should google Sagarin and Cuban, Matt.

Sagarin's algorithms are closely guarded, though apparently he has willed them to his alma mater, MIT, when he dies. In any case it will all be much of a muchness to what a whack of other guys are doing, and what you seem to be poised to do.

And, of course, the true measure of any evaluation is it's predictive value. And there are no magic bullets, just better tools.

Pretty clearly he is ignoring all the counting stats and just looking at basketball +/- rate for the players. And he would seem to be treating every group of five players as one unit, then separating them out once he has corrected for opposition quality. Which would be a computational nightmare, and would involve massive arrays with billions of values.

Wayne Winston, his cohort, used to claim that he used Excel and Sagarin used Fortran in parllel. Which is just nuts, Excel 2000 only had 256 columns, for starters.

In his latest book (published by Microsoft, btw, maybe Wayne is smarter than I thought :) ) Winston uses an example of Sagarin sending him processed information on NBA players in CSV format, and him uploading it into an Excel spreadsheet. That makes more sense.


I like the home and away split too. Even though I agree with Matt's point, it shows you the intent of the coaching staff.

So if you go back to 2000/2001, it looks like Marchant and Weight played a similar level of opposition, going by the only useful stats they published then (goals, and who was on the ice when they happened).

But very clearly, by the same numbers, on home ice it was Marchant who had the gig. And by memory too.

When they rolled into Dallas Hitchcock would try like hell to get Modano out against Weight's line, to the point that Modano was sometimes visibly pissed off. I think that ultimately the road coach can't avoid it, not if the home coach has made it his top priority. What Low COULD do, was start Weight with the puck going the right way already, or offensive zone draws. So tougher icetime for Modo than Doug. And tougher icetime for Marchant than Weight as well.

I don't know if that makes sense or not. More of a stream of consciousness than a clear argument. Anyhow, I think you capture more of this "hard minutes" thing when you understand the coach's intent. Especially when looking back on eras where there just isn't that much data publicly available.

Looks like you cracked open the ULTRA-grade secret that Pisani wasn't yet at 100% when they put him back in the lineup. The power of statistics!


Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?