Thursday, September 11, 2008
Rates and States
Preamble: David Staples has once again made the sensible suggestion that someone should collate the collected statistical knowledge & theories that have been hashed out over the past few years on the web. I have intended to do just that for a while, but it seems like work & life have intervened.
I'm going to give it a try. I'd like your questions, suggestions, and clarifications in the comments on the content; then, once I'm (we are) sufficiently satisfied that the explanations and examples are straightforward and illustrative, I'll delete the preamble (and possibly the comments), and leave the post on the sidebar as a "permanent" resource for newcomers who want to learn more, and for veterans who are frustrated with explaining something the nth time to refer to.
I'll try to be as clear and uncontroversial as possible, but of course some things will have my own spin or emphasis. Anyway, onward.
----------
Part I.
Everyone agrees that Goals-Assists-Points is not the definitive measure of the quality of a hockey player, but it's much less understood that it's not the definitive measure of a player's offensive ability either. There are (to start) two things that have a major impact on a player's production that are, strictly speaking, out of his control:
The average team has about a 2.5 times better chance of scoring on the PP than they do at EV. If you are getting 4:00 of powerplay ice time (PP TOI) per game, then you're probably going to put up more points than a comparable player who gets 2:00 PP TOI/game, or 0:30, or none.
Further, the average team has only a 1/4 to 1/3 as good a chance of scoring on the PK as they do at EV; roughly speaking, every minute you spend killing a penalty is a minute when you're not going to be scoring.
Beyond that, I hope it's accepted that the nature of PP offense is different than EV offense. By eye, it should be obvious: among other things, you can have uncontested possession of the puck on the PP, because the defenders are more concerned about their position, and what you might do with the puck, than with taking it away from you right now. And by numbers, we know from many years of statistics that there are certain players who have made their living by being terrific on the PP without being any great shakes at EV.
As such, it makes all sorts of sense to express scoring as a rate, or rather, rates. The two you will see most often is EVPts/60 (or EVP/60) and PPPts/60 (PPP/60), which are both the # of points scored per 60 minutes of ice time in that segment of the game. Advantages:
1) Distinguishes between EV and PP offense, which is interesting to know, and I think, important. Alexei Kovalev may well have "come to play every night" last season, and backchecked harder, and shown more leadership, etc., but the biggest boost he gave to the conference champ Habs was producing at a Mario-esque 8.07 Pts/60 on the PP: no one else in the league was over 6.00! Meanwhile, he scored 1.88Pts/60 at EV: same as Chad Kilger and Derek Armstrong.
2) Accounts for variations in ice time. No one would object if I said that a player who played 80 games should score more than a similar player who played 60 games, so likewise, it should be uncontroversial to say that someone who gets 4:00 PPTOI per game should score more than a similar player who gets 3:00 PPTOI per game. Why not correct for that variation to get a clearer picture?
3) Rates are useful almost right away. How many PP points should a good offensive player have after 33 games? I have no earthly idea. But if you have a line in the sand expressed as a rate, you can compare, and evaluate, parts of seasons much more easily. Speaking of:
Lines in the Sand
Something that has become more clear to me over the past year or so is that one of the biggest reasons people are reluctant to embrace newer (and to my mind, better) statistics is that you lose some of the familiar reference points -- especially at the upper and lower limits.
If I tell you I'm thinking of a guy who scored 20 goals, it's hard for you to tell me how good an offensive player he is. He's probably a legitimate NHLer with decent hands, but he may have done it centering a couple of rookies on the 3rd line, which would be impressive, or he may have done it with Crosby on the Pens' PP, which would be somewhat less so.
But if I tell you I'm thinking of a guy who scored 7 goals in a full season, you know that he's lousy offensively, regardless of circumstances. On the flip side, the guy who scores 40 is a very good offensive player; we can say that with confidence even without knowing much else. So:
Even Strength:
Forwards
>3.00EVPts/60 = excellent, elite, fantastic. There were only 5 players who scored at that rate this season (Crosby, Malkin, Stastny, Alfie, Ovechkin). Another 21 were >2.50EVPts/60; as you go down that list, you start to see more guys for whom "circumstances" were clearly a benefit (e.g. good linemates), but all good players nonetheless.
~2.00EVPts/60 = the mushy middle. No one in this range is embarrassing themselves, but there is a wide range in what we might call perceived player quality. A non-star player in this range who you know had mediocre linemates has done well to hit this mark; conversely, if you get much below 2.00 and you see a player who has the reputation of creating offense, it might be time to reconsider that reputation.
1.00EVPts/60 = Lowetide's Mendoza line for hockey players. If you are near, at, or below this level, you stink at creating offense. Unless you are outstanding defensively, or good at punching other guys in the head, your days of a regular NHL shift are numbered at this level.
Defensemen
A totally different story here; d-men are a relative non-factor when it comes to EV offense. Here's the list of guys who played at least 40GP and 10:00 EVTOI/game. Only 4 guys are above 1.20EVPts/60: Lidstrom, Kronvall, and 2 guys who actually played forward a lot of the year.
The top 30 scoring forwards in the league last season scored a bit under 37% of their points on the PP; the top 30 defensemen scored over 57% of theirs on the PP.
Also note: 10:00 EVTOI/game is a pretty good dividing line between players who are getting rolled out on a regular shift and those who aren't.
Power Play:
Forwards
>6.00PPP/60 = WOW, >5.00PPP/60 = excellent, >4.00PPP/60 = capable
Here's where we need to make the obligatory note that common sense still needs to be applied. The PPTOI totals, and thus the sample size, are relatively small in a given season; small enough that a handful of lucky goals or heroic saves can produce a big swing in someone's rate. (Accordingly, rating a player's PP chops based on a single season's results is probably unwise.) Here's hoping that no one clicks on this and thinks because the section of the list around 4.50 goes Ovechkin, Nash, Lecavalier, Peca, M.Richards that Mike Peca is seriously as good on the PP as those other guys.
Defensemen
Knock 1.00 off of each of the fwd categories, and you have it about right. I'd suggest that someone who is nominally a shooter, but is below 3.00PPP/60, is probably not shooting enough, or at least not getting it through, where rebounds, deflections, etc. can happen.
TOI rule of thumb: over 5:00 PPTOI/gm means that player is leaned on heavily, probably to the point where they skip the shift change. Over 4:00 = regular 1st-unit type; over 3:30 (maybe 3:00 for d-men) equates to plenty of opportunity, but maybe not always on the #1 unit. Less than 3:00 means that player is a secondary option.
----------
Post-amble: Once again, I would appreciate suggestions (and corrections) in the Comments.
Assuming I carry some momentum forward, future parts of this "series" will deal with things like context (quality of linemates and competition), Corsi # and shooting/save percentages, and skill vs. luck in general, but this (straightforward modification of counting stats) seemed like a logical place to start.
I'm going to give it a try. I'd like your questions, suggestions, and clarifications in the comments on the content; then, once I'm (we are) sufficiently satisfied that the explanations and examples are straightforward and illustrative, I'll delete the preamble (and possibly the comments), and leave the post on the sidebar as a "permanent" resource for newcomers who want to learn more, and for veterans who are frustrated with explaining something the nth time to refer to.
I'll try to be as clear and uncontroversial as possible, but of course some things will have my own spin or emphasis. Anyway, onward.
----------
Part I.
Everyone agrees that Goals-Assists-Points is not the definitive measure of the quality of a hockey player, but it's much less understood that it's not the definitive measure of a player's offensive ability either. There are (to start) two things that have a major impact on a player's production that are, strictly speaking, out of his control:
- How much ice time he gets
- How that ice time is divided up between the PP, the PK, and even strength (EV)
The average team has about a 2.5 times better chance of scoring on the PP than they do at EV. If you are getting 4:00 of powerplay ice time (PP TOI) per game, then you're probably going to put up more points than a comparable player who gets 2:00 PP TOI/game, or 0:30, or none.
Further, the average team has only a 1/4 to 1/3 as good a chance of scoring on the PK as they do at EV; roughly speaking, every minute you spend killing a penalty is a minute when you're not going to be scoring.
Beyond that, I hope it's accepted that the nature of PP offense is different than EV offense. By eye, it should be obvious: among other things, you can have uncontested possession of the puck on the PP, because the defenders are more concerned about their position, and what you might do with the puck, than with taking it away from you right now. And by numbers, we know from many years of statistics that there are certain players who have made their living by being terrific on the PP without being any great shakes at EV.
As such, it makes all sorts of sense to express scoring as a rate, or rather, rates. The two you will see most often is EVPts/60 (or EVP/60) and PPPts/60 (PPP/60), which are both the # of points scored per 60 minutes of ice time in that segment of the game. Advantages:
1) Distinguishes between EV and PP offense, which is interesting to know, and I think, important. Alexei Kovalev may well have "come to play every night" last season, and backchecked harder, and shown more leadership, etc., but the biggest boost he gave to the conference champ Habs was producing at a Mario-esque 8.07 Pts/60 on the PP: no one else in the league was over 6.00! Meanwhile, he scored 1.88Pts/60 at EV: same as Chad Kilger and Derek Armstrong.
2) Accounts for variations in ice time. No one would object if I said that a player who played 80 games should score more than a similar player who played 60 games, so likewise, it should be uncontroversial to say that someone who gets 4:00 PPTOI per game should score more than a similar player who gets 3:00 PPTOI per game. Why not correct for that variation to get a clearer picture?
3) Rates are useful almost right away. How many PP points should a good offensive player have after 33 games? I have no earthly idea. But if you have a line in the sand expressed as a rate, you can compare, and evaluate, parts of seasons much more easily. Speaking of:
Lines in the Sand
Something that has become more clear to me over the past year or so is that one of the biggest reasons people are reluctant to embrace newer (and to my mind, better) statistics is that you lose some of the familiar reference points -- especially at the upper and lower limits.
If I tell you I'm thinking of a guy who scored 20 goals, it's hard for you to tell me how good an offensive player he is. He's probably a legitimate NHLer with decent hands, but he may have done it centering a couple of rookies on the 3rd line, which would be impressive, or he may have done it with Crosby on the Pens' PP, which would be somewhat less so.
But if I tell you I'm thinking of a guy who scored 7 goals in a full season, you know that he's lousy offensively, regardless of circumstances. On the flip side, the guy who scores 40 is a very good offensive player; we can say that with confidence even without knowing much else. So:
Even Strength:
Forwards
>3.00EVPts/60 = excellent, elite, fantastic. There were only 5 players who scored at that rate this season (Crosby, Malkin, Stastny, Alfie, Ovechkin). Another 21 were >2.50EVPts/60; as you go down that list, you start to see more guys for whom "circumstances" were clearly a benefit (e.g. good linemates), but all good players nonetheless.
~2.00EVPts/60 = the mushy middle. No one in this range is embarrassing themselves, but there is a wide range in what we might call perceived player quality. A non-star player in this range who you know had mediocre linemates has done well to hit this mark; conversely, if you get much below 2.00 and you see a player who has the reputation of creating offense, it might be time to reconsider that reputation.
1.00EVPts/60 = Lowetide's Mendoza line for hockey players. If you are near, at, or below this level, you stink at creating offense. Unless you are outstanding defensively, or good at punching other guys in the head, your days of a regular NHL shift are numbered at this level.
Defensemen
A totally different story here; d-men are a relative non-factor when it comes to EV offense. Here's the list of guys who played at least 40GP and 10:00 EVTOI/game. Only 4 guys are above 1.20EVPts/60: Lidstrom, Kronvall, and 2 guys who actually played forward a lot of the year.
The top 30 scoring forwards in the league last season scored a bit under 37% of their points on the PP; the top 30 defensemen scored over 57% of theirs on the PP.
Also note: 10:00 EVTOI/game is a pretty good dividing line between players who are getting rolled out on a regular shift and those who aren't.
Power Play:
Forwards
>6.00PPP/60 = WOW, >5.00PPP/60 = excellent, >4.00PPP/60 = capable
Here's where we need to make the obligatory note that common sense still needs to be applied. The PPTOI totals, and thus the sample size, are relatively small in a given season; small enough that a handful of lucky goals or heroic saves can produce a big swing in someone's rate. (Accordingly, rating a player's PP chops based on a single season's results is probably unwise.) Here's hoping that no one clicks on this and thinks because the section of the list around 4.50 goes Ovechkin, Nash, Lecavalier, Peca, M.Richards that Mike Peca is seriously as good on the PP as those other guys.
Defensemen
Knock 1.00 off of each of the fwd categories, and you have it about right. I'd suggest that someone who is nominally a shooter, but is below 3.00PPP/60, is probably not shooting enough, or at least not getting it through, where rebounds, deflections, etc. can happen.
TOI rule of thumb: over 5:00 PPTOI/gm means that player is leaned on heavily, probably to the point where they skip the shift change. Over 4:00 = regular 1st-unit type; over 3:30 (maybe 3:00 for d-men) equates to plenty of opportunity, but maybe not always on the #1 unit. Less than 3:00 means that player is a secondary option.
----------
Post-amble: Once again, I would appreciate suggestions (and corrections) in the Comments.
Assuming I carry some momentum forward, future parts of this "series" will deal with things like context (quality of linemates and competition), Corsi # and shooting/save percentages, and skill vs. luck in general, but this (straightforward modification of counting stats) seemed like a logical place to start.
# posted by Matt @ 1:20 pm 
Comments:
Yep, this is why you get paid the big bucks, my friend.
I particularly like the Lines in the Sand feature. Widely accepted/understood benchmarks are important.
One quick note that you may have already been planning about commenting on in a later post:
But if I tell you I'm thinking of a guy who scored 7 goals in a full season, you know that he's lousy offensively, regardless of circumstances.
1) Sometimes a player plays injured, and that can affect his counting numbers.
2) Sometimes the coach uses a player for a role when the player could be producing much more on a different line (Pisani). It's a matter of where the player is most effective with respect to the team.
Yes, this likely happens much less often than your other examples, and it might be right to push them into the margins, but I think they're good arguments as to where that particular stat falls apart.
12 or 15 goals, sure. 7... dunno. Find me an example of someone who played a full season and scored 7 goals -- with the exception of an 18/19 year old seeing limited ice time -- who is recognized as a decent offensive player, and maybe I'll change my mind.
Matt, if you're going to ask for feedback, you might want to give it some thought before dismissing it out of hand. Does Glencross mean anything to you? You know, before he went to the Oilers and Calgary bought him for the 2nd line?
Yes, this likely happens much less often than your other examples, and it might be right to push them into the margins, but I think they're good arguments as to where that particular stat falls apart.
...
Matt, if you're going to ask for feedback, you might want to give it some thought before dismissing it out of hand.
Agreed. This stat is not The Single Number By Which All Hockey Players Must Be Measured. That, of course, isn't even the right test to apply. The test is: does it tell us more about a player's offensive abilities than G-A-P. I honestly can't even believe that there would be a debate about such a thing.
Ok, I think I'm be horribly misunderstood. I think it's relevant to point how strengths AND weaknesses of tests.
Wow, I must be insane.
Glencross? That's the counter-example?
I mean, I had to go look it up, but that guy scored 7 goals in 45 NHL games before going to the Oilers.
Were you thinking that he was somehow on pace for 7 goals in 82 games?
Matt: Outstanding! Thank you.
Only one quibble:
roughly speaking, every minute you spend killing a penalty is a minute when you're not going to be scoring.
Some players do score in that situation. I think shorthanded scoring tends to be overlooked as irrelevant or irreproducible, when actually it's an important component to some players' games. So rather than exclude one category I consider the following six-pack:
ESTOI/60 | PPTOI/60 | SHTOI/60
ES Pts/60 | PP Pts/60 |SH Pts/ 60
... and guys like Marty St.Louis and Brian Rolston who put crooked numbers in every category can be truly appreciated for their all-around games.
What's a line in the sand? Last year there were 179 forwards who achieved 1 minute SH TOI/GP in at least 40 GP. Here's the distribution in SHP/60:
8 @ 2.00+
7 @ 1.50-1.99
34 @ 1.00-1.49
44 @ 0.50-0.99
39 @ 0.25-0.49
47 @ 0.00-0.24
suggesting the following lines in the sand:
1.50 elite
1.00 positive
0.50 Mendoza Line
As for defencemen, there were 142 who met the same criteria:
1@ 1.00+
19 @ 0.50-0.99
44 @ 0.25-0.49
78 @ 0.00-0.24
... suggesting 0.50 is elite and 0.25 the Mendoza line.
All data courtesy of the wonderful Behind the Net, needless to say.
@earl sleek
The stats on yahoo have him at 6 over that span, not 7, but it's a moot point. I personally think that you're assuming that his only playing 36 of the first 56 games is unimportant, especially when you're talking about a 25 year old player. We can ignore the reasons why he wasn't on the ice if you'd like, and we can ignore that it took him until 25 to crack the nhl in anything resembling an offensive player.
Unfortunately (or fortunately for you, whichever you prefer), at that point, I stop talking because when people ask for feedback, they don't want it, and we have the "Stats guys" once again disregarding anything they don't agree with, and treat people with contempt for not agreeing.
Wasn't the point to try to look at the sport as though it was definable? Like a science? Good luck with that.
Matt,
Very nice post. I am a statzi by nature (what hockey obsessed engineer isn't) and I think this series is a very good idea.
I'm not sure what your intent is, but if it is to hold the interest of a non-statzi and possible convert them, you would probably do well to eliminate some of the comments like "and to my mind, better". It left a bit of a bad taste in my mouth, and I agreed with you going into the read.
My 2 cents as I look forward to the next in the series.
we have the "Stats guys" once again disregarding anything they don't agree with, and treat people with contempt for not agreeing.
Dude, you are looking for contempt in a thread that's barely 10 comments old. There is none. People are allowed to disagree with you, however.
Wasn't the point to try to look at the sport as though it was definable? Like a science? Good luck with that.
We're talking about probability, not absolutes, no? If GF/GA matches actual standings 9 times out of ten, we're on to something, but there will always be outliers and exceptions.
But if I tell you I'm thinking of a guy who scored 7 goals in a full season, you know that he's lousy offensively, regardless of circumstances.
@ Ender
I have trouble following your comments then. I thought you were bringing up Glencross as part of your criticism of this "full season" line.
If you had another point with Glencross, I'm not quite sure what it was, but I'll give you the benefit of the doubt.
I have to say that was an excellent start, just the kind of "Hockey Stats for Dummies" I've been hoping to see, both my own edification and for others.
Myself, I got the most out of the "Lines in the Sand," feature.
P.S. Just talking to vet sports columnist John Mackinnon and he tells me that NHL teams have long been using all kinds of advanced stats to judge their players. So my notion about that was incorrect.
P.S. Just talking to vet sports columnist John Mackinnon and he tells me that NHL teams have long been using all kinds of advanced stats to judge their players. So my notion about that was incorrect.
I'd like to know what advanced means here. I've heard some stuff about arbitration systems and, uniformly, they sound pretty asinine. You'd think that there if was advanced stuff that valued a player's role properly, it would show up there.
Also: Can someone explain what's meant by the Glencross reference? I can't figure it out. I'm not all sure what Ender means by that. I also can't figure out what he's talking about in terms of the weakness in the stat. Is it a weakness compared to traditional G-A-P that he's talking about? Does anyone understand this objection?
That should read "arbitration submissions" and, from what I've heard, they're painfully stone age.
I've heard some stuff about arbitration systems and, uniformly, they sound pretty asinine.
I have yet to hear a GM refer to something like PTS per 60, or rarely things like his value at even strength...
@mike w
I think my issue is that me saying "it'd be a good idea to point out the limits of what that stat says" met with the fight that it did. Some wording of posts, noteably Matt, Tyler, and Earl (though retrospectively, I think I misread Earl) hit me the wrong way. I'll grant that I overreacted, but I think the main point still stands: if you're going to put the advantages, put the limits.
@earl
I realize that I wasn't clear. Let me put it this way. Glencross, at 25 (not 18 or 19), couldn't string together a "full" season. He played 36 out of 56 games, which, had it continued at that rate, would be about 52 games played in a full season. His 6 goals would forecast to 8 goals. It's not a perfect example, obviously, but I think it's a valid one.
@mc79
I'm basically just saying that the margins exist, and in order to make any stat better you need to recognize that, and work with it, not just say "it's in the margin, so it doesn't really matter anyway"
And before somebody jumps on it, I'll add that I'm looking at his stats from this season, with columbus. not full nhl record.
Don't point the finger at me Ender. You were complaining that Matt was dismissive of your point about before I said a single thing in this thread.
And I might be tired, or stupid, or something, but I still can't figure out what your point is. He had 7 goals in 45 games before coming to the Oilers. Maybe there was a bunch of press when got picked up saying that he was an offensive player but I must have missed it. How does he fit the parameters of what Matt was talking about?
FWIW, I don't think that anyone would disagree that the stats might not tell the whole story on the guy and that information like who he was playing with and whether he was injured is relevant in understanding what can be expected going forward. I just don't think that Glencross is a guy who supports your point.
Ender, I meant to put a note at the end to the effect of, "I'll talk about context next time, but this seems like a logical place to start." Sorry if I sounded snarky before: of course context and injuries matter.
As to the idea that 7 goals in a season means lousy on O, your counter example does not meet the criteria.
Ok, so just so we're clear, it's ok for you to ignore the reasons why he didn't play a full season. It's ok for you to ignore him being in the pressbox. It's ok for you to ignore his age while looking at his stats. It's is not ok for me to project a single season had he continued playing for Columbus for the amount that he was for that season?
And you're telling me that "This stat is not The Single Number By Which All Hockey Players Must Be Measured. That, of course, isn't even the right test to apply." is not derogatory at all? Seriously?
Look. My point is the margins exist. Period.
But enough about that. It may be useful to me, or people like me if someone defined what exactly you're looking for. Are you looking to understand the game better? Are you looking to find better ways of evaluating your team? Are you looking to show how the on-ice realities don't reflect counting numbers? Who's team is the best? What, exactly?
@matt
It does not meet the criteria. I see. And what exactly is the criteria again? And why is the criteria the criteria? what age group does a player need to be in to meet your criteria? How many games is a full season? How many years does the player need to be in the league for? Is there an exemption for players on shitty teams?
I'll come out and admit that I don't understand any of what's going on in the comments here, so maybe I'm way off, but: why are we arguing about a more-or-less throwaway example used to explain a larger idea about reference points?
why are we arguing about a more-or-less throwaway example used to explain a larger idea about reference points?
Believe it or not, I'm wondering the same thing myself.
And thanks for doing this Matt.
I think by far the biggest problem with respect to the new stats is that most people don't understand them, and if they do, the numbers are meaningless to them because they have no reference points for comparison.
Actually explaining what these stats are and how they can be interpreted will help immensely with respect to increasing their acceptance by the hockey community. As of right now, I wouldn't be surprised if 75% of the people who read a post at mc79hockey have no fucking idea what Tyler is talking about. And that sometimes includes me, someone who has been bugging the shit out of you guys since day one.
Given this, I'm convinced that far more people will start embracing these new stats once they fully understand what they mean, so I appreciate that Matt is taking he time to do this. Godspeed.
Apologies to anyone who read my post before I deleted it. On reading it over, it seemed a bit overly caustic.
Someone message me if people are starting to play nice, because if you post after this note, I won't be back to read it. Not out of spite - there's just not really a point.
What did you assholes go and do to Ender?
P.S. Matt do you think you could make an exclusion for Gilbert Brule in the goal scoring line in the sand, because I sincerely and emphatically believe that he was mishandled in Ohio, and I don't feel it's fair to blemish his reputation here and elsewhere for anyone new to these concept.
Maybe just an asterick with "excluding Gilbert Brule" beside it. Thanks.
Mackinnon is talking about things like scoring chances and the errors stat, and about tracking time on ice. I don't know if these are considered "advanced" or not, as opposed to say Desjardins', who starts with regular plus/minus and then makes new stats out of it, such as qualcomp.
More explanation may well come from Mackinnon, as he had some interesting stories, which I would now share, but I encouraged him to write a blog post about this himself, and he may well do so.
When I say "tracking time on ice," I mean he says the Habs were doing this back in the early '80s, that they had a stats person with a stopwatch for every player, and when they would come on and off the ice, that stats person would click their watch. This was an example of his that NHL teams have been doing some interesting things for some time, that this isn't all that new, which was news to me.
I'm all for profanity, but you can't tell someone to f-off here. Sorry. If you don't like it, f-off.
Also, I prefer that the phrase Godspeed be reserved for important things, like bombing missions and beer runs.
Here's a serious question. Why did 60 minutes get chosen as the base? Just because it adds up to a "game"?
The thing that makes stats like ppg so appealing, despite their obvious flaws, is that they are calibrated to a meaningful scale for the viewer.
For each game, I can think "Hey, XXX has a 1.25 ppg scoring average, but hasn't put up any points in the last 4 games, what gives?".
With a base of 60 minutes, no matter how useful the stat is, it just isn't easily convertable to an immediately relevant scale. "Hey, according to Fenwick, Dvorak has dropped from a 3.2 to a 2.8 EVPTS/60. That really sucks -- at least I think it does -- let's see, how bad is it, well -0.4 over 60 minutes means he hasn't scored in, wait, how many minutes per game does he play again, ahhh fuck it."
I'm not sure there is any real solution, but there it is.
The thing that makes stats like ppg so appealing, despite their obvious flaws, is that they are calibrated to a meaningful scale for the viewer.
For each game, I can think "Hey, XXX has a 1.25 ppg scoring average, but hasn't put up any points in the last 4 games, what gives?".
That's a very good point. I suppose it could be based on 15 ES minutes a game or the median or whatever. The thing I like about 60 is that it does seem to have a couple of natural breaks: over 3.0 is awesome, 2.0 is good, 1.0 and it's "Fuckin' Chrysler plant, here I come."
I like the per 60 metric myself: what does the player produce per full game of ice time? It's easy to calculate if it takes him 4, 5, or 6 GP to actually achieve that ice time based on his TOI; the whole point is to level that playing field between players of different opportunity levels.
It's particularly valuable to use per 60 in the GF/GA ON category, where the numbers relate to actual game scores, goals against averages and other familiar standards. (Not perfect due to the ES / PP dichotomy, but close enough to be useful.)
mc79 (or someone else),
I seem to recall that you have the percentiles for scoring rates stored away somewhere in a chart for the last several consecutive seasons. I think it would be good to take an average of say, the last ten or so seasons and express them in terms of percentile. So if someone is scoring 2.89pts/60 we could know that he was in the 94th percentile or whatever it would be. Seems more helpful and precise than just using round numbers.
Not a bad idea. My only counterpoint is that it fluctuates with leaguewide scoring levels: see here.
I can see the appeal of showing EV results as per 10 minutes, where (A) it would resemble BAA or OBP a bit more (.199 is the suck, .400 is excellent), and (B) the duration is in the range of what someone actually plays in the game.
The downside (or at least, the one I'm thinking of) is that those natural things don't translate to the PP at all: not the TOI per game or the figure. And I'd rather have PP and EV rates expressed in the same terms than not. I'll pretty much second Tyler's comment.
As for Scott's, while there's no question that's more rigourous, I think a lot of the previous comments (and my own inclination) say, No Thanks. Round numbers are handy. Really, who was in the 94th percentile in Goals this season?
That said, I think there's no doubt that going backwards will help re: acceptance. I doubt anything ever sold the OPS stat better than the fact that someone figured out that Babe Ruth had the best season of the 20th century by that measure.
Here's the other thing about the much maligned PPG statistic, and just one way that folks using the EVPTS/Game need to be careful about their terminology.
EVPTS/60 mins is often phrased as a measure of "productivity". But I think that's misleading.
Yes, of course, PPG is dependent upon how much ice time a player gets. But I see this as a feature, not a bug. Players are not capable of playing the same amount and their ice-time reflects this.
Sure, somebody who produces 2.0EVPTS/60 mins by playing only 2 mins a game is equally "productive" as someone who produces 2.0EVPTS/60 mins who plays 15 mins a game. But at the end of the day, the guy who plays 15 mins a game puts up more points on the scoreboard.
Given that there is no guarantee that the 2-minute man would be able to keep up his rate of production were his ice-time bumped up (in fact, I'd argue that the reason his ice-time hasn't been bumped up is because he isn't capable), I find it hard to call the two players equally "productive".
By not controlling for ice-time, the ole-fashioned PPG does take into account just how much the guy can actually play and how much he is actually capable of producing at a meaningful scale of reference (i.e., a game).
Now I know that the EVPTS/60mins is not intended to be a measure of "productivity" so to speak, and is only intended to be a measure of the "rate of productivity per minute of ice-time", but if that's the case, we all need to take care to spell this out and not use the shorthand of saying Dvorak's absurdly high EVPTS/60mins demonstrates he is highly "productive".
I think you're half right...
For starters, the raw numbers of goals and assists are the building blocks; not going anywhere. So a forward who can play 25 minutes a game will generally score more goals and get at least his fair share of credit (e.g. Lecavalier).
By not controlling for ice-time, the ole-fashioned PPG does take into account just how much the guy can actually play and how much he is actually capable of producing at a meaningful scale of reference (i.e., a game).
I think 'can' and 'capable' are misplaced there. I suppose this is one of the points of this whole exercise; I don't want to take the coach's word for it as to who is 'capable' of what, I'm interested in results.
What I would say needs to be acknowledged somehow is that we don't know what the point of diminishing returns is for ice time (how much is too much), it's different for different players, and there's plenty of reason to believe that (e.g.) the Tampa forwards have better totals but worse rates on account of them being on the ice the entire game.
I can certainly understand the confusion with the per 60 metric...it can also lead to a misleading inflation/deflation of the stat in question. I remember a post at Tylers site where he pointed out that Lupul was getting outshot at ES last season at a rate of 8 shots/60 which looks pretty terrible at first glance, but only works out to about 1.5shots per game with Lupuls actual ice time.
The reasons that I think the percentiles will help are:
1. People are accustomed to using percentiles all through their lives and it's easy for most people to understand their value. I think that they will help people to decide for themselves what round numbers are good and bad. Most people know that 30% is terrible, 60% is mediocre, 80% is a pretty good mark, while 95% is outstanding without anyone needing to do any explaining.
2. It really is more specific and effectively compares people to their peers automatically.
3. I think it will help those of us familiar with the stastic get a better idea of what is good and how good it is, as well as what is bad and how bad it is (1 pt/60 is around the 20the percentile. Is that an accurate reflection of the "Mendoza line").
4. As for the fluctuations, that is going to happen, but placing people into a range of ten years worth of compiled data would help to situate each individual's performance in historical context. Plus, it's not really an issue that's taken into account by the current system either (i.e. how good is 3EVPts/60 if 25 guys do it instead of 5? I think there's the same problem either way.
I also liked Bruce's suggestion that short-handed points be included, but again, I think it would be better to express them by both a number and a percentile. Especially here it seems like people are flying by the seat of their pants w.r.t. what is considered good.
I agree that the per sixty minute format is good. People already measure a lot of things "per hour" so it seems like there would be a veneer of familiarity, and as Matt said, part of the purpose of this is to be different from G-A-P by taking out some of the influence of the coach.
Thanks for doing all of this work Matt!
OM's comments about Lupul are part of what I'm talking about. If getting outshot by 8/60 is only 1.5 shots per game then maybe it's not that bad? If we could also say, he was in the negative by 8shots/60 and that this is in the 17th percentile (or whatever it would actually be) I think that it would demonstrate more clearly how terrible that is.
While I'm not sure if there's any point in making this comment now, I figure I might as well.
First, let me say that I like a lot of the stuff done here, contrary to recent evidence. The situational per-60s are a repurposing of existing data to try to account for context at the most basic level, and it seems to work quite well. The continuing search for context -- Desjardins quality stats, faceoff zones, etc. -- further helps quantify the sort of player we're looking at, and the sort of circumstances they are good (and not good) in. On the other hand, there's some stuff here that really bugs me, like the continuing unwillingness/inability to separate sportscaster cliche/crutch from genuine sport science, and the general hostility towards new ideas from outside the 'sphere and its influences. I also think some of the stuff Vic's posted lately is kind of crazy, but that's a subject that was beaten to death in the previous thread.
Now, I consider Ender to be a close friend, and have for some time, I was in his wedding party this past spring, all that good stuff. And as a general rule, he's a very reasonable man. He's got a philosophy degree and was only a handful of courses from a computer science degree, so really, if anyone understands logic and math, it's him. I do think he raised some valid objections and questions, both here and at CinO, and I don't know that many of them have been addressed all that well (or even at all). I am, however, surprised at how badly his ideas and counterpoints, and those of others like me, Bruce, and Staples, have been received. That being said, I talked to Ender on MSN while this thread was ongoing and I have to admit, I was totally baffled. I didn't get where this hostility was coming from, when I thought everything Fenwick and the others had posted here was fairly reasonable. I finally just gave up around 5:00, which is also when I gave up on trying to jerry-rig a wireless connection for my brand-new Xbox 360 and just play a bloody game. (Incidentally, he sent me a better example of what he was getting at this morning: Dan Cleary.)
But the way I see it, everyone has bad days. Everyone has their triggers for shit that sends them up the wall. For me, ironically, one of them is all the Goddamned hostility that the stats vs. no stats "debates" kick up, and that makes me as guilty as anyone about this. But really, is there any need for all the baiting and sniping and condescension and ridiculous nonsense? Not everyone who objects to the Oilersphere way of thinking is like the people who drove this group off HFBoards three years ago. Not everyone who looks at numbers other than the basic counting stats is a stuck-up twat with one eye on the TV and the other on a laptop. Just as there was no cause for Ender to go off here yesterday, there was no cause for Dennis to go off on Bruce for the fiftieth fucking time on CinO a few days ago. There's just no need for any of it. There's nothing wrong with healthy debate, but this here? This has not been healthy debate.
I've had the same issue that Sacamano has with EV/60. It's just not a very elegant or easy to understand way to describe point production.
I've been reading this particular way of expressing stats for almost a year now, and it still seems alien to me.
But is there a better way to do it?
Could we not describe this same idea on a point per even strength minutes basis?
Such as, Hemsky gets one point for every 14 es minutes, while Stoll gets one every 66 es minute.
In short, it would be expressed as Hemsky 1/28 p-esm, Stoll 1/76 p-esm.
For the power play, Hemsky 1-7 p-ppm, and Stoll 1/13 p-ppm.
Most players play about 10-15 minutes of even strength play per game, and most fans know this, so if the stat is expressed this way, the average fan can quickly figure out how far away a guy is from scoring one point per 15 es minutes.
I don't know if this improves things or not . . .
And, of course, most folks get these EV/60 stats from Desjardins, do they not, so unless they want to do their own calculations, this issue is really up to him.
It has been a difficult debate, for sure, doogie2K, marked by a lot of hostility and defensiveness.
Perhaps, though, with Matt's honest and genous attempt to work through some of the basic questions and issues about these new stats, the tide is turning.
I've interviewed hundreds of scientists and researchers over the years, and found almost all of them to be helpful, even as I've pushed them to describe their complex work in terms that a layman like me can comprehend. Almost to a person, these scientists and researchers have been extremely helpful.
On the Oilogosphere, for whatever reason I'm not sure, this attitude of happily explaining your ideas to others isn't the general order of business. Instead things can often be combative, hostile, childish. It has partly to do, I think, with the nature of the Internet, and the fact we don't see each other's faces and don't hear one another's voices. We have little idea of how we're coming across, what buttons we might be pushing, what buttons are being pushed on us.
I know I lost it myself in one debate simply because a few buttons of my own got pushed, and I saw red for about a half hour.
In any case, this isn't an easy placed to discuss ideas. And it's going to take no small amount of work to get past that problem.
But some steps are being taken.
I see I wrote "genous" in the last post.
That, of course, isn't a typo, as I never, ever make typos.
It is a deliberate combination of two words, genius and generous. ;)
Bah, D2K. You're going to make me comment again. Again, as I said before, I apologize if I overreacted. However, I still don't think that I was jumping at nothing. As DMFB said, I presented a throwaway example to make a point about weaknesses of the stat. I thought it would be a nothing comment, warranting little to no reply aside from a "yes, the margins exist and we should probably say something about them too if we're trying to be thorough." That was all. Things didn't work out that way, and I got really frustrated, as we ran into an issue with onus of proof.
It seems that on this blog, in order to be taken seriously, you need to present proof. Proof involves stats. I'm not sure why that is when you're not trying to argue for something, but rather against the exclusion of something else, but whatever. Moot point. Like I said, I thought a throwaway example would be enough, as I do still think Glencross fits the bill, but it was unaccepted, and somehow the idea of "don't exclude things" was lost. Dan Cleary is probably a better example, but as Matt said, Glencross didn't fit the criteria. Would Cleary?
The issue is that people seem to use stats when it helps them, ignore stats when they either disagree with them or it involves more work. No criteria here are ever defined rigorously enough to not have those built in loopholes in them, and they're exploited. I've been reading blogs on the sphere for years, and as D2K said, I know my math and logic. I still have no idea what anyone is really trying to get out of the stats that they're using, and it frustrates me how cavalier people are being about blatantly disregarding information. Especially when those same people are saying that the people who disagree are being intentionally ignorant.
Look. None of this is meant as an attack. I've been told that I came across really poorly in this thread, in an attacking way. Text is funny that way. it was never my intent. Like I said, I over-reacted to some, and probably wasn't watching my words as carefully as I should have. This post is not intended as an arguing point. It's intended as a something similar to a confession/apology. Forget when people play nice, I'm not going to be in again, period.
I thought it would be a nothing comment, warranting little to no reply aside from a "yes, the margins exist and we should probably say something about them too if we're trying to be thorough."
I still can't understand what is meant by this comment about the margins. I swear to god that I'm not trying to be intentionally stupid, but I haven't the foggiest as to a) what it means and b) what the point is.
What I would say needs to be acknowledged somehow is that we don't know what the point of diminishing returns is for ice time (how much is too much), it's different for different players, and there's plenty of reason to believe that (e.g.) the Tampa forwards have better totals but worse rates on account of them being on the ice the entire game.
Precisely.
I've actually occasionally cited scoring rates as points per X minutes for the sake of greater understanding. One point for every ten minutes of power-play time for Ales Hemsky is a lot easier to understand than 5.93 PPP/60.
I still can't understand what is meant by this comment about the margins. I swear to god that I'm not trying to be intentionally stupid, but I haven't the foggiest as to a) what it means and b) what the point is.
I think he was just trying to say that there are always exceptions to the rule. Dan Cleary had a number of 3-7 goal seasons before the lockout, but has since put up some real solid numbers. There were extenuating circumstances behind it, but clearly, he wasn't a bad offensive player, full stop: he just needed to change his circumstances.
Incidentally, I love Andy's post, because it advocates a lot of the things that the objectors have been talking about for the last week, but have been dismissed for one reason or another.
Just catching up on the past two threads, damn this is hilarious.
Doogie2K may very well have invented a brand new kind of stupid, it's hilarious. Seriously, why has nobody ripped him apart, were you waiting for me. It's comic gold.
And seriously, does Staples genuinely not realize what he has been measuring here? Has chubby undergrad and avid video game player "clutch" Doogie not understood his own words? It's not possible that he could have.
Is pathetisad a word?
Mr. Fenwick.
Permission to attack, sir!
I think the other thread might be dead by now, but I thought it was relevant, for anyone who deosn't feed of every blog, that Brownlee responded to some of the recent debates about stats.
http://www.oilersnation.com/index2.php/2008/09/12/by-the-numbers/#more-2760
re: SH scoring. The post-lockout year to year correlation in SH/60 is below .20 so it is almost random league wide at the player level. That said there are handful of people who do score SH such as St. Louis and Madden but they are outliers.
Someone asked to see where the percentile rankings fell over the last several seasons. So here we go.
FORWARDS 90%
ES/60
1998 2.35 (S. KAPANEN)
1999 2.38 (ELIAS)
2000 2.44 (S. RICHER)
2001 2.54 (CASSELS)
2002 2.34 (ALFREDSSON)
2003 2.39 (ZUBRUS)
2004 2.35 (RYDER)
2006 2.69 (STILLMAN)
2007 2.55 (HORTON)
2008 2.32 (ANTROPOV)
DEFENSE 90%
ES/60
1998 0.97 (ZHITNIK)
1999 0.98 (S. COTE)
2000 1.10 (J. NORTON)
2001 1.06 (GONCHAR)
2002 0.96 (DESJARDINS)
2003 1.00 (MARA)
2004 0.99 (EHRHOFF)
2006 1.21 (JOVANOVSKI)
2007 1.14 (MESZAROS)
2008 1.02 (PHANEUF)
Are you going to keep up the ad hominem bullshit or do you have a fucking point, Vic? I'm really not interested in arguing unless you have something substantial.
Scott:
I finally put all this together this summer.
The more recent years are easy, you can just copy and paste stuff off of NHL.com, or if you know how to write code make a script (I can't do that yet).
The older years involved time consuming manual data entry from either old editions of The Hockey News Yearbooks or using the NHL.com super stats machine. I'm still missing data on a couple of players.
It is pretty sweet that we know have this many seasons of data to look at. My next step is working on individual players projection models-now that I have this data all together I can move forward.
Post a Comment
<< Home
Yep, this is why you get paid the big bucks, my friend.
I particularly like the Lines in the Sand feature. Widely accepted/understood benchmarks are important.
One quick note that you may have already been planning about commenting on in a later post:
But if I tell you I'm thinking of a guy who scored 7 goals in a full season, you know that he's lousy offensively, regardless of circumstances.
1) Sometimes a player plays injured, and that can affect his counting numbers.
2) Sometimes the coach uses a player for a role when the player could be producing much more on a different line (Pisani). It's a matter of where the player is most effective with respect to the team.
Yes, this likely happens much less often than your other examples, and it might be right to push them into the margins, but I think they're good arguments as to where that particular stat falls apart.
12 or 15 goals, sure. 7... dunno. Find me an example of someone who played a full season and scored 7 goals -- with the exception of an 18/19 year old seeing limited ice time -- who is recognized as a decent offensive player, and maybe I'll change my mind.
Matt, if you're going to ask for feedback, you might want to give it some thought before dismissing it out of hand. Does Glencross mean anything to you? You know, before he went to the Oilers and Calgary bought him for the 2nd line?
Yes, this likely happens much less often than your other examples, and it might be right to push them into the margins, but I think they're good arguments as to where that particular stat falls apart.
...
Matt, if you're going to ask for feedback, you might want to give it some thought before dismissing it out of hand.
Agreed. This stat is not The Single Number By Which All Hockey Players Must Be Measured. That, of course, isn't even the right test to apply. The test is: does it tell us more about a player's offensive abilities than G-A-P. I honestly can't even believe that there would be a debate about such a thing.
# posted by 
: September 11, 2008 3:43 pm
: September 11, 2008 3:43 pm Ok, I think I'm be horribly misunderstood. I think it's relevant to point how strengths AND weaknesses of tests.
Wow, I must be insane.
Glencross? That's the counter-example?
I mean, I had to go look it up, but that guy scored 7 goals in 45 NHL games before going to the Oilers.
Were you thinking that he was somehow on pace for 7 goals in 82 games?
Matt: Outstanding! Thank you.
Only one quibble:
roughly speaking, every minute you spend killing a penalty is a minute when you're not going to be scoring.
Some players do score in that situation. I think shorthanded scoring tends to be overlooked as irrelevant or irreproducible, when actually it's an important component to some players' games. So rather than exclude one category I consider the following six-pack:
ESTOI/60 | PPTOI/60 | SHTOI/60
ES Pts/60 | PP Pts/60 |SH Pts/ 60
... and guys like Marty St.Louis and Brian Rolston who put crooked numbers in every category can be truly appreciated for their all-around games.
What's a line in the sand? Last year there were 179 forwards who achieved 1 minute SH TOI/GP in at least 40 GP. Here's the distribution in SHP/60:
8 @ 2.00+
7 @ 1.50-1.99
34 @ 1.00-1.49
44 @ 0.50-0.99
39 @ 0.25-0.49
47 @ 0.00-0.24
suggesting the following lines in the sand:
1.50 elite
1.00 positive
0.50 Mendoza Line
As for defencemen, there were 142 who met the same criteria:
1@ 1.00+
19 @ 0.50-0.99
44 @ 0.25-0.49
78 @ 0.00-0.24
... suggesting 0.50 is elite and 0.25 the Mendoza line.
All data courtesy of the wonderful Behind the Net, needless to say.
@earl sleek
The stats on yahoo have him at 6 over that span, not 7, but it's a moot point. I personally think that you're assuming that his only playing 36 of the first 56 games is unimportant, especially when you're talking about a 25 year old player. We can ignore the reasons why he wasn't on the ice if you'd like, and we can ignore that it took him until 25 to crack the nhl in anything resembling an offensive player.
Unfortunately (or fortunately for you, whichever you prefer), at that point, I stop talking because when people ask for feedback, they don't want it, and we have the "Stats guys" once again disregarding anything they don't agree with, and treat people with contempt for not agreeing.
Wasn't the point to try to look at the sport as though it was definable? Like a science? Good luck with that.
Matt,
Very nice post. I am a statzi by nature (what hockey obsessed engineer isn't) and I think this series is a very good idea.
I'm not sure what your intent is, but if it is to hold the interest of a non-statzi and possible convert them, you would probably do well to eliminate some of the comments like "and to my mind, better". It left a bit of a bad taste in my mouth, and I agreed with you going into the read.
My 2 cents as I look forward to the next in the series.
we have the "Stats guys" once again disregarding anything they don't agree with, and treat people with contempt for not agreeing.
Dude, you are looking for contempt in a thread that's barely 10 comments old. There is none. People are allowed to disagree with you, however.
Wasn't the point to try to look at the sport as though it was definable? Like a science? Good luck with that.
We're talking about probability, not absolutes, no? If GF/GA matches actual standings 9 times out of ten, we're on to something, but there will always be outliers and exceptions.
But if I tell you I'm thinking of a guy who scored 7 goals in a full season, you know that he's lousy offensively, regardless of circumstances.
@ Ender
I have trouble following your comments then. I thought you were bringing up Glencross as part of your criticism of this "full season" line.
If you had another point with Glencross, I'm not quite sure what it was, but I'll give you the benefit of the doubt.
I have to say that was an excellent start, just the kind of "Hockey Stats for Dummies" I've been hoping to see, both my own edification and for others.
Myself, I got the most out of the "Lines in the Sand," feature.
P.S. Just talking to vet sports columnist John Mackinnon and he tells me that NHL teams have long been using all kinds of advanced stats to judge their players. So my notion about that was incorrect.
P.S. Just talking to vet sports columnist John Mackinnon and he tells me that NHL teams have long been using all kinds of advanced stats to judge their players. So my notion about that was incorrect.
I'd like to know what advanced means here. I've heard some stuff about arbitration systems and, uniformly, they sound pretty asinine. You'd think that there if was advanced stuff that valued a player's role properly, it would show up there.
Also: Can someone explain what's meant by the Glencross reference? I can't figure it out. I'm not all sure what Ender means by that. I also can't figure out what he's talking about in terms of the weakness in the stat. Is it a weakness compared to traditional G-A-P that he's talking about? Does anyone understand this objection?
# posted by : September 11, 2008 4:22 pm
: September 11, 2008 4:22 pm That should read "arbitration submissions" and, from what I've heard, they're painfully stone age.
# posted by : September 11, 2008 4:22 pm
: September 11, 2008 4:22 pm I've heard some stuff about arbitration systems and, uniformly, they sound pretty asinine.
I have yet to hear a GM refer to something like PTS per 60, or rarely things like his value at even strength...
@mike w
I think my issue is that me saying "it'd be a good idea to point out the limits of what that stat says" met with the fight that it did. Some wording of posts, noteably Matt, Tyler, and Earl (though retrospectively, I think I misread Earl) hit me the wrong way. I'll grant that I overreacted, but I think the main point still stands: if you're going to put the advantages, put the limits.
@earl
I realize that I wasn't clear. Let me put it this way. Glencross, at 25 (not 18 or 19), couldn't string together a "full" season. He played 36 out of 56 games, which, had it continued at that rate, would be about 52 games played in a full season. His 6 goals would forecast to 8 goals. It's not a perfect example, obviously, but I think it's a valid one.
@mc79
I'm basically just saying that the margins exist, and in order to make any stat better you need to recognize that, and work with it, not just say "it's in the margin, so it doesn't really matter anyway"
And before somebody jumps on it, I'll add that I'm looking at his stats from this season, with columbus. not full nhl record.
Don't point the finger at me Ender. You were complaining that Matt was dismissive of your point about before I said a single thing in this thread.
And I might be tired, or stupid, or something, but I still can't figure out what your point is. He had 7 goals in 45 games before coming to the Oilers. Maybe there was a bunch of press when got picked up saying that he was an offensive player but I must have missed it. How does he fit the parameters of what Matt was talking about?
FWIW, I don't think that anyone would disagree that the stats might not tell the whole story on the guy and that information like who he was playing with and whether he was injured is relevant in understanding what can be expected going forward. I just don't think that Glencross is a guy who supports your point.
# posted by : September 11, 2008 4:50 pm
: September 11, 2008 4:50 pm Ender, I meant to put a note at the end to the effect of, "I'll talk about context next time, but this seems like a logical place to start." Sorry if I sounded snarky before: of course context and injuries matter.
As to the idea that 7 goals in a season means lousy on O, your counter example does not meet the criteria.
# posted by : September 11, 2008 4:53 pm
: September 11, 2008 4:53 pm Ok, so just so we're clear, it's ok for you to ignore the reasons why he didn't play a full season. It's ok for you to ignore him being in the pressbox. It's ok for you to ignore his age while looking at his stats. It's is not ok for me to project a single season had he continued playing for Columbus for the amount that he was for that season?
And you're telling me that "This stat is not The Single Number By Which All Hockey Players Must Be Measured. That, of course, isn't even the right test to apply." is not derogatory at all? Seriously?
Look. My point is the margins exist. Period.
But enough about that. It may be useful to me, or people like me if someone defined what exactly you're looking for. Are you looking to understand the game better? Are you looking to find better ways of evaluating your team? Are you looking to show how the on-ice realities don't reflect counting numbers? Who's team is the best? What, exactly?
@matt
It does not meet the criteria. I see. And what exactly is the criteria again? And why is the criteria the criteria? what age group does a player need to be in to meet your criteria? How many games is a full season? How many years does the player need to be in the league for? Is there an exemption for players on shitty teams?
I'll come out and admit that I don't understand any of what's going on in the comments here, so maybe I'm way off, but: why are we arguing about a more-or-less throwaway example used to explain a larger idea about reference points?
why are we arguing about a more-or-less throwaway example used to explain a larger idea about reference points?
Believe it or not, I'm wondering the same thing myself.
And thanks for doing this Matt.
I think by far the biggest problem with respect to the new stats is that most people don't understand them, and if they do, the numbers are meaningless to them because they have no reference points for comparison.
Actually explaining what these stats are and how they can be interpreted will help immensely with respect to increasing their acceptance by the hockey community. As of right now, I wouldn't be surprised if 75% of the people who read a post at mc79hockey have no fucking idea what Tyler is talking about. And that sometimes includes me, someone who has been bugging the shit out of you guys since day one.
Given this, I'm convinced that far more people will start embracing these new stats once they fully understand what they mean, so I appreciate that Matt is taking he time to do this. Godspeed.
Apologies to anyone who read my post before I deleted it. On reading it over, it seemed a bit overly caustic.
Someone message me if people are starting to play nice, because if you post after this note, I won't be back to read it. Not out of spite - there's just not really a point.
What did you assholes go and do to Ender?
P.S. Matt do you think you could make an exclusion for Gilbert Brule in the goal scoring line in the sand, because I sincerely and emphatically believe that he was mishandled in Ohio, and I don't feel it's fair to blemish his reputation here and elsewhere for anyone new to these concept.
Maybe just an asterick with "excluding Gilbert Brule" beside it. Thanks.
Mackinnon is talking about things like scoring chances and the errors stat, and about tracking time on ice. I don't know if these are considered "advanced" or not, as opposed to say Desjardins', who starts with regular plus/minus and then makes new stats out of it, such as qualcomp.
More explanation may well come from Mackinnon, as he had some interesting stories, which I would now share, but I encouraged him to write a blog post about this himself, and he may well do so.
When I say "tracking time on ice," I mean he says the Habs were doing this back in the early '80s, that they had a stats person with a stopwatch for every player, and when they would come on and off the ice, that stats person would click their watch. This was an example of his that NHL teams have been doing some interesting things for some time, that this isn't all that new, which was news to me.
I'm all for profanity, but you can't tell someone to f-off here. Sorry. If you don't like it, f-off.
Also, I prefer that the phrase Godspeed be reserved for important things, like bombing missions and beer runs.
Here's a serious question. Why did 60 minutes get chosen as the base? Just because it adds up to a "game"?
The thing that makes stats like ppg so appealing, despite their obvious flaws, is that they are calibrated to a meaningful scale for the viewer.
For each game, I can think "Hey, XXX has a 1.25 ppg scoring average, but hasn't put up any points in the last 4 games, what gives?".
With a base of 60 minutes, no matter how useful the stat is, it just isn't easily convertable to an immediately relevant scale. "Hey, according to Fenwick, Dvorak has dropped from a 3.2 to a 2.8 EVPTS/60. That really sucks -- at least I think it does -- let's see, how bad is it, well -0.4 over 60 minutes means he hasn't scored in, wait, how many minutes per game does he play again, ahhh fuck it."
I'm not sure there is any real solution, but there it is.
The thing that makes stats like ppg so appealing, despite their obvious flaws, is that they are calibrated to a meaningful scale for the viewer.
For each game, I can think "Hey, XXX has a 1.25 ppg scoring average, but hasn't put up any points in the last 4 games, what gives?".
That's a very good point. I suppose it could be based on 15 ES minutes a game or the median or whatever. The thing I like about 60 is that it does seem to have a couple of natural breaks: over 3.0 is awesome, 2.0 is good, 1.0 and it's "Fuckin' Chrysler plant, here I come."
# posted by : September 11, 2008 10:04 pm
: September 11, 2008 10:04 pm I like the per 60 metric myself: what does the player produce per full game of ice time? It's easy to calculate if it takes him 4, 5, or 6 GP to actually achieve that ice time based on his TOI; the whole point is to level that playing field between players of different opportunity levels.
It's particularly valuable to use per 60 in the GF/GA ON category, where the numbers relate to actual game scores, goals against averages and other familiar standards. (Not perfect due to the ES / PP dichotomy, but close enough to be useful.)
mc79 (or someone else),
I seem to recall that you have the percentiles for scoring rates stored away somewhere in a chart for the last several consecutive seasons. I think it would be good to take an average of say, the last ten or so seasons and express them in terms of percentile. So if someone is scoring 2.89pts/60 we could know that he was in the 94th percentile or whatever it would be. Seems more helpful and precise than just using round numbers.
Not a bad idea. My only counterpoint is that it fluctuates with leaguewide scoring levels: see here.
# posted by : September 11, 2008 11:08 pm
: September 11, 2008 11:08 pm I can see the appeal of showing EV results as per 10 minutes, where (A) it would resemble BAA or OBP a bit more (.199 is the suck, .400 is excellent), and (B) the duration is in the range of what someone actually plays in the game.
The downside (or at least, the one I'm thinking of) is that those natural things don't translate to the PP at all: not the TOI per game or the figure. And I'd rather have PP and EV rates expressed in the same terms than not. I'll pretty much second Tyler's comment.
As for Scott's, while there's no question that's more rigourous, I think a lot of the previous comments (and my own inclination) say, No Thanks. Round numbers are handy. Really, who was in the 94th percentile in Goals this season?
That said, I think there's no doubt that going backwards will help re: acceptance. I doubt anything ever sold the OPS stat better than the fact that someone figured out that Babe Ruth had the best season of the 20th century by that measure.
Here's the other thing about the much maligned PPG statistic, and just one way that folks using the EVPTS/Game need to be careful about their terminology.
EVPTS/60 mins is often phrased as a measure of "productivity". But I think that's misleading.
Yes, of course, PPG is dependent upon how much ice time a player gets. But I see this as a feature, not a bug. Players are not capable of playing the same amount and their ice-time reflects this.
Sure, somebody who produces 2.0EVPTS/60 mins by playing only 2 mins a game is equally "productive" as someone who produces 2.0EVPTS/60 mins who plays 15 mins a game. But at the end of the day, the guy who plays 15 mins a game puts up more points on the scoreboard.
Given that there is no guarantee that the 2-minute man would be able to keep up his rate of production were his ice-time bumped up (in fact, I'd argue that the reason his ice-time hasn't been bumped up is because he isn't capable), I find it hard to call the two players equally "productive".
By not controlling for ice-time, the ole-fashioned PPG does take into account just how much the guy can actually play and how much he is actually capable of producing at a meaningful scale of reference (i.e., a game).
Now I know that the EVPTS/60mins is not intended to be a measure of "productivity" so to speak, and is only intended to be a measure of the "rate of productivity per minute of ice-time", but if that's the case, we all need to take care to spell this out and not use the shorthand of saying Dvorak's absurdly high EVPTS/60mins demonstrates he is highly "productive".
I think you're half right...
For starters, the raw numbers of goals and assists are the building blocks; not going anywhere. So a forward who can play 25 minutes a game will generally score more goals and get at least his fair share of credit (e.g. Lecavalier).
By not controlling for ice-time, the ole-fashioned PPG does take into account just how much the guy can actually play and how much he is actually capable of producing at a meaningful scale of reference (i.e., a game).
I think 'can' and 'capable' are misplaced there. I suppose this is one of the points of this whole exercise; I don't want to take the coach's word for it as to who is 'capable' of what, I'm interested in results.
What I would say needs to be acknowledged somehow is that we don't know what the point of diminishing returns is for ice time (how much is too much), it's different for different players, and there's plenty of reason to believe that (e.g.) the Tampa forwards have better totals but worse rates on account of them being on the ice the entire game.
I can certainly understand the confusion with the per 60 metric...it can also lead to a misleading inflation/deflation of the stat in question. I remember a post at Tylers site where he pointed out that Lupul was getting outshot at ES last season at a rate of 8 shots/60 which looks pretty terrible at first glance, but only works out to about 1.5shots per game with Lupuls actual ice time.
The reasons that I think the percentiles will help are:
1. People are accustomed to using percentiles all through their lives and it's easy for most people to understand their value. I think that they will help people to decide for themselves what round numbers are good and bad. Most people know that 30% is terrible, 60% is mediocre, 80% is a pretty good mark, while 95% is outstanding without anyone needing to do any explaining.
2. It really is more specific and effectively compares people to their peers automatically.
3. I think it will help those of us familiar with the stastic get a better idea of what is good and how good it is, as well as what is bad and how bad it is (1 pt/60 is around the 20the percentile. Is that an accurate reflection of the "Mendoza line").
4. As for the fluctuations, that is going to happen, but placing people into a range of ten years worth of compiled data would help to situate each individual's performance in historical context. Plus, it's not really an issue that's taken into account by the current system either (i.e. how good is 3EVPts/60 if 25 guys do it instead of 5? I think there's the same problem either way.
I also liked Bruce's suggestion that short-handed points be included, but again, I think it would be better to express them by both a number and a percentile. Especially here it seems like people are flying by the seat of their pants w.r.t. what is considered good.
I agree that the per sixty minute format is good. People already measure a lot of things "per hour" so it seems like there would be a veneer of familiarity, and as Matt said, part of the purpose of this is to be different from G-A-P by taking out some of the influence of the coach.
Thanks for doing all of this work Matt!
OM's comments about Lupul are part of what I'm talking about. If getting outshot by 8/60 is only 1.5 shots per game then maybe it's not that bad? If we could also say, he was in the negative by 8shots/60 and that this is in the 17th percentile (or whatever it would actually be) I think that it would demonstrate more clearly how terrible that is.
While I'm not sure if there's any point in making this comment now, I figure I might as well.
First, let me say that I like a lot of the stuff done here, contrary to recent evidence. The situational per-60s are a repurposing of existing data to try to account for context at the most basic level, and it seems to work quite well. The continuing search for context -- Desjardins quality stats, faceoff zones, etc. -- further helps quantify the sort of player we're looking at, and the sort of circumstances they are good (and not good) in. On the other hand, there's some stuff here that really bugs me, like the continuing unwillingness/inability to separate sportscaster cliche/crutch from genuine sport science, and the general hostility towards new ideas from outside the 'sphere and its influences. I also think some of the stuff Vic's posted lately is kind of crazy, but that's a subject that was beaten to death in the previous thread.
Now, I consider Ender to be a close friend, and have for some time, I was in his wedding party this past spring, all that good stuff. And as a general rule, he's a very reasonable man. He's got a philosophy degree and was only a handful of courses from a computer science degree, so really, if anyone understands logic and math, it's him. I do think he raised some valid objections and questions, both here and at CinO, and I don't know that many of them have been addressed all that well (or even at all). I am, however, surprised at how badly his ideas and counterpoints, and those of others like me, Bruce, and Staples, have been received. That being said, I talked to Ender on MSN while this thread was ongoing and I have to admit, I was totally baffled. I didn't get where this hostility was coming from, when I thought everything Fenwick and the others had posted here was fairly reasonable. I finally just gave up around 5:00, which is also when I gave up on trying to jerry-rig a wireless connection for my brand-new Xbox 360 and just play a bloody game. (Incidentally, he sent me a better example of what he was getting at this morning: Dan Cleary.)
But the way I see it, everyone has bad days. Everyone has their triggers for shit that sends them up the wall. For me, ironically, one of them is all the Goddamned hostility that the stats vs. no stats "debates" kick up, and that makes me as guilty as anyone about this. But really, is there any need for all the baiting and sniping and condescension and ridiculous nonsense? Not everyone who objects to the Oilersphere way of thinking is like the people who drove this group off HFBoards three years ago. Not everyone who looks at numbers other than the basic counting stats is a stuck-up twat with one eye on the TV and the other on a laptop. Just as there was no cause for Ender to go off here yesterday, there was no cause for Dennis to go off on Bruce for the fiftieth fucking time on CinO a few days ago. There's just no need for any of it. There's nothing wrong with healthy debate, but this here? This has not been healthy debate.
I've had the same issue that Sacamano has with EV/60. It's just not a very elegant or easy to understand way to describe point production.
I've been reading this particular way of expressing stats for almost a year now, and it still seems alien to me.
But is there a better way to do it?
Could we not describe this same idea on a point per even strength minutes basis?
Such as, Hemsky gets one point for every 14 es minutes, while Stoll gets one every 66 es minute.
In short, it would be expressed as Hemsky 1/28 p-esm, Stoll 1/76 p-esm.
For the power play, Hemsky 1-7 p-ppm, and Stoll 1/13 p-ppm.
Most players play about 10-15 minutes of even strength play per game, and most fans know this, so if the stat is expressed this way, the average fan can quickly figure out how far away a guy is from scoring one point per 15 es minutes.
I don't know if this improves things or not . . .
And, of course, most folks get these EV/60 stats from Desjardins, do they not, so unless they want to do their own calculations, this issue is really up to him.
It has been a difficult debate, for sure, doogie2K, marked by a lot of hostility and defensiveness.
Perhaps, though, with Matt's honest and genous attempt to work through some of the basic questions and issues about these new stats, the tide is turning.
I've interviewed hundreds of scientists and researchers over the years, and found almost all of them to be helpful, even as I've pushed them to describe their complex work in terms that a layman like me can comprehend. Almost to a person, these scientists and researchers have been extremely helpful.
On the Oilogosphere, for whatever reason I'm not sure, this attitude of happily explaining your ideas to others isn't the general order of business. Instead things can often be combative, hostile, childish. It has partly to do, I think, with the nature of the Internet, and the fact we don't see each other's faces and don't hear one another's voices. We have little idea of how we're coming across, what buttons we might be pushing, what buttons are being pushed on us.
I know I lost it myself in one debate simply because a few buttons of my own got pushed, and I saw red for about a half hour.
In any case, this isn't an easy placed to discuss ideas. And it's going to take no small amount of work to get past that problem.
But some steps are being taken.
I see I wrote "genous" in the last post.
That, of course, isn't a typo, as I never, ever make typos.
It is a deliberate combination of two words, genius and generous. ;)
Bah, D2K. You're going to make me comment again. Again, as I said before, I apologize if I overreacted. However, I still don't think that I was jumping at nothing. As DMFB said, I presented a throwaway example to make a point about weaknesses of the stat. I thought it would be a nothing comment, warranting little to no reply aside from a "yes, the margins exist and we should probably say something about them too if we're trying to be thorough." That was all. Things didn't work out that way, and I got really frustrated, as we ran into an issue with onus of proof.
It seems that on this blog, in order to be taken seriously, you need to present proof. Proof involves stats. I'm not sure why that is when you're not trying to argue for something, but rather against the exclusion of something else, but whatever. Moot point. Like I said, I thought a throwaway example would be enough, as I do still think Glencross fits the bill, but it was unaccepted, and somehow the idea of "don't exclude things" was lost. Dan Cleary is probably a better example, but as Matt said, Glencross didn't fit the criteria. Would Cleary?
The issue is that people seem to use stats when it helps them, ignore stats when they either disagree with them or it involves more work. No criteria here are ever defined rigorously enough to not have those built in loopholes in them, and they're exploited. I've been reading blogs on the sphere for years, and as D2K said, I know my math and logic. I still have no idea what anyone is really trying to get out of the stats that they're using, and it frustrates me how cavalier people are being about blatantly disregarding information. Especially when those same people are saying that the people who disagree are being intentionally ignorant.
Look. None of this is meant as an attack. I've been told that I came across really poorly in this thread, in an attacking way. Text is funny that way. it was never my intent. Like I said, I over-reacted to some, and probably wasn't watching my words as carefully as I should have. This post is not intended as an arguing point. It's intended as a something similar to a confession/apology. Forget when people play nice, I'm not going to be in again, period.
I thought it would be a nothing comment, warranting little to no reply aside from a "yes, the margins exist and we should probably say something about them too if we're trying to be thorough."
I still can't understand what is meant by this comment about the margins. I swear to god that I'm not trying to be intentionally stupid, but I haven't the foggiest as to a) what it means and b) what the point is.
# posted by : September 12, 2008 1:10 pm
: September 12, 2008 1:10 pm What I would say needs to be acknowledged somehow is that we don't know what the point of diminishing returns is for ice time (how much is too much), it's different for different players, and there's plenty of reason to believe that (e.g.) the Tampa forwards have better totals but worse rates on account of them being on the ice the entire game.
Precisely.
I've actually occasionally cited scoring rates as points per X minutes for the sake of greater understanding. One point for every ten minutes of power-play time for Ales Hemsky is a lot easier to understand than 5.93 PPP/60.
I still can't understand what is meant by this comment about the margins. I swear to god that I'm not trying to be intentionally stupid, but I haven't the foggiest as to a) what it means and b) what the point is.
I think he was just trying to say that there are always exceptions to the rule. Dan Cleary had a number of 3-7 goal seasons before the lockout, but has since put up some real solid numbers. There were extenuating circumstances behind it, but clearly, he wasn't a bad offensive player, full stop: he just needed to change his circumstances.
Incidentally, I love Andy's post, because it advocates a lot of the things that the objectors have been talking about for the last week, but have been dismissed for one reason or another.
Just catching up on the past two threads, damn this is hilarious.
Doogie2K may very well have invented a brand new kind of stupid, it's hilarious. Seriously, why has nobody ripped him apart, were you waiting for me. It's comic gold.
And seriously, does Staples genuinely not realize what he has been measuring here? Has chubby undergrad and avid video game player "clutch" Doogie not understood his own words? It's not possible that he could have.
Is pathetisad a word?
Mr. Fenwick.
Permission to attack, sir!
I think the other thread might be dead by now, but I thought it was relevant, for anyone who deosn't feed of every blog, that Brownlee responded to some of the recent debates about stats.
http://www.oilersnation.com/index2.php/2008/09/12/by-the-numbers/#more-2760
re: SH scoring. The post-lockout year to year correlation in SH/60 is below .20 so it is almost random league wide at the player level. That said there are handful of people who do score SH such as St. Louis and Madden but they are outliers.
Someone asked to see where the percentile rankings fell over the last several seasons. So here we go.
FORWARDS 90%
ES/60
1998 2.35 (S. KAPANEN)
1999 2.38 (ELIAS)
2000 2.44 (S. RICHER)
2001 2.54 (CASSELS)
2002 2.34 (ALFREDSSON)
2003 2.39 (ZUBRUS)
2004 2.35 (RYDER)
2006 2.69 (STILLMAN)
2007 2.55 (HORTON)
2008 2.32 (ANTROPOV)
DEFENSE 90%
ES/60
1998 0.97 (ZHITNIK)
1999 0.98 (S. COTE)
2000 1.10 (J. NORTON)
2001 1.06 (GONCHAR)
2002 0.96 (DESJARDINS)
2003 1.00 (MARA)
2004 0.99 (EHRHOFF)
2006 1.21 (JOVANOVSKI)
2007 1.14 (MESZAROS)
2008 1.02 (PHANEUF)
Are you going to keep up the ad hominem bullshit or do you have a fucking point, Vic? I'm really not interested in arguing unless you have something substantial.
Scott:
I finally put all this together this summer.
The more recent years are easy, you can just copy and paste stuff off of NHL.com, or if you know how to write code make a script (I can't do that yet).
The older years involved time consuming manual data entry from either old editions of The Hockey News Yearbooks or using the NHL.com super stats machine. I'm still missing data on a couple of players.
It is pretty sweet that we know have this many seasons of data to look at. My next step is working on individual players projection models-now that I have this data all together I can move forward.
Post a Comment
<< Home
