Wednesday, September 10, 2008
Me? I'm not the least bit interested in the opinions of someone who refuses to learn anything new
I'm actually surprised at how kind and measured the reaction to David Staples' post about stats has been. Maybe it's just me: this type of piece -- that purports to soberly assess competing viewpoints about hockey, while mischaracterizing one side near-totally -- bugs me more than a straight-out blast about guys in their PJ's in Mom's basement, calculating away. I'd characterize the "debate" a lot differently, and I don't think it's any less fair:
On the one hand, you have a bunch of guys who love hockey and watch every game. Beyond that, they are intellectually curious, and seek out or create ways to numerically assess what is happening in the game as well as possible. Some of these ways will affirm their perceptions, some will challenge them, and some will prove to be so discordant with apparent reality that they get tossed out.
They will find that certain metrics mesh best with what they see, so that when (say) UFA season comes up, and most of their team's potential additions are guys they haven't seen play near as much as they would like, they can look at how those potential additions fare based on said favoured metrics.
They look at something like traditional +/- and see a flawed statistic, so they seek out ways to mitigate the flaws: take out EN and SH goals, try to quantitatively assess whether they were playing against tough or soft competition, try to quantitatively assess whether they had decent linemates, and look at whether they were starting the majority of their shifts in their own end, or the other guy's.
On the other hand, you have a bunch of guys who love hockey and watch every game, but simply do not have the curiosity, humility, and/or stomach to have their perceptions challenged. They look at something like traditional +/- and see a flawed statistic, so they decide that being on the ice for goals for and against is meaningless in and of itself. And when UFA season comes up, their opinion of potential team additions is based on the handful of times they've seen the guy play, or on those same statistics they claim to hate/ignore, or on what Darren Pang and Nick Kypreos say about the guy.
This debate will rage on, surely! Dave had a good response, I think, particularly regarding this idea that the stats guys don't watch (or is it appreciate) hockey as much as the traditionalists on account of their statsyness. Garbage. Just as galling to me is the attribution of so much certainty on the part of the "stats guys". This despite the near-constant acknowledgement by the likes of Vic, Gabe, and Tyler that nothing they are doing is definitive, and they are not claiming to be the biggest hockey geniuses in the world.
I think this quote says a lot:
That's Vic Ferrari, in the comments (#35) at mc79hockey.
I also have a factual quibble with Staples:
This is just false. While we don't have a comprehensive idea of what NHL teams are doing statistically (they're about as eager to share that stuff as they are injuries in the playoffs), there are anecdotes from all over the place that they are doing a lot more than what is reported. Marc Crawford says they tracked scoring chances for & against all year with the Kings; the Wild's director of hockey ops was hired away from the Red Sox; Tim Hunter had a frickin' laptop behind the Sharks bench!
Junior/amateur scouting will no doubt mostly remain a matter left to two-eyes-and-a-gut (and as Lowetide has pointed out many times, the success rate of NHL teams as a whole is quite commendable). But clubs are definitely doing a lot in terms of quantitative analysis of their own teams (efforts at in-season improvement), and as they continue to discover and settle on analysis that proves effective -- predictive -- they will further apply it to pro scouting.
(Also, since it's all speculation anyway, I'd guess that the biggest problem Gillis had this summer achieving what he wanted to achieve was not that the other GMs all want to screw over the new jerk. It was that a lot of this breakthrough he thought he could make -- unappreciated value he thought he could exploit, by looking at things a new way -- turned out to be ground that is already being covered by 10 or 20 other teams.)
Now, back to my spreadsheets and my fantasy baseball league.
Coda: Pensee du jour, per Vic -- what if the fact that hockey has more numerous and complex interactions than baseball makes it more statistically predictable, not less?
Food for thought.
On the one hand, you have a bunch of guys who love hockey and watch every game. Beyond that, they are intellectually curious, and seek out or create ways to numerically assess what is happening in the game as well as possible. Some of these ways will affirm their perceptions, some will challenge them, and some will prove to be so discordant with apparent reality that they get tossed out.
They will find that certain metrics mesh best with what they see, so that when (say) UFA season comes up, and most of their team's potential additions are guys they haven't seen play near as much as they would like, they can look at how those potential additions fare based on said favoured metrics.
They look at something like traditional +/- and see a flawed statistic, so they seek out ways to mitigate the flaws: take out EN and SH goals, try to quantitatively assess whether they were playing against tough or soft competition, try to quantitatively assess whether they had decent linemates, and look at whether they were starting the majority of their shifts in their own end, or the other guy's.
On the other hand, you have a bunch of guys who love hockey and watch every game, but simply do not have the curiosity, humility, and/or stomach to have their perceptions challenged. They look at something like traditional +/- and see a flawed statistic, so they decide that being on the ice for goals for and against is meaningless in and of itself. And when UFA season comes up, their opinion of potential team additions is based on the handful of times they've seen the guy play, or on those same statistics they claim to hate/ignore, or on what Darren Pang and Nick Kypreos say about the guy.
This debate will rage on, surely! Dave had a good response, I think, particularly regarding this idea that the stats guys don't watch (or is it appreciate) hockey as much as the traditionalists on account of their statsyness. Garbage. Just as galling to me is the attribution of so much certainty on the part of the "stats guys". This despite the near-constant acknowledgement by the likes of Vic, Gabe, and Tyler that nothing they are doing is definitive, and they are not claiming to be the biggest hockey geniuses in the world.
I think this quote says a lot:
For me at least it’s very little of “these new stats made me look at things differently” and a whole helluva lot of “these nutters on this message board must’ve been watching a different game than me, that’s just crazy”, and then going and trying to find if there is information there to prove it.
People can piss on the numbers all they want, but when I was trying to make the point that Smyth was playing way more against the good players than Moreau, I was almost universally being called a fool (Smyth isn’t a checker, dummy!). Even though I can’t imagine how it could have been more obvious.
That's Vic Ferrari, in the comments (#35) at mc79hockey.
I also have a factual quibble with Staples:
As smart as a hockey statsmeister Gabriel Desjardins is, as innovative as his work at Behind the Net is, it is a long way from being used by NHL GMs. Maybe Mike Gillis in Vancouver will give it a look, but the rest of them will go with their guts and their scouting reports every time.
This is just false. While we don't have a comprehensive idea of what NHL teams are doing statistically (they're about as eager to share that stuff as they are injuries in the playoffs), there are anecdotes from all over the place that they are doing a lot more than what is reported. Marc Crawford says they tracked scoring chances for & against all year with the Kings; the Wild's director of hockey ops was hired away from the Red Sox; Tim Hunter had a frickin' laptop behind the Sharks bench!
Junior/amateur scouting will no doubt mostly remain a matter left to two-eyes-and-a-gut (and as Lowetide has pointed out many times, the success rate of NHL teams as a whole is quite commendable). But clubs are definitely doing a lot in terms of quantitative analysis of their own teams (efforts at in-season improvement), and as they continue to discover and settle on analysis that proves effective -- predictive -- they will further apply it to pro scouting.
(Also, since it's all speculation anyway, I'd guess that the biggest problem Gillis had this summer achieving what he wanted to achieve was not that the other GMs all want to screw over the new jerk. It was that a lot of this breakthrough he thought he could make -- unappreciated value he thought he could exploit, by looking at things a new way -- turned out to be ground that is already being covered by 10 or 20 other teams.)
Now, back to my spreadsheets and my fantasy baseball league.
Coda: Pensee du jour, per Vic -- what if the fact that hockey has more numerous and complex interactions than baseball makes it more statistically predictable, not less?
In hockey we know where it comes from though. The spread of the players away from their career averages will be nearly identical the the spread of a bunch of dice rollers (each throwing the same number of times as one player's shots, and having dice weighted to that player's average EV shooting% over the past three years).
It's just spooky how close the results are, in terms of the right number of guys overachieving and underachieving on the year, and by what amount. Plot it out and it's two bell shaped curves almost on top of one another.
Do that with most baseball stats and they are a mile apart.
Food for thought.
# posted by Matt @ 11:44 am 
Comments:
Beauty post about an absolute dog's breakfast of an article by Staples.
Among the many many many ridiculous assertions was this:
The NHL is now tracking "hits," but the traditionalist knows that what is counted as a hit in Dallas, might not count as one in Montreal. The NHL also tracks "face off wins," but on many draws, it's almost impossible to tell who won, so how do you know if the numbers are accurate?
Except that David Johnson at Hockey Analysis (http://stats.hockeyanalysis.com/) recognizes that fact and has tried to account for it. I imagine he's not alone in doing so.
If anything, the characterization of the traditionalist as too lazy to try to understand anything new is actually accurate. I have tonnes of friends that, in any sport, react to a new statistical measure as some sort of poison. God help anyone that tells them that plus/minus is garbage or that the passer rating is not the be all that they believe it to be.
Gah, I'd probably respond to Staples if I even bothered reading the guy any more.
I know he's been a boost to the Oilogosphere in some ways, but he sure comes off as a very two-edged sword from my (distant) perspective.
The thing that jars, that I didn't quite elucidate in my post, is the stuff about the self-confidence exhibited by the stats guys.
"I'm not the least bit interested in these numbers. I know what I see and I know what I think. I'll go with that over pages of statistics any day."
Have you ever seen the opposite statement made by a so-called stats guy, even half as categorically as Brownlee makes his? And Dellow is the arrogant one... riiiiight.
To use Vic Ferrari as an example:
Years back, on the forums, he was one of the first guys bringing to my awareness the EV+ EV- (with shorties and empty netters) idea as a method of evaluating player ability.
He didn't sit back and boast about how genius he was and hang his hat on EV+ EV-.
He began tracking shooting percentage and save percentage to express whether any of these stats were ballooning or anchoring a player's results in other areas.
He crunched where goals were coming from and shot quality.
As a way to further the cause he began to raise the idea of shot differential, and shots directed at the net, as another method to define a player's ability. In this case, mostly in regard to possession.
All the while the dude has argued that these things should jive with what we see on the ice. As it should.
Every four months someone new DB has to re-raise the question of the validity of these methods and the character of the people who ascribe to using them, in 4,000 word or less. I don't see why they can't just fuck off and accept that a sub-set of people have decided not to buy into to traditional fan interests. I'd rather bash my head in with a hammer than discuss momentum, clutch and big goals from game to game, or how MacTavish should have played Garon form day one of last season.
I want to know who's playing against who, in what end, where they are getting the chances from, who's taking the defnsive/attack zone draws, etc, etc.
That's my interest in the game of hockey. If you can't accept that, why the fuck are you reading places like this? I'm far too disinterested to be bothered to blast people on hockey forums about every ridiculous topic or concept they raise, or how I find their fanatic, obsessive interest in under-age hockey players sorta disturbing. So what the hell is up with people who rail against these stats continuing to post in these parts?
I think a little of the MSM attention might be part to blame. We are talking about a community of aspiring writers, after all. I guess if one David Staples post wouldn't illicit 10 rebuttals across the Oilogosphere there would be no conversation in the first place.
Could someone help me out and correct the grammar in that last sentence there? Thanks.
It's "elicit," not "illicit." Thank you for giving me license to play Grammar Nazi.
Now, I see your larger point, and I don't disagree that certain methods of data analysis are producing useful information. Contrary to appearances, I'm not here just to shit-disturb. I just don't think much of the attitude that comes from these places sometimes.
I'm sorry Matt, but I do find that some "stats guys" can be very closed-minded. Look at how long it took for Bruce's suggestion (and data!) that outshooting doesn't necessarily result in victory to even be half-assed accepted. Hell, look at the continued animosity from Dennis to this day, and it's been almost a year since Bruce has been around. For that matter, I found it pretty damned rich that Tyler could crow about correctly predicting a 20th-overall power play, without really accounting for how vicious the swing was between the early and late part of the year. How very Adam Proteau of him. (I also seem to recall him giving a pretty generous range in the first place, though I could be mistaken. Sorry, but after four months, I'm not spending two hours sifting through every comment thread on LT's blog.) And then there's the near-universal dismissal (or in Vic's case, total intentional ignorance -- irony!) of the Staples Error, rather than trying to make it better or replace it with something that better correlates to winning/scoring/whatever. Oh, and I disagree with your characterization of people who don't like/use the new-age stats: the CalPuck and HF retards might be like that, but I know people who have no use for the new-age stats, but perfectly understand the game itself, the sorts of players you need to win, and the roles they play. The fact that they're not interested in sub-analyzing it is not a sin.
So don't even try to tell me that the stats men can't be arrogant and closed-minded. You're deluding yourself.
P.S. What is the statistical significance of Vic's discoveries? Correlation is lovely, but it doesn't really say much, at the end of the day. Unless I've totally missed something, I don't see any p values related to the null hypothesis, and how he controlled for confounding variables.
Junior/amateur scouting will no doubt mostly remain a matter left to two-eyes-and-a-gut
http://hitthepost.blogspot.com/2008/07/prospect-camp-day-4-evaluation.html
Paragraph #3.
The characterization of the two sides of the debate -- which I think is a misnomer anyway -- was obviously exaggerated deliberately as a counterpoint.
The statistical significance of Vic's discoveries? Considerable, if you're coming at it from his (or the gambler's or the pro scout's) angle of trying to figure out what is most likely to happen in the future. At least insofar as I've absorbed it:
EV+/- is a better predictor of future record than is past record.
Best predictor of EV outscoring is where the (past) end-of-shift faceoffs occurred, followed somewhat closely by past Corsi #, followed somewhat less closely by past EV outscoring.
That shit is nothing to sneeze at. Now, all the while, there are young players getting old and old players getting worse, nagging injuries, d-men that are better at stopping certain types of scoring chances than others, etc. Like most statistical models, there is all sorts of interesting things happening out at either end of the bell (if you're Mario Lemieux's coach do you worry about him getting slightly outshot at EV? not so much). But it sure seems to form an awfully sound basis for what players and teams are headed for success and which are not.
And no, it ain't no sin to pass on subanalyzing anything -- unless you are trading on being an expert on it. I would be appalled at a legal scholar who decided he didn't need to read any more law articles, or a historian who announced he was done reading history books. At the very least, I can't imagine that they would wear their disinterest in learning more as a fucking badge of honour.
As somebody who is by no means a stathead, I found Staples's post far fairer than yours, Matt.
Which makes sense. You are a stats guy. You think you're right. Therefore, everybody who's not a stats guy is wrong, and should probably go back to boning their sisters or whatever it is we do.
But there is more to evaluating hockey using ones' senses than sticking two sweaters in your ears and yelling "LA LA LA" as loud as you can whenever anybody who isn't Don Cherry is talking.
I saw an interesting comment on Covered in Oil recently: the stats we end up accepting are the ones that agree with what we already thought. For example, a stat showing "Joffrey Lupul is an awesome frickin' hockey player" would be thrown out the window because we all know Joffrey Lupul is a waste of carbon atoms. Which, to me, carries the interesting consequence that when we sit down and work the numbers on players, what we see is what we wanted to see anyway.
Worth some thought.
Look at how long it took for Bruce's suggestion (and data!) that outshooting doesn't necessarily result in victory to even be half-assed accepted
What seems to repeatedly go over the heads of some contributers is that nobody once suggested that dominating at any one of these underlying numbers is a guarantee of victory.
We're evaluating who the better team is. I think it's widely accepted that the better team doesn't always win in sports. But jumping on that accepted fact and doing a little dance because an unforseen variable (often luck) won through the day doens't make you right.
Afterall, it is sports, and much of the results are dependant on chance. Something that's widely accepted and echoed the very people who openly ascribe to an interest in "new stats".
But really, haven't we all made these arguments a thousand times before?
For that matter, I found it pretty damned rich that Tyler could crow about correctly predicting a 20th-overall power play, without really accounting for how vicious the swing was between the early and late part of the year. How very Adam Proteau of him.
I don't think that I was "crowing" about predicting a 20th overall power play. If I have the right LT thread in mind, I'm pretty sure that I was pointing out that, for as much as AO wanted to kick the stats guys for not expecting the OIlers to have an unbelievable amount of shootouts AND an unbelievable amount of shootout wins, the team still sucked.
(I also seem to recall him giving a pretty generous range in the first place, though I could be mistaken.
I think that this all stems from this post. I wrote:
If I had to guess, I’d guess that Edmonton ends up slotting somewhere between 15th and 20th on the PP. As you can see from the chart above, it’s basically a cluster outside of the extremes and that’s going to have a minimal effect in the grand scheme of things.
If you look at the chart in that post, I also made the point that, outside of the real extremes, there really isn't much difference between team's PP abilities. For example, if you look at the chart that I posted, you'll see that, when the 2006-07 PP's are ranked by goal differential, there was a difference of all of 7 goal differential between the number 26 PP and the number 13 PP. I was up front with all of that when I wrote my post - the information was there for anyone who wanted to see it. I certainly didn't claim that there was much of a substantive difference between 15th and 20th. If you think that that's a wide range or that if there was anything to closely examining this stuff, it could be predicted more closely than that...well, I'm sorry but I think that if that's the standard you're applying, nobody's actually able to meet it.
I still haven't taken a close look at the Oilers 2007-08 PP. We'll see what it looks like when I do but having taken a very quick look, it seems to me that they weren't able to buy a goal during the bad period and seemingly everything went in during the good period and that the shots and all that didn't really change too much.
True Fact: Of the bottom 15 teams in PP shooting percentage through game 615 of last season, 13 of them improved in the second half. Of the top 15 teams in PP shooting percentage for the first half of the season, 12 of them got worse, one stayed the same and the two got better (Pittsburgh and Carolina, for those scoring at home).
Edmonton was of course, Very Bad during the first half of the season at shooting percentage on the PP. Seventh worst in the league. In the second half, they led the league, while other teams fell back to earth.
The PP is a volatile thing. Shooting percentage seems to matter a lot and, last year at least, it was pretty damned volatile. All of the Oilers fans with hardons for how awesome the PP is going to be this year might want to consider that before concluding that the bad games don't count and that it'll be at the top of the league this year. Not saying that's you Doogie2K and I'm not really making any comment on what I think the PP will be yet; just saying that, while the Oilers had a huge swing last year from the first half to the second, they weren't alone and as tantalizing as it is to to take that the second half resurgence, tie it to other things that we know happened and assume that they're causative (Souray's return, Gagner doing well), I'm hesitant to do so.
I know the one you're talking about LB, but I don't think that's quite what was meant -- it was worded inelegantly.
If you're coming up with a new player ranking of the 07/08 Flames and it doesn't show that Iginla >> Nolan >> Godard, then it's garbage. But if you have one that does show that, and it also shows (say) Conroy surprisingly high (to you) and Lombardi surprisingly low, that's not a reason to throw it out... There has to be a reasonableness test. (Whole concept is obviously not limited to hockey or even sports.)
For example, a stat showing "Joffrey Lupul is an awesome frickin' hockey player" would be thrown out the window because we all know Joffrey Lupul is a waste of carbon atoms. Which, to me, carries the interesting consequence that when we sit down and work the numbers on players, what we see is what we wanted to see anyway.
I think that that arose out of a Bill James quote. As I recall, he was willing to be surprised, but within limits. If the stat says that Joffrey Lupul sucks while guys like Crosby and Ovechkin stink, then sure, I think you throw the stat. If the stat tends to peg other guys where you think that they should be...I think you re-examine Lupul and, possibly, confine future jibes to how he looks so fucking mopey all the time.
You call it mopey, I call it dreamy. He has bedroom eyes!
In all honesty, as an in-between gut and stat guy, I think the 'stat guys' can be a bit obnoxious which comes off a bit as arrogant. I don't think it's so much 'if you're too stupid to see how right I am, you're not worth talking to it,' but closer to 'goddammit, I'm not arguing about the validity of EV/60 as a predictor again, piss off.' Yeah, it's a bit dickish, but you can't blame them for not wanting to retate Stats 101 when they're trying to write their thesis.
And I think gut guys can be a lot more arrogant (the Brownlee quote comes to mind). There's a level of disinterest in being sophisicated, so they're going to tell you Souray is worth $5m and it doesn't matter what your fancy stats say. I'm not going to shit on them for having that belief (it's more fun to ignore stats sometimes, particularly when they predict your team is fucked), but if you're going to have that attitude don't come around blogs that are pretty big proponents of them, pick fights, and then refuse to argue on a level above 'did you see that time Souray scored that goal???'
Fantastic post, Matt.
As somebody who is by no means a stathead, I found Staples's post far fairer than yours, Matt.
I'm just as biased here, but I don't think the characterization of this as being between people with intellectual curiosity and the willfully ignorant is really that far off the mark. It's not necessarily nice, but how else do you explain a comment like Brownlee's?
Further, it's not as though we're saying you have to agree that the new stats are an effective way of examining the game or anything: a guy like Brownlee isn't even interested in looking into them, which is something more than just having different outlooks on the game.
I assure you, Mr. Fenwick, I am an expert at the art of inelegant wording.
Anyway, your answer only moves the problem down a level. Any idiot knows that Iginla is better than Nolan who is better than Godard. You could ask a million Calgary fans who can't even add two and two and they'd tell you that. So saying that a statistic is good at addressing the self-evident is nice, but not helpful.
What statisticians are looking for - and correct me if I'm wrong on this, but I don't think I am - is to explain or predict that which isn't obvious; Conroy and Lombardi, as in your example. Saying that because a given statistic can tell the difference between Jarome Iginla and Eric Godard and therefore it can tell the difference between Matthew Lombardi and Craig Conroy is, to me, a considerable leap.
(I'm obviously simplifying, but I'm at work here. :P)
I'm not trying to say that anything that requires a calculator has no place in hockey. When you're looking at a player you're not familiar with, statistics are the best way to do it. "Ah, we got Joe Blow for two first-round picks, but Joe Blow's shooting percentage this year is ten percent better than his previous career best. I will now shoot myself." Stuff like that. And they can provide handy ways to get kernels of information and quantify things that are otherwise difficult to quantify. But they're not The Answer, any more than a hammer is The Tool.
(By the way, re-reading your post, I think you were more aghast at people who say 'lol NERDS' every time EV+/- comes up, and not against - to put it arrogantly - people like I think I am. But, from a quick reading at work, it did come off differently the first time.)
(By the by the way, writing this post was not at all quick. Why yes I am paid by the hour, in fact!)
Also, ex-Pleasure Motors: Robin Brownlee has a brain like a cinder block. Nobody is arguing this. :P
Anyway, your answer only moves the problem down a level. Any idiot knows that Iginla is better than Nolan who is better than Godard. You could ask a million Calgary fans who can't even add two and two and they'd tell you that. So saying that a statistic is good at addressing the self-evident is nice, but not helpful.
Matt's point in this regard was: noting how a stat syncs up with the obvious is a short-hand way of testing a metrics validity. Therefore if it does do that reasonably well then the less obvious findings that emerge in the gray areas and are potentially counter-intuitive are probably accurate also (and of value, if the inquirer is interested in learning something new).
And I think gut guys can be a lot more arrogant (the Brownlee quote comes to mind).
They also happen to be the guys in charge or working in hockey media, so if anyone should have a chip on their shoulder it's the reasonably intelligent fan tired of hearing "What these guys need is a big momentum swing right now..." pass for analysis.
Is it so crazy to make PTS/60 a mainstream stat?
The tone of many people on HF Boards (and media) is almost-literally "Oh no! Stats!," with the same grumpy befuddlement that you might expect from seniors setting up a recording on their VCRs...
Some items, on the post and the comments:
1) When it comes to gut guys/stat guys, neither has a monopoly on arrogance. I've seen plenty of high-handed and obnoxious responses from guys on both sides of the fence - and I'd guess it's because there's very little correlation between willingness to use math and playing well with others. I haven't run the numbers, but that is my suspicion.
2) As for all the knocking of the fellows at HFBoards, lets not forget that there's good content being generated out of there as well. Calling them all "retards" or whatever is in the same range as lumping analytical bloggers in with Eklund.
"The NHL is now tracking 'hits', but the traditionalist knows that what is counted as a hit in Dallas, might not count as one in Montreal."
This from the guy who at one point last season tabulated "defensive points" by adding hits and blocked shots together.
That Staples has come a long way.
Marc Crawford says they tracked scoring chances for & against all year with the Kings;
I just read an article somewhere that noted that the Oilers were tracking EV+ EV- in the AHL this year, as well.
Not to get all Dr. Phil, but I think a real part of the problem has been a lack of respect from both sides of this rather ridiculous disagreement. I'm far from a pure stats guy...I'm not gonna lie, I've seen some probably brilliant articles from Vic and MC that I didn't read because the math bored the shit out of me (and I should add, I've done about 4 million years of school with a PhD and a faculty job to prove it, so I'm very used to being bored)...but I still get pissed off by people like Brownlee who dismiss the stats entirely. The suggestion that the 'stats guys' don't WATCH hockey, or lack insight/understanding of the game, is frankly insulting.
That said, I think some of the "non-stats" guys are reactionary due to the tone of some of these blogs. I've seen some really insulting comments aimed at individuals or entire message boards in these comment sections. Most often these comments are unjustified, hyperbole, and flat out immature, and most often they originate from bloggers and frequent commenters (i.e. members of the blogging community).
Frankly, people take this shit too seriously. I found Staples piece to be condescending, and I find that both sides of this debate show a real reluctance to consider and respect other opinions. It's the internet, you don't always have to be right.
@Matt: I dunno, maybe what Vic is doing is just different than what I'm used to, but none of the posts have had the sort of things I've absolutely had to have in my manuscripts and lab reports in order to not get laughed out of science forever, like those that I mentioned. And if this shit has been done for years already and long since validated, then fine, but when he gets into dice rolls and stuff, he just loses me.
And no, it ain't no sin to pass on subanalyzing anything -- unless you are trading on being an expert on it. I would be appalled at a legal scholar who decided he didn't need to read any more law articles, or a historian who announced he was done reading history books. At the very least, I can't imagine that they would wear their disinterest in learning more as a fucking badge of honour.
And that's fair. I never said Brownlee was right: I think that attitude is absurd. I'm just saying, you're talking down to people who don't care about the numbers, and tarring them all with the same brush, and that's exactly what gets people's backs up. And now that I've said that totally un-ironically:
As for all the knocking of the fellows at HFBoards, lets not forget that there's good content being generated out of there as well. Calling them all "retards" or whatever is in the same range as lumping analytical bloggers in with Eklund.
I was referring to the sorts who would troll Vic, Dennis, Tyler, et al for their stats work, the sort who rip LT for his baseball references. Not the whole board. I'm sorry that wasn't clear.
I don't think that I was "crowing" about predicting a 20th overall power play. If I have the right LT thread in mind, I'm pretty sure that I was pointing out that, for as much as AO wanted to kick the stats guys for not expecting the OIlers to have an unbelievable amount of shootouts AND an unbelievable amount of shootout wins, the team still sucked.
It seemed that way from where I was sitting, and it surprised me, since you had (justifiably) ripped Proteau a new one for his 12th place "success" the previous summer.
The PP is a volatile thing. Shooting percentage seems to matter a lot and, last year at least, it was pretty damned volatile. All of the Oilers fans with hardons for how awesome the PP is going to be this year might want to consider that before concluding that the bad games don't count and that it'll be at the top of the league this year. Not saying that's you Doogie2K and I'm not really making any comment on what I think the PP will be yet; just saying that, while the Oilers had a huge swing last year from the first half to the second, they weren't alone and as tantalizing as it is to to take that the second half resurgence, tie it to other things that we know happened and assume that they're causative (Souray's return, Gagner doing well), I'm hesitant to do so.
I understand that. It just feels to me like the bad period is being considered a bit more indicative than the good period, when frankly, I don't think either say much by themselves, and certainly, are of only limited predictive value for the '09 Oilers, all things considered.
@Dubya: You just said what I've been trying to get at all this time, but continually let my Scottish temper get in the way of. Thank you.
Matt's point in this regard was: noting how a stat syncs up with the obvious is a short-hand way of testing a metrics validity. Therefore if it does do that reasonably well then the less obvious findings that emerge in the gray areas and are potentially counter-intuitive are probably accurate also (and of value, if the inquirer is interested in learning something new).
If only that were true. If a test passes on an obvious comparison that does not logically imply that less obvious comparisons will be even "probably" accurate. It just does not. While it may challenge your opinions, and it may force you to look at things in a different light, you're making a huge jump between obvious and not, and I'm sorry but it's not supported by logic.
Actually, now that I think on it more:
what if the fact that hockey has more numerous and complex interactions than baseball makes it more statistically predictable, not less?
What if the fact that hockey has more numerous and complex interactions than baseball makes it less statistically predictable, not more?
Isn't that more logical?
Ender, the word "possibly" would have been a better choice there... a reasonableness test, like you say, means unreasonable = incorrect, but it does not mean reasonable = correct.
And Doogie, the question was posed quite honestly (not rhetorically). I don't think the answer is Yes or No necessarily (logically). The key is not really whether one is more complex than the other; it's whether there is more luck involved in one than the other, and I don't think either A or B is an obvious answer there (the spread around .500 in baseball is awfully tight too).
Ender, the word "possibly" would have been a better choice there... a reasonableness test, like you say, means unreasonable = incorrect, but it does not mean reasonable = correct.
And that's fair, but if so, I didn't see any actual reply to Lord Bob's very relevant statement:
So saying that a statistic is good at addressing the self-evident is nice, but not helpful.
@Matt: I don't think you're seeing my point here. Take a step back for a minute and ask yourself, does it really make sense to assert that a system with "more numerous and complex interactions" can actually be modelled more simply, particularly when there appears to be only a handful of correlations and some unpublished bell curves driving such a statement? How do you model luck? How can hockey possibly be modelled, in any facet, with dice?
Maybe Staples doesn't like the stats that bloggers use because he didn't think of the stats himself first.
Hey lookit, another debate on lies, damned lies, and Statzis. By golly, someone even mentioned my name. What with the benches being cleared already and a big pile of gloves on the ice, I would be out of form to not wade in. Especially with all those juicy quotes to choose from ...
I'm sorry Matt, but I do find that some "stats guys" can be very closed-minded. Look at how long it took for Bruce's suggestion (and data!) that outshooting doesn't necessarily result in victory to even be half-assed accepted.
Thanks for that, D2K. It was an interesting discussion, wasn't it?
What seems to repeatedly go over the heads of some contributers is that nobody once suggested that dominating at any one of these underlying numbers is a guarantee of victory.
Yes of course, Slipper. Anybody who has seen a goalie, three iron pipes, and/or a referee steal a game knows this. What surprised me -- which is why I posted it -- was how weak is the correlation between outshooting and winning in the four separate, admittedly targeted studies that I did. What also surprised me was how the most statistically-minded on the 'sphere seemed to be the most reluctant to accept this new information.
We're evaluating who the better team is. I think it's widely accepted that the better team doesn't always win in sports.
Of course. It should also go without saying, Slipper, that you and I won't always agree who actually was the better team. You might go for the team with the shots and Corsi numbers and possession, and I might go for the team with the hot goaltending and the shot blocks and the opportunism. Fortunately we do have the scoreboard to settle the argument as to which team was, if not "better", more successful. I will always keep hearkening back to that, even as we advance the discussion about possession and goaltending and all those good things that make up this grand game.
As an accident of birth, I'm a numbers guy myself. I can cite you hockey statistics out of my ass, and no doubt I do all too frequently. It's a great way to try to get a handle on the game, getting better all the time with all the new methods you guys have been churning out. But I know all too well that there are many aspects of the game that can't be captured by a number or a ratio. Even if we could figure out what's important to measure, the actual compilation of the data is too labour-intensive to be practical. So we observe, we sample, and we theorize. The process is necessarily far from complete, so our theories are sometimes wrong ... at least mine can be. It is through the invalidation of the old theory, or its refinement, that understanding advances. So it is with any science.
Rereading that, my current theory is that I'm getting too full of myself and my theories and it's time to change the subject. But I'll leave that out there for reflection and ridicule as suits your taste.
I'd rather bash my head in with a hammer than discuss momentum, clutch and big goals from game to game, or how MacTavish should have played Garon form day one of last season.
I want to know who's playing against who, in what end, where they are getting the chances from, who's taking the defnsive/attack zone draws, etc, etc.
So you want to know who's playing against who, unless it's Garon?
:D
That's my interest in the game of hockey. If you can't accept that, why the fuck are you reading places like this?
Yes, of course I can accept that, Slipper. Can you accept that others who benefit from your insights may have a different but nonetheless serious interest in the game of hockey?
But really, haven't we all made these arguments a thousand times before?
I have no doubt that you probably have, Slipper. It seems that statement is always part of your argument. To some of us this is a fairly new thing, and while we all need to be educated in turn in this big one-room-schoolhouse we call the Oilogosphere, it is nonetheless possible that a newcomer may bring a fresh perspective.
Maybe Staples doesn't like the stats that bloggers use because he didn't think of the stats himself first.
Bill Needle: Actually, turn that right around and it's a logical explanation of the icy reception Staples Errors have received among the Statzis. To this lifelong fan of the game the formal assignment of blame for a bad result is an interesting concept, yet all some ever do is simultaneously "ignore" it and beat it up. At times it seems like an attack on the source of the stat rather than the stat itself.
I've seen plenty of high-handed and obnoxious responses from guys on both sides of the fence
How about these two recent beauts from the opposing corners?
"I'm not the least bit interested in these numbers. I know what I see and I know what I think. I'll go with that over pages of statistics any day."
"Errors (whatever the hell they even are, I've never been a Staples reader) and scoring chances are both subjective, leading some people to think that they somehow equate."
Ignorance and arrogance in equal measure make a potent cocktail.
I'd guess it's because there's very little correlation between willingness to use math and playing well with others. I haven't run the numbers, but that is my suspicion.
Jonathan, you are a joker, but there is truth in what you say. I have seen plenty of sticks come up on the 'sphere -- it can be quite entertaining at times -- but those who have been rude to me personally are mostly among the Statzis. Having been an unpopular math whiz myself as a child I thought I might find an affinity with such folk, but that hasn't proven to be the case. I won't name any names, but let's just say that for six months I completely withdrew as a commenter from IOF because I was getting gang-banged every time I went in there. Call it the Philly Flu.
How can hockey possibly be modelled, in any facet, with dice?
D2K: W-a-a-a-y back in the day I used to play Strat-O-Matic hockey. Alas, it wasn't too realistic.
W-a-a-a-y back in the day I used to play Strat-O-Matic hockey. Alas, it wasn't too realistic.
Yeah, I preferred the card game I had when I was a wee lad. I don't remember much about it, other than the fact that it was like Mille Borne, it had a Habs deck and a Nordiques deck, and there was no "leave the bench" card, which made no damned sense, because that's just what the Habs and Nords did back in the day. My dad and I only played it a couple of times because I think I was too young to get the rules at the time.
Matt:
1. First off, if in my original post on this matter I implied that the stats guys lack passion for watching hockey and aren't mad dog fans of the game, I apologize. That's certainly how you took my post, so even if wasn't my intent, I'm guessing there was some line or word that triggered that response.
The truth is I see you and Dennis, Tyler and the rest of the Stats Guys as totally obsessed with hockey, maybe not soft-hearted, fun-loving fans, but nonetheless enraptured by the beauty and skill of the game, along with the question of how can we best rate high performance in the NHL.
2. Slipper, I can see that you're tired of dealing with the same old questions, and with the no small amount of scorn directed at the statistical work of the Stats Guys.
Fair enough.
I think that scorn is very often out of line as well.
That said, I don't see the kind of questions asked by folks like Bruce, Performance Oil and Doogie2K as being generally scornful.
It's no doubt tedious to have to answer the same old questions and deal with the same old doubts, especially if you're convinced that major breakthroughs in thinking have been made. But what's needed is a F.A.Q. on all the major new stats and concepts that the Stats Guys talk about, so they can simply refers doubters and newbies to that and have all their questions answered.
Perhaps this is something Lowetide could produce, as he's got that educator's streak to him.
3. Matt, you write about some major findings of the Stats Pack: "At least insofar as I've absorbed it:
EV+/- is a better predictor of future record than is past record.
Best predictor of EV outscoring is where the (past) end-of-shift faceoffs occurred, followed somewhat closely by past Corsi #, followed somewhat less closely by past EV outscoring."
I'm guessing if the average fan could fully and completely understand this short summary, he might well be very excited by some of these discoveries.
So I'd ask you, or Lowetide, or anyone else who has a full grasp of all this, that as a favour to that average fan (me) -- and if you want to take up this challenge -- to pretend that there's an exam question that states, "The major findings of the Edmonton School of Hockey Analysis from 2000 to 2008 was that EV+/- is a better predictor of future record than is past record. The best predictor of EV outscoring is where the (past) end-of-shift faceoffs occurred, followed somewhat closely by past Corsi #, followed somewhat less closely by past EV outscoring.
To answer this question, first define the terms used, then spell out the main arguments used to support each finding, along with a brief summary of the facts that support these finding."
If only that were true. If a test passes on an obvious comparison that does not logically imply that less obvious comparisons will be even "probably" accurate. It just does not. While it may challenge your opinions, and it may force you to look at things in a different light, you're making a huge jump between obvious and not, and I'm sorry but it's not supported by logic.
Yes, yes, of course it's not necessarily true. As Matt says, it's a "rule of thumb" method of testing the reasonableness of a given measure without running numerous trials, discovering p values etc. Quick and easy, it's hardly full proof or, as you, totally logical.
I discussed this issue with a member of the Calagary media recently and my comment to him was we are just learning which of these new measures have real predictive of descriptive power. For the guys like yourself who seem totally opposed to statistical anaylsis, I wonder:
1.) Is it your contention that hockey can't be honestly or reasonably analyzed beyond qualitative judgements and counting numbers (goals and assists) because of it's nature (too fluid, too complex, etc)?
Or...
2.) Are you simply unimpressed by the current metrics being developed and employed by the bloggers and such right now (you see them as invalid or not powerful) and believe that better measures will be or could be created?
I think I can certainly accept and discuss the issue with anyone who has the second attitude. The first attitude - and I think that's the one Matt is critical of in this post - is just willful ignorance.
Well said Kent. The fact that so much happens in the 60(+) minutes of a hockey game may well be a feature, not a bug, when it comes to modelling it statistically. No model will ever predict whether a goal will go in off of Peter Sykora's leg tomorrow, but it's entirely conceivable that with detailed data collection and enough brain & computing power, you could predict the number of goals-off-legs in the NHL this season with remarkable accuracy.
And David, it's funny you bring that up (again), as I was just mulling that this morning. It's time -- I'll try to spend some of the next couple of weeks doing just that, barring something like life intervening.
@Kent
Both, actually. As I see it, statistically, you will never have a large enough sample space to say anything with any sort of rigor about any player or team, due to so few games per season where the rosters of both teams are even mostly the same (4 games of edmonton vs detroit, for example). As I said on a thread in CiO, all of the current stats can be made irrelevant due to particular styles of play, so you can't really use a stat to say something about a team over a season...
...aside from wins. And at the end of the day, goals are the only thing that give you wins, and even GF/GA is not a good indicator of how successful a team is. Hypothetically, a team could win 75% of their matches by 1 goal, and lose the rest by 3. Not really going to tell you a lot.
My real issue, and the reason I've been posting about this topic is that I find (whether intentional or not) that the Statzis (as Bruce puts it) are treating their number work as law, and anyone who disagrees with them are close-minded, arrogant, ignorant dumbasses. The issue is that their numbers are not law. There are some real issues with them.
You want a good offensive stat? Measure the shot percentage by player in particular places on the ice vs either certain goalies or goalies with similar styles. Split it into whether the player is covered or uncovered.
You want a good defensive stat? Use the Staples Error, but have 20-30 people all evaluating it and cross-reference the result. Goalies are exempt from this stat.
You want a good Goalie stat? Short side goals + rebound goals are errors. Miraculous saves don't balance it out because generally miraculous saves are there because the goalie was out of position in the first place.
None of these stats I'm suggesting are perfect, and some are *gasp* subjective, but I'm fairly confident they'll give you much more meaningful results as far as success of a team goes, and as far as how players are used by the coach.
Knowing how many goals will go in off of legs will never tell you which game they'll come in or whether or not they'll be a game-winning goal, so to me, it's a moot point.
the Statzis (as Bruce puts it)
Not my term. It was raised in one of the long debates last winter by one of the Statzis themselves (Tyler?), and it sounded like it had been around for awhile. I have to admit it made me laugh.
What surprised me -- which is why I posted it -- was how weak is the correlation between outshooting and winning in the four separate, admittedly targeted studies that I did. What also surprised me was how the most statistically-minded on the 'sphere seemed to be the most reluctant to accept this new information.
We could exchange targeted studies or anecdotal evidence until our fingers are numb. I don't recall all four examples you speak of, but based on previous defenses, I'm guessing the stellar, yet flukey, 2006 cup run was in there, right?
The team was good, the goalie was fucking great. The lesser team won. Almost.
That's how we should remember it, amirite?
These are the redundant exercises in logic and wit that these discussions always reduce down to, and I'm equally at fault. I know this.
But the goalie is part of the team!
And they acquired him knowing that this was the spring he was going to outperform his career averages in order to successfully out "counter" the obviously superior Detroit team.
I could just as smugly say how unsurprised I was when there was a signifigant correlation between winning and outshooting and outscoring at even strength in the 2007 playoffs. 11 or 12 out of 15 series if I recall correctly, and one or tow were so close it was a coin flip either way.
I don't think anyone would be surprised that the same correlation would exist for this past spring, ecspecially consideirng who won. Why? Because there were few to no underdog, fluke wins. Just like in 2007.
So there aren't any defiant fans specualting wildy about how their teams won because they won't accept what's evident: their team got lucky at the right time.
Afterall, when someone describes something as "hot", isn't luck what they're usually refering to? If I have a pair red hot dice in my hand, am I just getting better at throwing them?
Lastly, it's now even the Bruce M's (only David Staples is allowed to refer to him by his full last name;D) that I was refering to in the first place. Although sometimes I do feel that he's set to automatically awaken from cryogenic sleep as soon as I hit enter on my keyboard so he can dissect my posts to the very sentence and respond in kind, his points both bulleted and italicized.
If anything I was trying to express that the processes undertaken around here are constantly being refined by their proponents. People weren't satisfied with EV+ EV-, and began looking at on and off ice SV% and Shooting%, shot differential, Corsi, shot distance, etc.
It's a work in progress and those within the niche seem to accept that the methods aren't infallible. Yet, that doesn't stop those outside the niche from getting their panties in a knot when they find some anecdotal evidence that runs contrary to these methods and then acting shocked that in light of this "evidence" the whole process doesn't come to a crashing halt.
and even GF/GA is not a good indicator of how successful a team is.
Are you sure about that?
IIRC, the Hockey Analytics dude was able to predict standings based on GF/GA alone rather accurately.
@Slipper
I don't know if it's just me or not, but that's not really the point. I would never say "Stop it! It's useless!"
I might think it, but that's another story.
My point is, and always has been that it doesn't matter who the "better" team is. Good coaching an team play can generally trump "better." The only thing that matters is who wins. Detroit was the better team this year, because they won. They didn't win because they were the better team. That's what I'm always arguing. It's fallacious. Same as "fake wins (SO)." It's a win. It doesn't fit into your numbers, but it's a win, and at the end of the game, it's all that matters. Deal with it.
IIRC, the Hockey Analytics dude was able to predict standings based on GF/GA alone rather accurately.
Teams outperform and underperform the GF/GA yearly. Whether or not it's an adequate (and who is defining adequate?) metric for the regular season, well, I say meh. It doesn't tell you who will win the next game of Detroit vs Edmonton, so explain to me why I should care. I mean, we're talking about using retroactive stats to argue "my dad could beat up your dad" here, aren't we?
Just a question:
Does anyone else read these debates and immediately think, man, these Statzis sound just like the early proponents of the behavioral revolution?
When it comes to gut guys/stat guys, neither has a monopoly on arrogance. I've seen plenty of high-handed and obnoxious responses from guys on both sides of the fence - and I'd guess it's because there's very little correlation between willingness to use math and playing well with others. I haven't run the numbers, but that is my suspicion.
Now that's funny
They brought a SPORTS PSYCHOLOGIST in, everyone got injured, and they were on with a bunch of people younger than me leading the way. I personally assume the big change wasn’t the injuries, but rather it just took that long for the COUNSELING to really set in.
Why are you posting here, Ender?
...goals are the only thing that give you wins, and even GF/GA is not a good indicator of how successful a team is.
I mean, seriously, why?
Detroit was the better team this year, because they won. They didn't win because they were the better team. That's what I'm always arguing.
WHY? Why are you wasting your own time always arguing that point?
Ender said: "I would never say 'Stop it! It's useless!'"
Ender said: "and I’m ok throwing their opinion right out the window."
Fallacious- adjective
3.disappointing; delusive: a fallacious peace.
You are fallacious.
But the goalie is part of the team!
And they acquired him knowing that this was the spring he was going to outperform his career averages in order to successfully out "counter" the obviously superior Detroit team.
I can specifically remember a lot of talk about the Oilers wanting Roloson because of his rebound control - that they had the personnel on D to pick up a rebound and get the puck moving out of the zone (centered around Pronger). So isn't it possible that they built a gameplan around this and that Roloson's stellar save percentage in that run was due to the concept of allowing the low percentage shot and regaining possession, not regaining possession because all of a sudden Roloson is a SP freak?.....maybe someone in the Oilers organization ran some stats?
@slipper
Thank you for that excellent example of using data (or stats, as they call them in sports) out of context to create a point.
Is it possible, Oilman?
I guess so, anything is possible.
I doubt that it's true though. I don't think any wise team gameplans to get outshot or outchanced 1.5 or 2 to 1. It just sounds ridiculous.
I love how Slipper pounced on that sports psych bit like a rabid tiger.
It's science, bitches. It works. Just ask the Australians.
Ender wrote, "You want a good defensive stat? Use the Staples Error, but have 20-30 people all evaluating it and cross-reference the result. Goalies are exempt from this stat."
Ender, this would be a helluva good way to score an error, and it's what I've been attempting to do by posting my results and my thinking every Oilers game on who should get an error and who should not.
That way, when I get it wrong, or when I'm not following the set criteria, or if the criteria needs adjusting, I can make the necessary changes.
No doubt about it, if 20 or 30 people spent the time to go over the criteria for assigning errors (I'm going to update it soon), then watched the video and also assigned errors, and we all took a vote on who deserved the error, the end result would be more credible, and we'd all learn a lot about the strengths and weaknesses of the error.
But to do all that, that's asking a lot of people . . .
Matt, there's no rush in producing such F.A.Q.s or answering the hypothetical exam question, as that kind of thing takes a ton of work. But if you get to it sometime this season, much appreciated . . .
P.S. On another blog, someone posted under my name, "david staples," a pretty low thing to do, but the kind of thing I've come to expect from various skunks.
So if you read some post and it sounds highly intelligent, not shallow, not condescending and doesn't smack of cowardice, then it's almost certainly not me that posted said comment, but some other guy pretending to be me and posting under my name.
Oilman, if that's true, it makes a lot of things make way more sense to me. I always thought Roli was a shitty goalie, and the only reason he stood up in net was because of the defensive style.
Roli got re-signed before Pronger left as well. Food for thought.
I mean, we're talking about using retroactive stats to argue "my dad could beat up your dad" here, aren't we?
Um... huh?
Teams outperform and underperform the GF/GA yearly. Whether or not it's an adequate (and who is defining adequate?) metric for the regular season, well, I say meh.
You're right, a formula that can predict a team's results with an accuracy of about 94 per cent is a total waste of our time.
I can specifically remember a lot of talk about the Oilers wanting Roloson because of his rebound control - that they had the personnel on D to pick up a rebound and get the puck moving out of the zone (centered around Pronger). So isn't it possible that they built a gameplan around this and that Roloson's stellar save percentage in that run was due to the concept of allowing the low percentage shot and regaining possession, not regaining possession because all of a sudden Roloson is a SP freak?.....maybe someone in the Oilers organization ran some stats?
Probably stats like these, Oilman:
2002-03: 50 GP, .927
2003-04: 48 GP, .933
Roloson had had the top Sv% in the NHL the previous season, the second best in the history of the stat in fact, but after the lockout Manny Fernandez got a leg up on him and Roli languished on the bench. He couldn't win a game to save his soul but he wasn't playing that bad (.910 Sv%) in his one game a week, he just wasn't doing enough to win favour with Uncle Jacques. :) So going for Roli was a pretty goddamned good gamble, it just required a GM/organization willing to look beyond that 6-17-1 record at the bigger picture.
I remember doing our pre-trade deadline predictions on the old Officepools Powerboard that Roloson would be a perfect fit in Ottawa, who were a goalie away from making a run at the Cup (Hasek having gone down in Torino). Roloson was solid enough to succeed on a legitimate contender, I figured. Turned out that I was both wrong, and right.
I never said "a total waste of [your] time." It all depends on what you're looking for. Me, I want to know who will win X game. You, apparently (and I'm sorry if I'm misrepresenting you) want to know who will rise to the top of the standings. My issue is that the latter gives you who is the "better" team but it doesn't give you the answer of who will win between team A and team B. Only in the playoffs do teams play each other often enough in a short enough time period to get a meaningful data table out of, ranking Team A's style vs Team B's style.
Long story short, the regular season is a very long exhibition round, and since everyone plays different teams a different amount it's not as easy as throwing a weighted die. It's throwing 30 weighted die (if rosters were exactly the same from beginning to the end of the season).
No stat is useless. It's all about context. but yes, I will go out on a limb here and say that I do strongly feel that the stats people use on these boards don't necessarily give any of the information people seem to think that they do.
Roli got re-signed before Pronger left as well. Food for thought.
I figure Roloson getting resigned was a foregone conclusion - he played extremely well - and with the shit kicking Edmonton was taking, not signing Roloson would have been another PR nightmare (unless you could have traded Pronger for Luongo).
Slipper: I don't think the plan was to get outshot and outchanced by a wide margin - that's that's a shortsighted simplification. I think the plan was to limit the high percentage chances and let other teams fire off as many low percentage shots as they were willing to take (Detroit is more willing than almost any other team) knowing you had a good SP goalie with good rebound control and one of the best D-men in the world at regaining possession off the rebound and starting the play moving the other way. As I recall, it worked to near perfection, and if this high SP, rebound controlling goalie had not been knocked out of Finals game one, we could remove the near part.
We could exchange targeted studies or anecdotal evidence until our fingers are numb. I don't recall all four examples you speak of, but based on previous defenses, I'm guessing the stellar, yet flukey, 2006 cup run was in there, right?
Absolutely. It was pretty conclusive from team "total shots" data -- that I compiled that heady spring long before I heard Jim Corsi's name associated with them -- that based on shots and by extension possession, the Oilers outplayed just one of their four opponents that spring: Carolina. The teams that won the four series out-Corsied their opponents in exactly one of the 24 games. It was pretty goddamned astonishing. The Oilers clearly departed from their game plan in the Carolina series, perhaps due to Roli's injury, and went back to the "outplay but don't outscore" formula of the Fricklin and Frakkanen duo that had frustrated Oiler fans from October to February.
The team was good, the goalie was fucking great. The lesser team won. Almost.
I'll repeat myself since you apparently didn't read it the first time:
It should also go without saying, Slipper, that you and I won't always agree who actually was the better team. You might go for the team with the shots and Corsi numbers and possession, and I might go for the team with the hot goaltending and the shot blocks and the opportunism. Fortunately we do have the scoreboard to settle the argument as to which team was, if not "better", more successful.
The other examples were all Oiler-related, which is what I mean by "targeted"; I'm an Oiler fan first and I want to understand what makes them tick even as I will cheerfully admit that they are probably not very representative of the league as a whole. They included a detailed analysis of the Oilers' last 20 games of 2007-08 (of which 16 were won by the team being outshot, usually by a large margin); an analysis of Oilers record when outshooting or outshot over MacTavish's coaching career (.535 in 273 GP when outshooting, .551 in 277 GP when outshot); and of the Oilers five Stanley Cup seasons, in which they were outshot by their collective opposition in all five regular seasons. From these four negative-to-strongly-negative correlations between shots and winning, I drew the rather straightforward conclusion that to the Oilers at least, outshooting isn't necessarily a big part of the game plan.
But when I put all that out there, I didn't get so much as a "Hmmm, that's interesting" from certain quarters that didn't seem too receptive to any new evidence that didn't support their pet theories. Or so it seemed at the time.
But based on that evidence, this Oiler fan isn't going to get my shirt in a knot when this player or that has a big red Corsi number. It's one way to analyze the team, sure, and it's interesting, I'm just not sure how relevant it is.
Afterall, when someone describes something as "hot", isn't luck what they're usually refering to? If I have a pair red hot dice in my hand, am I just getting better at throwing them?
You guys and your dice. When the word "hot" is followed directly by the word "goaltending", luck is only part of it. Goddamned right the goalie is part of the team. Flow-of-play stats like shots, Corsi, Fenwick, or the hopefully-soon-available scoring chances do not adequately account for the most important player in the defensive zone. So they're not entirely reliable in my view, either as analytical or predictive tool. (Maybe they would work better in basketball!)
Bruce, you have some interesting points on Corsi, although I don't have the full history of debate about this with you and Vic, Tyler or whoever. Another reason why you may want to start a blog or join an existing one -- it's easier to be heard.
That said, some other dudes on this thread seem to be mixing that up with the usefulness of stats-as-predictions in general, which is completely weird. Asking for certitude when the matter is probability seems to be missing the point.
I mean, if the variance of predicting top goal scorers based on Corsi numbers is too weird or doesn't match up, it's maybe not the best tool. I may be reading this wrong, but I don't think it's even a go-to stat for the bloggers that brought up anyway. I'd still like to see more about Corsi numbers in general.
Either way, it should be able to predict something relatively well, like GF/GA does almost all of the time.
a formula that can predict a team's results with an accuracy of about 94 per cent
When you guys figure that out, sign me up. I've got some money to win.;o)
But when I put all that out there, I didn't get so much as a "Hmmm, that's interesting" from certain quarters that didn't seem too receptive to any new evidence that didn't support their pet theories.
I recall that at the time that myself and perhaps Tyler both commented that the team outshooting or out Corsi-ing wasn't neccessarily the better team, but for cases where there was a commanding lead. Other things must be considered with this "work in progress" including on ice sooting and save percentage.
I don't know what else would have satisfied your ego in this case, considering you presented what you admit is a selective sample.
I'll repeat myself since you apparently didn't read it the first time:
Yet, that doesn't stop those outside the niche from getting their panties in a knot when they find some anecdotal evidence that runs contrary to these methods and then act shocked that in light of this "evidence" the whole process doesn't come to a crashing halt.
Any number of guys around here, if compelled to do so, could do a selective sample of the '07 Ducks and '08 Wings and present what would appear to be a slam dunk in favour of Corsi. It wouldn't be any less fair or relevant than your argument.
I don't know what else would have satisfied your ego in this case, considering you presented what you admit is a selective sample.
Slipper: Said sample covered, in one way or another, 12 of the Oilers 28 seasons. So it's pretty significant w.r.t. the Oilers if not the league as a whole.
Yet, that doesn't stop those outside the niche from getting their panties in a knot when they find some anecdotal evidence that runs contrary to these methods and then act shocked that in light of this "evidence" the whole process doesn't come to a crashing halt.
The last thing I want is for the process to come to a screeching halt. I want to advance the discussion. But when I try, there is a certain niche that would disagree with "black is black" depending on who said so. I would have expected this process to be a little more inclusionary.
Slipper makes a series of good points there. I vaguely recall, thinking back on it, that I was objecting on the basis that the "outshooting" focus of the statzis is largely related to ES play, which, in fairness to Bruce, he wasn't around for in 2006 when we started kicking at it.
Vic made the point that looking at the Gretzky led Oilers might not be the best way to suss out whether teams that get outshot usually end up winning the game, which also strikes me as a fair point.
But when I put all that out there, I didn't get so much as a "Hmmm, that's interesting" from certain quarters that didn't seem too receptive to any new evidence that didn't support their pet theories.
Well, you got a thoroughly researched post out of me, showing that over the past 15 years, there's been a real advantage for the team that outshoots.
The last thing I want is for the process to come to a screeching halt. I want to advance the discussion. But when I try, there is a certain niche that would disagree with "black is black" depending on who said so. I would have expected this process to be a little more inclusionary.
Must just be you. I've been welcomed with open arms.
12 of the Oilers 28 seasons.
Seven of which are post loser point which, as I pointed out in my post, has been a huge benefit for the teams that get outshot.
I suppose an interesting question would be whether there are any consistently very good teams, not featuring guys who are in the Hockey News Top One Player of All Time, that get consistently outshot. I'd guess no, but I don't really know the answer.
This is my conclusion:
1. Hire a sports psychologist
2. Get outshot 2 to 1
3. While not committing alot of errors
4. Draft Wayne Gretzky
Seven of which are post loser point which, as I pointed out in my post, has been a huge benefit for the teams that get outshot.
MC: On this we entirely agree. As I recall it was one of my conclusions at the time; certainly I made numerous posts about MacT's ability to play the system, and how the system does not reward attacking hockey. The Bettman Point is truly asinine.
The comparisons between 2000-08 Oilers and 1983-90 Oilers could hardly be more different ... yet both indicated outshooting was at best a non-factor, for an extended sample (1/4 of franchise history in each case).
I suppose an interesting question would be whether there are any consistently very good teams, not featuring guys who are in the Hockey News Top One Player of All Time, that get consistently outshot. I'd guess no, but I don't really know the answer.
Neither do I, but it's an interesting question worth further thought. I'm certainly ready to agree that the Oilers of the 1980s were Exceptional for the reason you state. If nothing else they/he proved that it is possible to win without dominating the shot clock. But maybe they really are the exception that proves the rule.
Too bad we don't have team shots data before 1983, at least not on Hockey-reference.com which is my touchstone source. The same data shortfall really cripples historic Sv% comparisons. Do you know of an alternate source? Shots have been counted for as long as I can remember.
On this we entirely agree. As I recall it was one of my conclusions at the time; certainly I made numerous posts about MacT's ability to play the system, and how the system does not reward attacking hockey.
I wouldn't say that MacT necessarily knows how to play the system. I'd say that the new system is superficially kinder to mediocre teams than the old one. It's not like MacT is pumping out 100 point season after 100 point season.
I still think that it's worth noting that the only MacT team that did a damned thing in the playoffs is the one that really put the boots to the opposition in terms of shots in the regular season. Harder to do that in the playoffs because you're playing other teams that can do that too but it's noticeable to me.
I wouldn't say that MacT necessarily knows how to play the system. I'd say that the new system is superficially kinder to mediocre teams than the old one. It's not like MacT is pumping out 100 point season after 100 point season.
Nope, he's scraping to get every point he can get.
-- In 2005-06 Oilers made the playoffs by virtue of playing more overtime games than anybody and winning half of those bonus points.
-- In 2006-07 the team couldn't get to OT and was uncompetitive.
-- In 2007-08 the team led the league in both # of overtime games and overtime Pts%, a double that is not likely to be repeated for a very long time. In fact they had the fewest number of OTL in the league despite playing the most OT games. If you look it up I'm sure they were outshot in the lion's share of those games but still scored 1 or more often 2 points. What appeared to be another uncompetitive team very nearly made the dance as a result.
Sure there was happenstance: not all of those OTs were a matter of choice to sya the least, and both the shootout success and Cogliano's OT hat trick seemed an act of the hockey gods; but I'd still call that playing the system.
Sure there was happenstance: not all of those OTs were a matter of choice to sya the least, and both the shootout success and Cogliano's OT hat trick seemed an act of the hockey gods; but I'd still call that playing the system.
I see there as being a rather large difference between "benefiting" from the system and "playing" the system. I'm not convinced that MacT is playing it.
This is gotta be one of my biggest points of contention.
A is true. A in this case being that the Oilers played in overtime lots. One of the reasons is B. B being that they didn't win in regulation.
My question is, how is it people come to the various conclusions for C?
C for particular argument ebing that they played to get to overtime.
Or C is a strategy employd by the coach.
All by virtue of result A occuring, people divine all sorts of myths as an explanation.
It's baffling. What did MacT really do in order to reachOT more? Press the swithc in Huddy's ass the prevents goals form entering eother net after the score is tied? Is he a God? WhaT?
WHAT!?!?!
The same goes for theOilers being dominated by the Red Wings. OH! It must be a strategy!
Precisely.
Lowe: We really need to atleast contend for a playoff spot this year Mackie, for the sake of my reputation and the franchise's
Huddy: Well Bubba, I'm on board with you. The key difference I saw between the two seasons was the one year we had Pronger, Peca, Spacek, and depth at forward, while the next year those players defected and then we were decimated by inj...
MacT: Shut you pie-hole you useless twat! The obvious difference between seasons was that we didn't play to overtime more in '07.
Lowe: Oh Lord Craig, you're right! I'm calling Montreal and trading Horcoff for three assets and then calling up Rob Schremp!
Huddy: Boyz, hold on just for a sec. I mean, playing for the tie!? How can we insure ourselves against the bounces. Wouldn't we be better off just trying to win gamesin regulation?
MacT: Winning games in regulation is for pussies. Besides, we want to spread some points around to make it more exciting for the fans the final few weeks of the game.
Huddy: But I... I mean... err if we can play to overtime, it would entail almsot scoring and not allowing goals by force of will alone... why wouldn't we just do that earlier and win the game out right... I mean...
Lowe and MacT: Shut-up you useless twat!!!!
You want a good defensive stat? Use the Staples Error, but have 20-30 people all evaluating it and cross-reference the result.
I think 5 would be enough. If 3 score an error for the same guy, it's an Error. Kinda like how Olympic boxing works, leaving absolutely no room for controversy. :)
But as, say, a targeted project (a few specific games, or even a bunch of selected goals) it would be cool to have as many people as will try to do it honestly and see what kind of consensus there was on who was responsible for the goal. Saboteurs need not apply.
I shadowed Staples for a dozen games or so this past year as a test to see how similarly or differently two different observers judged the same plays, and we probably had 80+% agreement on initial (before consultation) assignment of errors. This raised my level of confidence that this sort of stat could be measured to at least an RTSS level of consistency.
The irony of those flying the flag of neutral science hurling insults like 12 year olds is really too rich. Are we supposed to be more confident that stats guys are neutral or clear headed when they start making infantile interpretations of others' arguments?
What are you saying? That the Nazis and the early proponents of the behavioral revolution were more high brow?
Not so sure about high brow http://coveredinoil.blogspot.com/2008/03/david-staples-is-making-error.html
wonder what Bill James' approach to getting his ideas accepted was, and how much bitch slappin' and dick lickin' was involved?
Bill James did not tolerate fools easily.
Understandable though, to me, anyway. I mean, you put a lot of work into to something, and then when you show people they tell you to put away the calculator and actually watch a game.
I'd rather read an asshole write something intelligent than a nice guy write something stupid.
I am saying, pretty obviously of course, that insofar as individual stats need to be put into their proper context, and this requires synoptic thinking, that I would trust those authors who can defend their stated point of view without excessive displays of indignation or passion. But if one does not adhere to a belief in the ultimate neutrality of statistical analysis--and particularly if one thinks that sports are shot through with phenomena such as anger and the like, and this will have a discernible but not repeatable impact on the data--one is not held to such a standard.
So I do not need to be surprised by how much a discussion about neutral stats revolves around indignation and insult, because I am quite willing to concede the power of emotions in both sport and our analyses of sport.
That is a nice thread Oilman. I made some well thought out posts and some fascinating points in a fair and well argued, albeit passionate, debate.
But where did I write what now?
I am above reproach, yo.
Post a Comment
<< Home
Beauty post about an absolute dog's breakfast of an article by Staples.
Among the many many many ridiculous assertions was this:
The NHL is now tracking "hits," but the traditionalist knows that what is counted as a hit in Dallas, might not count as one in Montreal. The NHL also tracks "face off wins," but on many draws, it's almost impossible to tell who won, so how do you know if the numbers are accurate?
Except that David Johnson at Hockey Analysis (http://stats.hockeyanalysis.com/) recognizes that fact and has tried to account for it. I imagine he's not alone in doing so.
If anything, the characterization of the traditionalist as too lazy to try to understand anything new is actually accurate. I have tonnes of friends that, in any sport, react to a new statistical measure as some sort of poison. God help anyone that tells them that plus/minus is garbage or that the passer rating is not the be all that they believe it to be.
# posted by 
: September 10, 2008 1:03 pm
: September 10, 2008 1:03 pm Gah, I'd probably respond to Staples if I even bothered reading the guy any more.
I know he's been a boost to the Oilogosphere in some ways, but he sure comes off as a very two-edged sword from my (distant) perspective.
The thing that jars, that I didn't quite elucidate in my post, is the stuff about the self-confidence exhibited by the stats guys.
"I'm not the least bit interested in these numbers. I know what I see and I know what I think. I'll go with that over pages of statistics any day."
Have you ever seen the opposite statement made by a so-called stats guy, even half as categorically as Brownlee makes his? And Dellow is the arrogant one... riiiiight.
To use Vic Ferrari as an example:
Years back, on the forums, he was one of the first guys bringing to my awareness the EV+ EV- (with shorties and empty netters) idea as a method of evaluating player ability.
He didn't sit back and boast about how genius he was and hang his hat on EV+ EV-.
He began tracking shooting percentage and save percentage to express whether any of these stats were ballooning or anchoring a player's results in other areas.
He crunched where goals were coming from and shot quality.
As a way to further the cause he began to raise the idea of shot differential, and shots directed at the net, as another method to define a player's ability. In this case, mostly in regard to possession.
All the while the dude has argued that these things should jive with what we see on the ice. As it should.
Every four months someone new DB has to re-raise the question of the validity of these methods and the character of the people who ascribe to using them, in 4,000 word or less. I don't see why they can't just fuck off and accept that a sub-set of people have decided not to buy into to traditional fan interests. I'd rather bash my head in with a hammer than discuss momentum, clutch and big goals from game to game, or how MacTavish should have played Garon form day one of last season.
I want to know who's playing against who, in what end, where they are getting the chances from, who's taking the defnsive/attack zone draws, etc, etc.
That's my interest in the game of hockey. If you can't accept that, why the fuck are you reading places like this? I'm far too disinterested to be bothered to blast people on hockey forums about every ridiculous topic or concept they raise, or how I find their fanatic, obsessive interest in under-age hockey players sorta disturbing. So what the hell is up with people who rail against these stats continuing to post in these parts?
I think a little of the MSM attention might be part to blame. We are talking about a community of aspiring writers, after all. I guess if one David Staples post wouldn't illicit 10 rebuttals across the Oilogosphere there would be no conversation in the first place.
Could someone help me out and correct the grammar in that last sentence there? Thanks.
It's "elicit," not "illicit." Thank you for giving me license to play Grammar Nazi.
Now, I see your larger point, and I don't disagree that certain methods of data analysis are producing useful information. Contrary to appearances, I'm not here just to shit-disturb. I just don't think much of the attitude that comes from these places sometimes.
I'm sorry Matt, but I do find that some "stats guys" can be very closed-minded. Look at how long it took for Bruce's suggestion (and data!) that outshooting doesn't necessarily result in victory to even be half-assed accepted. Hell, look at the continued animosity from Dennis to this day, and it's been almost a year since Bruce has been around. For that matter, I found it pretty damned rich that Tyler could crow about correctly predicting a 20th-overall power play, without really accounting for how vicious the swing was between the early and late part of the year. How very Adam Proteau of him. (I also seem to recall him giving a pretty generous range in the first place, though I could be mistaken. Sorry, but after four months, I'm not spending two hours sifting through every comment thread on LT's blog.) And then there's the near-universal dismissal (or in Vic's case, total intentional ignorance -- irony!) of the Staples Error, rather than trying to make it better or replace it with something that better correlates to winning/scoring/whatever. Oh, and I disagree with your characterization of people who don't like/use the new-age stats: the CalPuck and HF retards might be like that, but I know people who have no use for the new-age stats, but perfectly understand the game itself, the sorts of players you need to win, and the roles they play. The fact that they're not interested in sub-analyzing it is not a sin.
So don't even try to tell me that the stats men can't be arrogant and closed-minded. You're deluding yourself.
P.S. What is the statistical significance of Vic's discoveries? Correlation is lovely, but it doesn't really say much, at the end of the day. Unless I've totally missed something, I don't see any p values related to the null hypothesis, and how he controlled for confounding variables.
Junior/amateur scouting will no doubt mostly remain a matter left to two-eyes-and-a-gut
http://hitthepost.blogspot.com/2008/07/prospect-camp-day-4-evaluation.html
Paragraph #3.
The characterization of the two sides of the debate -- which I think is a misnomer anyway -- was obviously exaggerated deliberately as a counterpoint.
The statistical significance of Vic's discoveries? Considerable, if you're coming at it from his (or the gambler's or the pro scout's) angle of trying to figure out what is most likely to happen in the future. At least insofar as I've absorbed it:
EV+/- is a better predictor of future record than is past record.
Best predictor of EV outscoring is where the (past) end-of-shift faceoffs occurred, followed somewhat closely by past Corsi #, followed somewhat less closely by past EV outscoring.
That shit is nothing to sneeze at. Now, all the while, there are young players getting old and old players getting worse, nagging injuries, d-men that are better at stopping certain types of scoring chances than others, etc. Like most statistical models, there is all sorts of interesting things happening out at either end of the bell (if you're Mario Lemieux's coach do you worry about him getting slightly outshot at EV? not so much). But it sure seems to form an awfully sound basis for what players and teams are headed for success and which are not.
And no, it ain't no sin to pass on subanalyzing anything -- unless you are trading on being an expert on it. I would be appalled at a legal scholar who decided he didn't need to read any more law articles, or a historian who announced he was done reading history books. At the very least, I can't imagine that they would wear their disinterest in learning more as a fucking badge of honour.
As somebody who is by no means a stathead, I found Staples's post far fairer than yours, Matt.
Which makes sense. You are a stats guy. You think you're right. Therefore, everybody who's not a stats guy is wrong, and should probably go back to boning their sisters or whatever it is we do.
But there is more to evaluating hockey using ones' senses than sticking two sweaters in your ears and yelling "LA LA LA" as loud as you can whenever anybody who isn't Don Cherry is talking.
I saw an interesting comment on Covered in Oil recently: the stats we end up accepting are the ones that agree with what we already thought. For example, a stat showing "Joffrey Lupul is an awesome frickin' hockey player" would be thrown out the window because we all know Joffrey Lupul is a waste of carbon atoms. Which, to me, carries the interesting consequence that when we sit down and work the numbers on players, what we see is what we wanted to see anyway.
Worth some thought.
Look at how long it took for Bruce's suggestion (and data!) that outshooting doesn't necessarily result in victory to even be half-assed accepted
What seems to repeatedly go over the heads of some contributers is that nobody once suggested that dominating at any one of these underlying numbers is a guarantee of victory.
We're evaluating who the better team is. I think it's widely accepted that the better team doesn't always win in sports. But jumping on that accepted fact and doing a little dance because an unforseen variable (often luck) won through the day doens't make you right.
Afterall, it is sports, and much of the results are dependant on chance. Something that's widely accepted and echoed the very people who openly ascribe to an interest in "new stats".
But really, haven't we all made these arguments a thousand times before?
For that matter, I found it pretty damned rich that Tyler could crow about correctly predicting a 20th-overall power play, without really accounting for how vicious the swing was between the early and late part of the year. How very Adam Proteau of him.
I don't think that I was "crowing" about predicting a 20th overall power play. If I have the right LT thread in mind, I'm pretty sure that I was pointing out that, for as much as AO wanted to kick the stats guys for not expecting the OIlers to have an unbelievable amount of shootouts AND an unbelievable amount of shootout wins, the team still sucked.
(I also seem to recall him giving a pretty generous range in the first place, though I could be mistaken.
I think that this all stems from this post. I wrote:
If I had to guess, I’d guess that Edmonton ends up slotting somewhere between 15th and 20th on the PP. As you can see from the chart above, it’s basically a cluster outside of the extremes and that’s going to have a minimal effect in the grand scheme of things.
If you look at the chart in that post, I also made the point that, outside of the real extremes, there really isn't much difference between team's PP abilities. For example, if you look at the chart that I posted, you'll see that, when the 2006-07 PP's are ranked by goal differential, there was a difference of all of 7 goal differential between the number 26 PP and the number 13 PP. I was up front with all of that when I wrote my post - the information was there for anyone who wanted to see it. I certainly didn't claim that there was much of a substantive difference between 15th and 20th. If you think that that's a wide range or that if there was anything to closely examining this stuff, it could be predicted more closely than that...well, I'm sorry but I think that if that's the standard you're applying, nobody's actually able to meet it.
I still haven't taken a close look at the Oilers 2007-08 PP. We'll see what it looks like when I do but having taken a very quick look, it seems to me that they weren't able to buy a goal during the bad period and seemingly everything went in during the good period and that the shots and all that didn't really change too much.
True Fact: Of the bottom 15 teams in PP shooting percentage through game 615 of last season, 13 of them improved in the second half. Of the top 15 teams in PP shooting percentage for the first half of the season, 12 of them got worse, one stayed the same and the two got better (Pittsburgh and Carolina, for those scoring at home).
Edmonton was of course, Very Bad during the first half of the season at shooting percentage on the PP. Seventh worst in the league. In the second half, they led the league, while other teams fell back to earth.
The PP is a volatile thing. Shooting percentage seems to matter a lot and, last year at least, it was pretty damned volatile. All of the Oilers fans with hardons for how awesome the PP is going to be this year might want to consider that before concluding that the bad games don't count and that it'll be at the top of the league this year. Not saying that's you Doogie2K and I'm not really making any comment on what I think the PP will be yet; just saying that, while the Oilers had a huge swing last year from the first half to the second, they weren't alone and as tantalizing as it is to to take that the second half resurgence, tie it to other things that we know happened and assume that they're causative (Souray's return, Gagner doing well), I'm hesitant to do so.
# posted by : September 10, 2008 3:21 pm
: September 10, 2008 3:21 pm I know the one you're talking about LB, but I don't think that's quite what was meant -- it was worded inelegantly.
If you're coming up with a new player ranking of the 07/08 Flames and it doesn't show that Iginla >> Nolan >> Godard, then it's garbage. But if you have one that does show that, and it also shows (say) Conroy surprisingly high (to you) and Lombardi surprisingly low, that's not a reason to throw it out... There has to be a reasonableness test. (Whole concept is obviously not limited to hockey or even sports.)
For example, a stat showing "Joffrey Lupul is an awesome frickin' hockey player" would be thrown out the window because we all know Joffrey Lupul is a waste of carbon atoms. Which, to me, carries the interesting consequence that when we sit down and work the numbers on players, what we see is what we wanted to see anyway.
I think that that arose out of a Bill James quote. As I recall, he was willing to be surprised, but within limits. If the stat says that Joffrey Lupul sucks while guys like Crosby and Ovechkin stink, then sure, I think you throw the stat. If the stat tends to peg other guys where you think that they should be...I think you re-examine Lupul and, possibly, confine future jibes to how he looks so fucking mopey all the time.
# posted by : September 10, 2008 3:28 pm
: September 10, 2008 3:28 pm You call it mopey, I call it dreamy. He has bedroom eyes!
In all honesty, as an in-between gut and stat guy, I think the 'stat guys' can be a bit obnoxious which comes off a bit as arrogant. I don't think it's so much 'if you're too stupid to see how right I am, you're not worth talking to it,' but closer to 'goddammit, I'm not arguing about the validity of EV/60 as a predictor again, piss off.' Yeah, it's a bit dickish, but you can't blame them for not wanting to retate Stats 101 when they're trying to write their thesis.
And I think gut guys can be a lot more arrogant (the Brownlee quote comes to mind). There's a level of disinterest in being sophisicated, so they're going to tell you Souray is worth $5m and it doesn't matter what your fancy stats say. I'm not going to shit on them for having that belief (it's more fun to ignore stats sometimes, particularly when they predict your team is fucked), but if you're going to have that attitude don't come around blogs that are pretty big proponents of them, pick fights, and then refuse to argue on a level above 'did you see that time Souray scored that goal???'
Fantastic post, Matt.
As somebody who is by no means a stathead, I found Staples's post far fairer than yours, Matt.
I'm just as biased here, but I don't think the characterization of this as being between people with intellectual curiosity and the willfully ignorant is really that far off the mark. It's not necessarily nice, but how else do you explain a comment like Brownlee's?
Further, it's not as though we're saying you have to agree that the new stats are an effective way of examining the game or anything: a guy like Brownlee isn't even interested in looking into them, which is something more than just having different outlooks on the game.
I assure you, Mr. Fenwick, I am an expert at the art of inelegant wording.
Anyway, your answer only moves the problem down a level. Any idiot knows that Iginla is better than Nolan who is better than Godard. You could ask a million Calgary fans who can't even add two and two and they'd tell you that. So saying that a statistic is good at addressing the self-evident is nice, but not helpful.
What statisticians are looking for - and correct me if I'm wrong on this, but I don't think I am - is to explain or predict that which isn't obvious; Conroy and Lombardi, as in your example. Saying that because a given statistic can tell the difference between Jarome Iginla and Eric Godard and therefore it can tell the difference between Matthew Lombardi and Craig Conroy is, to me, a considerable leap.
(I'm obviously simplifying, but I'm at work here. :P)
I'm not trying to say that anything that requires a calculator has no place in hockey. When you're looking at a player you're not familiar with, statistics are the best way to do it. "Ah, we got Joe Blow for two first-round picks, but Joe Blow's shooting percentage this year is ten percent better than his previous career best. I will now shoot myself." Stuff like that. And they can provide handy ways to get kernels of information and quantify things that are otherwise difficult to quantify. But they're not The Answer, any more than a hammer is The Tool.
(By the way, re-reading your post, I think you were more aghast at people who say 'lol NERDS' every time EV+/- comes up, and not against - to put it arrogantly - people like I think I am. But, from a quick reading at work, it did come off differently the first time.)
(By the by the way, writing this post was not at all quick. Why yes I am paid by the hour, in fact!)
Also, ex-Pleasure Motors: Robin Brownlee has a brain like a cinder block. Nobody is arguing this. :P
Anyway, your answer only moves the problem down a level. Any idiot knows that Iginla is better than Nolan who is better than Godard. You could ask a million Calgary fans who can't even add two and two and they'd tell you that. So saying that a statistic is good at addressing the self-evident is nice, but not helpful.
Matt's point in this regard was: noting how a stat syncs up with the obvious is a short-hand way of testing a metrics validity. Therefore if it does do that reasonably well then the less obvious findings that emerge in the gray areas and are potentially counter-intuitive are probably accurate also (and of value, if the inquirer is interested in learning something new).
And I think gut guys can be a lot more arrogant (the Brownlee quote comes to mind).
They also happen to be the guys in charge or working in hockey media, so if anyone should have a chip on their shoulder it's the reasonably intelligent fan tired of hearing "What these guys need is a big momentum swing right now..." pass for analysis.
Is it so crazy to make PTS/60 a mainstream stat?
The tone of many people on HF Boards (and media) is almost-literally "Oh no! Stats!," with the same grumpy befuddlement that you might expect from seniors setting up a recording on their VCRs...
Some items, on the post and the comments:
1) When it comes to gut guys/stat guys, neither has a monopoly on arrogance. I've seen plenty of high-handed and obnoxious responses from guys on both sides of the fence - and I'd guess it's because there's very little correlation between willingness to use math and playing well with others. I haven't run the numbers, but that is my suspicion.
2) As for all the knocking of the fellows at HFBoards, lets not forget that there's good content being generated out of there as well. Calling them all "retards" or whatever is in the same range as lumping analytical bloggers in with Eklund.
"The NHL is now tracking 'hits', but the traditionalist knows that what is counted as a hit in Dallas, might not count as one in Montreal."
This from the guy who at one point last season tabulated "defensive points" by adding hits and blocked shots together.
That Staples has come a long way.
# posted by : September 10, 2008 4:29 pm
: September 10, 2008 4:29 pm Marc Crawford says they tracked scoring chances for & against all year with the Kings;
I just read an article somewhere that noted that the Oilers were tracking EV+ EV- in the AHL this year, as well.
Not to get all Dr. Phil, but I think a real part of the problem has been a lack of respect from both sides of this rather ridiculous disagreement. I'm far from a pure stats guy...I'm not gonna lie, I've seen some probably brilliant articles from Vic and MC that I didn't read because the math bored the shit out of me (and I should add, I've done about 4 million years of school with a PhD and a faculty job to prove it, so I'm very used to being bored)...but I still get pissed off by people like Brownlee who dismiss the stats entirely. The suggestion that the 'stats guys' don't WATCH hockey, or lack insight/understanding of the game, is frankly insulting.
That said, I think some of the "non-stats" guys are reactionary due to the tone of some of these blogs. I've seen some really insulting comments aimed at individuals or entire message boards in these comment sections. Most often these comments are unjustified, hyperbole, and flat out immature, and most often they originate from bloggers and frequent commenters (i.e. members of the blogging community).
Frankly, people take this shit too seriously. I found Staples piece to be condescending, and I find that both sides of this debate show a real reluctance to consider and respect other opinions. It's the internet, you don't always have to be right.
# posted by : September 10, 2008 4:47 pm
: September 10, 2008 4:47 pm @Matt: I dunno, maybe what Vic is doing is just different than what I'm used to, but none of the posts have had the sort of things I've absolutely had to have in my manuscripts and lab reports in order to not get laughed out of science forever, like those that I mentioned. And if this shit has been done for years already and long since validated, then fine, but when he gets into dice rolls and stuff, he just loses me.
And no, it ain't no sin to pass on subanalyzing anything -- unless you are trading on being an expert on it. I would be appalled at a legal scholar who decided he didn't need to read any more law articles, or a historian who announced he was done reading history books. At the very least, I can't imagine that they would wear their disinterest in learning more as a fucking badge of honour.
And that's fair. I never said Brownlee was right: I think that attitude is absurd. I'm just saying, you're talking down to people who don't care about the numbers, and tarring them all with the same brush, and that's exactly what gets people's backs up. And now that I've said that totally un-ironically:
As for all the knocking of the fellows at HFBoards, lets not forget that there's good content being generated out of there as well. Calling them all "retards" or whatever is in the same range as lumping analytical bloggers in with Eklund.
I was referring to the sorts who would troll Vic, Dennis, Tyler, et al for their stats work, the sort who rip LT for his baseball references. Not the whole board. I'm sorry that wasn't clear.
I don't think that I was "crowing" about predicting a 20th overall power play. If I have the right LT thread in mind, I'm pretty sure that I was pointing out that, for as much as AO wanted to kick the stats guys for not expecting the OIlers to have an unbelievable amount of shootouts AND an unbelievable amount of shootout wins, the team still sucked.
It seemed that way from where I was sitting, and it surprised me, since you had (justifiably) ripped Proteau a new one for his 12th place "success" the previous summer.
The PP is a volatile thing. Shooting percentage seems to matter a lot and, last year at least, it was pretty damned volatile. All of the Oilers fans with hardons for how awesome the PP is going to be this year might want to consider that before concluding that the bad games don't count and that it'll be at the top of the league this year. Not saying that's you Doogie2K and I'm not really making any comment on what I think the PP will be yet; just saying that, while the Oilers had a huge swing last year from the first half to the second, they weren't alone and as tantalizing as it is to to take that the second half resurgence, tie it to other things that we know happened and assume that they're causative (Souray's return, Gagner doing well), I'm hesitant to do so.
I understand that. It just feels to me like the bad period is being considered a bit more indicative than the good period, when frankly, I don't think either say much by themselves, and certainly, are of only limited predictive value for the '09 Oilers, all things considered.
@Dubya: You just said what I've been trying to get at all this time, but continually let my Scottish temper get in the way of. Thank you.
Matt's point in this regard was: noting how a stat syncs up with the obvious is a short-hand way of testing a metrics validity. Therefore if it does do that reasonably well then the less obvious findings that emerge in the gray areas and are potentially counter-intuitive are probably accurate also (and of value, if the inquirer is interested in learning something new).
If only that were true. If a test passes on an obvious comparison that does not logically imply that less obvious comparisons will be even "probably" accurate. It just does not. While it may challenge your opinions, and it may force you to look at things in a different light, you're making a huge jump between obvious and not, and I'm sorry but it's not supported by logic.
Actually, now that I think on it more:
what if the fact that hockey has more numerous and complex interactions than baseball makes it more statistically predictable, not less?
What if the fact that hockey has more numerous and complex interactions than baseball makes it less statistically predictable, not more?
Isn't that more logical?
Ender, the word "possibly" would have been a better choice there... a reasonableness test, like you say, means unreasonable = incorrect, but it does not mean reasonable = correct.
And Doogie, the question was posed quite honestly (not rhetorically). I don't think the answer is Yes or No necessarily (logically). The key is not really whether one is more complex than the other; it's whether there is more luck involved in one than the other, and I don't think either A or B is an obvious answer there (the spread around .500 in baseball is awfully tight too).
Ender, the word "possibly" would have been a better choice there... a reasonableness test, like you say, means unreasonable = incorrect, but it does not mean reasonable = correct.
And that's fair, but if so, I didn't see any actual reply to Lord Bob's very relevant statement:
So saying that a statistic is good at addressing the self-evident is nice, but not helpful.
@Matt: I don't think you're seeing my point here. Take a step back for a minute and ask yourself, does it really make sense to assert that a system with "more numerous and complex interactions" can actually be modelled more simply, particularly when there appears to be only a handful of correlations and some unpublished bell curves driving such a statement? How do you model luck? How can hockey possibly be modelled, in any facet, with dice?
Maybe Staples doesn't like the stats that bloggers use because he didn't think of the stats himself first.
Hey lookit, another debate on lies, damned lies, and Statzis. By golly, someone even mentioned my name. What with the benches being cleared already and a big pile of gloves on the ice, I would be out of form to not wade in. Especially with all those juicy quotes to choose from ...
I'm sorry Matt, but I do find that some "stats guys" can be very closed-minded. Look at how long it took for Bruce's suggestion (and data!) that outshooting doesn't necessarily result in victory to even be half-assed accepted.
Thanks for that, D2K. It was an interesting discussion, wasn't it?
What seems to repeatedly go over the heads of some contributers is that nobody once suggested that dominating at any one of these underlying numbers is a guarantee of victory.
Yes of course, Slipper. Anybody who has seen a goalie, three iron pipes, and/or a referee steal a game knows this. What surprised me -- which is why I posted it -- was how weak is the correlation between outshooting and winning in the four separate, admittedly targeted studies that I did. What also surprised me was how the most statistically-minded on the 'sphere seemed to be the most reluctant to accept this new information.
We're evaluating who the better team is. I think it's widely accepted that the better team doesn't always win in sports.
Of course. It should also go without saying, Slipper, that you and I won't always agree who actually was the better team. You might go for the team with the shots and Corsi numbers and possession, and I might go for the team with the hot goaltending and the shot blocks and the opportunism. Fortunately we do have the scoreboard to settle the argument as to which team was, if not "better", more successful. I will always keep hearkening back to that, even as we advance the discussion about possession and goaltending and all those good things that make up this grand game.
As an accident of birth, I'm a numbers guy myself. I can cite you hockey statistics out of my ass, and no doubt I do all too frequently. It's a great way to try to get a handle on the game, getting better all the time with all the new methods you guys have been churning out. But I know all too well that there are many aspects of the game that can't be captured by a number or a ratio. Even if we could figure out what's important to measure, the actual compilation of the data is too labour-intensive to be practical. So we observe, we sample, and we theorize. The process is necessarily far from complete, so our theories are sometimes wrong ... at least mine can be. It is through the invalidation of the old theory, or its refinement, that understanding advances. So it is with any science.
Rereading that, my current theory is that I'm getting too full of myself and my theories and it's time to change the subject. But I'll leave that out there for reflection and ridicule as suits your taste.
I'd rather bash my head in with a hammer than discuss momentum, clutch and big goals from game to game, or how MacTavish should have played Garon form day one of last season.
I want to know who's playing against who, in what end, where they are getting the chances from, who's taking the defnsive/attack zone draws, etc, etc.
So you want to know who's playing against who, unless it's Garon?
:D
That's my interest in the game of hockey. If you can't accept that, why the fuck are you reading places like this?
Yes, of course I can accept that, Slipper. Can you accept that others who benefit from your insights may have a different but nonetheless serious interest in the game of hockey?
But really, haven't we all made these arguments a thousand times before?
I have no doubt that you probably have, Slipper. It seems that statement is always part of your argument. To some of us this is a fairly new thing, and while we all need to be educated in turn in this big one-room-schoolhouse we call the Oilogosphere, it is nonetheless possible that a newcomer may bring a fresh perspective.
Maybe Staples doesn't like the stats that bloggers use because he didn't think of the stats himself first.
Bill Needle: Actually, turn that right around and it's a logical explanation of the icy reception Staples Errors have received among the Statzis. To this lifelong fan of the game the formal assignment of blame for a bad result is an interesting concept, yet all some ever do is simultaneously "ignore" it and beat it up. At times it seems like an attack on the source of the stat rather than the stat itself.
I've seen plenty of high-handed and obnoxious responses from guys on both sides of the fence
How about these two recent beauts from the opposing corners?
"I'm not the least bit interested in these numbers. I know what I see and I know what I think. I'll go with that over pages of statistics any day."
"Errors (whatever the hell they even are, I've never been a Staples reader) and scoring chances are both subjective, leading some people to think that they somehow equate."
Ignorance and arrogance in equal measure make a potent cocktail.
I'd guess it's because there's very little correlation between willingness to use math and playing well with others. I haven't run the numbers, but that is my suspicion.
Jonathan, you are a joker, but there is truth in what you say. I have seen plenty of sticks come up on the 'sphere -- it can be quite entertaining at times -- but those who have been rude to me personally are mostly among the Statzis. Having been an unpopular math whiz myself as a child I thought I might find an affinity with such folk, but that hasn't proven to be the case. I won't name any names, but let's just say that for six months I completely withdrew as a commenter from IOF because I was getting gang-banged every time I went in there. Call it the Philly Flu.
How can hockey possibly be modelled, in any facet, with dice?
D2K: W-a-a-a-y back in the day I used to play Strat-O-Matic hockey. Alas, it wasn't too realistic.
W-a-a-a-y back in the day I used to play Strat-O-Matic hockey. Alas, it wasn't too realistic.
Yeah, I preferred the card game I had when I was a wee lad. I don't remember much about it, other than the fact that it was like Mille Borne, it had a Habs deck and a Nordiques deck, and there was no "leave the bench" card, which made no damned sense, because that's just what the Habs and Nords did back in the day. My dad and I only played it a couple of times because I think I was too young to get the rules at the time.
Matt:
1. First off, if in my original post on this matter I implied that the stats guys lack passion for watching hockey and aren't mad dog fans of the game, I apologize. That's certainly how you took my post, so even if wasn't my intent, I'm guessing there was some line or word that triggered that response.
The truth is I see you and Dennis, Tyler and the rest of the Stats Guys as totally obsessed with hockey, maybe not soft-hearted, fun-loving fans, but nonetheless enraptured by the beauty and skill of the game, along with the question of how can we best rate high performance in the NHL.
2. Slipper, I can see that you're tired of dealing with the same old questions, and with the no small amount of scorn directed at the statistical work of the Stats Guys.
Fair enough.
I think that scorn is very often out of line as well.
That said, I don't see the kind of questions asked by folks like Bruce, Performance Oil and Doogie2K as being generally scornful.
It's no doubt tedious to have to answer the same old questions and deal with the same old doubts, especially if you're convinced that major breakthroughs in thinking have been made. But what's needed is a F.A.Q. on all the major new stats and concepts that the Stats Guys talk about, so they can simply refers doubters and newbies to that and have all their questions answered.
Perhaps this is something Lowetide could produce, as he's got that educator's streak to him.
3. Matt, you write about some major findings of the Stats Pack: "At least insofar as I've absorbed it:
EV+/- is a better predictor of future record than is past record.
Best predictor of EV outscoring is where the (past) end-of-shift faceoffs occurred, followed somewhat closely by past Corsi #, followed somewhat less closely by past EV outscoring."
I'm guessing if the average fan could fully and completely understand this short summary, he might well be very excited by some of these discoveries.
So I'd ask you, or Lowetide, or anyone else who has a full grasp of all this, that as a favour to that average fan (me) -- and if you want to take up this challenge -- to pretend that there's an exam question that states, "The major findings of the Edmonton School of Hockey Analysis from 2000 to 2008 was that EV+/- is a better predictor of future record than is past record. The best predictor of EV outscoring is where the (past) end-of-shift faceoffs occurred, followed somewhat closely by past Corsi #, followed somewhat less closely by past EV outscoring.
To answer this question, first define the terms used, then spell out the main arguments used to support each finding, along with a brief summary of the facts that support these finding."
If only that were true. If a test passes on an obvious comparison that does not logically imply that less obvious comparisons will be even "probably" accurate. It just does not. While it may challenge your opinions, and it may force you to look at things in a different light, you're making a huge jump between obvious and not, and I'm sorry but it's not supported by logic.
Yes, yes, of course it's not necessarily true. As Matt says, it's a "rule of thumb" method of testing the reasonableness of a given measure without running numerous trials, discovering p values etc. Quick and easy, it's hardly full proof or, as you, totally logical.
I discussed this issue with a member of the Calagary media recently and my comment to him was we are just learning which of these new measures have real predictive of descriptive power. For the guys like yourself who seem totally opposed to statistical anaylsis, I wonder:
1.) Is it your contention that hockey can't be honestly or reasonably analyzed beyond qualitative judgements and counting numbers (goals and assists) because of it's nature (too fluid, too complex, etc)?
Or...
2.) Are you simply unimpressed by the current metrics being developed and employed by the bloggers and such right now (you see them as invalid or not powerful) and believe that better measures will be or could be created?
I think I can certainly accept and discuss the issue with anyone who has the second attitude. The first attitude - and I think that's the one Matt is critical of in this post - is just willful ignorance.
Well said Kent. The fact that so much happens in the 60(+) minutes of a hockey game may well be a feature, not a bug, when it comes to modelling it statistically. No model will ever predict whether a goal will go in off of Peter Sykora's leg tomorrow, but it's entirely conceivable that with detailed data collection and enough brain & computing power, you could predict the number of goals-off-legs in the NHL this season with remarkable accuracy.
And David, it's funny you bring that up (again), as I was just mulling that this morning. It's time -- I'll try to spend some of the next couple of weeks doing just that, barring something like life intervening.
@Kent
Both, actually. As I see it, statistically, you will never have a large enough sample space to say anything with any sort of rigor about any player or team, due to so few games per season where the rosters of both teams are even mostly the same (4 games of edmonton vs detroit, for example). As I said on a thread in CiO, all of the current stats can be made irrelevant due to particular styles of play, so you can't really use a stat to say something about a team over a season...
...aside from wins. And at the end of the day, goals are the only thing that give you wins, and even GF/GA is not a good indicator of how successful a team is. Hypothetically, a team could win 75% of their matches by 1 goal, and lose the rest by 3. Not really going to tell you a lot.
My real issue, and the reason I've been posting about this topic is that I find (whether intentional or not) that the Statzis (as Bruce puts it) are treating their number work as law, and anyone who disagrees with them are close-minded, arrogant, ignorant dumbasses. The issue is that their numbers are not law. There are some real issues with them.
You want a good offensive stat? Measure the shot percentage by player in particular places on the ice vs either certain goalies or goalies with similar styles. Split it into whether the player is covered or uncovered.
You want a good defensive stat? Use the Staples Error, but have 20-30 people all evaluating it and cross-reference the result. Goalies are exempt from this stat.
You want a good Goalie stat? Short side goals + rebound goals are errors. Miraculous saves don't balance it out because generally miraculous saves are there because the goalie was out of position in the first place.
None of these stats I'm suggesting are perfect, and some are *gasp* subjective, but I'm fairly confident they'll give you much more meaningful results as far as success of a team goes, and as far as how players are used by the coach.
Knowing how many goals will go in off of legs will never tell you which game they'll come in or whether or not they'll be a game-winning goal, so to me, it's a moot point.
the Statzis (as Bruce puts it)
Not my term. It was raised in one of the long debates last winter by one of the Statzis themselves (Tyler?), and it sounded like it had been around for awhile. I have to admit it made me laugh.
What surprised me -- which is why I posted it -- was how weak is the correlation between outshooting and winning in the four separate, admittedly targeted studies that I did. What also surprised me was how the most statistically-minded on the 'sphere seemed to be the most reluctant to accept this new information.
We could exchange targeted studies or anecdotal evidence until our fingers are numb. I don't recall all four examples you speak of, but based on previous defenses, I'm guessing the stellar, yet flukey, 2006 cup run was in there, right?
The team was good, the goalie was fucking great. The lesser team won. Almost.
That's how we should remember it, amirite?
These are the redundant exercises in logic and wit that these discussions always reduce down to, and I'm equally at fault. I know this.
But the goalie is part of the team!
And they acquired him knowing that this was the spring he was going to outperform his career averages in order to successfully out "counter" the obviously superior Detroit team.
I could just as smugly say how unsurprised I was when there was a signifigant correlation between winning and outshooting and outscoring at even strength in the 2007 playoffs. 11 or 12 out of 15 series if I recall correctly, and one or tow were so close it was a coin flip either way.
I don't think anyone would be surprised that the same correlation would exist for this past spring, ecspecially consideirng who won. Why? Because there were few to no underdog, fluke wins. Just like in 2007.
So there aren't any defiant fans specualting wildy about how their teams won because they won't accept what's evident: their team got lucky at the right time.
Afterall, when someone describes something as "hot", isn't luck what they're usually refering to? If I have a pair red hot dice in my hand, am I just getting better at throwing them?
Lastly, it's now even the Bruce M's (only David Staples is allowed to refer to him by his full last name;D) that I was refering to in the first place. Although sometimes I do feel that he's set to automatically awaken from cryogenic sleep as soon as I hit enter on my keyboard so he can dissect my posts to the very sentence and respond in kind, his points both bulleted and italicized.
If anything I was trying to express that the processes undertaken around here are constantly being refined by their proponents. People weren't satisfied with EV+ EV-, and began looking at on and off ice SV% and Shooting%, shot differential, Corsi, shot distance, etc.
It's a work in progress and those within the niche seem to accept that the methods aren't infallible. Yet, that doesn't stop those outside the niche from getting their panties in a knot when they find some anecdotal evidence that runs contrary to these methods and then acting shocked that in light of this "evidence" the whole process doesn't come to a crashing halt.
and even GF/GA is not a good indicator of how successful a team is.
Are you sure about that?
IIRC, the Hockey Analytics dude was able to predict standings based on GF/GA alone rather accurately.
@Slipper
I don't know if it's just me or not, but that's not really the point. I would never say "Stop it! It's useless!"
I might think it, but that's another story.
My point is, and always has been that it doesn't matter who the "better" team is. Good coaching an team play can generally trump "better." The only thing that matters is who wins. Detroit was the better team this year, because they won. They didn't win because they were the better team. That's what I'm always arguing. It's fallacious. Same as "fake wins (SO)." It's a win. It doesn't fit into your numbers, but it's a win, and at the end of the game, it's all that matters. Deal with it.
IIRC, the Hockey Analytics dude was able to predict standings based on GF/GA alone rather accurately.
Teams outperform and underperform the GF/GA yearly. Whether or not it's an adequate (and who is defining adequate?) metric for the regular season, well, I say meh. It doesn't tell you who will win the next game of Detroit vs Edmonton, so explain to me why I should care. I mean, we're talking about using retroactive stats to argue "my dad could beat up your dad" here, aren't we?
Just a question:
Does anyone else read these debates and immediately think, man, these Statzis sound just like the early proponents of the behavioral revolution?
# posted by : September 11, 2008 11:38 am
: September 11, 2008 11:38 am When it comes to gut guys/stat guys, neither has a monopoly on arrogance. I've seen plenty of high-handed and obnoxious responses from guys on both sides of the fence - and I'd guess it's because there's very little correlation between willingness to use math and playing well with others. I haven't run the numbers, but that is my suspicion.
Now that's funny
They brought a SPORTS PSYCHOLOGIST in, everyone got injured, and they were on with a bunch of people younger than me leading the way. I personally assume the big change wasn’t the injuries, but rather it just took that long for the COUNSELING to really set in.
Why are you posting here, Ender?
...goals are the only thing that give you wins, and even GF/GA is not a good indicator of how successful a team is.
I mean, seriously, why?
Detroit was the better team this year, because they won. They didn't win because they were the better team. That's what I'm always arguing.
WHY? Why are you wasting your own time always arguing that point?
Ender said: "I would never say 'Stop it! It's useless!'"
Ender said: "and I’m ok throwing their opinion right out the window."
Fallacious- adjective
3.disappointing; delusive: a fallacious peace.
You are fallacious.
But the goalie is part of the team!
And they acquired him knowing that this was the spring he was going to outperform his career averages in order to successfully out "counter" the obviously superior Detroit team.
I can specifically remember a lot of talk about the Oilers wanting Roloson because of his rebound control - that they had the personnel on D to pick up a rebound and get the puck moving out of the zone (centered around Pronger). So isn't it possible that they built a gameplan around this and that Roloson's stellar save percentage in that run was due to the concept of allowing the low percentage shot and regaining possession, not regaining possession because all of a sudden Roloson is a SP freak?.....maybe someone in the Oilers organization ran some stats?
@slipper
Thank you for that excellent example of using data (or stats, as they call them in sports) out of context to create a point.
Is it possible, Oilman?
I guess so, anything is possible.
I doubt that it's true though. I don't think any wise team gameplans to get outshot or outchanced 1.5 or 2 to 1. It just sounds ridiculous.
I love how Slipper pounced on that sports psych bit like a rabid tiger.
It's science, bitches. It works. Just ask the Australians.
Ender wrote, "You want a good defensive stat? Use the Staples Error, but have 20-30 people all evaluating it and cross-reference the result. Goalies are exempt from this stat."
Ender, this would be a helluva good way to score an error, and it's what I've been attempting to do by posting my results and my thinking every Oilers game on who should get an error and who should not.
That way, when I get it wrong, or when I'm not following the set criteria, or if the criteria needs adjusting, I can make the necessary changes.
No doubt about it, if 20 or 30 people spent the time to go over the criteria for assigning errors (I'm going to update it soon), then watched the video and also assigned errors, and we all took a vote on who deserved the error, the end result would be more credible, and we'd all learn a lot about the strengths and weaknesses of the error.
But to do all that, that's asking a lot of people . . .
Matt, there's no rush in producing such F.A.Q.s or answering the hypothetical exam question, as that kind of thing takes a ton of work. But if you get to it sometime this season, much appreciated . . .
P.S. On another blog, someone posted under my name, "david staples," a pretty low thing to do, but the kind of thing I've come to expect from various skunks.
So if you read some post and it sounds highly intelligent, not shallow, not condescending and doesn't smack of cowardice, then it's almost certainly not me that posted said comment, but some other guy pretending to be me and posting under my name.
Oilman, if that's true, it makes a lot of things make way more sense to me. I always thought Roli was a shitty goalie, and the only reason he stood up in net was because of the defensive style.
Roli got re-signed before Pronger left as well. Food for thought.
I mean, we're talking about using retroactive stats to argue "my dad could beat up your dad" here, aren't we?
Um... huh?
Teams outperform and underperform the GF/GA yearly. Whether or not it's an adequate (and who is defining adequate?) metric for the regular season, well, I say meh.
You're right, a formula that can predict a team's results with an accuracy of about 94 per cent is a total waste of our time.
I can specifically remember a lot of talk about the Oilers wanting Roloson because of his rebound control - that they had the personnel on D to pick up a rebound and get the puck moving out of the zone (centered around Pronger). So isn't it possible that they built a gameplan around this and that Roloson's stellar save percentage in that run was due to the concept of allowing the low percentage shot and regaining possession, not regaining possession because all of a sudden Roloson is a SP freak?.....maybe someone in the Oilers organization ran some stats?
Probably stats like these, Oilman:
2002-03: 50 GP, .927
2003-04: 48 GP, .933
Roloson had had the top Sv% in the NHL the previous season, the second best in the history of the stat in fact, but after the lockout Manny Fernandez got a leg up on him and Roli languished on the bench. He couldn't win a game to save his soul but he wasn't playing that bad (.910 Sv%) in his one game a week, he just wasn't doing enough to win favour with Uncle Jacques. :) So going for Roli was a pretty goddamned good gamble, it just required a GM/organization willing to look beyond that 6-17-1 record at the bigger picture.
I remember doing our pre-trade deadline predictions on the old Officepools Powerboard that Roloson would be a perfect fit in Ottawa, who were a goalie away from making a run at the Cup (Hasek having gone down in Torino). Roloson was solid enough to succeed on a legitimate contender, I figured. Turned out that I was both wrong, and right.
I never said "a total waste of [your] time." It all depends on what you're looking for. Me, I want to know who will win X game. You, apparently (and I'm sorry if I'm misrepresenting you) want to know who will rise to the top of the standings. My issue is that the latter gives you who is the "better" team but it doesn't give you the answer of who will win between team A and team B. Only in the playoffs do teams play each other often enough in a short enough time period to get a meaningful data table out of, ranking Team A's style vs Team B's style.
Long story short, the regular season is a very long exhibition round, and since everyone plays different teams a different amount it's not as easy as throwing a weighted die. It's throwing 30 weighted die (if rosters were exactly the same from beginning to the end of the season).
No stat is useless. It's all about context. but yes, I will go out on a limb here and say that I do strongly feel that the stats people use on these boards don't necessarily give any of the information people seem to think that they do.
Roli got re-signed before Pronger left as well. Food for thought.
I figure Roloson getting resigned was a foregone conclusion - he played extremely well - and with the shit kicking Edmonton was taking, not signing Roloson would have been another PR nightmare (unless you could have traded Pronger for Luongo).
Slipper: I don't think the plan was to get outshot and outchanced by a wide margin - that's that's a shortsighted simplification. I think the plan was to limit the high percentage chances and let other teams fire off as many low percentage shots as they were willing to take (Detroit is more willing than almost any other team) knowing you had a good SP goalie with good rebound control and one of the best D-men in the world at regaining possession off the rebound and starting the play moving the other way. As I recall, it worked to near perfection, and if this high SP, rebound controlling goalie had not been knocked out of Finals game one, we could remove the near part.
We could exchange targeted studies or anecdotal evidence until our fingers are numb. I don't recall all four examples you speak of, but based on previous defenses, I'm guessing the stellar, yet flukey, 2006 cup run was in there, right?
Absolutely. It was pretty conclusive from team "total shots" data -- that I compiled that heady spring long before I heard Jim Corsi's name associated with them -- that based on shots and by extension possession, the Oilers outplayed just one of their four opponents that spring: Carolina. The teams that won the four series out-Corsied their opponents in exactly one of the 24 games. It was pretty goddamned astonishing. The Oilers clearly departed from their game plan in the Carolina series, perhaps due to Roli's injury, and went back to the "outplay but don't outscore" formula of the Fricklin and Frakkanen duo that had frustrated Oiler fans from October to February.
The team was good, the goalie was fucking great. The lesser team won. Almost.
I'll repeat myself since you apparently didn't read it the first time:
It should also go without saying, Slipper, that you and I won't always agree who actually was the better team. You might go for the team with the shots and Corsi numbers and possession, and I might go for the team with the hot goaltending and the shot blocks and the opportunism. Fortunately we do have the scoreboard to settle the argument as to which team was, if not "better", more successful.
The other examples were all Oiler-related, which is what I mean by "targeted"; I'm an Oiler fan first and I want to understand what makes them tick even as I will cheerfully admit that they are probably not very representative of the league as a whole. They included a detailed analysis of the Oilers' last 20 games of 2007-08 (of which 16 were won by the team being outshot, usually by a large margin); an analysis of Oilers record when outshooting or outshot over MacTavish's coaching career (.535 in 273 GP when outshooting, .551 in 277 GP when outshot); and of the Oilers five Stanley Cup seasons, in which they were outshot by their collective opposition in all five regular seasons. From these four negative-to-strongly-negative correlations between shots and winning, I drew the rather straightforward conclusion that to the Oilers at least, outshooting isn't necessarily a big part of the game plan.
But when I put all that out there, I didn't get so much as a "Hmmm, that's interesting" from certain quarters that didn't seem too receptive to any new evidence that didn't support their pet theories. Or so it seemed at the time.
But based on that evidence, this Oiler fan isn't going to get my shirt in a knot when this player or that has a big red Corsi number. It's one way to analyze the team, sure, and it's interesting, I'm just not sure how relevant it is.
Afterall, when someone describes something as "hot", isn't luck what they're usually refering to? If I have a pair red hot dice in my hand, am I just getting better at throwing them?
You guys and your dice. When the word "hot" is followed directly by the word "goaltending", luck is only part of it. Goddamned right the goalie is part of the team. Flow-of-play stats like shots, Corsi, Fenwick, or the hopefully-soon-available scoring chances do not adequately account for the most important player in the defensive zone. So they're not entirely reliable in my view, either as analytical or predictive tool. (Maybe they would work better in basketball!)
Bruce, you have some interesting points on Corsi, although I don't have the full history of debate about this with you and Vic, Tyler or whoever. Another reason why you may want to start a blog or join an existing one -- it's easier to be heard.
That said, some other dudes on this thread seem to be mixing that up with the usefulness of stats-as-predictions in general, which is completely weird. Asking for certitude when the matter is probability seems to be missing the point.
I mean, if the variance of predicting top goal scorers based on Corsi numbers is too weird or doesn't match up, it's maybe not the best tool. I may be reading this wrong, but I don't think it's even a go-to stat for the bloggers that brought up anyway. I'd still like to see more about Corsi numbers in general.
Either way, it should be able to predict something relatively well, like GF/GA does almost all of the time.
a formula that can predict a team's results with an accuracy of about 94 per cent
When you guys figure that out, sign me up. I've got some money to win.;o)
But when I put all that out there, I didn't get so much as a "Hmmm, that's interesting" from certain quarters that didn't seem too receptive to any new evidence that didn't support their pet theories.
I recall that at the time that myself and perhaps Tyler both commented that the team outshooting or out Corsi-ing wasn't neccessarily the better team, but for cases where there was a commanding lead. Other things must be considered with this "work in progress" including on ice sooting and save percentage.
I don't know what else would have satisfied your ego in this case, considering you presented what you admit is a selective sample.
I'll repeat myself since you apparently didn't read it the first time:
Yet, that doesn't stop those outside the niche from getting their panties in a knot when they find some anecdotal evidence that runs contrary to these methods and then act shocked that in light of this "evidence" the whole process doesn't come to a crashing halt.
Any number of guys around here, if compelled to do so, could do a selective sample of the '07 Ducks and '08 Wings and present what would appear to be a slam dunk in favour of Corsi. It wouldn't be any less fair or relevant than your argument.
I don't know what else would have satisfied your ego in this case, considering you presented what you admit is a selective sample.
Slipper: Said sample covered, in one way or another, 12 of the Oilers 28 seasons. So it's pretty significant w.r.t. the Oilers if not the league as a whole.
Yet, that doesn't stop those outside the niche from getting their panties in a knot when they find some anecdotal evidence that runs contrary to these methods and then act shocked that in light of this "evidence" the whole process doesn't come to a crashing halt.
The last thing I want is for the process to come to a screeching halt. I want to advance the discussion. But when I try, there is a certain niche that would disagree with "black is black" depending on who said so. I would have expected this process to be a little more inclusionary.
Slipper makes a series of good points there. I vaguely recall, thinking back on it, that I was objecting on the basis that the "outshooting" focus of the statzis is largely related to ES play, which, in fairness to Bruce, he wasn't around for in 2006 when we started kicking at it.
Vic made the point that looking at the Gretzky led Oilers might not be the best way to suss out whether teams that get outshot usually end up winning the game, which also strikes me as a fair point.
But when I put all that out there, I didn't get so much as a "Hmmm, that's interesting" from certain quarters that didn't seem too receptive to any new evidence that didn't support their pet theories.
Well, you got a thoroughly researched post out of me, showing that over the past 15 years, there's been a real advantage for the team that outshoots.
# posted by : September 11, 2008 4:41 pm
: September 11, 2008 4:41 pm The last thing I want is for the process to come to a screeching halt. I want to advance the discussion. But when I try, there is a certain niche that would disagree with "black is black" depending on who said so. I would have expected this process to be a little more inclusionary.
Must just be you. I've been welcomed with open arms.
12 of the Oilers 28 seasons.
Seven of which are post loser point which, as I pointed out in my post, has been a huge benefit for the teams that get outshot.
# posted by : September 11, 2008 4:44 pm
: September 11, 2008 4:44 pm I suppose an interesting question would be whether there are any consistently very good teams, not featuring guys who are in the Hockey News Top One Player of All Time, that get consistently outshot. I'd guess no, but I don't really know the answer.
# posted by : September 11, 2008 4:51 pm
: September 11, 2008 4:51 pm This is my conclusion:
1. Hire a sports psychologist
2. Get outshot 2 to 1
3. While not committing alot of errors
4. Draft Wayne Gretzky
Seven of which are post loser point which, as I pointed out in my post, has been a huge benefit for the teams that get outshot.
MC: On this we entirely agree. As I recall it was one of my conclusions at the time; certainly I made numerous posts about MacT's ability to play the system, and how the system does not reward attacking hockey. The Bettman Point is truly asinine.
The comparisons between 2000-08 Oilers and 1983-90 Oilers could hardly be more different ... yet both indicated outshooting was at best a non-factor, for an extended sample (1/4 of franchise history in each case).
I suppose an interesting question would be whether there are any consistently very good teams, not featuring guys who are in the Hockey News Top One Player of All Time, that get consistently outshot. I'd guess no, but I don't really know the answer.
Neither do I, but it's an interesting question worth further thought. I'm certainly ready to agree that the Oilers of the 1980s were Exceptional for the reason you state. If nothing else they/he proved that it is possible to win without dominating the shot clock. But maybe they really are the exception that proves the rule.
Too bad we don't have team shots data before 1983, at least not on Hockey-reference.com which is my touchstone source. The same data shortfall really cripples historic Sv% comparisons. Do you know of an alternate source? Shots have been counted for as long as I can remember.
On this we entirely agree. As I recall it was one of my conclusions at the time; certainly I made numerous posts about MacT's ability to play the system, and how the system does not reward attacking hockey.
I wouldn't say that MacT necessarily knows how to play the system. I'd say that the new system is superficially kinder to mediocre teams than the old one. It's not like MacT is pumping out 100 point season after 100 point season.
I still think that it's worth noting that the only MacT team that did a damned thing in the playoffs is the one that really put the boots to the opposition in terms of shots in the regular season. Harder to do that in the playoffs because you're playing other teams that can do that too but it's noticeable to me.
# posted by : September 11, 2008 5:17 pm
: September 11, 2008 5:17 pm I wouldn't say that MacT necessarily knows how to play the system. I'd say that the new system is superficially kinder to mediocre teams than the old one. It's not like MacT is pumping out 100 point season after 100 point season.
Nope, he's scraping to get every point he can get.
-- In 2005-06 Oilers made the playoffs by virtue of playing more overtime games than anybody and winning half of those bonus points.
-- In 2006-07 the team couldn't get to OT and was uncompetitive.
-- In 2007-08 the team led the league in both # of overtime games and overtime Pts%, a double that is not likely to be repeated for a very long time. In fact they had the fewest number of OTL in the league despite playing the most OT games. If you look it up I'm sure they were outshot in the lion's share of those games but still scored 1 or more often 2 points. What appeared to be another uncompetitive team very nearly made the dance as a result.
Sure there was happenstance: not all of those OTs were a matter of choice to sya the least, and both the shootout success and Cogliano's OT hat trick seemed an act of the hockey gods; but I'd still call that playing the system.
Sure there was happenstance: not all of those OTs were a matter of choice to sya the least, and both the shootout success and Cogliano's OT hat trick seemed an act of the hockey gods; but I'd still call that playing the system.
I see there as being a rather large difference between "benefiting" from the system and "playing" the system. I'm not convinced that MacT is playing it.
# posted by : September 11, 2008 6:11 pm
: September 11, 2008 6:11 pm This is gotta be one of my biggest points of contention.
A is true. A in this case being that the Oilers played in overtime lots. One of the reasons is B. B being that they didn't win in regulation.
My question is, how is it people come to the various conclusions for C?
C for particular argument ebing that they played to get to overtime.
Or C is a strategy employd by the coach.
All by virtue of result A occuring, people divine all sorts of myths as an explanation.
It's baffling. What did MacT really do in order to reachOT more? Press the swithc in Huddy's ass the prevents goals form entering eother net after the score is tied? Is he a God? WhaT?
WHAT!?!?!
The same goes for theOilers being dominated by the Red Wings. OH! It must be a strategy!
Precisely.
Lowe: We really need to atleast contend for a playoff spot this year Mackie, for the sake of my reputation and the franchise's
Huddy: Well Bubba, I'm on board with you. The key difference I saw between the two seasons was the one year we had Pronger, Peca, Spacek, and depth at forward, while the next year those players defected and then we were decimated by inj...
MacT: Shut you pie-hole you useless twat! The obvious difference between seasons was that we didn't play to overtime more in '07.
Lowe: Oh Lord Craig, you're right! I'm calling Montreal and trading Horcoff for three assets and then calling up Rob Schremp!
Huddy: Boyz, hold on just for a sec. I mean, playing for the tie!? How can we insure ourselves against the bounces. Wouldn't we be better off just trying to win gamesin regulation?
MacT: Winning games in regulation is for pussies. Besides, we want to spread some points around to make it more exciting for the fans the final few weeks of the game.
Huddy: But I... I mean... err if we can play to overtime, it would entail almsot scoring and not allowing goals by force of will alone... why wouldn't we just do that earlier and win the game out right... I mean...
Lowe and MacT: Shut-up you useless twat!!!!
You want a good defensive stat? Use the Staples Error, but have 20-30 people all evaluating it and cross-reference the result.
I think 5 would be enough. If 3 score an error for the same guy, it's an Error. Kinda like how Olympic boxing works, leaving absolutely no room for controversy. :)
But as, say, a targeted project (a few specific games, or even a bunch of selected goals) it would be cool to have as many people as will try to do it honestly and see what kind of consensus there was on who was responsible for the goal. Saboteurs need not apply.
I shadowed Staples for a dozen games or so this past year as a test to see how similarly or differently two different observers judged the same plays, and we probably had 80+% agreement on initial (before consultation) assignment of errors. This raised my level of confidence that this sort of stat could be measured to at least an RTSS level of consistency.
The irony of those flying the flag of neutral science hurling insults like 12 year olds is really too rich. Are we supposed to be more confident that stats guys are neutral or clear headed when they start making infantile interpretations of others' arguments?
# posted by : September 12, 2008 1:30 am
: September 12, 2008 1:30 am What are you saying? That the Nazis and the early proponents of the behavioral revolution were more high brow?
Not so sure about high brow http://coveredinoil.blogspot.com/2008/03/david-staples-is-making-error.html
wonder what Bill James' approach to getting his ideas accepted was, and how much bitch slappin' and dick lickin' was involved?
Bill James did not tolerate fools easily.
Understandable though, to me, anyway. I mean, you put a lot of work into to something, and then when you show people they tell you to put away the calculator and actually watch a game.
I'd rather read an asshole write something intelligent than a nice guy write something stupid.
I am saying, pretty obviously of course, that insofar as individual stats need to be put into their proper context, and this requires synoptic thinking, that I would trust those authors who can defend their stated point of view without excessive displays of indignation or passion. But if one does not adhere to a belief in the ultimate neutrality of statistical analysis--and particularly if one thinks that sports are shot through with phenomena such as anger and the like, and this will have a discernible but not repeatable impact on the data--one is not held to such a standard.
So I do not need to be surprised by how much a discussion about neutral stats revolves around indignation and insult, because I am quite willing to concede the power of emotions in both sport and our analyses of sport.
# posted by : September 12, 2008 11:19 am
: September 12, 2008 11:19 am That is a nice thread Oilman. I made some well thought out posts and some fascinating points in a fair and well argued, albeit passionate, debate.
But where did I write what now?
I am above reproach, yo.
Post a Comment
<< Home
