An NHL scout's traditional toolkit includes a reliable car, a good pen and a tolerance for countless hours on rough roads and in seedy hotel rooms. They are anonymous to the public when they succeed and often have to find a new employer when they fail.
The job is a classic love-of-the-game cliche, perhaps romantic to some with the image of hockey in its purest form: free from huge salaries, no-trade clauses and athletic super-stardom. The NHL salary cap has privileged drafting more than ever.
Whether or not new methodologies produce more accurate results, scouts will be more accountable for the performance of their prospects because of a major shift in scouting methodology that has occurred recently. The new world of scouting places more accountability on scouts and affects how teams forecast players' careers. Advanced statistical models, increased emphasis on measured puck possession skills and more precise and accountable rankings have created this change.
Pat Steinberg, radio host on the Fan 960, said some new advanced NHL statistics have crept into scouting techniques.
"I know of at least five NHL teams that have their own metrics or stats made up and [applied] to scouting," said Steinberg in a message to the Gauntlet. "It's teams embracing the newest wave of legitimate statistical analysis."
Another breakthrough in the advanced statistical analysis of players is an emphasis on possession-based metrics. Essentially, this measures a player's ability to keep offensive play flowing. This is the main focus of many new NHL statistical models and has carried over to the amateur scouting level.
"I've probably talked to seven to nine teams who express that line of thinking, even if they don't state the word 'possession'," said Cory Pronman, a blogger from HockeyProspectus.com. "They indicate it through the way they evaluate talent. I have not gotten close to asking all 30 teams that question."
This clinical approach contrasts sharply with focus on nebulous terms like 'character' that often influences drafting and player rankings. Pronman notes that some teams still prioritize "leadership and character," but argues these "intangibles" may not necessarily lead to actual success.
In this sense, the emphasis on objective, measurable data is important because it potentially signifies a shift away from some traditional means of scouting. For instance, personal interviews with players are usually a factor in finalizing teams' rankings -- these have never been foolproof and can give either overly positive or negative impressions. Similarly, every year there are players who drop in consensus rankings on vague, sometimes rumour-based intangible issues.
Another new way amateur players' potential is being measured is NHL equivalency. Developed by Calgary-based stats guru Rob Vollman, NHLE attempts to estimate how a player's performance will translate to the big leagues. Generally applied to the American Hockey League using a rule of 0.45 NHL points for every AHL point, there have recently been attempts to use it for undrafted players. Major junior hockey and european elite leagues are also now included in NHLE evaluation.
Gabriel Desjardins, an NHL statistics pioneer who has consulted several NHL teams, used a similar system last year to criticize the Winnipeg Jets's selection of Mark Scheifele over Sean Couturier. A year later, this prophecy seems to be coming to fruition with the emergence of Couturier in Philadelphia.
Although the eyes and notes of scouts are still the primary source for NHL teams, it is not certain to remain this way. Several years ago, the Buffalo Sabres even switched to a primarily video-based scouting system, but its success is still being determined. PUCKS is a video system used by 16 NHL teams, including the Calgary Flames, combining detailed video organization with statistical analysis. While many scouts have said that video is at best supplementary, the growth of PUCKS may suggest otherwise.
The Flames, along with a few other teams, have also recently employed Decision Lens, an outside analytics system to help them more precisely rank wild-card players. The Flames used this to help draft undersized but skilled forward John Gaudreau, who currently plays for Boston College, in the fourth round of the 2011 draft. The season after his draft, Gaudreau was named Hockey East's MVP and helped Boston College win the NCAA Frozen Four tournament in his freshman season. The selection of Gaudreau in a shrewd, risk-free spot is a symbol of a different priority with the Flames in recent years. Faced with bare cupboards, the Flames have attempted to prioritize offensive skill since 2007.
Prior to this, the Flames drafted a third and fourth line projected player far ahead of his consensus ranking, Kris Chucko in 2004. Advanced methods and analytics are only part of the reason for this -- Jay Feaster has only been general manager for two years, after all. However, heading into a tumultuous era in franchise history, it seems undeniable that the Flames will need all the help they can get.
The Flames show why teams might resort to new, untraditional approaches to the draft. Despite being a team that spends a lot and therefore expects to win, the Flames have gradually declined to their present limbo between mediocrity and basement- dwelling. One key reason is a lack of value in younger players. Part of this stemmed from former GM Darryl Sutter's aversion to younger players, but this occurred in conjunction with poor drafting.
What is clear from PUCKS and Decision Lens is that Feaster recognizes the Flames's past problems, and perhaps sees a need to modernize in more ways than one. How this will clash or integrate with traditional scouting methods is still to be determined. However, if the Flames embark on their second rebuild phase, history suggests change can only be a good thing. If it works for the Flames, they might be a case study for modernization in scouting and drafting.
While the image of a scouts determining NHL hopefuls' futures based on instinct alone might be fading, statistical and video analysis is paving the way for a new drafting system. The Flames have generally failed to draft well and get little value for younger players, but may see an improvement in young stock if stats instead of gut feelings are used to choose prospects. The NHL will likely have to adapt to the changing landscape of statistical analysis or risk becoming irrelevant.