U of C disputes Maclean’s Rankings

By Katy Anderson

The University of Calgary finished low in the Maclean’s rankings despite releasing an internal survey last week claiming a number of improvements.


Maclean’s magazine published its 16th annual University Rankings issue Thur., Nov. 2 and the U of C remained near the bottom, inching up a spot to 13th out of 15 in the medical doctoral universities category. The Maclean’s survey includes data on student satisfaction, class sizes, reputation, library and finances at every Canadian university.


This year’s rankings did not go off without a hitch. Many universities claimed that Maclean’s’ methods are careless, accusing Maclean’s of only trying to sell magazines. Maclean’s responded by accusing the schools of silencing their graduates.


This August, 11 Canadian university presidents collectively sent a letter to Maclean’s indicating they would no longer supply the data required to create the magazine’s rankings.


“In short, the ranking methodology used by Maclean’s is oversimplified and arbitrary,” the letter stated. “We do find it ironic that universities are being asked to subsidize and legitimize this flawed methodology, when many faculty, staff and students at our institutions are dedicated in their research to ensuring that data are collected rigorously and analyzed meticulously.”


In response, Maclean’s used the Freedom of Information and Protection of Privacy Act to obtain results of national student surveys already conducted by the 11 universities. Maclean’s then published aspects of three surveys, the 2004 National Survey of Student Engagement, the 2005 Canadian Undergraduate Survey Consortium, and their own survey, comparing the student experience across Canadian universities.


“As you may have heard, this issue has been the focus of considerable controversy,” said the Maclean’s editorial accompanying the rankings. “Last spring our decision to ask for universities’ student satisfaction surveys, with the intention of making them public, was met with opposition from many schools. In August and September, in the run-up to this issue, half of Canada’s universities abruptly announced they would withhold the information about their operations that they have been releasing to Maclean’s, and to the general public, for the past 15 years. They have since claimed that the methodology we employ is flawed, and that rankings are inherently unfair.”


The editorial claimed Maclean’s had developed rankings only after close consultations with the very same universities now expressing dissatisfaction, about which data to use, and how much weight to give each particular measure.


“They essentially bundle up a whole bunch of different measures and try to deduce a single ranking out of that,”said U of C provost and vice-president academic


Dr. Alan Harrison. “We think that the methodology they use is fundamentally flawed. We’ve been telling them that for many years. They’ve never demonstrated any willingness to do anything about it. Eventually we said, ‘Enough, if you’re not going to listen to us we’re not going to cooperate.’ It takes us about a month to do all the work involved in producing all the data for them and we didn’t think it was a good use of students’ money or our own time.”


Harrison said the rankings ignore the specific strengths and weaknesses of different institutions.


Rankings can, however, be a useful earmark, noted U of C Students’ Union vice-president academic Shannon O’Connor.


“I think [the survey] definitely shows that we need to be putting more focus on the quality of student education, which we have been doing,”said O’Connor. “But, it doesn’t mean that there still isn’t a way to go from administration and the SU to make quality better at the U of C.”


The University of Alberta also declined to provide data to Maclean’s. U of A SU president Samantha Powers said the problem may be not with Maclean’s itself, but that Canada’s only national ranking system is conducted by a


private business designed to make profits.


“We have concerns that Maclean’s is the only ranking process,” said Powers. “That’s a problem. I think that the federal


government should look at it.”


O’Connor pointed to the need for more routine surveys.


“We need to be doing regular surveys over a number of years and compare those results with what we’re trying to accomplish at the institution in order to tell if they are accurate,” said O’Connor. “I think that there has been a major push towards the student experience, towards improving quality for students at the university. I think that it is accurate, for lack of a better term, in that we’re doing better, but we still have a way to go.”


In an attempt to make rankings more issue-specific for individuals, Maclean’s has introduced a new feature on their website, allowing users to create their own ranking by means of a customized


evaluation of the data.


The U of C online questionnaire had more than 4,300 responses. Overall, satisfaction was up in three main areas: integration of services, online services and customer service.


According to the Maclean’s University Rankings, Calgary came in dead last when asked to respond to ‘I am satisfied with my decision to attend this university.’ Only 14 per cent answered ‘agree strongly,’ leaving the U of C seven per cent behind the next university.


We also came last with senior students when asked if they would attend the same institution if they had to start over. However, the


U of C had the fifth largest operating budget per student and tied McGill University for sixth place in library holdings.