Education

  Star Education Fair

Sunday November 11, 2007

Whither rankings?

By KAREN CHAPMAN, TAN SHIOW CHIN and HARIATI AZIZAN


educate@thestar.com.my

StarEducation takes a look at response to the THES-QS World University Rankings 2007.

RANKINGS and ratings seem an inevitable aspect of today's higher education scene in most parts of the world. And while there is much debate about them, they can provide a useful basis for comparison of institutions.

As such, the publication of the Times Higher Education Supplement (THES)-Quacquarelli Symonds (QS) World University Rankings has been awaited with interest in recent years.

This year, for the first time since the rankings began in 2004, no Malaysian university made it to the top 200.

But before jumping to discussions on how local universities have done, it is important that rankings are not seen as the end all and be all.

Higher Education Minister Datuk Mustapa Mohamed has said previously that rankings are useful guides and should be treated as such.

Foreign students at Universiti Malaya, which is once again the country's top university according to the THES-QS World University Rankings 2007.
“It would be wrong to rely on them entirely as a measure of a university’s quality. Instead, they should be used alongside other, sometimes less tangible, criteria to assess a university’s quality and value,” he says.

International Association of Universities (IAU) secretary-general Eva Egron-Polak believes rankings make institutions more competitive today, noting that universities had traditionally been guided by the spirit of collaboration.

“We must do everything we can to improve and diversify the criteria and allow for the diverse and specific nature of institutions to be reflected and not try to fit everyone into the same mould.

“If you have one set of criteria against which you measure everyone, then they will slowly drift to a single model, but I don't think that is what we want,” she notes.

Instead, she adds, the criteria used should reflect the culture and national priorities of each country.

Down but not out

In terms of the rankings, QS Quacquarelli Symonds Limited managing director Nunzio Quacquarelli says Universiti Malaya (UM) is the highest ranked at 246, followed by Universiti Sains Malaysia (USM) at 307, Universiti Kebangsaan Malaysia (UKM) at 309 and Universiti Putra Malaysia (UPM) at 364.

“This in no way reflects their quality as we know they have been working hard,” he observes.

Calculations for rankings are based on data gathered under peer review (40%), recruiter review (10%), international faculty ratio (5%), international students ratio (5%), student-faculty ratio (5%), and citations per faculty (20%).

According to a report by QS head of research, Ben Sowter, rankings attract a lot of attention but “nowhere has keener interest been taken than in Malaysia”.

In his report entitled “Implications of methodological enhancements for Malaysian institutions”, he notes that Malaysian universities are not listed in the top 200 but adds:

“The purpose of this report is to point out, in no uncertain terms, that this result is less reflective of the actual performance, profile or quality of Malaysian institutions – which have been working ceaselessly to enhance their international reputation – but is almost entirely attributable to the combination of methodological enhancements and improved response dynamics in the rankings themselves.”

He also says there are five Malaysian universities in the 2007 rankings exercise but since Universiti Teknologi Malaysia was only included this year, the report only deals with the other four.

According to him, the single biggest factor suppressing the performance of Malaysian universities in the world rankings is their current profiles in both ESI (Thomson) and Scopus citation databases.

“The citations/faculty aspect of our rankings is the area in which Malaysian universities are consistently weak.”

He points out that UM, for instance, showed improvements in the international indicators and the recruiter review.

However, he adds, these gains have been offset by the peer review and citations per faculty score, where minor setbacks have been experienced but in more highly weighted indicators.

UM vice-chancellor Datuk Rafiah Salim does not deny that the university has a long way to go.

“We need to focus on peer review and be more involved in international conferences, not just in terms of attendance, but our people have to be the lead presenters.

“We need to have more working relationships with more universities,” she notes.

Sowter says USM's performance in each indicator has remained fairly stable in comparison with last year and as a result it has experienced less of an impact from methodological changes than the other three universities.

USM vice-chancellor Prof Datuk Dzulkifli Abdul Razak says the university tries to remain as consistent as possible.

“We hope THES-QS will stabilise their criteria so that we know what we are doing,” he adds.

As for the switch from Scopus to ESI (Thomson) for citation data, Prof Dzulkifli says the university will have to study the matter to see if Scopus will meet its needs.

“We can't switch just to suit the rankings,” he adds.

According to Sowter, all Malaysian universities have suffered in the peer review by at least 20 places, but UKM has experienced the most profound impact in this area.

UKM vice-chancellor Prof Datuk Dr Sharifah Hapsah Syed Hasan Shahabuddin says the university is giving serious attention to the drop in the rankings.

“However, we maintain that the drop in ranking, as acknowledged by THES-QS, is in no way reflective of actual performance, profile or quality of our institution and must not be equated with a drop in performance.

“We have always maintained that it should be used as a motivation tool for self -improvement rather than as an indicator of worth,” she says.

Prof Sharifah Hapsah believes the ranking results will vary from year to year because methodology is in its infancy and is subject to continuous improvement.

In terms of citations, she says, the change to Scopus is good as the database is bigger and includes papers written in other languages.

“Where citation index is concerned, we are better under Scopus but the problem is other people are also better,” she notes.

As for UPM, Sowter says the university has made good gains on student-faculty ratio – perhaps connected with the implementation of full-time equivalent numbers for student and faculty across the board.

UPM vice-chancellor Prof Datuk Dr Nik Mustapha Raja Abdullah acknowledges that the university's weakest areas are the peer review and citations per faculty.

“We have taken steps by setting up key performance indicators for our staff beginning two years ago and expect to get more people to conduct research and publish their findings.

“But we have to accept that we are not there yet. Give us time, and we will get there,” he adds.

Sowter also says the changes made this year should mean that improvements in universities will be more accurately reflected in their ranking.

The larger picture

To improve rankings, Egron-Polak believes there should be more categories so that institutions that are research-intensive can be ranked together.

“Institutions are not a monolith; they are different so there is a need to have the kinds of rankings or ratings that look at institutions which are similar,” she explains.

She argues that it is critical to keep in mind that universities are not built overnight as many universities in developing countries were created in post-colonial times.

“The purpose of universities in developing countries is also different as they may need to look into research that is locally oriented to solve local dilemmas and problems,” she says.

It may be more difficult, she adds, to get research on local issues published in international journals but that does not make the studies any less relevant.

Malaysian Qualifications Agency deputy chief executive officer Assoc Prof Zita Mohd Fahmi says there are many rating and ranking systems in the world.

“So we shouldn't rely on one,” she adds.

The changes in some of the methodology this year, she says, shows how dynamic rankings can be and universities entering the lists should be aware of the many variables involved.

“What is important is that the universities keep improving themselves. Quality targets are never static. They are always moving.”

Related Stories:
A choice of signposts
Key changes

  • E-mail this story
  • Print this story