Education

Sunday September 6, 2009

Playing a numbers game

BY SIMRIT KAUR

The use of KPIs in public universities promises to transform Malaysian academia.

KEY performance indicators (KPIs) at universities are not new with different terms being used to describe the measurement of an academic’s performance.

Prof Ghauth: Good research will have an impact on teaching.

However, given the current higher education landscape, Malaysian universities are setting even more stringent KPI targets for their staff.

KPIs are now being used to not only gauge an academic’s yearly performance, but also his eligibility for promotion.

Universiti Malaya (UM), for one has set the bar very high. Vice-chancellor Prof Datuk Dr Ghauth Jasmon makes no apologies for this.

“If our academics can meet the standard we have set, they can become professors anywhere in the world.

Prof Sharifah Hapsah: It will not be fair to have the same target for everyone.

“If UM had introduced these targets 20 years ago, we would have been among the top 50 universities today,” he declares. But, how fair are the KPIs and can the targets be met?

Setting the KPIs

At UM, academics are evaluated in six major areas, namely research publication, number of research students supervised, amount of research funding, teaching quality, consultancy and administration. (See chart).

Although no weightage is given, the biggest challenge is the research publication target.

Prof Ghauth says that with the KPIs the “goalposts” that academics must aim for are clearly defined. And a 100% success rate is expected. “The reaction has been more positive than I thought. It proves to me that people in UM want to work.”

UM came up with the KPIs based on the curriculum vitae of academics from the top 100 universities in the world.

Prof Zaini is of the opinion that KPIs help create a performance-based work culture

“We looked at their ISI publications when they were lecturers, associate professors and professors to come up with the guidelines.

(The research publication of an academic is measured by databases of articles. Two such sources are Scopus and Thomson ISI.)

From this year, Scopus is no longer accepted, only ISI. Prof Ghauth cites the higher impact factor for the latter, which should result in more citations.

Even within ISI, UM has set a target that 40% of publication must be in Tier One and Tier Two journals and 60% Tier Three and Tier Four. This, says Prof Ghauth, addresses the issue of quality over quantity.

He says that one of the reasons publication was chosen over citations was because “to generate citations would take a few years, not a few months.”

All authors will receive equal recognition for joint papers. “We want to promote teamwork and get academics to collaborate with each other and also internationally.”

Prof Ghauth says concerns that those from the social sciences are unfairly disadvantaged if only ISI is used has been addressed.

Prof Nik Mustapha says that staff are more receptive to the idea of being monitored.

“That is why we put the condition that 30% of the research publication can be from books with the proviso that there must be three international expert referees.”

He also dismisses criticism that teaching will take a backseat in the research-dominated KPIs. “Good research will have an impact on teaching. Academics should bring in their current research findings into their lectures. If they do not apply new theories. the only thing they would be doing is teaching what they have learnt 20 years go.”

Prof Ghauth says there has been an exponential increase in ISI publications since the current KPIs were introduced and cites dentistry as an example.“The dean informed me that the faculty already had 18 ISI papers by July this year compared to only five last year.”

Another view

Although Universiti Kebangsaan Malaysia (UKM) first instituted KPIs in 2000, they have gone through several modifications since then.

There are three main areas evaluated in UKM: education, research and services. The weightage for university-level KPIs are education (30%), research (50%), and services (20%) .

The KPIs of individual academics and faculties make up the university’s KPIs. The goal is to ensure that everyone works towards achieving UKM’s overall KPI. (See chart)

“There are several layers and stages to the KPIs. We don’t expect 100% achievement immediately. By 2018, the targets we have set should have been met,’’ says UKM vice-chancellor Prof Tan Sri Dr Sharifah Hapsah Syed Hasan Shahabudin.

The weightage for the KPIs vary. For research institutes, the research weightage will be higher compared to a faculty’s.

Academics’ KPIs have also been customised. Lecturers with a heavy teaching load would have a higher weightage in that area. Similarly, deans who do administrative duties would have a higher weightage under services.

“It would not be fair to have the same target for everyone. We want to make sure people contribute in all the three areas,, that is education, research and services,’’ says Prof Sharifah Hapsah.

“If an academic were to achieve 85% and above, then he will be considered excellent. Below that, he will be considered average. However if it is below 50%, he will be categorised as a low performer and will receive no annual increment,’’ she adds.

UKM uses both ISI and Scopus for research publication under the KPIs. For co-authored papers, authors are given proportional marks based on the number of writers per paper.

“Ultimately, academics must understand why the KPIs are relevant to them. For that to happen, it is important that we listen and consult them.”

        Transforming a university

The benefits of implementing KPIs in academia are manifold, says Universiti Teknologi Malaysia vice-chancellor Prof Datuk Dr Ir Zaini Ujang.

“They help create a performance-based work culture, provide a more transparent criteria for annual appraisal and promotion and set clear targets for individuals to aim towards.”

At UTM, the Balanced Scorecard approach is used to “ensure that the performance targets of individual employees are aligned with those of the university,” he says.

Earlier this year, UTM began to develop performance indicators for the annual appraisal of academic staff. To be considered excellent, they must achieve more than 80% in each area.

The criteria used are teaching, postgraduate supervision, research input, research output, professional services, personal quality, leadership and other contributions. (See chart).

For promotion, a different weightage is used. For example, a lecturer in the DS45 scale’s weightage for teaching is 60% compared to 45% for a senior lecturer or associate professor.

“The KPIs have created a new mindset among the academic and non-academic staff as we move towards a more performance-based work culture,’’ says Prof Zaini.

He stresses that teaching will not be forgotten in the university’s quest to attain research university status and improve its position in the Times Higher Education (THE) World University Rankings

“THE also does an employers’ review where major recruiters evaluate the quality of graduates. This reflects the quality of teaching. In fact, in the last two THE rankings, we were the top-ranked Malaysian university in this area.”

UTM’s research publication KPIs take into account both ISI and Scopus. “We use the number of indexed publications as well as citations. We give special emphasis to the cumulative impact factor and differentiate between low, medium and high impact journals.”

To address the difficulty those from the social sciences face to publish in ISI indexed journals, UTM has set them a lower target.

“We recognise this ‘unfair disadvantage’ and our KPIs address this issue by requiring a slightly lower publications target.”

As for co-written papers, UTM does not distinguish between the first and second authors, with every team-member getting the same mark.       

Teaching is core

Universiti Putra Malaysia (UPM) instituted KPIs for teaching and learning in 1992 but they were only devised for research following the attainment of research university status in 2006.

“KPIs enable us to monitor the achievement of individuals as well as our organisation’s objectives.

“The response at UPM has been encouraging so far. Staff are more receptive to the idea of being monitored as the KPIs are measurable and transparent.’’ says vice-chancellor Prof Datuk Dr Nik Mustapha R. Abdullah.

Academics are evaluated based on teaching and learning (40%), research (40%) and services 20% and are required to obtain 80% as stipulated under the Sistem Saraan Malaysia (SSM, the civil service’s remuneration system).

“Researchers with low KPIs will be denied promotion while those who perform well will be rewarded with monetary incentives,’’adds Prof Nik Mustapha.

He says that UPM has always considered teaching its core business.

“Excellent research is the basis for excellent teaching. Based on a recent teaching evaluation by our students, 98.4% of the staff achieved a score greater than 3.5 on a 5-point scale. This indicates that the teaching meets students’ expectations.”

UPM’s Centre for Academic Development helps to ensure that the quality of teaching is continuously improved and lecturers have enough time to concentrate on both teaching and research .

Prof Nik Mustapha says that UPM uses both ISI and Scopus when assessing the research publication of its academics.

The implementation of KPIs are aimed at making public universities and academics world-class. Malaysians like Prof Wang Gungwu and Prof Ho Peng Yoke, who went on to become professors in the Australian National University and Cambridge University respectively, show that the challenge can be met.

·See
StarEducation next week for more stories.

Related Story:
A big task

  • E-mail this story
  • Print this story