Saturday, September 24, 2016

Ranking Mania

Once I have attended a workshop by Kevin Downing who told us, despite what we think of university rankings, we will not be able to ignore them simply because of public opinion is centred around them. Thus, the involvement of our universities in the rankings. My own views on the matter have always been not to ignore them but cautiously use them for our own progress. The rankings are after all one-dimensional projections (hence allowing ordering in mathematical sense) and are not designed to capture the total worth of an organization. I don't really pay attention to the individual ranking number too much but the magnitude of the number will probably signify similar class of universities.

First and foremost,  there are probably more than 30,000 universities in the whole world, most of which are unranked. Now ranked universities are probably in the magnitude of one or two thousand. The number of ranked universities could also vary from year to year and thus this may spoil temporal rank comparison.

Next, there are many different university rankings, each with different ranking criteria. Kevin mentioned the big three:
The latter two actually began together in 2004 but they went separate ways in 2009 for which QS decided to use Scopus database while THE uses Thomson-Reuters database. I follow at least another ranking for cross check namely
  • Centre for World University Rankings (CWUR), which is based in Jeddah and began in 2012.
QS & THE rankings involve academic peer review while ARWU and CWUR claim they are purely data-based. With different criteria and input dataset, certainly the rankings they produced will be different and it is interesting to see the way they differ. On the surface, I would prefer those without the peer review, which tends to be subjective.

Our local universities seem to have favoured the QS rankings and perhaps prepared well according to their criteria. Hence better performance for our local universities. This is our recent QS rankings:
  • Universiti Malaya - ranked 133
  • Universiti Putra Malaysia - ranked 270
  • Universiti Teknologi Malaysia - ranked 288
  • Universiti Kebangsaan Malaysia - ranked 302
  • Universiti Sains Malaysia - ranked 330
UPM made a substantial leap of 61 spots (from 331st position) bringing it to within top 300 but I consider we made the same leap as UTM (from 303rd position). Next came the QS ranking of Top 50 universities under 50:
  • Universiti Putra Malaysia - ranked 17
  • Universiti Teknologi Malaysia - ranked 25
  • Universiti Kebangsaan Malaysia - ranked 26
  • Universiti Sains Malaysia - ranked 33
We have also made some achievements in Reuters top innovative university rankings:
  • Universiti Putra Malaysia - ranked 73
  • Universiti Malaya - ranked 75
The breakdown for UPM marks is given here.

All these improvements are very much welcomed. One can see our university staff committed in helping the university to improve but much more meaningful is the research culture  and the science we build that have improved over the years.

Having these results, we should not go over our heads with such improvement and achievements. Once we accept rankings, we should accept them even if we are not doing good. If we pick and choose, it may not reflect us that well. Just several days after all these good news, THES came out with their rankings and they gave
  • Universiti Kebangsaan Malaysia
  • Universiti Putra Malaysia
  • Universiti Sains Malaysia
  • Universiti Teknologi Malaysia
  • Universiti Teknologi Petronas
all ranked within 601-800 category. Universiti Malaya seems conspicuously missing from it (I do not know why) and we noticed Universiti Teknologi Petronas is placed within the same rank as the other research universities. When I first highlighted these results, I guess some are unhappy. But as I said, if one accepts QS ranking, one ought not ignore the other rankings on what they say. The very least we need to see is what they have analysed, why are the ranking results different and in what way can we improve from them. In any case, I find the ranking is consistent with the other rankings. For the CWUR 2016 rankings, we have the following list:
  • Universiti Malaya - ranked 539
  • Universiti Sains Malaysia - ranked 694
  • Universiti Putra Malaysia - ranked 832
Note that CWUR has not been highlighted in any of our media probably due to it being lesser known. It is interesting however to see the score for UPM for which we scored well in the patent criterion, and this further supports the Reuters top innovative ranking results. Now for the ARWU ranking, UPM didn't get to be ranked but the following universities are ranked as 401-500:
  • Universiti Malaya
  • Universiti Sains Malaysia
  • Universiti Kebangsaan Malaysia
UPM has at least been ranked 151-200 in 2015 for the Mathematics subject ranking. We are looking forward to the 2016 subject ranking, whether we are able to maintain this ranking. Note that both CWUR and ARWU rankings are harder. ARWU ranking for instance, considers alumni of the university that wins the Nobel Prize and Fields Medal as one of the criteria.

As I said earlier, I would rather look at these rankings as tools. One ought to be careful of how we think about them. If top rankings are being made into ultimate goals, they can lead to undesirable and unethical behaviour; ends justify the means mode. The ultimate goals should be those of more noble and intrinsic nature.

We had already seen one academic scandal earlier where local academics duplicated and manipulate images from their experiments to be sent as three different papers:
Recently, a bit less severe is the accusation of salami slicing results into multiple papers:
Why is this happening is simply because of the terrible pressure that an academic experiencing today to the extent that some resort to manipulative means and worse, deceit to achieve their key performance indicators and at times unnatural demands. Part of these are indeed fueled by the university's desire to achieve better rankings. One should really reflect the kind of academic ethos that we are trying to build. Let's try to keep good academic ideals while trying to achieve better rankings. The rankings are not permanent, but the science we do is.

No comments: