Calculating mean score can distort interview results

By MUNGAI KIHANYA

The Sunday Nation

Nairobi,

10 December 2017

Last week, the Moi University Council was widely and loudly criticised for the manner in which it evaluated the performance of applicants for the position of Vice Chancellor. Six aspirants were interviewed by eight members and, according to media reports, the results for each of the candidates were as follows:

Prof Kosgei: 69; 65; 61; 92; 69; 90; 99; 64 (mean = 76);

Prof Ayiro: 89; 89; 91; 45; 73; 51; 41; 80 (mean = 70);

Prof Nangulu: 61; 91; 68; 72; 56; 69; 74; 68 (mean = 70);

Prof Amutabi: 63; 71; 69; 70; 51; 72; 89; 46 (mean = 66);

Prof Kibwage: 67; 78; 65; 63; 59; 56; 48; 77 (mean = 64);

Prof Chacha: 58; 59; 61; 52; 45; 67; 88; 55 (mean = 61)

From this evaluation, Prof Kosgei emerges top of the ranking followed by Prof Ayiro while Prof Nangulu takes the third position. However, this is a good illustration of how using the wrong averaging method can lead to the erroneous conclusions. Let me explain.

The most commonly used average is the mean and it is quite easy to calculate: you simply add up the data and divide by the number of values. Unfortunately, it has a major drawback that can make it quite misleading especially when there are just a few numbers to be averaged.

In the case of the VC interviews, there were eight council members and so only eight values of scores per applicant. Now, this is too small a data set for the mean to be a reliable average. The reason is that, if one of the values is much smaller or much larger than the rest, it can distort the outcome significantly.

This is exactly what happened to the interview results. Three council members heavily under-marked one applicant and over-marked another one. The outcome of these actions was that the two mean scores were pulled down and pushed up, respectively.

How then, can we ensure that such action by a minority of interviewers does not affect the final outcome? The answer in the type of average we use. With only eight data values to work with, the mean is inappropriate. The median is the better average for this situation.

To get the median, we arrange the data in ascending order and pick the one that falls in the middle of the series. That way, even if one of the data values is unusually high or low, it will not affect the number that comes in the middle!

The results are shown below:

Prof Ayiro: 41; 45; 51; 73; 80; 89; 89; 91 (median = 76.5)

Prof Amutabi: 46; 51; 63; 69; 70; 71; 72; 89 (median = 69.5)

Prof Kosgei: 61; 64; 65; 69; 69; 90; 92; 99 (median = 69)

Prof Nangulu: 56; 61; 68; 68; 69; 72; 74; 91 (median = 68.5)

Prof Kibwage: 48; 56; 59; 63; 65; 67; 77; 78 (median = 64.0)

Prof Chacha: 45; 52; 55; 58; 59; 61; 67; 88 (median = 58.5)

We see a dramatic change in the top three applicants. Clearly, using the mean yielded distorted results.