Marks scored in an interview are not important!

By MUNGAI KIHANYA

The Sunday Nation

Nairobi,

17 December 2017

 

As data collection methods, job interviews are complete waste of time because their results are unreliable and invalid. An interview cannot measure what it is intended to (ability to do the job) and the outcome is not reproducible.

For these reasons, the data collected from a job interview must be evaluated with care in order to select the most suited applicant. Unfortunately, many interviewers treat the process as if it was an examination where marks are awarded objectively and consistently.

Last week we saw how evaluating the mean score can lead to erroneous results. This mistake arises from the assumption that the scores awarded by the interviewing panel were objective quantities.

But this is not so: The reason members of an interviewing panel are asked to award marks (based on a guideline of points to look out for) is to help the interviewers rank the applicants. The actual scores are not important at all! It is only the ranking that matters.

Therefore, since the University Council is expected to recommend three applicants to the Cabinet Secretary, the best approach to producing the shortlist would be to ask each member of the interviewing panel to indicate their favourite three applicants.

Think of it as a sports tournament, say, the Olympics. Countries are ranked in order of medals won; that is, how many of Gold, Silver and Bronze, successively. A similar method can be used in aggregating the scores awarded in an interview. Let me illustrate using the same data from last week; that is:

Prof Kosgei: 69; 65; 61; 92; 69; 90; 99; 64 (mean = 76);

Prof Ayiro: 89; 89; 91; 45; 73; 51; 41; 80 (mean = 70);

Prof Nangulu: 61; 91; 68; 72; 56; 69; 74; 68 (mean = 70);

Prof Amutabi: 63; 71; 69; 70; 51; 72; 89; 46 (mean = 66);

Prof Kibwage: 67; 78; 65; 63; 59; 56; 48; 77 (mean = 64);

Prof Chacha: 58; 59; 61; 52; 45; 67; 88; 55 (mean = 61)

In this list, the scores are in the order of Council members; that is, the first score for each candidate is what was awarded by the first Council member, the second score is what was awarded by the second member and so on.

 Accordingly, we find that first Council member ranked Prof Ayiro on top (89 = Gold medal), followed by Prof Kosgei (69 = Silver) and Prof Kibwage (67 = Bronze) was third. We can evaluate the rankings by all the other interviewers and then aggregate the results.

It then turns out that Prof Ayiro leads with 4 Gold medals and 1 Silver followed by Prof Kosgei (3 Gold, 2 Silver) and Prof Nangulu comes third with 1 Gold, 1 Silver and 3 Bronze medals.

We notice that both Ayiro and Kosgei have five medals each. However, in such a case preference goes to the person who has more Gold.

In my view, this is the most appropriate evaluation method for the job interview data. A simple calculation of the mean score is bound to give misleading results.

 
     
  Back to 2017 Articles  
     
 
World of Figures Home About Figures Consultancy