Marks scored in an interview are not important!
By MUNGAI KIHANYA
The Sunday Nation
Nairobi,
17 December 2017
As data collection methods, job interviews are
complete waste of time because their results are unreliable and invalid.
An interview cannot measure what it is intended to (ability to do the
job) and the outcome is not reproducible.
For these reasons, the data collected from a job
interview must be evaluated with care in order to select the most suited
applicant. Unfortunately, many interviewers treat the process as if it
was an examination where marks are awarded objectively and consistently.
Last week we saw how evaluating the mean score can
lead to erroneous results. This mistake arises from the assumption that
the scores awarded by the interviewing panel were objective quantities.
But this is not so: The reason members of an
interviewing panel are asked to award marks (based on a guideline of
points to look out for) is to help the interviewers rank the applicants.
The actual scores are not important at all! It is only the ranking that
matters.
Therefore, since the University Council is expected
to recommend three applicants to the Cabinet Secretary, the best
approach to producing the shortlist would be to ask each member of the
interviewing panel to indicate their favourite three applicants.
Think of it as a sports tournament, say, the
Olympics. Countries are ranked in order of medals won; that is, how many
of Gold, Silver and Bronze, successively. A similar method can be used
in aggregating the scores awarded in an interview. Let me illustrate
using the same data from last week; that is:
Prof Kosgei: 69; 65; 61; 92; 69; 90; 99; 64 (mean =
76);
Prof Ayiro: 89; 89; 91; 45; 73; 51; 41; 80 (mean =
70);
Prof Nangulu: 61; 91; 68; 72; 56; 69; 74; 68 (mean =
70);
Prof Amutabi: 63; 71; 69; 70; 51; 72; 89; 46 (mean =
66);
Prof Kibwage: 67; 78; 65; 63; 59; 56; 48; 77 (mean =
64);
Prof Chacha: 58; 59; 61; 52; 45; 67; 88; 55 (mean =
61)
In this list, the scores are in the order of Council
members; that is, the first score for each candidate is what was awarded
by the first Council member, the second score is what was awarded by the
second member and so on.
Accordingly,
we find that first Council member ranked Prof Ayiro on top (89 = Gold
medal), followed by Prof Kosgei (69 = Silver) and Prof Kibwage (67 =
Bronze) was third. We can evaluate the rankings by all the other
interviewers and then aggregate the results.
It then turns out that Prof Ayiro leads with 4 Gold
medals and 1 Silver followed by Prof Kosgei (3 Gold, 2 Silver) and Prof
Nangulu comes third with 1 Gold, 1 Silver and 3 Bronze medals.
We notice that both Ayiro and Kosgei have five medals
each. However, in such a case preference goes to the person who has more
Gold.
In my view, this is the most appropriate evaluation
method for the job interview data. A simple calculation of the mean
score is bound to give misleading results.
|