image: An award-winning study by a global team featuring researchers from SMU presents a more comprehensive and integrated method for assessing prospective talent. view more
Credit: Singapore Management University
SMU Office of Research & Tech Transfer - Selecting the right person for a job can be tricky. A potential candidate may present well initially, but it's hard to predict how they will perform once they are operating within an organisation. Talent recruitment becomes expensive if you get it wrong.
A popular assessment method used by employers is the situational judgment test (SJT), which poses multiple-choice questions to capture the behavioural decision-making and thought processes of a candidate when they are introduced to hypothetical work-related scenarios.
Another approach for predicting work performance is to consider a candidate's personality traits, commonly described as habitual patterns of behaviour, thought and emotion.
According to trait theory, which dates back to pioneering work by American psychologist Gordon Allport in the 1920s, the five broad personality traits are openness to experience, conscientiousness, extroversion, agreeableness and neuroticism.
Now, in a project that has taken five years to reach fruition, a team of researchers from Singapore Management University (SMU) and Ghent University in Belgium has demonstrated a way to combine the strengths of both approaches - an adjusted form of SJT with a contemporary rethinking of trait theory - and the results are promising.
The team's paper, The Predictive Power of People's Intra-individual Variability Across Situations: Implementing Whole Trait Theory in Assessment, won the 2020 Joyce and Robert Hogan Award for Personality and Work Performance, presented by the U.S.-based Society for Industrial & Organizational Psychology.
The award is given to the authors of a paper judged to have the highest potential to further the understanding of personality as it relates to work performance.
"The idea for our project stemmed from the SJT field," says research partner Filip Lievens, the Lee Kong Chian Professor of Human Resources at SMU.
"However, SJT studies often do not pay a lot of attention to what is being measured. Therefore, we enriched the SJT field by drawing on insights from recent developments in the personality domain."
Dynamic trait constructs
The traditional view of traits - as aspects of personality - has been that they are relatively stable over time, differ across individuals, are relatively consistent over situations, and influence behaviour.
But this static interpretation misses the subtleties of behaviour springing from human adaptability to different situations - the way we can vary our responses to be appropriate for different contexts. In academic language it is known as intra-individual, or within-person, variability.
For decades there was a clash in the psychology field between those for whom consistency and stability in the definition of traits was pivotal, and those favouring a situationist view.
These two perspectives are no longer seen as irreconcilable and there is some consensus that traits are most useful for predicting behaviour over a longer time span, whereas situations make individual behaviour vary in the short run.
According to the researchers: "The overarching idea is that while traits provide a useful summary of a person's general behavioural tendencies - for example, a level of sociability - across many situations, additional information can be obtained if we know how much variability in trait expressions the person displays across various situations."
And this additional information can have a strong impact.
"In lay terms, we found that people's flexibility across a situation trumps their consistency for predicting future performance. So, for example, it is better to be extroverted in some situations and not in others, and we found a straightforward approach for assessing this," says Professor Lievens.
Measuring variation
The researchers conducted three studies. For the first, 235 third-year psychology students from a large West-European university were invited to participate for a course credit. The average age was 21, the cohort was 31 percent male and the questions were in Dutch, the participants' native language.
An SJT was specifically designed to test two traits: sociability - for interpersonal performance (such as working in small-group class projects) - and dutifulness, for intra-person performance, given its consistent validity in academic and training contexts. A new statistical tree-model based on item response theory was created to analyse the results.
"Our studies presented test-takers with a large set of written situations (like in an SJT) followed by response options that varied in terms of trait levels (higher vs lower)," says Professor Lievens.
"This allowed the computing of people's mean level scores as well as their variation across situations and [enabled] using these mean levels as well as variability scores for prediction purposes."
One key finding was that people's intra-individual variability was related to their self-rated functional flexibility - that is, their perception of their ability to adjust to the needs of different situations.
For the second study, participants from the first round were later contacted by email and invited to engage in a daily diary survey where they would self-report their personality states each evening across 10 weekdays. The results provided the researchers with one of the project's 'aha' moments:
"We established evidence that intra-individual variability in people's responses to written SJT situations predicted actual intra-individual variability [in real life] as captured by a diary study two years later," says Professor Lievens.
"We found this remarkable because we are the first to link SJT scores to future diary study ratings."
Improved assessment
The third study accessed 126 employees in the financial industry who completed an SJT designed to measure personal initiative - the tendency to behave proactively in work settings.
Supervisor ratings of employees' job performance were collected concurrently and the combined data validated intra-individual variability as an indicator of performance.
"At a practical level, all three studies present convincing evidence for a different and more comprehensive approach to assessing personality traits," Professor Lievens says.
The team is already planning to examine other assessment methods and a related project deals with multiple-speed assessment in which candidates go through many short interpersonal simulations in a carousel-like approach. This project obtained a Ministry of Education Academic Research Funding (AcRF) Tier 2 grant.
"In the future, we also recommend linking intra-individual variability to adaptive and leadership performance," Professor Lievens says.
###
By Alistair Jones
The research team comprised Filip Lievens and Ronald Bledow from SMU, and Jonas Lang, Filip De Fruyt, Jan Corstjens, and Myrjam Van de Vijver from Ghent University.