News Release

High variability among experts when assessing claimants for work disability benefits

More research to improve reliability is urgently needed, say experts

Peer-Reviewed Publication


Healthcare professionals have high variation in judgement when assessing the same claimant for disability benefits, finds a review published by The BMJ today.

The authors warn that the current evidence base is limited, and the results are "disconcerting," and call for "substantial investment in research to improve assessment of disability."

Many workers seek wage replacement benefits due to a disabling illness or injury. Public and private insurance systems provide wage replacement benefits for such employees, as long as eligibility criteria are met.

Insurers often arrange for evaluation of eligibility by medical professionals, but there are concerns regarding low quality evaluations and poor reliability between medical experts assessing the same claimant.

So a group of European and Canadian researchers, led by Regina Kunz from the University of Basel, Switzerland, explored agreement among healthcare professionals when assessing claimants with ill health for work disability benefits.

They carried out a systematic review of 23 studies, conducted between 1992-2016, from 12 countries in Europe, North America, Australia, the Middle East, and northeast Asia.

The studies include those carried out in an insurance setting, with medical experts assessing claimants for work disability benefits, and in a research setting, where evaluation of patients took place outside of actual assessments, for example, for rehabilitation.

In 63% (10/16) of studies, medical experts reached only low to moderate agreement in their judgement of capacity to work. The results also show higher agreement was strongly associated with the use of a standardised approach to guide judgment.

"Despite their widespread use, medical evaluations of work disability show high variability and often low reliability. Use of standardised and validated instruments to guide the process could improve reliability," say the authors.

They explain that few countries have standards to guide assessments. Promising targets include formal training in evaluation of capacity to work, use of standardised instruments to guide disability evaluations, and addressing the conflict of interest that arises when insurers select their own experts.

Furthermore, they say that there might be greater need for strategies to improve agreement when patients present with subjective complaints, and improved knowledge of individual factors that contribute to variability in evaluation of capacity to work.

They conclude by saying "there is an urgent need for high quality research, conducted in actual insurance settings, to explore promising strategies to improve agreement in evaluation of capacity to work."


Research: Inter-rater agreement in evaluation of disability: systematic review of reproducibility studies

About BMJ

BMJ is a healthcare knowledge provider that aims to advance healthcare worldwide by sharing knowledge and expertise to improve experiences, outcomes and value. For a full list of BMJ products and services, please visit

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.