In this week's BMJ, Nigel Hawkes, freelance journalist and Director of Straight Statistics, a campaign for the better use of statistical data, investigates how the way that patients are allocated diagnostic codes by a hospital can have a big effect on a hospital's performance.
It follows two articles published by the BMJ last week arguing that using death rates to judge hospital performance is a bad idea.
It began, says Hawkes, when Mid Staffordshire Foundation NHS Trust was branded "appalling" by the Care Quality Commission, yet was rated in the top ten for quality of care by health analysts Dr Foster in their 2009 Good Hospital Guide. Further investigations into lower than average death rates from broken hips at Mid Staffordshire was put down to "improved coding procedures."
Could such coding changes be affecting Dr Foster's performance evaluations, flattering hospital-standardised mortality ratios (HMSRs) and making hospitals appear much better than they are, asks Hawkes?
Diagnostic coding is used by hospitals to record the conditions suffered by patients admitted to them for treatment, he explains. If a hospital uses an increasing number of codes year by year, it implies that more patients with more severe conditions are being treated. But if the death rate remains constant while the severity index increases, it appears that the hospital is doing better at keeping people alive.
Research by CHKS, a rival health analytical company, shows that the number of codes has been creeping up over recent years, yet crude death rates in English hospitals show virtually no change over the past five years. There is also a big variation in the average number of diagnostic codes per patient from hospital to hospital.
How has this happened, asks Hawkes? One explanation might be the increasing proportion of patients classified as needing palliative care. CHKS figures show the number of deaths coded Z51.5 (the code used for palliative care) was under 400 a month in 2004, but had reached more than 1,800 a month by June 2009.
Patients coded Z51.5 are assumed to have come into hospital to die, so performance calculations make allowance for that. This means that a few heavy users of the Z51.5 code could have reduced their HSMRs from 110 (above average) to 90 (below average) simply by increasing the frequency of use of the palliative care code.
Although Hawkes cannot prove that any hospital is doing this, he says "there is no question that the situation is open to manipulation by trusts."
Dr Foster Intelligence, the company responsible for the Good Hospital Guide, believes that HSMRs remain a good measure of the quality of care, if used alongside other measures. CHKS, by contrast, believes that while they may be a useful tool within a hospital, they are unsuitable on their own for comparisons between hospitals while such big coding variations exist.
The clash of evidence between the Good Hospital Guide and the Care Quality Commission embarrassed the Department of Health and a review of the use of HSMRs in England is now underway.