Study finds concerning variations in care between physicians of the same specialty and in the same city, delivering care in the same clinical scenarios

Some physicians are far more likely to deliver appropriate care than others in the same geographic area or health care system, according to a new study led by Zirui Song of the Department of Health Care Policy in the Blavatnik Institute at Harvard Medical School, with colleagues at Massachusetts General Hospital, Brigham and Women’s Hospital, Embold Health, University of Michigan, and Vanderbilt University.

In the study, the researchers looked at whether physicians applied evidence-based guidelines to choices they made in common clinical scenarios. The analysis, published Jan. 28 in JAMA Health Forum, found significant, sometimes substantial, differences in how often individual physicians chose the recommended specific treatments or course of action.  

“We looked at a set of situations where clear-cut guidelines have been in place for years, with the hope of limiting variation in physician decision-making and promoting the use of the most appropriate care, based on rigorous evidence,” said study lead author Zirui Song, HMS associate professor of health care policy and a general internist at Mass General. “In some of the cases we looked at, physicians who made the most clinically appropriate decisions were five to 10 times more likely to use the recommended standard of care than peers in the same specialties and cities whose decisions tended to be the least appropriate. The differences we found are a cause for concern.”

Many clinical situations are complex, requiring physicians to make clinical decisions without the help of guidelines with a strong evidence base and choose from options that involve substantial uncertainty for patient outcomes. These are not the type of scenarios the researchers analyzed. Instead, they looked at more straightforward, simpler situations with a clear clinical decision and guideline-recommended pathway of care.

Based on evidence from randomized clinical trials and long-term observational studies, there are many clinical scenarios where a physician can provide effective care that maximizes clinical benefit and minimizes risk to the patient by adhering to well-known clinical guidelines, Song said. Even if a given patient’s unique circumstances might make the physician deviate from those guidelines, on average across many patients, those guidelines would generally point to higher value care.

For example, most pregnant patients should receive a common set of prenatal screenings, and patients with nonemergency hip or knee problems should receive some physical therapy before the decision is made to undergo surgery. 

The researchers examined medical insurance records from 2016 to 2019, involving care provided in 14 common clinical scenarios by 8,788 physicians from seven different specialties across five municipal areas across the United States. To minimize the chance that variation in the care received may be due to patient differences rather than physician decision-making, the researchers compared similar patients with similar clinical and demographic characteristics.

“One clear example is arthroscopic knee surgery for new osteoarthritis,” Song said. “Several randomized clinical trials have shown that the surgery is no better than no surgery or placebo treatments.”

With such clear-cut evidence, Song said he was surprised and concerned to see a marked variation in arthroscopic knee surgery rates on similar patients with new osteoarthritis among surgeons in the same cities. For this study, the researchers divided physicians into five quintiles based on how likely they were to follow the guidelines and provide the recommended care. In the arthroscopic surgery for new osteoarthritis scenario, the top 20 percent of surgeons in the study performed the surgery on only 2 to 3 percent of their patients. By contrast, between 26 and 31 percent of patients with the same condition in the same cities got surgery if they saw a surgeon from the bottom 20 percent.

The first observations of widespread differences in clinical-decision making were published two decades ago, including research by the Dartmouth Atlas of Health Care, but focused on geographic variation between different regions of the country. A 2013 National Academies of Medicine committee led by Alan Garber, provost of Harvard University, and Joseph Newhouse, HMS professor of health care policy, called for a greater emphasis on understanding differences in individual physician decision-making, rather than geography, for improving quality of care.

“In the last decade, however, rigorous measurement of within-area, between-physician variations in decision-making remains sparse,” Song said. “This study aims to help fill that gap.”

Song noted that the variations in the study were most likely related to differences in individual physicians, not differences in practices or health systems, as the variation between individual doctors working in the same organization was greater than the differences in performance between organizations.

Further research into how and why doctors differ in the decisions they make can help medical schools, teaching hospitals, and continuing education programs fine-tune their training and improve value of care. This type of evidence, if replicated in a rigorous way, may also empower patients, employers, and payers to choose physicians who provide higher value care in their region, allowing consumers voting with their feet to stimulate practice improvement and competition on quality, Song said.

For example, right now some employers and insurers require higher co-pays for patients who go to higher-priced hospitals to encourage use of lower-priced hospitals. Generally, appropriateness of clinical decisions is not factored in. In the future, Song said, it would make more sense to develop more clinically nuanced incentives to encourage patients to choose physicians who provide higher value care that ensures better patient outcomes aligned with evidence-based approaches.

Some estimates suggest that approximately 25 percent of health care spending in the U.S. is wasteful. Reducing this waste will require addressing multiple sources of unnecessary spending, one of which is inappropriate clinical decision-making, researchers noted.

“Researchers have long suspected that much of that waste lies in the variation of the appropriateness of care from one physician to another,” Song said. “This study offers evidence that this problem is large and widespread across specialties.”

This research was supported by grants from the National Institutes of Health, Arnold Ventures, the National Institute of Diabetes and Digestive and Kidney Disease, and Harvard Catalyst.