"An overconfident physician will not seek help or may ignore help that is offered; someone underconfident may, in extreme cases, be talked out of what was an accurate assessment," said Charles P. Friedman, Ph.D., professor of medicine at the University of Pittsburgh School of Medicine, in an article published in the April issue of the Journal of General Internal Medicine. Dr. Friedman reported that confidence plays a major role in clinicians' use of outside resources in determining diagnoses and making other medical decisions. These resources include colleagues, medical books, journals and computer-based decision support systems (DSSs).
"Decision support occurs in two modes: either clinicians must seek medical information to supplement what they already know, or if it is 'pushed' to them in the form of a computer-generated alert or reminder, they must be open to the advice," Dr. Friedman said.
"The whole decision support process presupposes that clinicians know when to seek help or pay attention to help that is offered," he said, noting that this study is not about how often physicians are correct, but rather, when they are correct, whether they are aware of it.
"Dr. Friedman, who is on leave from Pitt to work as a senior scholar and program officer in the National Library of Medicine at the National Institutes of Health in Bethesda, Md., and his colleagues developed detailed written synopses from 36 detailed diagnostically challenging cases from patient records at the University of Illinois at Chicago, the University of Michigan and the University of North Carolina. They recruited 216 volunteer subjects from those three institutions: 72 fourth-year medical students, 72 second- and third-year internal medicine residents and 72 general internists with faculty appointments and at least two years of post-residency experience.
Each subject worked on nine cases. For each case, subjects were asked to provide a diagnosis and a measure of their confidence in the diagnosis.
"Focusing on the residents and faculty physicians, what we found was that correctness and confidence were not aligned in about 1/3 of the cases," Dr. Friedman said. "For those cases where confidence and correctness were not aligned, physicians were more likely to lack confidence in a correct diagnosis than the reverse."
Nonetheless, he added, residents and more experienced internists were "overconfident," believing they are correct when in fact they were not, in 12 to 15 percent of the cases overall.
"Overconfidence is the zone of greatest concern when one thinks about how to improve practice using information technology," Dr. Friedman said. "While this was a study performed in the laboratory and not in the clinic, the apparent prevalence of overconfidence represents a major challenge to designers of decision support systems--if the advice of these systems is to be sought and attended to when it is really needed."
Also participating in the study were Guido G. Gatti, M.S., center for Biomedical Informatics, University of Pittsburgh; Timothy M. Franz, Ph.D., department of psychology, St. John Fisher College; Gwendolyn C. Murphy, Ph.D., Division of Community Health, Duke University; Fredric M. Wolf, Ph.D., department of medical education and informatics, University of Washington; Paul S. Heckerling, M.D., department of medicine, University of Illinois at Chicago; Paul L. Fine, M.D., department of medicine, University of Michigan; Thomas M. Miller, M.D., department of medicine, University of North Carolina; and Arthur S. Elstein, Ph.D., department of medical education, University of Illinois at Chicago.
Journal
Journal of General Internal Medicine