AI scribes promise faster emergency care — but at what cost?
As hospitals adopt AI tools to reduce documentation burden, UVA’s Tom Hartvigsen warns that efficiency gains may come at the expense of care quality and patient data privacy
image: Tom Hartvigsen, assistant professor of data science at the University of Virginia
Credit: University of Virginia School of Data Science
Season two of HBO’s Emmy award-winning show “The Pitt” is bringing renewed attention to the role of artificial intelligence in emergency medicine, featuring a storyline in which emergency room staff consider adopting AI-based tools to accelerate clinical documentation.
In the show, a new attending physician, Baran Al Hashimi, proposes using AI systems to speed up the process of charting patient conditions — reflecting real-world technologies known as AI scribes. These tools are designed to record and summarize clinician-patient interactions, potentially reducing administrative workload for healthcare professionals.
To better understand the implications of these systems, we spoke with Tom Hartvigsen, assistant professor of data science at the University of Virginia, whose work focuses on making machine learning systems more trustworthy and responsible in high-stakes environments.
AI scribes are increasingly being adopted in healthcare settings, in part because of clinician burnout and staffing pressures. However, Hartvigsen notes that adoption is also shaped by economic constraints in healthcare systems.
“AI scribes are pitched as solutions to a problem that is very real: Clinicians are overworked, and their burnout rates are high. Plus, the risks are hard to measure because most AI scribe errors won’t cause catastrophes,” he said. “When hiring new health care professionals is more expensive than AI subscriptions, administrators can choose AI.”
While these systems may improve documentation efficiency, Hartvigsen cautions that efficiency gains could reshape care delivery in unintended ways.
“This is a big risk of AI scribes. Efficiency is more easily measured than quality of care, so therefore easier to optimize,” he said. “I worry that AI scribes may indeed accelerate care, but towards mediocrity instead of excellence.”
Another major concern involves patient privacy and the handling of sensitive medical data when it is processed by artificial intelligence systems.
“Patient data is absolutely at risk of being trained on and sold when ingested by AI scribes,” he said. “Any time a generative machine learning model is trained on patient data, there is a risk that the data is then exposed in the future in inappropriate contexts.”
Hartvigsen emphasizes that, given the early stage of widespread adoption, patients and providers are still actively navigating how these tools should be governed and used responsibly.
“If patients want to protect their data, they should ask their care providers whether their data are being ingested by AI. If so, ask about any benefits they’ve observed and what protections are in place,” he suggested. “It’s such new technology that many of these conversations start at the ground up, as we’re still quickly identifying opportunities and challenges.”
Hartvigsen is an assistant professor of data science at the University of Virginia. He works to make machine learning trustworthy, robust, and socially responsible enough for deployment in high-stakes, dynamic settings.
Prior to joining UVA, Hartvigsen was a postdoctoral associate at MIT’s Computer Science and Artificial Intelligence Laboratory. He holds a doctorate and master’s degree in data science from Worcester Polytechnic Institute and a bachelor’s degree in applied math from SUNY Geneseo.