News Release

How does my therapist rate?

New machine-learning software developed by USC, University of Washington and University of Utah researchers detects empathy in therapy sessions

Peer-Reviewed Publication

University of Southern California

"And how does that make you feel? Empathy is the foundation of therapeutic intervention. But how can you know if your therapist is or will be empathetic? Technology developed by researchers from USC, University of Washington, and the University of Utah can tell you.

Leveraging developments in automatic speech recognition, natural language processing and machine learning, researchers Bo Xiao (Ming Hsieh Department of Electrical Engineering at the USC Viterbi School of Engineering), Zac E. Imel (Department of Educational Psychology at the University of Utah), Panayiotis G. Georgiou (Ming Hsieh Department of Electrical Engineering at the USC Viterbi School of Engineering), David C. Atkins (Department of Psychiatry and Behavioral Sciences at the University of Washington) and Shrikanth S. Narayanan (Ming Hsieh Department of Electrical Engineering at the USC Viterbi School of Engineering), developed software to detect "high-empathy" or "low-empathy" speech by analyzing more than 1,000 therapist-patient sessions. The researchers designed a machine-learning algorithm that takes speech as its input to generate an empathy score for each session automatically.

Their methodology is documented in a forthcoming article titled, "'Rate My Therapist': Automated Detection of Empathy in Drug and Alcohol Counseling via Speech and Language Processing," and according to the authors, is the first study of its kind to record therapy sessions and automatically determine the quality of a therapy session based on a single characteristic. The study appears in the December issue of PLoS ONE.

Currently, there are very few ways to assess the quality of a therapy session. In fact, according the researchers, the methods for evaluating therapy have remained unchanged for seventy years. Methods requiring third-party human evaluators are time-consuming and affect the privacy of each session.

Instead, imagine a natural language processing app like SIRI listening in for the right phrases and vocal qualities. The researchers are building on a emerging a new field in engineering and computer science called behavioral signal processing, which "utilizes computational methods to assist in human decision-making about behavioral phenomena."

The authors taught their algorithm to recognize empathy via data from training sessions for therapists, specifically looking at therapeutic interactions with individuals coping with addiction and alcoholism. Using automatic speech recognition and machine learning based models, the algorithm then automatically identified select phrases that would indicate whether a therapist demonstrated high or low empathy.

Key phrases such as: "it sounds like," "do you think," and "what I'm hearing," indicated high empathy, while phrases such as "next question," "you need to," and "during the past," were perceived as low-empathy by the computational model.

Speaking about this innovation Shri Narayanan, Andrew J. Viterbi Professor of Engineering at USC and the senior author on this study, said, "Technological advances in human behavioral signal processing and informatics promise not only scale up and provide cost savings through automation of processes that are typically manual, but enable new insights by offering tools for discovery. This particular study gets at a hidden mental state and what this shows is that computers can be trained to detect constructs like empathy using observational data."

Narayanan's team in the Signal Analysis and Interpretation Lab at USC continues to develop more advanced models -- giving the algorithm the capacity to analyze diction, the tone of voice, the musicality of one's speech (prosody) as well as how the cadence of one speaker in conversation is echoed with another (for example when a person talks fast and the listener's oral response mirrors the rhythm with quick speech).

In the near term, the researchers are hoping to use this tool to train aspiring therapists.

"Being able to assess the quality of psychotherapy is critical to ensuring that patients receive quality treatment, said David Atkins, a University of Washington research professor of psychiatry.

"...The sort of technology our team of engineers and psychologists is developing may offer one way to help providers get immediate feedback on what they are doing - and ultimately improve the effectiveness of mental health care," said Zac Imel, a University of Utah professor of educational psychology and the paper's corresponding author.

In the long run, the team hopes to create software that provides real-time feedback or rates a therapy session on the spot. In addition, the researchers want to incorporate additional elements into their empathy rating algorithm, including acoustic channels and the frequency with which a therapist or patient speaks.


About USC Viterbi School of Engineering:

Engineering Studies began at the University of Southern California in 1905. Nearly a century later, the Viterbi School of Engineering received a naming gift in 2004 from alumnus Andrew J. Viterbi, inventor of the Viterbi algorithm now key to cell phone technology and numerous data applications. USC Viterbi is ranked among the top graduate programs in the world and enrolls more than 6,500 undergraduate and graduate students taught by 185 tenured and tenure-track faculty, with 73 endowed chairs and professorships.

About UW Medicine

UW Medicine?s mission is to improve the health of the public by advancing medical knowledge, providing patient care, and training the next generation of health professionals. Our system includes Harborview Medical Center, Northwest Hospital & Medical Center, Valley Medical Center, UW Medical Center, UW Neighborhood Clinics, UW Physicians, UW School of Medicine and Airlift Northwest.

About the University of Utah

The University of Utah, located in Salt Lake City in the foothills of the Wasatch Range, is the flagship institution of higher learning in Utah. Founded in 1850, it serves more than 33,000 students from across the United States and the world. With more than 100 major subjects at the undergraduate level and more than 90 major fields of study at the graduate level, including law and medicine, the university prepares students to live and compete in the global workplace. Learn more about all the U has to offer online at .

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.