Sophia Antipolis, France, 14 June: The first study to use artificial intelligence (AI) technology to generate podcasts about research published in scientific papers has shown the results were so good that half of the papers’ authors thought the podcasters were human.
In research published today (Saturday) in the European Journal of Cardiovascular Nursing (EJCN) [1], researchers led by Professor Philip Moons from the University of Leuven, Belgium, used Google NotebookLM, a personalised AI research assistant created by Google Labs, to make podcasts explaining research published recently in the EJCN.
Prof. Moons, who also presented the findings at the Association of Cardiovascular Nursing and Allied Professions (ACNAP) conference in Sophia Antipolis today, France, said: “In September 2024, Google launched a new feature in NotebookLM that enables users to make AI-generated podcasts. It made me think about how it could be used by researchers and editors.
“When I did a first test case with one of my own articles, I was flabbergasted by the high quality and how natural it sounded. At that moment, I realised that such a system could have the potential to be used by journals for science communication. Of course, the quality and accuracy needed to be evaluated. Therefore, we designed this study to evaluate its potential.”
Prof. Moons, who is Professor of Healthcare and Nursing Science at KU Leuven and editor-in-chief of the EJCN, and his colleagues selected ten different types of articles [2] and contacted the authors to ask if they would agree to take part in the study. The participants were not informed that the podcast about their study would be generated by AI.
The researchers sent the AI-generated podcasts to the authors to evaluate for engagement, trustworthiness and AI detection. They followed up with a questionnaire (generated with the help of ChatGPT) and a 30-minute interview via Microsoft Teams.
Engagement
The authors reported that the podcasts captured the key points of their papers in ‘very simple, easy to understand terms’, were well-structured, had a good balance in terms of length and depth, and that the podcast ‘hosts’ were professional, with some authors even assuming the speakers had a background in nursing or medicine. The conversational interaction of the hosts was a significant asset.
Trustworthiness
Most of the authors said the podcasts were reliable sources of information. However, some commented on the American accent and style, including some hyping of research findings with the use of words such as ‘amazing’, ‘groundbreaking’ and ‘totally’. They said there were some inaccuracies and misrepresentations, there was sometimes lack of context, incorrect use of medical terminology and mispronunciation of medical words. The podcasts would need to be checked carefully for accuracy before their release.
AI detection
When they were asked: ‘Would you be surprised if I told you this podcast was generated through Artificial Intelligence?’, five authors said ‘yes’, including one author who had conducted research on AI previously. They said they were ‘shocked’, ‘amazed’, or even ‘having an existential crisis’. Five others said ‘no’. The participants said that if AI podcasts were released, they should make it clear that they were AI-generated, acknowledge the authors and reference the original article.
All the authors said they thought that patients and the general public would be the most appropriate target for the podcasts, mainly because of the tone and the ability of the podcasts to explain the articles in an easy-to-understand manner. However, some commented that the podcasts might also be useful for enabling healthcare workers to stay up to date with the latest research, access it more easily and make the original research article more visible.
Some of the authors suggested that podcasts could be tailored for particular audiences according to age, interests or ethnic background as the technology develops and improves. At present, it is not possible to change the voices or the language of the AI ‘hosts’, but these features will probably be enabled in the future.
Prof. Moons said: “It was striking how accurate the podcasts were in general. Knowing that we are just at the beginning of this kind of AI-generated podcasts, the quality will become better over time, probably within the next few months. Another important aspect is that these podcasts seem most suitable for a non-technical audience, for instance, the general public or patients.
“If podcasts could be generated by AI, that could really be a game-changer. Podcasts could be made with very little work, just by uploading the article and maybe a bit of prompting. This could be a sustainable model to get the message out to people who do not typically read scientific journals.
“I think that this will be a technology that will allow editors, journals and researchers to communicate about science to the general public. It will not make human podcasters redundant. There will always be a market for human-made podcasts, probably because not all topics are capable of being addressed accurately or appropriately by AI. I can even imagine that there will be hybrid podcasts, in which human podcasters and AI come together in different sections of a podcast episode.”
Now, the researchers plan to explore further possibilities of these podcasts for science communication, including seeing what patients and other members of the public think of them. They also want to investigate if it would be possible to use AI-generated podcasts for scientific conference sessions. “For instance, making a podcast on the content of sessions at the ESC conferences, as a summary for those who did not participate and want to get a blurb of it,” said Prof. Moons.
Evaluate AI-generated podcasts for yourself
The researchers have made an AI-podcast on their research paper about AI-generated podcasts. A link to it is contained in the online published paper.
ENDS
Notes:
[1] “Artificial intelligence-generated podcasts open new doors to make science accessible: a mixed-method evaluation of quality and the potential for science communication”, by Chloé Desmedt et al. European Journal of Cardiovascular Nursing, doi:10.1093/eurjcn/zvaf074
[2] The papers included five original research articles, two reviews, one patient perspective, one methods corner and one discussion paper.
Journalists can download an embargoed pdf of the full research paper and editorial from the European Journal of Cardiovascular Nursing before the publication date at: zvaf074_PM.pdf
N.B. Journalists: please don’t use this link in any stories that you publish. It is for the exclusive use of the media BEFORE publication. After publication the paper will be replaced, so please provide the following permanent link to the published research paper on the European Journal of Cardiovascular Nursing website in your stories:
https://academic.oup.com/eurjcn/article-lookup/doi/10.1093/eurjcn/zvaf074
Please acknowledge the journal as a source in any articles.
When obtaining outside comment, journalists are requested to ensure that their contacts are aware of the embargo on this release.
Funding: The study received no specific funding. The first author, Chloé Desmedt, has a PhD position funded by the Frans Van De Werf Fund for cardiovascular research.
ESC Press Office:
Mobile: +33 (0) 6 61 40 18 84
Email: press@escardio.org
Follow us on X @ESCardioNews
|
About the European Society of Cardiology
|
|
Journal
European Journal of Cardiovascular Nursing