Public Release: 

Seeing with your ears

Novel acoustics project aims to improve virtual reality and explore ear shape effects on 3-D sound

Acoustical Society of America

IMAGE

IMAGE: This is a screenshot of a visual rendering model of Notre Dame Cathedral, created based on plans, laser scan data and site visits. view more 

Credit: Ghost Orchestra Project/LIMSI

WASHINGTON, D.C., June 25, 2017 -- Paris' Cathedral of Notre Dame has a ghost orchestra that is always performing, thanks to a sophisticated, multidisciplinary acoustics research project that will be presented during Acoustics '17 Boston, the third joint meeting of the Acoustical Society of America and the European Acoustics Association being held June 25-29, in Boston, Massachusetts.

In the project, computer models use recordings from a live concert held at the cathedral and detailed room acoustic simulations to produce a novel type of audience experience: a virtual recreation of the live performance using spatial audio and virtual reality.

Researchers reproduced the recordings using computerized acoustical data and enhanced it with computer-generated virtual navigation -- 3-D visualizations made with immersive architectural rendering that float the viewer through the complex acoustics of the acclaimed medieval gothic cathedral.

Combined, the multimodal sound and image footage of the ghost orchestra produce a spectral tour to the sounds of the 19th century opera "La Vierge" -- The Virgin -- performed live during the 2012-2013 concert season to celebrate the Cathedral's 850th anniversary.

Multimodal virtual reality integration is central to the project's significance, said Brian F.G. Katz, lead investigator and CNRS Research Director at the Institute Jean Le Rond d'Alembert, Pierre and Marie Curie University, in Paris.

"3D-audio is the hot topic today in virtual reality (VR) that is currently a very active subject in both academic and industrial research," Katz said. "With the commercialization of affordable VR systems -- the cheapest allowing for VR on smartphones -- spatial audio is rapidly immerging from the laboratory."

The next stage in spatial audio is personalized audio rendering that involves being able to adjust the rendering to match one's individual head and ear details.

"The importance of multimodal interactions, how visual and auditory cues balance in spatial perception, is key to VR and the sense of immersion, of being 'in' the VR world," Katz explained.

He envisions many applications emerging from the investigation.

"For me, spatial audio is a domain on the boundary of physical acoustics, psycho-acoustics, perception and cognition, and digital signal processing."

His work focuses on using each of the fields to learn more about the others -- from virtual audio scenes exploring how visually impaired people understand and remember architectural spaces, to improving understanding of spatial audition, to refining virtual reality rendering capabilities.

Created in the context of the French funded research projects FUI-BiLi, (Binaural Listening and ANR-ECHO), the acoustical foundation of the project is a 45 channel close-mic recording of the live concert made by the Conservatoire de Paris combined with a detailed geometrical room acoustics computational model that was created and calibrated based on in situ measurements of reverberation and clarity parameters.

Next, the team will apply the methodology to other complex multimodal environments such as theatre simulations.

"Aside from the acoustics in this project, I think we definitely learned a lot about computer graphics and VR content production, which has opened up a lot more dialogue with those communities that we intend to pursue," Katz said.

###

For more information about the project: https://groupeaa.limsi.fr/projets:ghostorch

Session 1aAAb1, "Experience with a virtual reality auralization of Notre-Dame Cathedral" by Brian F. Katz, is at 10:40-11:00 a.m. EDT, Sunday, June 25, 2017 in Room 208 of the John B. Hynes Veterans Memorial Convention Center.

MORE MEETING INFORMATION

Acoustics '17 Boston, the third joint meeting of the Acoustical Society of America and the European Acoustics Association.

The meeting is being held June 25-29, 2017 at the John B. Hynes Veterans Memorial Convention Center in Boston, Massachusetts.

USEFUL LINKS

Main meeting website: http://acousticalsociety.org/content/acoustics-17-boston

Technical program: http://acousticalsociety.org/content/program-acoustics-17-boston

Meeting/Hotel site: http://acousticalsociety.org/content/acoustics-17-boston#reservation

Press Room: http://acoustics.org

WORLD WIDE PRESS ROOM

In the coming weeks, ASA's World Wide Press Room will be updated with additional tips on dozens of newsworthy stories and with lay-language papers, which are 300-800 word summaries of presentations written by scientists for a general audience and accompanied by photos, audio, and video. You can visit the site during the meeting at: http://acoustics.org/world-wide-press-room/.

PRESS REGISTRATION

We will grant free registration to credentialed journalists and professional freelance journalists. If you are a reporter and would like to attend, contact Julia Majors (jmajors@aip.org) at AIP Media, 301-209-3090. For urgent requests, please contact media@aip.org who can also help with setting up interviews and obtaining images, sound clips, or background information.

LIVE MEDIA WEBCAST

A press briefing featuring will be webcast live from the conference on Monday, June 26, 2017 in the afternoon and Tuesday, June 27, 2017 in the morning in room 111 of the John B. Hynes Veterans Memorial Convention Center in Boston, Massachusetts.

Register at https://www1.webcastcanada.ca/webcast/registration/asa617.php to watch the live webcast. The schedule will be posted here as soon as it is available.

ABOUT THE ACOUSTICAL SOCIETY OF AMERICA

The Acoustical Society of America (ASA) is the premier international scientific society in acoustics devoted to the science and technology of sound. Its 7,000 members worldwide represent a broad spectrum of the study of acoustics. ASA publications include The Journal of the Acoustical Society of America (the world's leading journal on acoustics), Acoustics Today magazine, books, and standards on acoustics. The society also holds two major scientific meetings each year. For more information about ASA, visit our website at http://www.acousticalsociety.org.

ABOUT THE EUROPEAN ACOUSTICS ASSOCIATION

The European Acoustics Association (EAA) is a non-profit entity established in 1992 that includes in its membership societies predominantly in European countries interested in to promote development and progress of acoustics in its different aspects, its technologies and applications. EAA gathers 33 societies of acoustics and serves public citizens and more than 9000 individual members all over Europe with yearly events as well as scientific conferences and publications such as Acta Acustica united with Acustica and Acoustics in Practice. The European Acoustics Association (EAA) is an Affiliate Member of the International Commission for Acoustics (ICA) and of Initiative of Science in Europe ISE. Visit our website at https://euracoustics.org/.

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.