AI and extended reality help to preserve built cultural heritage

The Notre-Dame Cathedral in Lausanne was built between 1170 and 1235 and is the largest Gothic church in Switzerland. Over time, this imposing building has been altered, extended and restored on multiple occasions. The nearly 80-metre-tall lantern tower, for example, has undergone several redesigns over the years, the most recent of which took place at the end of the 19th century during an extensive restoration. The stone used to build this cathedral, and many other monuments in the Swiss plateau, is particularly susceptible to a variety of degradation mechanisms, all of which are enabled by the presence of water (such as freezing, swelling and atmospheric pollution). This indicates that conservation measures, including water management, are a constant concern.

Extended reality in built heritage conservation 

In several scientific publications, ETH Zurich researchers have now demonstrated how digital tools can support conservators in their work. The background to this is the “Heritage++” research project at the ETH Institute for Building Materials. This project is led by Robert Flatt, Professor of Physical Chemistry of Building Materials and is associated to the ETH Center Design++, which explores ways in which computer-assisted methods can be used in architecture, engineering and construction. The central focus of the “Heritage++” project, is currently the use of extended reality (XR) and Artificial Intelligence (AI) in the restoration of historic buildings.  

The main outcome of this research is an immersive co-pilot – essentially, a virtual assistant designed to help conservators in carrying out their tasks. This co-pilot is built on spatial computing technologies that analyse the physical space and overlay it with digital information, or holograms, which can be viewed through a tablet or an XR headset. “XR technologies have huge potential to facilitate interdisciplinary collaboration between experts in architecture, materials science, history and built heritage conservation,” says Robert Flatt.  

In their latest article in the International Journal of Architectural Heritage, the researchers show how information about the structural analysis of the monument can be integrated and augmented through XR to support decision-making and disseminate heritage knowledge. “Our article proposes a new way to display and interact with how the building works structurally. Experts from different disciplines as well as the general public can explore these insights, helping to ensure that structural safety and integrity are properly considered in future interventions,” explains Ricardo Maia Avelino, a postdoctoral fellow at Design++ who previously completed his doctorate at ETH Zurich on the structural analysis of historic buildings.

A digital twin of Lausanne Cathedral 

The Notre-Dame Cathedral in Lausanne serves as a case study for the development of the digital co-pilot, which took place in two stages. First, the researchers created 3D models of the cathedral based on data obtained from laser scanning and photogrammetry surveys. Then, they enhanced the model with detailed information about the stones in the cathedral – each stone element was assigned an age, a mineralogical composition and a type of degradation. This information can be used to determine when a stone block was first installed, where it came from and what level of damage it exhibits. The relevant data came from various sources, including geologists at the University of Lausanne. At present, it covers specific sections of the cathedral, and the researchers are looking to complete this multimodal mapping in the years to come.

Anyone who now visits with a tablet or an XR headset can view those sections of the cathedral and see an overlay of information on the age, material composition and damage to any element they choose (see video). “During on-site inspections, our digital co-pilot gives conservators an overview of a monument’s condition along with additional relevant information. Then, based on any weak points identified, they can determine the appropriate course of action,” explains Yamini Patankar, who is involved in the project as a doctoral student and benefits from a Swiss Government Excellence Scholarship. 

Climate-resilient restoration strategies 

When it comes to restoring historic buildings, the current guiding principle is to preserve as much of the original structure as possible. To achieve this, the ETH Zurich researchers are studying how exactly the sandstone becomes damaged. Water plays a key role in all degradation mechanisms, and much of this comes through rain. While vertical rain can be managed by architectural interventions, wind-driven rain cannot. Exposure to such rain, but in particular extreme events of fast and extreme wetting, followed by rapid drying appears to be a key factor in degradation and is very location-specific.  

During such events, moisture penetrates the clay-bearing sandstone used to build the cathedral and causes damage through processes such as swelling, salt crystallisation and freezing. To study how damage occurs and examine the influence of air pollution, the ETH Zurich researchers have been measuring wind-driven rain and the local microclimate, as well as moisture profiles within the stones over a number of years using a network of sensors around the cathedral.

Alongside this, ETH Zurich researchers are also developing a model to correlate local exposure, in particular wind-driven rain, to the observed forms and extents of damage. Experts could use this as a tool to predict how damage may change in the future, not just as a result of the course of time, but in relation to climate changes, in particular the nature, intensity and frequency of extreme events around the cathedral. They could then use this information as a basis to develop tailored restoration strategies and improve the durability of restoration measures, anticipating the consequences of climate change, which represents a real challenge for practitioners.  

Applicable to other buildings 

The use of digital tools such as 3D models or Heritage Building Information Modelling is nothing new in built heritage. However, the “Heritage++” project takes this one step further by integrating information from various disciplines, with a focus on conservation, allowing it to be used interactively and also making it possible to include conservators’ practical knowledge and expertise. In the future, the digital co-pilot, as applied in Lausanne Cathedral, could also be used for other sandstone monuments, such as bridges, castles or monasteries, and, in the medium term, for buildings made from other construction materials. 

Christophe Amsler, the architect responsible for Lausanne Cathedral, believes that digital technologies will become an integral part of conservation work: “Preserving historic monuments requires constant care and attention. Advances in technology can help us to repair and keep these buildings living, but at the same time, we need to remain faithful to the original spirit and heritage of the monument.” 

An app for the 750th anniversary 

In October, Lausanne celebrated the 750th anniversary of the Notre-Dame Cathedral. To mark the occasion, ETH researchers developed an app as part of “Heritage++” to provide the general public with an insight into the conservation work around the cathedral. “Visitors to the cathedral can use the app, which incorporates extended reality technologies, to access information on the building’s history and learn more about what conservation work is under way and how and why it is being carried out,” explains ETH doctoral student Camilla Tennenini, a member of Robert Flatt’s group. 

The app combines a wide range of specialist literature with video recordings in which experts give additional insight into the restoration work. This information is also visualised on the cathedral itself through the previously mentioned 3D model and augmented reality. Visitors can access this inside or outside the cathedral via a smartphone or tablet. 

Beyond the 750th anniversary, the co-pilot will be enhanced as part of the “Heritage++” project with a large language model that extracts information from specialist literature and videos. This would allow conservators to access information relevant to the restoration work directly on site. In the medium term, the public will also be able to use the app to ask questions about the cathedral's renovations.  

References

Avelino RM, Yang W, Weichbrodt A, Ochsendorf J, Flatt RJ: Augmented Reality for Structural Inspection of Historic Monuments: The Case of Lausanne Cathedral. International Journal of Architectural Heritage, 7. November 2025.  DOI: doi.org/10.1080/15583058.2025.2578318

Patankar Y, Tennenini C, Bischof R, Khatri I, Avelino RM, Yang W, Mahamaliyev N, Scotto F, Mitterberger D, Bickel B, Girardet F, Amsler C, Bomou B, Flatt RJ: Heritage ++, a Spatial Computing approach to Heritage Conservation. RILEM Technical Letters 9 (2024), pp. 50–60. DOI: doi.org/10.21809/rilemtechlett.2024.202

 

---

---