Artificial intelligence to help prevent illegal construction

Unmanned aerial vehicles, drones, are no longer a surprise to passers-by during events or on a sunny afternoon in the city centre. Drones, which have long been used in warfare, are now popular not only with professional and amateur filmmakers but also with researchers. Kaunas University of Technology scientist Rytis Maskeliūnas together with a team of researchers from other Lithuanian universities used UAV technology to detect changes in building facades against a crowded city background.

With the emergence of digitised cities and digital twins, which makes it possible to observe the various changes in buildings, both external and internal, there are more and more novel opportunities to capture such data. Outdoor image processing is difficult in typical European metropolitan areas due to various obstructions to perspectives such as wires, overhangs, poles, and other parts of a building as well as dynamically changing weather conditions affecting the quality of UAV imagery.

A team of researchers, including Rytis Maskeliūnas from the Kaunas University of Technology (KTU), Department of Multimedia Engineering, chose to study the facades of buildings in Vilnius Old Town and surrounding areas.

“Vilnius is a very unique place architecturally. The city’s Old Town is a UNESCO World Heritage Site and is full of buildings that vary in style, from Gothic church spires to state-of-the-art glass structures,” says Andrius Katkevičius professor at the Department of Electronic Systems, Vilnius Gediminas Technical University, stressing that this is what motivated them to investigate whether algorithms would be able to work across such a wide range of styles.

Such a challenge, according to Maskeliūnas, requires the use of smart signal processing solutions based on artificial intelligence to detect identify and classify buildings in the overall background of the city.

Drone assessment of the technical condition of buildings

“The facade is the identity of the building, the architectural face that allows the building to be assigned to one or another category or style. Imagine if suddenly part of the facade of an old building in the Old Town was glazed or redecorated in some avant-garde manner. A drone flying by would record such a change – perhaps it is illegal construction or urban blight,” says Darius Plonis, professor at the Department of Electronic Systems, Vilnius Gediminas Technical University.

As the drone flies through the city and photographs building facades, the data is sent wirelessly to a computing platform that can determine what class of facade a particular building belongs to.

Maskeliūnas believes that the drone’s photographs of the facade could also be used to assess the technical condition of buildings: “Water penetration through seams, cracks in the decor or structure, discolouration of facade elements or potentially even dents in the surfaces – all these determine the classification of a building and could potentially allow for a fairly accurate and automated assessment.”

The Kaunas University of Technology researcher stresses that the parts monitored by the drone can also be analysed independently of the facade as a whole, and their durability can be assessed by comparing the changes, both by visual inspection and concerning the building’s design documentation, construction materials, and the different extent and severity of the damage.

An overcrowded city is a challenge

According to the researchers, the study aims to detect the boundaries of a building’s facade in the face of changing weather and light conditions and to determine its actual style of it based on facade taxonomy. The complexity of the task depends on the shape of the building’s facade, the natural conditions and the urban background behind it. 

“For the algorithm, a congested city is the most difficult task to extract useful information, as buildings in city centres or old towns are very often obscured by various objects such as wires, trees and even festive decorations, and advertisements. Nevertheless, the UAV can collect a detailed, high-quality image of the facade from all angles,” the researchers note.

Previously, using rudimentary mathematical models, the researchers could only identify buildings with a simple facade structure. In addition, the buildings to be classified had to be separated from each other, i.e. not share any common walls or at least be distinguishable stylistically.

According to Maskeliūnas, identifying buildings with a similar facade structure poses many problems.

“Distance and environmental factors make it difficult to distinguish the contours of facade elements, and false contours can be caused by shadows or reflections, as well as camera angle and lens distortion. Simply extracting the correct shape does not guarantee successful further classification,” says the researcher, emphasizing that these are the reasons why artificial intelligence-based signal processing solutions were used in this study.

Images of the urban landscape captured by drones can also be used for visual landmarks and to identify defects in historic buildings, and urban heritage sites, or for planning urban development.

“This is a very promising technology for monitoring high-rise buildings and apartment blocks, as it is difficult to see the top from the ground,” the researchers add.