Article Highlight | 1-Feb-2024

First in-depth survey on the topic of the deep transfer learning for intelligent vehicle perception

Green Energy and Intelligent Transportation

A group of international scientists published a paper on the journal Green Energy and Intelligent Transportation on October 19th, 2023, summarizing a comprehensive review of deep transfer learning for intelligent vehicle perception.


In recent years, perception has been viewed as a critical component in intelligent vehicles for precise localization, safe motion planning, and robust control. The perception system provides intelligent vehicles with immediate environmental information about surrounding pedestrians, vehicles, traffic

signs, and other items and helps to avoid possible collisions.


Deep learning-based intelligent vehicle perception has been developing prominently to provide a reliable source for motion planning and decision making in autonomous driving. Many powerful deep learning-based methods can achieve excellent performance in solving various perception problems of autonomous driving.


However, these deep learning methods still have several limitations, for example, the assumption that lab-training (source domain) and real-testing (target domain) data follow the same feature distribution may not be practical in the real world. There is often a dramatic domain gap between them in many real-world cases. As a solution to this challenge, deep transfer learning can handle situations excellently by transferring the knowledge from one domain to another. Deep transfer learning aims to improve task performance in a new domain by leveraging the knowledge of similar tasks learned in another domain before.


There are currently no survey papers on the topic of deep transfer learning for intelligent vehicle perception. This survey paper aims to make contributions to introduce and explain the deep transfer learning techniques for intelligent vehicle perception, offering invaluable insights and directions for the future research.


For intelligent vehicles or autonomous driving, perception plays a crucial role in receiving data from sensors and extracting meaningful information from the surrounding environment, so as to make meaningful decisions for the precise motion planning by identifying obstacles, traffic signs/markers, and available driving areas. Researchers grouped these intelligent vehicle perception tasks into two classes (Object Detection, Semantic/Instance Segmentation).


Despite the remarkable achievements of the intelligent vehicle perception algorithms on benchmark datasets, there are still significant challenges in the real world due to the large variations in the sensor types and settings, data in diverse style, environment, weather and illumination, trained epoch, and architecture. Based on these observations, researchers divided the domain distribution discrepancy for intelligent vehicle perception into three types: sensor difference, data difference, and model difference.


With the rapid advancement of autonomous driving techniques, there is now an abundance of driving scene images available. Deep learning methods are booming in the application of autonomous driving with high performance perception. Transfer learning (TL) is a machine learning method to largely apply the knowledge acquired from one task or domain to another related task or domain. Researchers classified the deep transfer learning into several main types: Supervised TL, Unsupervised TL, Weakly-and-semi Supervised TL, Domain Generalization.


The main challenges of the deep transfer learning for the current intelligent vehicle perception include sensor robustness, methodology limitation, realism of synthetic data, scarcity of annotated benchmarks in complex scenarios, international standards for hardware sensors, and international standards for software packages.


In order to address the above challenges, the following work will be undertaken in the future. Firstly, more research should be focused on improving the sensor robustness. Secondly, researchers could make efforts to develop more advanced deep transfer learning methods. Thirdly, the realism of the synthetic data can be improved by more advanced computer game engines. What’s more, more high-quality benchmark datasets in complex driving scenarios could be collected and publicized. Finally, the multiple companies of different countries can collaborate together to promote the international standards for hardware sensors and software packages.






Authors: Xinyu Liu­a, Jinlong Li­a, Jin Ma­a, Huiming Sun­a, Zhigang Xu­b, Tianyun Zhang­a, Hongkai Yu­ a*

Title of original paper: Deep transfer learning for intelligent vehicle perception: A survey

Article link:

Journal: Green Energy and Intelligent Transportation

DOI: 10.1016/j.geits.2023.100125


a. Department of Electrical Engineering and Computer Science, Cleveland State University, Cleveland, OH 44115, USA

b. School of Information Engineering, Chang'an University, Xi'an 710064, China

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.