A new publication from Opto-Electronic Advances; DOI https://doi.org/10.29026/oea.2022.210021 discuss how deep learning makes single-shot, high-resolution structured light 3D imaging.
With the rapid development of optoelectronic information technology, three-dimensional (3D) imaging and sensing has become a research forefront in optical metrology. Following the transition from monochrome to color, from low-resolution to high-resolution, and from static photo to dynamic video, the evolution from “2D photography” to “3D sensing” has become the “fourth imaging revolution”. In this regard, fringe projection profilometry (FPP) is one of the most representative 3D imaging technologies due to its non-contact, high-resolution, high-speed, and full-field measurement capability, which has been widely applied in multiple fields, such as intelligent manufacturing, reverse engineering, industrial inspection, and heritage preservation.
In recent years, with the rapid advances of optoelectronic devices (e.g., digital light projectors, spatial light modulators, and high-speed image sensors) and digital signal processing units (e.g., high-performance computers and embedded processors), people subsequently set higher expectations on FPP: it should be both “high precision” and “high speed”. While these two aspects seem contradictory in nature, “speed” has gradually become a fundamental factor that must be taken into account when using FPP. For many fields, such as industrial manufacturing, “speed” means “efficiency”, and “efficiency” represents “productivity”. Under these circumstances, high-precision 3D reconstruction using only one single pattern has been the ultimate goal of structured light 3D imaging in perpetual pursuit.
The authors of this article report a deep-learning-enabled dual-frequency-multiplexing FPP technique, enabling single-shot unambiguous high-precision structured light 3D imaging. The proposed method constructed two parallel U-shaped networks with the same structure, one of which takes the well-designed dual-frequency composite fringe image as the network input, combined with the traditional phase-shifting physical model to predict the sine and cosine terms used to calculate the high-quality wrapped phase map, and the other is designed to predict the fringe order information from the input dual-frequency composite fringe image. Through “learning” from a large number of data sets, the properly trained neural networks can “de-multiplex” high-resolution, spectrum-crosstalk-free phases from the composite fringe and directly reconstruct a high-accuracy absolute phase map for single-shot, unambiguous 3D surface imaging.
The proposed deep-learning-enabled dual-frequency multiplexing FPP technique is expected to fill the speed “gap” between 3D imaging and 2D sensing, achieving true single-frame, high-precision, unambiguous 3D shape reconstruction. This research opens new avenues for single-shot, instantaneous 3D shape measurement of discontinuous and/or mutually isolated objects in fast motion. Nowadays, deep learning technology has fully “permeated” into almost all tasks of optical metrology. Despite the promising, in many cases impressive, results that have been reported in the literature, the underlying reasons behind these successes remain unclear at this stage. Many researchers are still skeptical and maintain a wait-and-see attitude towards its applications in high-risk scenarios, such as industrial inspection and medical care. But it can be envisaged that with the further development of artificial intelligence technology, the continuous improvement of computer hardware performance, and the further development of optical information processing techniques, these challenges will gradually be solved soon. Deep learning will thus play a more significant role and make a more far-reaching impact in optics and photonics.
Article reference: Li YX, Qian JM, Feng SJ, Chen Q, Zuo C. Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement. Opto-Electron Adv 5, 210021 (2022). doi: 10.29026/oea.2022.210021
Keywords: fringe projection profilometry (FPP) / phase unwrapping / deep learning
# # # # # #
The Smart Computational Imaging Laboratory (SCILab: www.scilaboratory.com) is part of the Nanjing University of Science and Technology and the Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense, Nanjing, Jiangsu Province, China. The group works at the intersection between Information Optics and Computer Vision, with focus on Computational Optical Imaging and Sensing Techniques such as Quantitative Phase Imaging, Optical Metrology, Phase Retrieval, and Digital Holography. They investigate how optical modulation and signal processing can be integrated into the optical system design, enabling new indirect imaging mechanisms for making the invisible visible and providing unprecedented perceptions that far exceeds the abilities of human and conventional image sensors.
# # # # # #
Opto-Electronic Advances (OEA) is a high-impact, open access, peer reviewed monthly SCI journal with an impact factor of 9.682 (Journals Citation Reports for IF 2020). Since its launch in March 2018, OEA has been indexed in SCI, EI, DOAJ, Scopus, CA and ICI databases over the time and expanded its Editorial Board to 36 members from 17 countries and regions (average h-index 49).
The journal is published by The Institute of Optics and Electronics, Chinese Academy of Sciences, aiming at providing a platform for researchers, academicians, professionals, practitioners, and students to impart and share knowledge in the form of high quality empirical and theoretical research papers covering the topics of optics, photonics and optoelectronics.
# # # # # #
More information: http://www.oejournal.org/oea
Editorial Board: http://www.oejournal.org/oea/editorialboard/list
All issues available in the online archive (http://www.oejournal.org/oea/archive).
Submissions to OEA may be made using ScholarOne (https://mc03.manuscriptcentral.com/oea).
ISSN: 2096-4579
CN: 51-1781/TN
Contact Us: oea@ioe.ac.cn
Twitter: @OptoElectronAdv (https://twitter.com/OptoElectronAdv?lang=en)
WeChat: OE_Journal
# # # # # #