Single-shot dynamic 3D shape measurement via triangular-wave spatial multiplexing and transformer-based phase retrieval

Single-shot absolute 3D measurement requires not only accurate phase recovery but also identification of the corresponding fringe period. With other conditions held constant, increasing the spatial frequency of sinusoidal fringes enhances phase sensitivity to surface-height variations. Their periodicity, however, prevents wrapped phase alone from determining fringe order. For isolated objects or surfaces with abrupt height changes, phase unwrapping based solely on spatial continuity cannot reliably resolve this ambiguity. Auxiliary coding information must therefore be incorporated within the available intensity modulation range to support both fine phase recovery and period localization.

To address this challenge, the research group led by Professors Qian Chen and Chao Zuo at Nanjing University of Science and Technology proposes a single-shot structured-light method combining triangular-wave spatial multiplexing with Transformer-based phase retrieval. Low-frequency triangular-wave modulation is embedded in high-frequency sinusoidal fringes, with periodic triangular-wave peaks providing spatial localization cues. At the decoding stage, two branches based on an enhanced Swin-Unet recover the sine- and cosine-related components required for accurate wrapped-phase calculation and a coarse absolute phase, respectively. Fringe order is then determined using the physical relationship between the recovered wrapped phase and the coarse absolute phase, enabling absolute phase recovery and 3D reconstruction from a single composite fringe image. The work was published in Advanced Devices & Instrumentation under the title “Deep Learning Enhanced Dynamic 3-Dimensional Shape Measurement Using Single-Shot Spatial Multiplexing and Transformer-Based Phase Retrieval.” Postdoctoral researcher Yixuan Li and master’s student Yile Xiao are co-first authors. Postdoctoral researcher Jiaming Qian and Professors Shijie Feng, Qian Chen, and Chao Zuo are co-corresponding authors.

As shown in Figure 1, a low-frequency triangular wave is superimposed on high-frequency sinusoidal fringes. The modulation weight of the auxiliary component is selected to introduce period-localization cues while retaining high-frequency phase information. The two waveform periods jointly determine the repetition range of the composite code. The experiments use sinusoidal and triangular-wave periods of 19 and 51 pixels, respectively, so that their combined repetition period covers the lateral projection range used for encoding. Compared with a single sinusoidal input, this design supplies additional spatial coding information for distinguishing fringe periods.

The decoding process separates fine phase recovery from fringe-order determination. The first branch predicts sine- and cosine-related components and calculates the high-frequency wrapped phase using a four-quadrant arctangent operation, avoiding the learning difficulty associated with discontinuities in direct wrapped-phase regression. The second branch estimates a coarse absolute phase to identify the corresponding fringe period. Fringe order is obtained by dividing the difference between the coarse absolute phase and the wrapped phase by 2π and rounding to the nearest integer. The corresponding integer multiple of 2π is then added to the wrapped phase to recover absolute phase, followed by calibrated reconstruction of 3D coordinates. Provided that fringe order is correctly determined, fine variations in the final absolute phase are retained by the accurate wrapped phase, while the coarse absolute phase primarily provides period localization.

To exploit both local fringe structures and spatial relationships over larger regions, the researchers adopt an enhanced Swin-Unet encoder–decoder. Windowed attention, shifted-window attention, multi-scale feature extraction, and skip connections jointly integrate local details with contextual information. The shape and size of the attention window are further adapted to the dimensions of the deepest feature map, expanding the coverage of each window to improve modeling of the overall fringe distribution and long-range spatial relationships. Phase-related components and absolute phase obtained from three-frequency, 12-step phase-shifting measurements provide supervision, with training optimized using a loss that combines mean squared error and the structural similarity index.

With the same triangular-wave composite encoding, the enhanced Swin-Unet achieves absolute-phase mean absolute errors (MAEs) of 0.0124 rad and 0.0324 rad in two test scenes. These are lower than those of UNet (0.0209 rad and 0.0374 rad) and the original Swin-Unet (0.1224 rad and 0.1896 rad), as shown in Figure 2. The lower errors obtained in these scenes indicate that adapting the attention range to fringe structures can improve phase retrieval accuracy.

The encoding comparison further verifies the role of auxiliary spatial information. As shown in Figure 3, single-frequency sinusoidal, dual-frequency sinusoidal, and triangular-wave-embedded encoding are compared in terms of phase recovery and 3D reconstruction across three scenes. For a scene containing two isolated plaster models, single-frequency encoding yields a fringe-order accuracy of 12.225% and an absolute-phase MAE of 25.0148 rad, with pronounced periodic ambiguity and reconstruction errors. Dual-frequency sinusoidal encoding increases fringe-order accuracy to 99.424% and reduces the absolute-phase MAE to 0.0516 rad. Triangular-wave-embedded encoding further improves these values to 99.543% and 0.0324 rad, respectively. These results demonstrate that triangular-wave auxiliary encoding supplies effective period-localization information while supporting reliable fringe-order determination and accurate absolute-phase recovery in this scene.

For 3D accuracy evaluation, the researchers built a monocular fringe-projection system comprising a digital light projector and an industrial camera, with a calibrated measurement volume of 200 mm × 200 mm × 100 mm. Single-shot reconstruction of two standard ceramic spheres yields surface root-mean-square (RMS) errors of 0.060 mm and 0.064 mm, respectively. This geometric evaluation verifies reconstruction accuracy under the tested conditions and complements the quantitative phase-domain assessment.

Dynamic experiments examine a rotating plaster model and a moving industrial workpiece. Figure 4 presents composite fringes captured at different positions and orientations and the corresponding 3D reconstructions, resolving surface details of the model and geometric features of the workpiece. Each reconstruction uses only one fringe image acquired in a single exposure, without jointly processing multiple phase-shifted images captured at successive times during motion. This avoids temporal mismatch associated with multi-frame acquisition. The experiments demonstrate the feasibility of the method for moving-object 3D shape acquisition and its potential relevance to dynamic inspection and online quality control.

This study combines triangular-wave spatial multiplexing with Transformer-based phase retrieval through a joint design that enables absolute phase recovery and 3D shape reconstruction from a single composite fringe image. Triangular-wave auxiliary encoding provides fringe-period localization information for single-shot phase unwrapping, while the enhanced Swin-Unet improves the accuracy and robustness of phase retrieval in complex scenes through self-attention and multi-scale feature learning. Experiments involving standard-object measurements, complex-object reconstruction, comparisons of encoding strategies, and dynamic-object measurements collectively validate the method. Because each 3D reconstruction requires only one fringe image acquired in a single exposure, the method avoids reconstruction artifacts caused by inter-frame motion mismatch in multi-frame phase-shifting measurements, offering a new approach to structured-light 3D measurement of dynamic objects and complex surfaces. With further advances in lightweight network design, hardware acceleration, and system-level real-time deployment, the method could be extended to applications including online inspection in smart manufacturing, robotic 3D perception, dynamic measurements in medical applications, human-computer interaction, and augmented reality. Adaptive optimization of encoding parameters, model compression, and improved cross-scene generalization across different hardware platforms and challenging environments remain important directions for future research.