Artificial intelligence-supported spatial scanning for enhanced real-time spectral analysis of heterogeneous media

Research on near-infrared (NIR) and mid-infrared (MIR) spectroscopy draws much attention in the development of rapid, non-destructive material analysis. In agriculture and the food industry, portable spectral sensors equipped with AI algorithms often serve as key functional components to assess soil health, crop quality, and product safety. Effective extraction of precise spectral data holds the promise for improving real-time decision-making in these fields. However, sample heterogeneity is one of the greatest challenges. Inhomogeneous granular samples, such as cereals, feature randomly packed particles that introduce air gaps and irregular scattering angles. These localized variations lead to "spectrospatial noise." While spatial averaging techniques like sample spinning or beam expansion are used in benchtop instruments, implementing them in miniaturized MEMS-based handheld sensors without sacrificing the Signal-to-Noise Ratio (SNR) or device portability has remained a crucial and long-standing challenge for spectral analysis.

A collaborative team comprising researchers from Si-Ware Systems, Ain Shams University, and Université Gustave Eiffel/CNRS (including Bassem Mortada, Yasser M. Sabry, Bassam Saadany, Diaa Khalil, and Tarik Bourouina) successfully achieved enhanced real-time spectral analysis of heterogeneous media through an AI-supported spatial scanning method. They compared miniaturized MEMS-based Fourier-Transform Near-Infrared (FT-NIR) spectral sensors with optical spot sizes of 3 mm, 6 mm, 10 mm, and 20 mm (Figure 1). To formalize the challenge of sample non-homogeneity, they developed an analytical model that treats the overall absorbance variability as a combination of spatial and system electrical noise. This model matched the practical results well, highlighting a critical trade-off: while a larger optical spot size captures more sample granules and reduces spatial noise, it simultaneously increases electrical noise due to reduced optical throughput. By manually and automatically scanning the sensors across the samples (Figure 2), they leveraged the high-speed micromotor of the MEMS Michelson core interferometer to collect and average hundreds of interferograms in real time. Mathematically, this scanning approach functions as a continuous moving average, integrating the fluctuating reflectance signals over the entire scanned distance to effectively negate localized spatial variations. This demonstration provides important insights for overcoming spectrospatial noise in miniaturized FTIR technology for field-deployable spectroscopic analysis.

While averaging multiple stationary refills improves repeatability, it requires significant testing time and manual effort. To overcome this, the researchers introduced the spatial scanning method. Theoretical illustrations of sample reflectance variations over time indicated that continuously sweeping the sample—coupled with the high speed of the MEMS interferometer mirror—effectively smooths out random spatial fluctuations without inducing spectral errors within a single mirror cycle (Figure 3).

To evaluate the fundamental performance of the varying optical heads, the researchers first analyzed their Signal-to-Noise Ratio (SNR). As predicted by the analytical model, the experimental data confirmed an inverse relationship between SNR and spot size for high-reflectance samples (Figure 4). While smaller spot sizes yield higher SNR, they capture less spatial information, making them highly susceptible to localized variations in heterogeneous samples. This trade-off was further investigated by examining the absorbance repeatability across multiple stationary measurements. When measuring inhomogeneous wheat samples with 6-mm, 10-mm, and 20-mm optical heads, larger spot sizes inherently yielded better absorbance repeatability due to the natural averaging of spatial noise (Figure 5).

To fully understand this dynamic, the team developed a theoretical model that combined the competing effects of both spatial and electrical noise. The model revealed an optimal spot size where the combined standard deviation of absorbance is minimized, and it perfectly matched the experimental data across varying numbers of sample refills (Figure 6).

This theoretical framework was then rigorously validated through experimental spectral measurements. The spatial scanning approach remarkably improved spectral repeatability compared to stationary modes. The scanned measurements demonstrated substantially lower absorbance standard deviations—improving by a factor of up to 5.75 for the 10-mm sensor—which remained consistent even after applying Standard Normal Variate (SNV) processing to correct for baseline and vertical sample shifts (Figure 7).

Furthermore, the researchers conducted rigorous validation of the physical significance of the spatial scanning approach through an AI-based chemometric testing. Figures 8 and 9 highlights the cross-validation performance of Partial Least Squares Regression (PLSR) and Artificial Neural Network (ANN) models when utilizing the spatial scanning technique. As illustrated, spatial scanning deeply improves the accuracy of the chemometric models. For a single-fill measurement using a 10 mm sensor, the protein RMSE was reduced by 60.2% and moisture RMSE by 43.5% compared to stationary modes. A two-sample F-test confirmed the statistical significance of these improvements, proving that dynamic spatial scanning effectively mimics the reliability of cumbersome, multi-refill stationary laboratory processes.

In this study, an effective spatial scanning method for minimizing spectrospatial noise in heterogeneous media using MEMS FT-NIR spectral sensors is reported. By utilizing sensors with varying optical spot sizes and continuously sweeping them across granular samples (such as wheat and hay), the researchers achieved highly accurate spatial averaging without sacrificing the high SNR of the device. The absorbance repeatability of the sensors was investigated based on theoretical noise modeling and experimental measurements, finding that the 10 mm sensor under scanning conditions improved repeatability by a factor of 5.75. This dependence indicates the dominant role of both spatial non-homogeneity and system electrical noise in miniaturized optical limits. The chemometric calculations based on PLSR and ANN models revealed that spatial scanning is key for achieving highly accurate predictions in non-uniform samples, reducing protein and moisture RMSE by up to 60%. NIR prediction repeatability of hay samples was found to be improved up to 4 times by the spinning dish compared to the stationary measurement mode (Figure 10). Looking forward, this demonstration of spatial scanning in miniature FT-NIR structures provides a foundational approach for the structural and operational design of portable sensors in precision agriculture and smart industry. Future integrations may even utilize advanced metasurfaces for optical manipulation and local mixing, further simplifying the analytical setup.