Article Highlight | 6-Feb-2024

Theoretical model for reliability assessment of machine learning systems

University of Tsukuba

Tsukuba, Japan—Machine learning systems for autonomous driving, diagnostic medical imaging, and other applications require reliable and safe output. One such system design is the N-version machine learning system. In this system, multiple machine learning models and input data are combined to deter inference errors in machine learning models from directly affecting the final output of the system. However, although it is known empirically that the diversity of machine learning models and input data affect the reliability of the output, a theoretical model to explain this has not yet been developed.

In this study, the researchers introduced the diversity metrics for machine learning models and input data with respect to inference errors of machine learning models and constructed a theoretical model to evaluate the reliability of the machine learning system output. The results showed that a configuration method that utilizes the diversity of machine learning models and input data is the most stable method for improving the reliability of a machine learning system under generally assumed situations.

The overhead and cost of performing multiple inference processes are other challenges in practical system design. Researchers will continue to investigate and develop methods to achieve high reliability in N-version machine learning systems while reducing the cost, power consumption, and overhead from both theoretical and experimental persupectives.

This work was supported by JSPS KAKENHI Grant Numbers 22K17871 and 19K24337.


Original Paper

Title of original paper:
Using Diversities to Model the Reliability of Two-version Machine Learning Systems

Transactions on Emerging Topics in Computing



Associate Professor MACHIDA, Fumio
Institute of Systems and Information Engineering, University of Tsukuba

Related Link

Institute of Systems and Information Engineering

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.