A Comprehensive Framework for Multimodal Sensor Fusion in Intelligent Manufacturing: Innovations, Interpretability, and Real-world Applications

Yue Zhu

doi:10.5281/zenodo.13905495

OPEN ACCESS |Research Article ||2 November 2024

A Comprehensive Framework for Multimodal Sensor Fusion in Intelligent Manufacturing: Innovations, Interpretability, and Real-world Applications

Yue Zhu

Georgia Institute of Technology

zechzhuyue@gmail.com

Georgia Institute of Technology, California, United States.

* Corresponding Author¹: Yue Zhu, E-Mail: zechzhuyue@gmail.com

Publication

Accepted Unknow ; Published 2024 November 2

Journal of Computer Technology and Applied Mathematics, 2024, 1(4), 3007-4126.

Abstract

This paper presents the novel work of developing an intelligent manufacturing framework based on multimodal sensor integration and computer vision. In this paper, we propose a hybrid fusion method that includes both early and late fusion with attention mechanisms to select the most important sensor data. Our system gathers visual, thermal, acoustic, and vibration data and offers accurate and interpretable predictions for fault identification, process enhancement, and product quality. (Liu et al. 2024) We meet the challenge of the opacity of AI systems by using explainable AI methods to help the user comprehend the results of the model. It shows that the proposed system is accurate, efficient, scalable, and can be applied to various types of data. Examples from the industry present real-life experiences and issues that may be encountered when implementing our system in different manufacturing contexts. It presents a new paradigm shift in smart manufacturing systems through the enhancement of efficiency, reliability, and interpretability for future research and industrial development.

Keywords

Multimodal Sensor Fusion , Smart Manufacturing , Explainable AI (XAI) , Industry 4.0 , Predictive Maintenance , Fault Detection , Attention Mechanisms , Deep Learning , Hybrid Fusion , Random Forest Classifier , Data Integration , Machine Learning , Feature Extraction , Real-Time Processing , Condition Monitoring .

Metadata

DOI:

10.5281/zenodo.13905495

ARK:

ark:/40704/JCTAM.v1n4a05

Pages: 36-46

References: 19

Disciplines: Computer Science

Subjects: Deep Learning

Cite This Article

APA Style

Zhu, Y. (2024). A comprehensive framework for multimodal sensor fusion in intelligent manufacturing: innovations, interpretability, and real-world applications. Journal of Computer Technology and Applied Mathematics, 1(4), 36-46. https://doi.org/10.5281/zenodo.13905495

Acknowledgments

The authors thank the editor and anonymous reviewers for their helpful comments and valuable suggestions.

FUNDING

Not applicable.

INSTITUTIONAL REVIEW BOARD STATEMENT

Not applicable.

DATA AVAILABILITY STATEMENT

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

INFORMED CONSENT STATEMENT

Not applicable.

CONFLICT OF INTEREST

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

AUTHOR CONTRIBUTIONS

Not applicable.

References

1.

Liu, Z., Xiao, G., Liu, H., & Wei, H. (2024). Multi-Sensor Measurement and Data Fusion. IEEE Transactions on Instrumentation and Measurement, 73, 234-245. https://ieeexplore.ieee.org/abstract/document/9693406?casa_token=40VH55MYDtEAAAAA:5M5RuueqkCua6ONFCVjNeNl8V0TlZ4-giZ8VQzK0U6CXh_TefMxhQaCBotzrZXsBrmaDEjnq

2.

Diez-Olivan, A., Del Ser, J., Galar, D., & Sierra, B. (2019). Data Fusion and Machine Learning for Industrial Prognosis: Trends and Perspectives Towards Industry 4.0. Journal of Manufacturing Processes, 40, 123-134. https://www.sciencedirect.com/science/article/pii/S1566253518304706?casa_token=Bi4TM627bvIAAAAA:FIhyFRBhR4K3o7_Ly5thLLoHChk7UYVPCZTvMDEK5aPxTYUJQuAPOI_Dn3ZHRIRz1TUbIaQ_

3.

Helwig, Nikolai, Pignanelli, Eliseo, and Schtze, Andreas. (2018). Condition monitoring of hydraulic systems. UCI Machine Learning Repository. https://doi.org/10.24432/C5CW21.

4.

Vergara, Alexander. (2013). Gas Sensor Array Drift at Different Concentrations. UCI Machine Learning Repository. https://doi.org/10.24432/C5MK6M.

5.

Buscema, M., Terzi, S., & Tastle, W. (2010). Steel Plates Faults. UCI Machine Learning Repository. https://doi.org/10.24432/C5J88N

6.

Zhang, Z., Farnsworth, M., Song, B., Tiwari, D., & Tiwari, A. (2024). Deep Transfer Learning With Self-Attention for Industry Sensor Fusion Tasks. IEEE Transactions on Industrial Informatics. https://ieeexplore.ieee.org/abstract/document/9813440

7.

Sensor Data Fusion in Industry 4.0. (2024). Farnell. https://si.farnell.com/sensor-data-fusion-in-industry-40-trc-ar

8.

Goyal, D., & Pabla, B. S. (2015). Condition-based maintenance of machine tools—A review. CIRP Journal of Manufacturing Science and Technology, 10, 24-35. https://doi.org/10.1016/j.cirpj.2015.05.002

9.

Kang, H. S., Lee, J. Y., Choi, S., Kim, H., Park, J. H., Son, J. Y., ... & Do Noh, S. (2016). Smart manufacturing: Past research, present findings, and future directions. International Journal of Precision Engineering and Manufacturing-Green Technology, 3(1), 111-128. https://doi.org/10.1007/s40684-016-0015-5

10.

Kusiak, A. (2019). Fundamentals of smart manufacturing: A multi-thread perspective. Annual Reviews in Control, 47, 214-220. https://doi.org/10.1016/j.arcontrol.2019.02.001

11.

Liu, Y., Li, S., Liu, C., & Wang, H. (2021). Explainable AI for intelligent manufacturing: A survey. AI Open, 2(1), 105-123. https://doi.org/10.1002/aiopen.76

12.

Monostori, L., Kádár, B., Bauernhansl, T., Kondoh, S., Kumara, S., Reinhart, G., ... & Ueda, K. (2016). Cyber-physical systems in manufacturing. CIRP Annals, 65(2), 621-641. https://doi.org/10.1016/j.cirp.2016.06.005

13.

Tao, F., Cheng, J., Qi, Q., Zhang, M., Zhang, H., & Sui, F. (2018). Digital twin-driven product design, manufacturing, and service with big data. The International Journal of Advanced Manufacturing Technology, 94(9-12), 3563-3576. https://doi.org/10.1007/s00170-017-0233-1

14.

Wuest, T., Weimer, D., Irgens, C., & Thoben, K. D. (2016). Machine learning in manufacturing: advantages, challenges, and applications. Production & Manufacturing Research, 4(1), 23-45. https://doi.org/10.1080/21693277.2016.1192517

15.

Yin, S., Ding, S. X., Xie, X., & Luo, H. (2014). A review on basic data-driven approaches for industrial process monitoring. IEEE Transactions on Industrial Electronics, 61(11), 6418-6428. https://doi.org/10.1109/TIE.2014.2301773

16.

Zheng, P., Wang, H., Sang, Z., Zhong, R. Y., Liu, Y., Liu, C., ... & Mubarok, K. (2018). Smart manufacturing systems for Industry 4.0: Conceptual framework, scenarios, and future perspectives. Frontiers of Mechanical Engineering, 13(2), 137-150. https://doi.org/10.1007/s11465-018-0499-5

17.

Zhong, R. Y., Xu, X., Klotz, E., & Newman, S. T. (2017). Intelligent manufacturing in the context of industry 4.0: a review. Engineering, 3(5), 616-630. https://doi.org/10.1016/J.ENG.2017.05.015

18.

Shao, Y., Yuan, X., Zhang, C., Song, Y., & Xu, Q. (2020). A Novel Fault Diagnosis Algorithm for Rolling Bearings Based on One-Dimensional Convolutional Neural Network and INPSO-SVM. Applied Sciences, 10(12), 4303. https://doi.org/10.3390/app10124303

19.

Qiu, S., Cui, X., Ping, Z., Shan, N., Li, Z., Bao, X., & Xu, X. (2023). Deep Learning Techniques in Intelligent Fault Diagnosis and Prognosis for Industrial Systems: A Review. Sensors, 23(3), 1305. https://doi.org/10.3390/s23031305

PUBLISHER'S NOTE

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Copyright © 2025 The Author(s). Published by Southern United Academy of Sciences.
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.