Abstract

This paper presents the novel work of developing an intelligent manufacturing framework based on multimodal sensor integration and computer vision. In this paper, we propose a hybrid fusion method that includes both early and late fusion with attention mechanisms to select the most important sensor data. Our system gathers visual, thermal, acoustic, and vibration data and offers accurate and interpretable predictions for fault identification, process enhancement, and product quality. (Liu et al. 2024) We meet the challenge of the opacity of AI systems by using explainable AI methods to help the user comprehend the results of the model. It shows that the proposed system is accurate, efficient, scalable, and can be applied to various types of data. Examples from the industry present real-life experiences and issues that may be encountered when implementing our system in different manufacturing contexts. It presents a new paradigm shift in smart manufacturing systems through the enhancement of efficiency, reliability, and interpretability for future research and industrial development.

Keywords

Multimodal Sensor FusionSmart ManufacturingExplainable AI (XAI)Industry 4.0Predictive MaintenanceFault DetectionAttention MechanismsDeep LearningHybrid FusionRandom Forest ClassifierData IntegrationMachine LearningFeature ExtractionReal-Time ProcessingCondition Monitoring

Publication

Accepted Unknow ; Published 2024 November 2

Journal of Computer Technology and Applied Mathematics, 2024, 1(4), 3007-4126

View PDF Download PDF

Metadata

DOI

10.5281/zenodo.13905495

ARK

ark:/40704/JCTAM.v1n4a05

Pages

36–46

References

19

Disciplines

Computer Science

Subjects

Deep Learning

Cite This Article

Acknowledgments

The authors thank the editor and anonymous reviewers for their helpful comments and valuable suggestions.

Funding

Not applicable.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Informed Consent Statement

Not applicable.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Author Contributions

Not applicable.

References

1. Liu, Z., Xiao, G., Liu, H., & Wei, H. (2024). Multi-Sensor Measurement and Data Fusion. IEEE Transactions on Instrumentation and Measurement, 73, 234-245. https://ieeexplore.ieee.org/abstract/document/9693406?casa_token=40VH55MYDtEAAAAA:5M5RuueqkCua6ONFCVjNeNl8V0TlZ4-giZ8VQzK0U6CXh_TefMxhQaCBotzrZXsBrmaDEjnq
2. Diez-Olivan, A., Del Ser, J., Galar, D., & Sierra, B. (2019). Data Fusion and Machine Learning for Industrial Prognosis: Trends and Perspectives Towards Industry 4.0. Journal of Manufacturing Processes, 40, 123-134. https://www.sciencedirect.com/science/article/pii/S1566253518304706?casa_token=Bi4TM627bvIAAAAA:FIhyFRBhR4K3o7_Ly5thLLoHChk7UYVPCZTvMDEK5aPxTYUJQuAPOI_Dn3ZHRIRz1TUbIaQ_
3. Helwig, Nikolai, Pignanelli, Eliseo, and Schtze, Andreas. (2018). Condition monitoring of hydraulic systems. UCI Machine Learning Repository. https://doi.org/10.24432/C5CW21.
4. Vergara, Alexander. (2013). Gas Sensor Array Drift at Different Concentrations. UCI Machine Learning Repository. https://doi.org/10.24432/C5MK6M.
5. Buscema, M., Terzi, S., & Tastle, W. (2010). Steel Plates Faults. UCI Machine Learning Repository. https://doi.org/10.24432/C5J88N
6. Zhang, Z., Farnsworth, M., Song, B., Tiwari, D., & Tiwari, A. (2024). Deep Transfer Learning With Self-Attention for Industry Sensor Fusion Tasks. IEEE Transactions on Industrial Informatics. https://ieeexplore.ieee.org/abstract/document/9813440
7. Sensor Data Fusion in Industry 4.0. (2024). Farnell. https://si.farnell.com/sensor-data-fusion-in-industry-40-trc-ar
8. Goyal, D., & Pabla, B. S. (2015). Condition-based maintenance of machine tools—A review. CIRP Journal of Manufacturing Science and Technology, 10, 24-35. https://doi.org/10.1016/j.cirpj.2015.05.002
9. Kang, H. S., Lee, J. Y., Choi, S., Kim, H., Park, J. H., Son, J. Y., ... & Do Noh, S. (2016). Smart manufacturing: Past research, present findings, and future directions. International Journal of Precision Engineering and Manufacturing-Green Technology, 3(1), 111-128. https://doi.org/10.1007/s40684-016-0015-5
10. Kusiak, A. (2019). Fundamentals of smart manufacturing: A multi-thread perspective. Annual Reviews in Control, 47, 214-220. https://doi.org/10.1016/j.arcontrol.2019.02.001
11. Liu, Y., Li, S., Liu, C., & Wang, H. (2021). Explainable AI for intelligent manufacturing: A survey. AI Open, 2(1), 105-123. https://doi.org/10.1002/aiopen.76
12. Monostori, L., Kádár, B., Bauernhansl, T., Kondoh, S., Kumara, S., Reinhart, G., ... & Ueda, K. (2016). Cyber-physical systems in manufacturing. CIRP Annals, 65(2), 621-641. https://doi.org/10.1016/j.cirp.2016.06.005
13. Tao, F., Cheng, J., Qi, Q., Zhang, M., Zhang, H., & Sui, F. (2018). Digital twin-driven product design, manufacturing, and service with big data. The International Journal of Advanced Manufacturing Technology, 94(9-12), 3563-3576. https://doi.org/10.1007/s00170-017-0233-1
14. Wuest, T., Weimer, D., Irgens, C., & Thoben, K. D. (2016). Machine learning in manufacturing: advantages, challenges, and applications. Production & Manufacturing Research, 4(1), 23-45. https://doi.org/10.1080/21693277.2016.1192517
15. Yin, S., Ding, S. X., Xie, X., & Luo, H. (2014). A review on basic data-driven approaches for industrial process monitoring. IEEE Transactions on Industrial Electronics, 61(11), 6418-6428. https://doi.org/10.1109/TIE.2014.2301773
16. Zheng, P., Wang, H., Sang, Z., Zhong, R. Y., Liu, Y., Liu, C., ... & Mubarok, K. (2018). Smart manufacturing systems for Industry 4.0: Conceptual framework, scenarios, and future perspectives. Frontiers of Mechanical Engineering, 13(2), 137-150. https://doi.org/10.1007/s11465-018-0499-5
17. Zhong, R. Y., Xu, X., Klotz, E., & Newman, S. T. (2017). Intelligent manufacturing in the context of industry 4.0: a review. Engineering, 3(5), 616-630. https://doi.org/10.1016/J.ENG.2017.05.015
18. Shao, Y., Yuan, X., Zhang, C., Song, Y., & Xu, Q. (2020). A Novel Fault Diagnosis Algorithm for Rolling Bearings Based on One-Dimensional Convolutional Neural Network and INPSO-SVM. Applied Sciences, 10(12), 4303. https://doi.org/10.3390/app10124303
19. Qiu, S., Cui, X., Ping, Z., Shan, N., Li, Z., Bao, X., & Xu, X. (2023). Deep Learning Techniques in Intelligent Fault Diagnosis and Prognosis for Industrial Systems: A Review. Sensors, 23(3), 1305. https://doi.org/10.3390/s23031305

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Copyright © 2024 The Author(s). Published by Southern United Academy of Sciences.
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.