Sign Language Recognition and Application Based on Graph Neural Networks: Innovative Integration in TV News Sign Language

Peilai Yu

doi:10.70393/6a6374616d.323636

OPEN ACCESS |Research Article ||1 March 2025

Sign Language Recognition and Application Based on Graph Neural Networks: Innovative Integration in TV News Sign Language

Peilai Yu

LMU Ludwig Maximilian University of Munich

peilai.yu@campus.lmu.de

LMU Ludwig Maximilian University of Munich, Germany.

* Corresponding Author¹: Peilai Yu, E-Mail: peilai.yu@campus.lmu.de

Publication

Accepted 2025 March 3 ; Published 2025 March 1

Journal of Computer Technology and Applied Mathematics, 2025, 2(2), 3007-4126.

Abstract

With the rapid development of information technology, sign language recognition plays an extremely important role in the communication among people with hearing impairments. Especially in the context of television news, the real-time and accuracy of sign language translation are very important. Traditional sign language translation technology faces challenges such as low accuracy of gesture recognition and poor real-time performance, which makes it difficult to meet the translation needs of daily complex news content. This paper proposes a sign language recognition method based on graph neural network (GNN). By constructing a graph structure of gesture nodes and joint connections, GNN can capture the relationship between gestures and efficiently transfer learning information. Through comparative experiments with traditional convolutional neural networks (CNN), the advantages of GNN in sign language recognition are proved, especially in the application of news broadcasting, which significantly improves the real-time and accuracy of sign language translation. Future research will focus on optimizing the generalization ability of the model and broadening its applicability to more languages and scenarios.

Keywords

Graph Neural Networks (GNN) , Sign Language Recognition , Television News , Real-time Translation , Automation .

Metadata

DOI:

10.70393/6a6374616d.323636

ARK:

ark:/40704/JCTAM.v2n2a02

Pages: 11-15

References: 28

Disciplines: Artificial Intelligence and Intelligence

Subjects: Machine Learning

Cite This Article

APA Style

Yu, P. (2025). Sign language recognition and application based on graph neural networks: innovative integration in tv news sign language. Journal of Computer Technology and Applied Mathematics, 2(2), 11-15. https://doi.org/10.70393/6a6374616d.323636

Acknowledgments

The authors thank the editor and anonymous reviewers for their helpful comments and valuable suggestions.

FUNDING

Not applicable.

INSTITUTIONAL REVIEW BOARD STATEMENT

Not applicable.

DATA AVAILABILITY STATEMENT

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

INFORMED CONSENT STATEMENT

Not applicable.

CONFLICT OF INTEREST

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

AUTHOR CONTRIBUTIONS

Not applicable.

References

1.

Dreuw, P., Rybach, D., Deselaers, T., Zahedi, M., & Ney, H. (2007). Speech recognition techniques for a sign language recognition system. hand, 60(80).

2.

Ma, Y., Zhou, G., Wang, S., Zhao, H., & Jung, W. (2018). SignFi: Sign language recognition using WiFi. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(1), 1-21.

3.

Chai, X., Li, G., Lin, Y., Xu, Z., Tang, Y., Chen, X., & Zhou, M. (2013, April). Sign language recognition and translation with kinect. In IEEE conf. on AFGR (Vol. 655, p. 4).

4.

Lyu, S. (2024). The Application of Generative AI in Virtual Reality and Augmented Reality. Journal of Industrial Engineering and Applied Science, 2(6), 1-9.

5.

Rastgoo, R., Kiani, K., & Escalera, S. (2021). Sign language recognition: A deep survey. Expert Systems with Applications, 164, 113794.

6.

Lyu, S. (2024). The Technology of Face Synthesis and Editing Based on Generative Models. Journal of Computer Technology and Applied Mathematics, 1(4), 21-27.

7.

Zaki, M. M., & Shaheen, S. I. (2011). Sign language recognition using a combination of new vision based features. Pattern Recognition Letters, 32(4), 572-577.

8.

Lyu, S. (2024). Machine Vision-Based Automatic Detection for Electromechanical Equipment. Journal of Computer Technology and Applied Mathematics, 1(4), 12-20.

9.

Rao, G. A., Syamala, K., Kishore, P. V. V., & Sastry, A. S. C. S. (2018, January). Deep convolutional neural networks for sign language recognition. In 2018 conference on signal processing and communication engineering systems (SPACES) (pp. 194-197). IEEE.

10.

Lin, W. (2024). A Review of Multimodal Interaction Technologies in Virtual Meetings. Journal of Computer Technology and Applied Mathematics, 1(4), 60-68.

11.

Pigou, L., Dieleman, S., Kindermans, P. J., & Schrauwen, B. (2015). Sign language recognition using convolutional neural networks. In Computer Vision-ECCV 2014 Workshops: Zurich, Switzerland, September 6-7 and 12, 2014, Proceedings, Part I 13 (pp. 572-578). Springer International Publishing.

12.

Lin, W. (2024). A Systematic Review of Computer Vision-Based Virtual Conference Assistants and Gesture Recognition. Journal of Computer Technology and Applied Mathematics, 1(4), 28-35.

13.

Wadhawan, A., & Kumar, P. (2021). Sign language recognition systems: A decade systematic literature review. Archives of computational methods in engineering, 28, 785-813.

14.

Luo, M., Zhang, W., Song, T., Li, K., Zhu, H., Du, B., & Wen, H. (2021, January). Rebalancing expanding EV sharing systems with deep reinforcement learning. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence (pp. 1338-1344).

15.

Sun, Y., & Ortiz, J. (2024). Data Fusion and Optimization Techniques for Enhancing Autonomous Vehicle Performance in Smart Cities. Journal of Artificial Intelligence and Information, 1, 42-50.

16.

Luo, M., Du, B., Zhang, W., Song, T., Li, K., Zhu, H., ... & Wen, H. (2023). Fleet rebalancing for expanding shared e-Mobility systems: A multi-agent deep reinforcement learning approach. IEEE Transactions on Intelligent Transportation Systems, 24(4), 3868-3881.

17.

Zhu, H., Luo, Y., Liu, Q., Fan, H., Song, T., Yu, C. W., & Du, B. (2019). Multistep flow prediction on car-sharing systems: A multi-graph convolutional neural network with attention mechanism. International Journal of Software Engineering and Knowledge Engineering, 29(11n12), 1727–1740.

18.

Li, K., Chen, X., Song, T., Zhang, H., Zhang, W., & Shan, Q. (2024). GPTDrawer: Enhancing Visual Synthesis through ChatGPT. arXiv preprint arXiv:2412.10429.

19.

Kumar, A., Thankachan, K., & Dominic, M. M. (2016, March). Sign language recognition. In 2016 3rd international conference on recent advances in information technology (RAIT) (pp. 422-428). IEEE.

20.

Von Agris, U., Zieren, J., Canzler, U., Bauer, B., & Kraiss, K. F. (2008). Recent developments in visual sign language recognition. Universal Access in the Information Society, 6, 323-362.

21.

Xu, Y., Lin, Y. S., Zhou, X., & Shan, X. (2024). Utilizing emotion recognition technology to enhance user experience in real-time. Computing and Artificial Intelligence, 2(1), 1388-1388.

22.

Cooper, H. M., Ong, E. J., Pugeault, N., & Bowden, R. (2012). Sign language recognition using sub-units. Journal of Machine Learning Research, 13, 2205-2231.

23.

Li, K., Liu, L., Chen, J., Yu, D., Zhou, X., Li, M., ... & Li, Z. (2024, November). Research on reinforcement learning based warehouse robot navigation algorithm in complex warehouse layout. In 2024 6th International Conference on Artificial Intelligence and Computer Applications (ICAICA) (pp. 296-301). IEEE.

24.

Bragg, D., Koller, O., Bellard, M., Berke, L., Boudreault, P., Braffort, A., ... & Ringel Morris, M. (2019, October). Sign language recognition, generation, and translation: An interdisciplinary perspective. In Proceedings of the 21st international ACM SIGACCESS conference on computers and accessibility (pp. 16-31).

25.

Li, K., Chen, J., Yu, D., Dajun, T., Qiu, X., Lian, J., ... & Han, J. (2024, October). Deep reinforcement learning-based obstacle avoidance for robot movement in warehouse environments. In 2024 IEEE 6th International Conference on Civil Aviation Safety and Information Technology (ICCASIT) (pp. 342-348). IEEE.

26.

Huang, X., Wu, Y., Zhang, D., Hu, J., & Long, Y. (2024, September). Improving Academic Skills Assessment with NLP and Ensemble Learning. In 2024 IEEE 7th International Conference on Information Systems and Computer Aided Education (ICISCAE) (pp. 37-41). IEEE.

27.

Cooper, H., Holt, B., & Bowden, R. (2011). Sign language recognition. In Visual Analysis of Humans: Looking at People (pp. 539-562). London: Springer London.

28.

Wu, S., & Huang, X. (2025). Psychological Health Prediction Based on the Fusion of Structured and Unstructured Data in EHR: a Case Study of Low-Income Populations. Preprints.

PUBLISHER'S NOTE

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Copyright © 2025 The Author(s). Published by Southern United Academy of Sciences.
This work is licensed under a Creative Commons Attribution 4.0 International License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.