Tai Chi Assisted Teaching System Based on Kinect and Unity3D

Zixuan Lu; Weibo Yang

doi:10.54097/867v5315

Authors

Zixuan Lu
Weibo Yang

DOI:

https://doi.org/10.54097/867v5315

Keywords:

Kinect, Unity3D, Joint Comparison, Mediapipe, Tai Chi

Abstract

With the rapid development of computer technology, the deep integration of traditional sports and cutting-edge technologies has gradually become a reality, and motion capture technology is increasingly being applied in sports education. This paper designs and implements an intelligent Tai Chi teaching system based on Kinect motion capture devices and the Unity3D platform, aiming to address the lack of interactivity and feedback in traditional teaching methods. The system captures user poses and compares them in real time with standard Tai Chi movements, offering diverse learning and evaluation features. In the beginner mode, the system provides instructional videos of standard movements, allowing users to learn the essentials through repeated viewing. Users can then compare their movements against the annotated key points in the colored data streams for targeted practice. In the advanced mode, the system introduces a motion scoring algorithm that evaluates the user's movements based on the number of matching key points, helping identify areas for improvement and optimize performance. To achieve these functionalities, the system integrates OpenCV and Mediapipe technologies to extract human key points and joint angles from videos, providing robust data support for algorithm design. This paper overcomes the limitations of traditional mouse-and-keyboard interactions by leveraging Kinect’s skeletal tracking technology and Unity3D’s virtual reality capabilities to create a novel motion-sensing interaction method. This enables users to intuitively control virtual interfaces, enhancing the immersive and engaging nature of the learning process. Experimental results demonstrate that the system can accurately recognize user poses and quantitatively assess learning outcomes. In the future, this system holds significant potential for promoting Tai Chi instruction, enriching sports education models, and contributing to the digital preservation of traditional culture.

Downloads

Download data is not yet available.

References

[1] Wang, Chenchen, Jean Paul Collet, and Joseph Lau. "The effect of Tai Chi on health outcomes in patients with chronic conditions: a systematic review." Archives of internal medicine 164.5 (2004): 493-501.

[2] Lan, Ching, et al. "Tai chi chuan in medicine and health promotion." Evidence‐Based Complementary and Alternative Medicine 2013.1 (2013): 502131.

[3] Caldwell, Karen, et al. "Effect of Pilates and taiji quan training on self-efficacy, sleep quality, mood, and physical performance of college students." Journal of bodywork and movement therapies 13.2 (2009): 155-163.

[4] Webster, Craig S., et al. "A systematic review of the health benefits of Tai Chi for students in higher education." Preventive medicine reports 3 (2016): 103-112.

[5] Xu, Rong. "Integrating the philosophy and practice of Tai Ji Quan in French university's curricula: reducing stress and improving well-being of students." (2018).

[6] Heng, Zou, and Pimurai Limpapath. "Intention to Use Innovative Instructional Methods for Teaching Tai Chi in Guangdong Province: Investigating Micro-Lecture and PAD Class." Library Progress International 44.3 (2024): 28033-28043.

[7] Wang, Zhen-Dong, and Feng-Yan Wang. "Ternary Taiji models of the traditional Chinese self: centered on Confucian, Taoist, and Buddhist cultures." Journal of Humanistic Psychology (2021): 00221678211016957.

[8] Menolotto, Matteo, et al. "Motion capture technology in industrial applications: A systematic review." Sensors 20.19 (2020): 5687.

[9] Lekova, Anna, D. Ryan, and Reggie Davidrajuh. "Fingers and gesture recognition with kinect v2 sensor." Information Technologies and Control 14.3 (2016): 24-30.

[10] Jawaid, Iqra, and Jawad Khalid Qureshi. "Advancements in medical imaging through Kinect: a review." 2017 International Symposium on Wireless Systems and Networks (ISWSN). IEEE, 2017.

[11] Lee, Jiann-Der, Chung-Hung Hsieh, and Ting-Yang Lin. "A Kinect-based Tai Chi exercises evaluation system for physical rehabilitation." 2014 IEEE International Conference on Consumer Electronics (ICCE). IEEE, 2014.

[12] He, Tianyu, et al. "Immersive and collaborative Taichi motion learning in various VR environments." 2017 IEEE Virtual Reality (VR). IEEE, 2017.

[13] Cao, Shufang, and Xiaozhou Huang. "The 24-Type Tai Chi Classic Motion System Based on Kinect Motion Capture in Mobile Edge Computing." IEEE Access (2024).

[14] Pang, Xun, and Bin Liang. "Motion recognition based on Kinect for human-computer intelligent interaction." Journal of Physics: Conference Series. Vol. 1187. No. 3. IOP Publishing, 2019.

[15] Shao, Ling, et al. "Computer vision for RGB-D sensors: Kinect and its applications [special issue intro.]." IEEE transactions on cybernetics 43.5 (2013): 1314-1317.

[16] Webb, Jarrett, and James Ashley. Beginning kinect programming with the microsoft kinect SDK. Apress, 2012.

[17] Wenlu Yang, Ying Peng, Hong Xie. Action Recognition Based on Kinect Deep Learning [J]. Journal of Electronics and Information Science, 2022, 7(1).

[18] Wang, Sa, et al. "A new method of virtual reality based on Unity3D." 2010 18th international conference on Geoinformatics. IEEE, 2010.

[19] Ren, Ran, et al. "Vehicle crash simulations for safety: Introduction of connected and automated vehicles on the roadways." Accident Analysis & Prevention 186 (2023): 107021.

[20] Papakostas, Christos, et al. "Exploration of augmented reality in spatial abilities training: a systematic literature review for the last decade." Informatics in Education 20.1 (2021): 107-130.

[21] Luckykumar Dwarkadas, Agrawal, et al. "A review of the application of virtual and augmented reality in physical and occupational therapy." Software: Practice and Experience 54.8 (2024): 1378-1407.

[22] Kaur, Deepti Prit, Narinder Pal Singh, and Bonny Banerjee. "A review of platforms for simulating embodied agents in 3D virtual environments." Artificial Intelligence Review 56.4 (2023): 3711-3753.

[23] Chauhan, Rahul, Ikshit Dhyani, and Himadri Vaidya. "A Review on Human Pose Estimation Using Mediapipe." 2023 3rd International Conference on Innovative Sustainable Computational Technologies (CISCT). IEEE, 2023.

[24] Harris, Moh, and Ali Suryaperdana Agoes. "Applying hand gesture recognition for user guide application using MediaPipe." 2nd International Seminar of Science and Applied Technology (ISSAT 2021). Atlantis Press, 2021.

[25] Marengoni, Maurício, and Stringhini Stringhini. "Tutorial: Introdução à visão computacional usando opencv." Revista de Informática Teórica e Aplicada 16.1 (2009): 125-160.

[26] Kavitha, D., et al. "Multiple Object Recognition Using OpenCV." REVISTA GEINTEC-GESTAO INOVACAO E TECNOLOGIAS 11.2 (2021): 1736-1747.