CNN and LSTM-Based Multimodal Data Fusion for Performance Optimization in Aerobics Using Wearable Sensors
Abstract
Aerobics is a high-intensity, multi-dimensional sport. Its motion evaluation places higher demands on data quality and time series modeling capabilities. This paper proposes a method for evaluating aerobics motion that integrates wearable sensors and motion tracking systems. It combines convolutional neural networks (CNNs) with long short-term memory networks (LSTMs) to perform fusion analysis on multimodal data from accelerometers, gyroscopes, magnetometers, and Kinect motion capture systems. To improve data quality, Kalman filtering, time synchronization, and wavelet transform techniques are introduced to preprocess the raw data. Experimental results show that this method performs well in motion classification tasks: in indoor low-intensity training scenarios, the accuracy of the CNN model increases from 74.5% to 87.1%; in high-intensity training scenarios, the accuracy increases from 75.0% to 88.2%. After combining with LSTM, the model further enhances the modeling capabilities of motion time series features and improves the recognition accuracy of complex motions. In different training scenarios, the average improvement rate of motion scores is 25.8%. The system feedback delay is controlled within 200 milliseconds, with good real-time and practical performance. This method provides aerobics athletes with high-precision movement assessment and personalized training suggestions, promoting the intelligent and personalized development of sports training.References
M. E. M. Simbolon, D. K. A. Firdausi, I. Dwisaputra, A. Rusdiana, C. Pebriandani, and R. Prayoga, “Utilization of Sensor technology as a Sport Technology Innovation in Athlete Performance Measurement,” Indonesian Journal of Electronics and Instrumentation Systems (IJEIS), 13(2): 147–158, 2023. https://doi.org/10.22146/ijeis.89581
Z. Mei, “3D images analysis of sports technical features and sports training methods based on artificial intelligence,” J Test Eval, 51(1): 189–200, 2023. https://doi.org/10.1520/JTE20210469
S. A. Kovalchik, “Player tracking data in sports,” Annu Rev Stat Appl, 10(1): 677–697, 2023. https://doi.org/10.1146/annurev-statistics-033021-110117
L. Yang, O. Amin, and B. Shihada, “Intelligent wearable systems: Opportunities and challenges in health and sports,” ACM Comput Surv, 56(7):1–42, 2024. https://doi.org/10.1145/3648469
W. Li, “Application of IoT-enabled computing technology for designing sports technical action characteristic model,” Soft comput, 27(17): 12807–12824, 2023. https://doi.org/10.1007/s00500-023-08966-4
Y. Fang, “Utilizing Wearable Technology to Enhance Training and Performance Monitoring in Indonesian Badminton Players,” Studies in Sports Science and Physical Education, 2(1): 11–23, 2024. DOI:10.1186/s40561-023-00247-9
J. Corban et al., “Using an affordable motion capture system to evaluate the prognostic value of drop vertical jump parameters for noncontact ACL injury,” Am J Sports Med, 51(4):1059–1066, 2023. https://doi.org/10.1177/03635465231151686
C. J. Rigozzi, G. A. Vio, and P. Poronnik, “Application of wearable technologies for player motion analysis in racket sports: A systematic review,” Int J Sports Sci Coach, 18(6): 2321–2346, 2023. https://doi.org/10.1177/17479541221138015
Y. Zhang, “Design of Wireless Motion Sensor Nodes based on the Kalman Filter Algorithm,” Recent Advances in Electrical & Electronic Engineering (Formerly Recent Patents on Electrical & Electronic Engineering), 16(3): 248–255, 2023. https://doi.org/10.2174/2352096515666220908152036
S. Akan and S. Varlı, “Use of deep learning in soccer videos analysis: survey,” Multimed Syst, 29(3): 897–915, 2023. https://doi.org/10.1007/s00530-022-01027-0
D. Gholamiangonabadi and K. Grolinger, “Personalized models for human activity recognition with wearable sensors: deep neural networks and signal processing,” Applied Intelligence, 53(5): 6041–6061, 2023. https://doi.org/10.1007/s10489-022-03832-6
A. Chakraborty and N. Mukherjee, “A deep-CNN based low-cost, multi-modal sensing system for efficient walking activity identification,” Multimed Tools Appl, 82(11): 16741–16766, 2023. https://doi.org/10.1007/s11042-022-13990-x
W. Liu, Y. Liu, and R. Bucknall, “Filtering based multi-sensor data fusion algorithm for a reliable unmanned surface vehicle navigation,” Journal of Marine Engineering & Technology, 22(2): 67–83, 2023. https://doi.org/10.1080/20464177.2022.2031558
L. Zhang and H. Dai, “Motion trajectory tracking of athletes with improved depth information-based KCF tracking method,” Multimed Tools Appl, 82(17): 26481–26493, 2023. https://doi.org/10.1007/s11042-023-14929-6
P. Hao and K. Qian, “The Integration of Personalized Training Program Design and Information Technology for Athletes,” Scalable Computing: Practice and Experience, 25(5): 4351–4359, 2024. https://doi.org/10.12694/scpe.v25i5.3083
V. Deepak, D. K. Anguraj, and S. S. Mantha, “An efficient recommendation system for athletic performance optimization by enriched grey wolf optimization,” Pers Ubiquitous Comput, 27(3): 1015–1026, 2023. https://doi.org/10.1007/s00779-022-01680-2
J. K. Urbanek et al., “Free-living gait cadence measured by wearable accelerometer: a promising alternative to traditional measures of mobility for assessing fall risk,” The Journals of Gerontology: Series A, 78(5): 802–810, 2023. https://doi.org/10.1093/gerona/glac013
A. Hussain, S. Ali, M.-I. Joo, and H.-C. Kim, “A deep learning approach for detecting and classifying cat activity to monitor and improve cat’s well-being using accelerometer, gyroscope, and magnetometer,” IEEE Sens J, 24(2): 1996–2008, 2023.
A. Spilz and M. Munz, “Synchronisation of wearable inertial measurement units based on magnetometer data,” Biomedical Engineering/Biomedizinische Technik, 68(3): 263–273, 2023. https://doi.org/10.1515/bmt-2021-0329
A. Liu, R. P. Mahapatra, and A. V. R. Mayuri, “Hybrid design for sports data visualization using AI and big data analytics,” Complex & Intelligent Systems, 9(3): 2969–2980, 2023. https://doi.org/10.1007/s40747-021-00557-w
C.-T. Lin, Y. Wang, S.-F. Chen, K.-C. Huang, and L.-D. Liao, “Design and verification of a wearable wireless 64-channel high-resolution EEG acquisition system with wi-fi transmission,” Med Biol Eng Comput, 61(11): 3003–3019, 2023. https://doi.org/10.1007/s11517-023-02879-y
X. Shi and H. Zou, “Data Collection and Analysis based on Sensor Technology in Sports Training,” Scalable Computing: Practice and Experience, 25(5): 4399–4406, 2024. https://doi.org/10.12694/scpe.v25i5.3200
M. Khodarahmi and V. Maihami, “A review on Kalman filter models,” Archives of Computational Methods in Engineering, 30(1): 727–747, 2023. https://doi.org/10.1007/s11831-022-09815-7
M. Azhar, S. Ullah, M. Raees, K. U. Rahman, and I. U. Rehman, “A real-time multi view gait-based automatic gender classification system using kinect sensor,” Multimed Tools Appl, 82(8): 11993–12016, 2023. https://doi.org/10.1007/s11042-022-13704-3
L. Lv, J. Yang, F. Gu, J. Fan, Q. Zhu, and X. Liu, “Validity and reliability of a depth camera–based quantitative measurement for joint motion of the hand,” J Hand Surg Glob Online, 5(1): 39–47, 2023. https://doi.org/10.1016/j.jhsg.2022.08.011
Y. Wu, Z. Sun, G. Ran, and L. Xue, “Intermittent control for fixed-time synchronization of coupled networks,” IEEE/CAA Journal of Automatica Sinica, 10(6): 1488–1490, 2023. DOI: 10.1109/JAS.2023.123363
A. Halidou, Y. Mohamadou, A. A. A. Ari, and E. J. G. Zacko, “Review of wavelet denoising algorithms,” Multimed Tools Appl, 82(27): 41539–41569, 2023. https://doi.org/10.1007/s11042-023-15127-0
M. Kang, C. L. Bentley, J. T. Mefford, W. C. Chueh, and P. R. Unwin, “Multiscale Analysis of Electrocatalytic Particle Activities: Linking Nanoscale Measurements and Ensemble Behavior,” ACS Nano, 17(21): 21493–21505, 2023. https://doi.org/10.1021/acsnano.3c06335
J. Sun, H. Zhang, X. Ma, R. Wang, H. Sima, and J. Wang, “Spectral–Spatial Adaptive Weighted Fusion and Residual Dense Network for hyperspectral image classification,” The Egyptian Journal of Remote Sensing and Space Sciences, 28(1): 21–33, 2025. https://doi.org/10.1016/j.ejrs.2024.11.001
J. P. Bharadiya, “A tutorial on principal component analysis for dimensionality reduction in machine learning,” Int J Innov Sci Res Technol, 8(5): 2028–2032, 2023. DOI:10.5281/zenodo.8002436
F. Bizzarri, D. Del Giudice, S. Grillo, D. Linaro, A. Brambilla, and F. Milano, “Inertia estimation through covariance matrix,” IEEE Transactions on Power Systems, 39(1): 947–956, 2023. DOI: 10.1109/TPWRS.2023.3236059
Y. He, C.-K. Zhang, H.-B. Zeng, and M. Wu, “Additional functions of variable-augmented-based free-weighting matrices and application to systems with time-varying delay,” Int J Syst Sci, 54(5): 991–1003, 2023. https://doi.org/10.1080/00207721.2022.2157198
T. Sharma, N. K. Verma, and S. Masood, “Mixed fuzzy pooling in convolutional neural networks for image classification,” Multimed Tools Appl, 82(6): 8405–8421, 2023. https://doi.org/10.1007/s11042-022-13553-0
M. Reyad, A. M. Sarhan, and M. Arafa, “A modified Adam algorithm for deep neural network optimization,” Neural Comput Appl, 35(23): 17095–17112, 2023. https://doi.org/10.1007/s00521-023-08568-z
Z. Mei et al., “Automatic loss function search for adversarial unsupervised domain adaptation,” IEEE Transactions on Circuits and Systems for Video Technology, 33(10): 5868–5881, 2023. DOI: 10.1109/TCSVT.2023.3260246
DOI:
https://doi.org/10.31449/inf.v49i16.9490Downloads
Published
How to Cite
Issue
Section
License
I assign to Informatica, An International Journal of Computing and Informatics ("Journal") the copyright in the manuscript identified above and any additional material (figures, tables, illustrations, software or other information intended for publication) submitted as part of or as a supplement to the manuscript ("Paper") in all forms and media throughout the world, in all languages, for the full term of copyright, effective when and if the article is accepted for publication. This transfer includes the right to reproduce and/or to distribute the Paper to other journals or digital libraries in electronic and online forms and systems.
I understand that I retain the rights to use the pre-prints, off-prints, accepted manuscript and published journal Paper for personal use, scholarly purposes and internal institutional use.
In certain cases, I can ask for retaining the publishing rights of the Paper. The Journal can permit or deny the request for publishing rights, to which I fully agree.
I declare that the submitted Paper is original, has been written by the stated authors and has not been published elsewhere nor is currently being considered for publication by any other journal and will not be submitted for such review while under review by this Journal. The Paper contains no material that violates proprietary rights of any other person or entity. I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in my article. I have informed the co-author(s) of the terms of this publishing agreement.
Copyright © Slovenian Society Informatika
