Lightweight 2D-1D CNN for Real-Time Driver Drowsiness Detection withFederated Learning on Edge Devices
Abstract
Driver drowsiness is one of the main leading causes of traffic crash accidents. Preventing these mishapsand limiting consequential damage and losses, whether human or material, has been the subject of manyresearch studies these recent years. Fulfilling this task requires a reliable drowsiness detection systeminstalled in the vehicle that can function on limited hardware such as edge devices with real-time executionspeed. The present study proposes a computer vision-based driver drowsiness detection system usingConvolutional Neural Networks. The traditional approach is to extract some features from the captureddriver’s face images such as PERCLOS, blink speed, or blink frequency, and determine the driver state usingmachine learning methods, predefined thresholds, or rules. However, deep learning techniques allowus to directly identify the driver state from the captured video, yet these techniques are usually computationallyintensive and unsuitable for edge computing applications. To overcome these obstacles, we proposea lightweight 2D-1D CNN model for driver drowsiness detection. We deployed a 2D-CNN model for theeyes feature extraction of both the left and right eyes of the driver from every captured frame. In addition,we applied a 1D-CNN model to capture the temporal information of the concatenated features overa defined period. We implemented and evaluated the model’s performance under both centralized learningand federated learning. Experimental results show that our approach achieves 99% accuracy on theDROZY dataset and 98.06% on subjects excluded from the training set in the CL, while in the FL it achieves95.27% on the same dataset and 77.86% on unseen subjects. Furthermore, the model runs in real time onan NVIDIA Jetson Nano, reaching 25.6 FPS. While FL offers enhanced privacy by enabling local modeltraining without sharing raw user data, it exhibits limited generalization capability. In contrast, CL provideshigher accuracy and a more robust model. Comparative analysis reveals a performance trade-off forFL, balanced by improved data privacy. This work highlights the potential of privacy-preserving, real-timedrowsiness detection systems using advanced deep learning architectures.References
World Health Organization. Road traffic injuries. https://www.who.int/news-room/fact-sheets/detail/road-traffic-injuries. Last accessed 2025-
-13.
National Center for Statistics and Analysis. Drowsy driving 2015 (crash.stats brief statistical summary. Report no. dot hs 812 446), National Highway Traffic Safety Administration, Washington, DC, 10 2017. E Grandjean. Fatigue in industry. Occupational and Environmental Medicine, 36(3):175–186, 1979.
David F Dinges. An overview of sleepiness and accidents. Journal of sleep research, 4:4–14, 1995.
Amina Guettas, Soheyb Ayad, and Okba Kazar. Driver state monitoring system: A review. Proceedings of the 4th International Conference on Big Data and Internet of Things, 2019.
Rustem Dautov and Erik Johannes Husom. Drowsiness detection using federated learning: lessons learnt from dealing with non-iid data. pages 285–292, 2024.
Jianfeng Hu and Jianliang Min. Automated detection of driver fatigue based on eeg signals using gradient boosting decision tree model. Cognitive neurodynamics, 12(4):431–440, 2018.
Ali Amer Hayawi and Jumana Waleed. Driver’s drowsiness monitoring and alarming auto-system based on eog signals. In 2019 2nd International
Conference on Engineering Technology and its Applications (IICETA), pages 214–218. IEEE, 2019.
Mohammad Mahmoodi and Ali Nahvi. Driver drowsiness detection based on classification of surface electromyography features in a driving simulator. Proceedings of the Institution of Mechanical Engineers, Part H: Journal of Engineering in Medicine, 233(4):395–406, 2019.
Rahul Bhardwaj, Priya Natrajan, and Venkatesh Balasubramanian. Study to determine the effectiveness of deep learning classifiers for ecg based driver fatigue classification. In 2018 IEEE 13th International Conference on Industrial and Information Systems (ICIIS), pages 98–102. IEEE, 2018.
Muhammad Awais, Nasreen Badruddin, and Micheal Drieberg. A hybrid approach to detect driver drowsiness utilizing physiological signals to improve system performance and wearability. Sensors, 17(9):1991, 2017.
Sazali Yaacob, Nur Afrina Izzati Affandi, Pranesh Krishnan, Amir Rasyadan, Muhyi Yaakop, and Firdaus Mohamed. Drowsiness detection using eeg and ecg signals. In 2020 IEEE 2nd International Conference on Artificial Intelligence in Engineering and Technology (IICAIET), pages 1–5. IEEE, 2020.
Sagara Sumathipala, Danodya Weerasinghe, Shashika Eranda, Thanuja Gunasekara, et al. Drowsiness detection and alert system for motorcyclist
safety. In 2020 20th International Conference on Advances in ICT for Emerging Regions (ICTer), pages 101–106. IEEE, 2020.
Sadegh Arefnezhad, Sajjad Samiee, Arno Eichberger, Matthias Fr¨uhwirth, Clemens Kaufmann, and Emma Klotz. Applying deep neural networks for multi-level classification of driver drowsiness using vehicle-based measures. Expert Systems with Applications, 162:113778, 2020.
Sadegh Arefnezhad, Sajjad Samiee, Arno Eichberger, and Ali Nahvi. Driver drowsiness detection based on steering wheel data applying adaptive neuro-fuzzy feature selection. Sensors, 19(4):943, 2019.
Mohit Dua, Ritu Singla, Saumya Raj, Arti Jangra, et al. Deep cnn models-based ensemble approach to driver drowsiness detection. Neural Computing and Applications, 33(8):3155–3168, 2021.
Suhandi Junaedi and Habibullah Akbar. Driver drowsiness detection based on face feature and perclos. In Journal of Physics: Conference Series, volume 1090, page 012037. IOP Publishing, 2018.
R Roopalakshmi, Jayantkumar A Rathod, Ashwitha S Shetty, K Supriya, et al. Driver drowsiness detection system based on visual features. In 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT), pages 1344–1347. IEEE, 2018.
Caio Bezerra Souto Maior, M´arcio Jose das Chagas Moura, Joao Mateus Marques Santana, and Isis Didier Lins. Real-time classification for autonomous drowsiness detection using eye aspect ratio. Expert Systems with Applications, 158:113505, 2020.
Heming Yao, Wei Zhang, Rajesh Malhan, Jonathan Gryak, and Kayvan Najarian. Filterpruned 3d convolutional neural network for drowsiness detection. In 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 1258–1262. IEEE, 2018.
Shuang Chen, Zengcai Wang, and Wenxin Chen. Driver drowsiness estimation based on factorized bilinear feature fusion and a long-short-term recurrent convolutional network. Information, 12(1):3, 2021.
Elena Magan, M Paz Sesmero, Juan Manuel Alonso-Weber, and Araceli Sanchis. Driver Drowsiness Detection by Applying Deep Learning Techniques to Sequences of Images. Applied Sciences, 12(3):1145, 2022.
Xiaofeng Li, Jiahao Xia, Libo Cao, Guanjun Zhang, and Xiexing Feng. Driver fatigue detection based on convolutional neural network and face alignment for edge computing device. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 235(10-11):2699–2711, 2021.
Hu He, Xiaoyong Zhang, Fu Jiang, Chenglong Wang, Yingze Yang, Weirong Liu, and Jun Peng. A real-time driver fatigue detection method based on two-stage convolutional neural network. IFACPapersOnLine, 53(2):15374–15379, 2020.
Shivam Khare, Sandeep Palakkal, TV Hari Krishnan, Chanwon Seo, Yehoon Kim, Sojung Yun, and Sankaranarayanan Parameswaran. Realtime driver drowsiness detection using deep learning and heterogeneous computing on embedded system. In International Conference on Computer Vision and Image Processing, pages 86–97. Springer, 2019.
William Lindskog, Valentin Spannagl, and Christian Prehofer. Federated learning for drowsiness detection in connected vehicles. pages 165–178, 2023.
Tran Anh Khoa, Nguyen Dang Trac, Vo Phuc Tinh, Nguyen Hoang Nam, Duc Ngoc Minh Dang, Hoang Hai Son, and Pham Duc Lam. Safety is our friend: a federated learning framework toward driver’s state and behavior detection. IEEE Transactions on Computational Social Systems, 11:2340–2358, 2023.
Jakub Konecny, H Brendan McMahan, Daniel Ramage, and Peter Richt´arik. Federated optimization: Distributed machine learning for on-device intelligence. arXiv preprint arXiv:1610.02527, 2016.
H Brendan McMahan, Eider Moore, Daniel Ramage, and Blaise Ag¨uera y Arcas. Federated learning of deep networks using model averaging. arXiv preprint arXiv:1602.05629, 2:15–18, 2016.
Jakub Konecny, H Brendan McMahan, Felix X Yu, Peter Richt´arik, Ananda Theertha Suresh, and Dave Bacon. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492, 2016.
Y Supriya and Thippa Reddy Gadekallu. A survey on soft computing techniques for federated learning-applications, challenges and future directions. ACM Journal of Data and Information Quality, 15:1–28, 2023.
Atiqa Zafar, Christian Prehofer, and Chih-Hong Cheng. Federated learning for driver status monitoring. pages 1463–1469, 2021.
Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. Federated learning: Challenges, methods, and future directions. IEEE signal processing magazine, 37:50–60, 2020.
Davis E King. Dlib-ml: A machine learning toolkit. The Journal of Machine Learning Research, 10:1755–1758, 2009.
Paul Viola and Michael Jones. Rapid object detection using a boosted cascade of simple features. In Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, volume 1, pages I—-I. IEEE, 2001.
Martin Danelljan, Gustav H¨ager, Fahad Khan, and Michael Felsberg. Accurate scale estimation for robust visual tracking. In British Machine Vision Conference, Nottingham, September 1-5, 2014. Bmva Press, 2014.
G Gamage, I Sudasingha, I Perera, and D Meedeniya. Reinstating dlib correlation human trackers under occlusions in human detection based tracking. In 2018 18th International Conference on Advances in ICT for Emerging Regions (ICTer), pages 92–98. IEEE, 2018.
Christos Sagonas, Georgios Tzimiropoulos, Stefanos Zafeiriou, and Maja Pantic. 300 faces in-the-wild challenge: The first facial landmark localization challenge. In Proceedings of the IEEE international conference on computer vision workshops, pages 397–403, 2013.
Vahid Kazemi and Josephine Sullivan. One millisecond face alignment with an ensemble of regression trees. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1867–1874, 2014.
Hatem Sindi, Majid Nour, Muhyaddin Rawa, Saban Ozturk, and Kemal Polat. Random fully connected layered 1d cnn for solving the z-bus loss allocation problem. Measurement, 171:108794, 2021.
DOI:
https://doi.org/10.31449/inf.v50i12.12530Downloads
Published
Issue
Section
License
Authors retain copyright in their work. By submitting to and publishing with Informatica, authors grant the publisher (Slovene Society Informatika) the non-exclusive right to publish, reproduce, and distribute the article and to identify itself as the original publisher.
All articles are published under the Creative Commons Attribution license CC BY 3.0. Under this license, others may share and adapt the work for any purpose, provided appropriate credit is given and changes (if any) are indicated.
Authors may deposit and share the submitted version, accepted manuscript, and published version, provided the original publication in Informatica is properly cited.
