Research on Sign Language Recognition for Hearing-Impaired People Through the Improved YOLOv5 Algorithm Combining CBAM with Focal CioU

Abstract

Sign language recognition has become increasingly important as the number of hearing-impaired people increases. This paper optimized the you only look once version 5 (YOLOv5) algorithm from perspectives of attention mechanism and loss function. The convolutional block attention module (CBAM) was added to the network, and the original intersection over union (IoU) loss function was improved to focal complete IoU (CIoU). Experimental analyses were performed on the American Sign Language (ASL) dataset in the Windows 10 environment. Moreover, the ten-fold cross-validation was used. The experiments found that adding the CBAM to the neck part of YOLOv5 showed the most effective sign language recognition results. The improved algorithm showed improvements of 0.95% in P value, 4.19% in R value, and 2.66% in mean average precision (mAP) compared to the baseline algorithm. When comparing different loss functions, the focal CIoU performed the best. Compared with other recognition algorithms, the improved YOLOv5 algorithm performed better in sign language recognition, achieving P value, R value, and mAP of 93.26%, 96.77%, and 98.12%, respectively. These results verify the reliability of the improved YOLOv5 algorithm in sign language recognition for hearing-impaired people. It can be applied in practice.