Attention-Based Bimodal Neural Network Speech Recognition System on FPGA

Abstract

To further improve the accuracy of speech recognition technology, a neural network speech recognition system based on field programmable gate array was designed. Firstly, a neural network audiovisual bimodal speech recognition algorithm based on attention mechanism was designed. Then, a speech recognition platform based on on-site programmable gate arrays was built. The proposed algorithm is proved to have the lowest word error rate and character error rate of 3.17% and 1.56%, with the fastest convergence speed and lower final loss value. The algorithm converges quickly when the raining rounds are less than 10, and tends to stabilize when it is 20. The proposed speech recognition platform uses many DSP units in its design, with a utilization rate of 83.2%, the lowest power consumption of 2.21W, the highest energy efficiency ratio of 26.15, and the shortest processing time and faster running speed. In summary, the research algorithm can reasonably allocate learning weights, improve training speed, and has certain feasibility and effectiveness because of introducing attention mechanism. It has good application effects in speech recognition, which helps to improve the accuracy of language recognition algorithms and promote communication between humans and machines.