Research on English Listening Comprehension Model Design Based on Transformer-ResNet Hybrid Model

Abstract

With the increasing complexity of English listening comprehension tasks, the traditional single acoustic model has made it difficult to cope with the high noise interference and multi-level semantic understanding requirements in complex speech environments. Based on the research on the design of the English listening comprehension model based on the Transformer-ResNet hybrid model, an innovative architecture combining residual convolutional network and self-attention mechanism is proposed, aiming to improve the model's performance in long-term dependency modeling and local acoustic pattern recognition. A parallel dual-stream feature extraction architecture is designed, using ResNet to extract fine-grained acoustic features and the Transformer self-attention mechanism to capture long-term semantic dependencies. In order to solve the alignment problem between phoneme-level and semantic-level features, a cross-layer connection strategy is proposed, and the robustness of the model is improved by multi-scale feature fusion. Due to the limitation of real-time and computing resources, model compression and distillation technology are adopted to optimize computing efficiency, and an efficient end-to-end speech understanding system is realized by combining the pre-trained language model. The optimized hybrid model performed outstandingly on the test set, with an overall accuracy rate of 78.9%, an increase of 10.1 percentage points compared with the baseline model. The Transformer module for long-term dependence modeling contributed a 32% performance gain. At the same time, ResNet's local feature extraction capability enabled the model to maintain a time series consistency score of 66.6 in 44 sets of consecutive speech frame processing. It is worth noting that the model can still maintain a low word error rate of 9.8 in 87% of multi-speaker scenarios, indicating its robustness advantage in complex auditory environments. The experimental data verify the effectiveness of the complementary design of Transformer and ResNet in improving English listening comprehension tasks.