Edge-Assisted CNN-Attention Model for Real-Time Multimodal Learner State Recognition in IoT-Enhanced English Language Learning Systems - Tong, Fangyin (Informatica, 2025)
major challenge in computer-assisted English learning lies in accurately perceiving learners’ cognitive and affective states, which limits adaptive feedback and personalized scaffolding. This study proposes an edge-assisted multimodal perception framework that integrates convolutional neural networks with an additive attention mechanism for real-time learner state recognition in IoT-enhanced English classrooms.Multimodal signals, including speech features, facial expressions, gaze, and body posture, are collected through IoT-enabled terminals and preprocessed at edge nodes to reduce latency and communication overhead. CNN layers extract local spatiotemporal features, and the attention module dynamically reweights salient behavioral cues for robust fusion. Learner states such as active oral interaction, passive listening, and distraction are classified, and real-time state outputs support personalized intervention and adaptive task assignment.Experiments on classroom recordings from 128 English learners demonstrated promising performance, achieving over 90% accuracy for major state categories and improved task adaptation efficiency. Results indicate that the proposed edge-assisted multimodal architecture enhances perceptive accuracy and responsiveness, offering a viable pathway toward fine-grained learner modeling and intelligent support in future English language learning environments.