Multi-Task BERT–BiLSTM–CNN–MHA Framework for Legal Sentiment Analysis and Judgment Prediction - Liu, Hongqi (Informatica, 2025)
This paper proposes a comprehensive multi-task learning (MTL) framework that integrates BERT, Bidirectional Long Short-Term Memory (BiLSTM), Convolutional Neural Network (CNN), and Multi-Head Attention (MHA) to jointly perform legal text sentiment analysis (SA) and predict judicial judgments. The architecture utilizes a shared BERT–BiLSTM encoder to generate contextualized representations, which are then extracted by a CNN module to capture fine-grained sentiment features. Meanwhile, an MHA module fuses information and guides the multi-task output for decision-related subtasks. To ensure effective joint optimization, a weighted loss function is designed to balance task-specific objectives and prevent dominance by any single task. Experiments are conducted on the public CAIL2018 dataset using ten-fold cross-validation to guarantee fairness and reproducibility. In this framework, BERT first encodes legal documents into deep semantic embeddings, which BiLSTM then processes to capture sequential dependencies. The CNN subnetwork extracts localized emotional features from the BiLSTM outputs, achieving 98.0% accuracy and an F1 score of 0.96 on sentiment classification (CAIL-big subset). Simultaneously, the MHA module leverages the shared encoder outputs to highlight and weight relevant legal clauses, supporting downstream tasks such as legal article recommendation and sentencing estimation. The recommendation task achieves 94.0% top-1 accuracy, and sentencing regression achieves a mean squared error (MSE) of 0.028. Compared with traditional baselines such as Word2Vec–BiLSTM and single-task CNN models, the proposed BERT–BiLSTM–CNN–MHA framework delivers over 5% improvement in both sentiment analysis and judgment prediction tasks. Its modular design, deep semantic representation, and robust empirical results validate its effectiveness and practical value for deployment in intelligent legal decision-support and judicial assistance systems.