Predicting Loess Collapsibility Using Grey Relational Analysis and Transformer Networks Zeliang Chen Lanzhou New Area Urban Construction Engineering Co., Ltd., Lanzhou 73000, China

Abstract

Subgrade and pavement diseases caused by loess collapsibility seriously affect the safety and durability of road engineering. Precise prediction of the collapsibility coefficient plays a pivotal role in engineering prevention and control in loess regions. To address the issues where loess collapsibility is influenced by the coupling effect of multiple factors and traditional forecasting approaches lack adequate generalization capability, this study presents a predictive model integrating grey relational analysis and Transformer network, based on the NCE7# Highway project in Lanzhou New Area. Firstly, indoor experiments were conducted to analyze the correlation between various indicators (such as sampling depth and compaction state) and the collapsibility coefficient. Grey relational degree was used to screen out saturation (0.79), natural density (0.78), void ratio (0.71), and compression coefficient (0.71) as core indicators, eliminating weakly correlated redundant information. Subsequently, an optimized Transformer model was constructed based on these core indicators, which captures the deep coupling relationships among indicators through the self-attention mechanism. Based on indoor experimental data, comparative verification was carried out with RF, BP, and SVM models. The results demonstrate that the Transformer model achieves an R² of 0.976 and an MSE of 0.0009 on the training set, and an R² of 0.953 and an MSE of 0.0015 on the test set. Compared with traditional models, the MSE is reduced by 57.1%– 67.9% and the R ² is improved by 5.7% – 8.9%. The model stabilizes after only 42 epochs, with a generalization performance decay of merely 8.3%, significantly outperforming the decay rate of over 35% observed in traditional models, effectively mitigating the overfitting problem. In summary, this study accurately screens core indicators through grey relational degree and combines the powerful feature extraction capability of the Transformer network to realize high-precision and stable prediction of the loess collapsibility coefficient. It provides a scientific basis and efficient technical support for the formulation of foundation treatment schemes, disease prevention and control, and service life extension of road engineering in loess areas.