Research on Time Series Forecasting Models Based on Hybrid Attention Mechanism and Graph Neural Networks

Abstract

In wireless data transmission, packet loss and missing data caused by environmental interference and network congestion significantly impact the stability of time series. To address these challenges, this study proposes a time series forecasting model named FGDLNet. FGDLNet is based on the Transformer architecture and integrates Graph Neural Networks (GNN) to enhance the performance of long sequence predictions, particularly in handling complex time series patterns. The model simplifies its structure and reduces computational complexity by removing the Decoder module from the traditional Transformer and replacing it with a linear layer for direct connection and prediction. To enhance the feature extraction capability of time series data, FGDLNet incorporates a multi-scale feature extraction module that extracts features at different temporal scales using multiple convolution kernels in parallel. Specifically, the model employs a single-channel processing approach to reduce interference between features and improve prediction accuracy. The introduced GNN module enables feature propagation and enhancement within the single channel, better capturing short-term fluctuations and long-term trends. In terms of the attention mechanism, this study designs a hybrid attention mechanism that combines global linear attention and local window attention. The global linear attention optimizes the computation to improve the efficiency of capturing global contextual information, while the local window attention strengthens the model’s ability to handle short-term dependencies. To evaluate the effectiveness of the model, we selected a dataset recorded during the flight of a specific aircraft, which includes longitude, latitude, and inertial navigation parameters, and conducted long-term trend forecasting. In the experiments, we used MAE (Mean Absolute Error), MSE (Mean Squared Error), and training time to assess the model’s performance. The experimental results show that FGDLNet outperforms traditional models such as Autoformer, Transformer, Informer, Reformer, DLinear, and ITransformer in long-term forecasting tasks. Specifically, FGDLNet achieves the following MAE values: 0.1400, 0.0595, 0.0092, 0.0324, 0.0493, and 0.122, which are significantly lower than those of the other models. In terms of MSE, FGDLNet also demonstrates lower errors: 0.0231, 0.0584, 0.0987, 0.0825, 0.0798, and 0.1925. Additionally, FGDLNet’s training time per epoch is 1156.47 seconds, which is about 7% faster than the Transformer model (1243.14 seconds).