AP-Traj2: Transformer-based Trajectory Prediction with Graph-Enhanced Attention Mechanism
Abstract
Trajectory prediction is essential for understanding human mobility patterns, with applications such as itinerary recommendation and urban planning. It involves analyzing sequences of visited locations to forecast the user’s next destination. Traditional approaches have often relied on Markov chains or recurrent
neural networks (RNNs). More recently, Transformer neural networks have gained attention for sequential prediction tasks due to their superior parallelization and training efficiency. In this study, we propose AP-Traj2 (Attention and Possible directions for TRAJectory prediction 2), a model designed to enhance prediction accuracy by leveraging attention mechanisms and graph-based movement modeling. AP-Traj2 employs self-attention to capture dependencies among visited locations, explores feasible next steps through a graph of possible directions, and incorporates contextual information via location embeddings. Experiments conducted on GPS, CDR, and WiFi datasets demonstrate that AP-Traj2 improves the average match
ratio by approximately 50% over state-of-the-art methods. Moreover, it achieves significantly faster training times, with reductions of up to 72% in the best-case scenario. Unlike existing approaches that focus primarily on neural network architecture, this work emphasizes the importance of data preprocessing and
filtering, highlighting their substantial impact on model performance.
Full Text:
PDFReferences
D. Bachir, G. Khodabandelou, V. Gauthier, M. El Yacoubi, and J. Puchinger, “Inferring dynamico rigin-destination flows by transport mode using mobile phone data,” Transportation Research Part C: Emerging Technologies, vol. 101, pp. 254–275, 2019.
F. Asgari, A. Sultan, H. Xiong, V. Gauthier, and M. A. El-Yacoubi, “Ct-mapper: Mapping sparse multimodal cellular trajectories using a multilayer transportation network,” Computer Communications, vol. 95, pp. 69–81, 2016, mobile Traffic Analytics.
L. Huang, J. Zhuang, X. Cheng, R. Xu, and H. Ma, “Sti-gan: Multimodal pedestrian trajectory prediction using spatiotemporal interactions and a generative adversarial network,” IEEE Access, vol. 9, pp. 50 846–50 856, 2021.
Y. Pang, X. Zhao, J. Hu, H. Yan, and Y. Liu, “Bayesian spatio-temporal graph transformer network (b-star) for multi-aircraft trajectory prediction,” Knowledge-Based Systems, p. 108998, 2022.
R. W. Liu, M. Liang, J. Nie, Y. Yuan, Z. Xiong, H. Yu, and N. Guizani, “Stmgcn: Mobile edge computing-empowered vessel trajectory prediction using spatio-temporal multi-graph convolutional network,” IEEE Transactions on Industrial Informatics, 2022.
K. Shao, Y. Wang, Z. Zhou, X. Xie, and G. Wang, “Trajforesee: How limited detailed trajectories enhance large-scale sparse information to predict vehicle trajectories?” in 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, 2021, pp. 2189–2194.
J. Bray, J. T. Feldblum, and I. C. Gilby, “Social bonds predict dominance trajectories in adult male chimpanzees,” Animal Behaviour, vol. 179, pp. 339–354, 2021.
S. Gambs, M.-O. Killijian, and M. N. del Prado Cortez, “Next place prediction using mobility markov chains,” in Proceedings of the first workshop on measurement, privacy, and mobility, 2012, pp. 1–6.
J. Feng, Y. Li, Z. Yang, Q. Qiu, and D. Jin, “Predicting human mobility with semantic motivation via multi-task attentional recurrent networks,” IEEE Transactions on Knowledge and Data Engineering, 2020.
Y.-C. Chen, T. Thaipisutikul, and T. K. Shih, “A learning-based poi recommendation with spatiotemporal context awareness,” IEEE Transactions on Cybernetics, vol. 52, no. 4, pp. 2453–2466, 2022.
J. Tang, J. Liang, T. Yu, Y. Xiong, and G. Zeng, “Trip destination prediction based on a deep integration network by fusing multiple features from taxi trajectories,” IET Intelligent Transport Systems, vol. 15, no. 9, pp. 1131–1141, 2021.
Y. Liang and Z. Zhao, “Nettraj: A network-based vehicle trajectory prediction model with directional representation and spatiotemporal attention mechanisms,” IEEE Transactions on Intelligent Transportation Systems, 2021.
Y. Suo, W. Chen, C. Claramunt, and S. Yang, “A ship trajectory prediction framework based on a recurrent neural network,” Sensors, vol. 20, no. 18, p. 5133, 2020.
T. Bogaerts, A. D. Masegosa, J. S. Angarita-Zapata, E. Onieva, and P. Hellinckx, “A graph cnn-lstm neural network for short and long term traffic forecasting based on trajectory data,” Transportation Research Part C: Emerging Technologies, vol. 112, pp. 62–77, 2020
M. A. Rahim, S. D. Khan, S. Khan, M. Rashid, R. Ullah, H. Tariq, and S. Czapp, “A novel spatio–temporal deep learning vehicle turns detection scheme using gps-only data,” IEEE Access, vol. 11, pp. 8727–8733, 2023.
J. Grigsby, Z. Wang, and Y. Qi, “Long-range transformers for dynamic spatiotemporal forecasting,” arXiv preprint arXiv:2109.12218, 2021.
Y. Li and J. M. Moura, “Forecaster: A graph transformer for forecasting spatial and time-dependent data,” arXiv preprint arXiv:1909.04019, 2019.
B. Yan, G. Zhao, L. Song, Y. Yu, and J. Dong, “Precln: Pretrained-based contrastive learning network for vehicle trajectory prediction,” World Wide Web, pp. 1–23, 2022.
Y. Hong, H. Martin, and M. Raubal, “How do you go where? improving next location prediction by learning travel mode information using transformers,” arXiv preprint arXiv:2210.04095, 2022.
D. Nguyen and R. Fablet, “Traisformer-a generative transformer for ais trajectory prediction,” arXiv preprint arXiv:2109.03958, 2021.
W. Jiang and J. Luo, “Graph neural network for traffic forecasting: A survey,” Expert Systems with Applications, vol. 207, p. 117921, 2022.
A.-P. Galarreta, H. Alatrista-Salas, and M. Nunez-del Prado, “Predicting next whereabouts using deep learning,” in International Conference on Modeling Decisions for Artificial Intelligence. Springer, 2023, pp. 214–225.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017.
J. Cheng, L. Dong, and M. Lapata, “Long short-term memory-networks for machine reading,” arXiv preprint arXiv:1601.06733, 2016.
M. Konrad. (2022) geovoronoi – a package to create and plot voronoi regions inside geographic areas. Accessed: January, 2023. [Online]. Available: https://pypi.org/project/geovoronoi/
A. Hagberg, P. Swart, and D. S Chult, “Exploring network structure, dynamics, and function using networkx,” Los Alamos National Lab.(LANL), Los Alamos, NM (United States), Tech. Rep., 2008.
U. Aslak and L. Alessandretti, “Infostop: Scalable stop-location detection in multi-user mobility data,” arXiv preprint arXiv:2003.14370, 2020.
Y. Lou, C. Zhang, Y. Zheng, X. Xie, W. Wang, and Y. Huang, “Map-matching for low-sampling-rate gps trajectories,” in Proceedings of the 17th ACM SIGSPATIAL international conference on advances in geographic information systems, 2009, pp. 352–361.
J. Yuan, Y. Zheng, X. Xie, and G. Sun, “Driving with knowledge from the physical world,” in Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, 2011, pp. 316–324.
Y. Zheng, X. Xie, W.-Y. Ma et al., “Geolife: A collaborative social networking service among user, location and trajectory.” IEEE Data Eng. Bull., vol. 33, no. 2, pp. 32–39, 2010.
V. D. Blondel, M. Esch, C. Chan, F. Cl´erot, P. Deville, E. Huens, F. Morlot, Z. Smoreda, and C. Ziemlicki, “Data for development: the d4d challenge on mobile phone data,” arXiv preprint arXiv:1210.0137, 2012.
Y.-A. de Montjoye, Z. Smoreda, R. Trinquart, C. Ziemlicki, and V. D. Blondel, “D4d-senegal: the second mobile phone data for development challenge,” arXiv preprint arXiv:1407.4885, 2014.
L. Pajevic, G. Karlsson, and V. Fodor, “Crawdad dataset kth/campus (v. 2019-07-01),” CRAWDAD Wirel Network Data Arch, vol. 10, 2019.
G. Boeing, “Osmnx: New methods for acquiring, constructing, analyzing, and visualizing complex street networks,” Computers, Environment and Urban Systems, vol. 65, pp. 126–139, 2017.
Manish, U. Dohare, and S. Kumar, “A survey of vehicle trajectory prediction based on deep learning models,” in Proceedings of Third International Conference on Sustainable Expert Systems: ICSES 2022. Springer, 2023, pp. 649–664.
DOI: https://doi.org/10.31449/inf.v49i21.8434
This work is licensed under a Creative Commons Attribution 3.0 License.