A Dual-Mode Conversational GIS for Proximity and Image-Inferred Category-Based Routing using CLIP

Medjon Hysenaj, Dritan Rustja

Abstract


This paper presents the design and implementation of a dual-mode conversational Geographic Information System (GIS) routing assistant integrated into the PrimeMap web platform. The system employs a chatbot interface to guide users through two distinct route-planning modes: Closest, which selects destinations based on nearest spatial proximity using sequential location calculations, and Theme, which builds routes according to place categories. In Theme mode, users can either manually select categories or upload an image for AI-assisted classification using OpenAI’s CLIP model. The image-driven approach automatically determines the relevant category by matching detected semantic labels to pre-defined groups in the GIS database, enabling a seamless visual search capability. The conversational interface, built with BotUI and integrated into a Leaflet-based map UI, supports dynamic user input, immediate visual feedback, and flexible route building. The backend, implemented in Spring Boot, manages category/group/place logic, image processing requests, and route computation using the Haversine formula for nearest location detection. The current implementation prioritizes simplicity and user experience, while planned future work includes more complex multi-criteria ranking (e.g., cosine similarity, opening hours, ratings), optional GPS-based starting locations, advanced search filters, and richer AI-assisted matching. Experimental evaluation demonstrated 92 % classification accuracy on a balanced 50-image test set, with average route-generation latency below one second and consistent thematic match performance across ten categories. This dual-mode chatbot demonstrates how conversational GIS can bridge the gap between traditional map interfaces and intelligent, user-adaptive routing, offering potential applications in tourism, urban mobility, and location-based services.


Full Text:

PDF

References


Goodchild, M.F. (2007). Citizens as sensors: the world of volunteered geography. GeoJournal 69, pp 211-221, https://doi.org/10.1007/s10708-007-9111-y.

Haklay, M. (2010). How Good is Volunteered Geographical Information? A Comparative Study of OpenStreetMap and Ordnance Survey Datasets. Environment and Planning B: Planning and Design, 37(4), pp 682-703, https://doi.org/10.1068/b35097 (Original work published 2010).

Radlinski, F., Craswell, N. (2017). A theoretical framework for conversational search. Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval, pp. 117-126. https://doi.org/10.1145/3020165.3020183.

Christakopoulou, K., Radlinski, F., Hofmann, K. (2016). Towards conversational recommender systems. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 815-824. https://doi.org/10.1145/2939672.2939746.

Ricci, F., Rokach, L., & Shapira, B. (Eds.). (2015). Recommender Systems Handbook (2nd ed.). Springer.

Zhao, S., King, I., & Lyu, M. R. (2016). A survey of point-of-interest recommendation in location-based social networks. arXiv preprint arXiv:1607.00647.

Zheng, Y. (2015). Trajectory data mining: An overview. ACM Transactions on Intelligent Systems and Technology, 6(3), pp. 1-41. https://doi.org/10.1145/2743025.

Sinnott, R. W. (1984). Virtues of the Haversine. Sky and Telescope, 68, 159.

Malczewski, J. (2006). GIS-based multicriteria decision analysis: A survey of the literature. International Journal of Geographical Information Science, 20(7), pp. 703-726. https://doi.org/10.1080/13658810600661508.

Delling, D., Sanders, P., Schultes, D., Wagner, D. (2009). Engineering Route Planning Algorithms. In: Lerner, J., Wagner, D., Zweig, K.A. (eds) Algorithmics of Large and Complex Networks. Lecture Notes in Computer Science, vol 5515. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02094-0_7.

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. arXiv. https://doi.org/10.48550/arXiv.2103.00020.

Arandjelović, R., Gronat, P., Torii, A., Pajdla, T., & Sivic, J. (2015). NetVLAD: CNN architecture for weakly supervised place recognition. arXiv. https://doi.org/10.48550/arXiv.1511.07247.

Weyand, T., Kostrikov, I., & Philbin, J. (2016). PlaNet – Photo Geolocation with Convolutional Neural Networks. arXiv. https://doi.org/10.1007/978-3-319-46484-8_3.

Kurashima, T., Iwata, T., Irie, G., & Fujimura, K. (2010). Travel route recommendation using geotags in photo sharing sites. In Proceedings of the 19th ACM international conference on Information and knowledge management pp. 579–588. ACM. https://doi.org/10.1145/1871437.1871513.

Gams M, Horvat T, Kolar Ž, Kocuvan P, Mishev K, Misheva (2025) MS. Evaluating a Nationally Localized AI Chatbot for Personalized Primary Care Guidance: Insights from the HomeDOCtor Deployment in Slovenia. Healthcare (Basel). 13(15): p. 1843. doi: 10.3390/healthcare13151843. PMID: 40805876; PMCID: PMC12346038.

Kolenik, T., & Gams, M. (2021). Intelligent Cognitive Assistants for Attitude and Behavior Change Support in Mental Health: State-of-the-Art Technical Review. Electronics, 10(11), 1250. https://doi.org/10.3390/electronics10111250.

D. Torkar., R. Murn, “GPS positioning and digital map processing in 2D and 3D terrain environment,” Journal of Computing and Information Technology, vol. 2, no. 3, pp. 233–241, 1994.

M. Hysenaj., R. Barjami (2012) “Web GIS Albania Platform, an informative technology for the Albanian territory,” Informatica, vol. 36, no. 4, pp. 431–439.

K. Christakopoulou, F. Radlinski., K. Hofmann. (2016) “Towards conversational recommender systems,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, pp. 815-824. https://doi.org/10.1145/2939672.2939746.

Y. Zheng, L. Capra, O. Wolfson., H. Yang, (2014) “Urban computing: Concepts, methodologies, and applications,” ACM Transactions on Intelligent Systems and Technology, vol. 5, no. 3, pp. 38:1-38:55, https://doi.org/10.1145/2629592.




DOI: https://doi.org/10.31449/inf.v49i32.11999

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 License.