Generative Artificial Intelligence Chatbot to Diagnose the Severity of Anxiety in Unsch Students Using the Hamilton Anxiety Scale (HAM- A), 2025

Abstract

Anxiety is a prevalent mental health problem among university students, creating the need for scalable and reliable tools for screening, severity classification, and decision support. This pilot study aimed to design, develop, and preliminarily evaluate a generative artificial intelligence chatbot for screening anxiety severity among Engineering students at the Universidad Nacional de San Cristóbal de Huamanga, Peru, during the 2025-II academic semester, using the Hamilton Anxiety Rating Scale (HAM-A). A pilot sample of 50 students was selected through cluster sampling. The chatbot, developed with Flutter and Dart and integrated with a generative language model API, automated HAM-A administration, score calculation, severity classification, result storage, and personalized feedback generation. The system was preliminarily validated by comparing chatbot-generated severity classifications with those assigned by a clinical mental health expert. Agreement was assessed using Cohen’s Kappa coefficient and a confusion matrix, while classification performance was analyzed using class-specific and multiclass metrics. The results showed 46 exact matches out of 50 cases, with an observed agreement of 92.0% and Cohen’s Kappa of κ = 0.8086, indicating almost perfect concordance in this pilot sample. Overall accuracy was 0.9200, with a Weighted F1-score of 0.9032, Macro F1-score of 0.7560, and Balanced Accuracy of 0.7197. The chatbot showed strong performance in moderate anxiety classification and robust identification of severe anxiety cases, with no false negatives in the severe category. However, the mild category showed lower recall, indicating a tendency to classify some mild cases into higher severity levels. These findings suggest that the chatbot has preliminary potential as a first-level screening and decision- support tool for university mental health programs. Nevertheless, due to the pilot nature of the study and the limited sample size, the results should be interpreted cautiously and validated in larger, more diverse, and preferably multicenter samples. The chatbot should be considered a complementary support system, not a substitute for formal clinical diagnosis or professional psychological evaluation.