Multimodal Depression Detection from WhatsApp Statuses Using Hybrid Feature Selection with HMO-RTH-OOA and KPCA-CCA - Begum, Shaik Rasheeda; Sait, Saad Yunus (Informatica, 2025)
Depression is a common and serious condition characterized by persistent sadness. Early detection is critical to leading a healthy and fulfilling life. Due to the rapid development of social media, more people are using it to share their thoughts and feelings. Therefore, social media sites like Facebook, Twitter, and Instagram are now developing into a significant data source that can be used for identifying depression and mental illness. In this work, we demonstrate the detection of depression using WhatsApp data. A dataset of 3 months' worth of images, text, and behavior of 100 users was collected with consent, labeled by three psychological professionals, and finalized by majority vote. We examined WhatsApp-based depression from two perspectives: (a) message-based, which uses a single WhatsApp status image with corresponding extracted text, and (b) user-based detection, which utilizes all status images, texts, and WhatsApp behavioral data of each user. BERT and ELECTRA models were employed for text feature extraction, while EfficientNetV2L was used for image feature extraction. We have proposed two feature selection approaches: a hybrid multi-objective Red-Tailed Hawk - Osprey Optimization Algorithm (HMORTH-OOA), which attempts to select an optimal set of features, and Kernel Principal Component Analysis - Canonical Correlation Analysis (KPCA-CCA), which reduces the dimensionality of the multi-modal dataset. Additionally, we employed data augmentation to improve the dataset count for message-based depression detection to 5514 samples. Classification was performed using standard machine learning models, including XGBoost (XGB), AdaBoost (ADA), Support Vector Machine (SVM), and Random Forest (RF). User-based depression detection achieved 93.79% accuracy with KPCA-CCA, while message-based depression detection achieved 90.46% accuracy with HMO-RTH-OOA, demonstrating significant improvements over depression detection performed on Twitter and Instagram datasets.