TAU-PSO: A Transferable Attention U-Net with Particle Swarm Optimization for Optic Cup-Disc Segmentation in Fundus Images

Abstract

The retina is impacted by diabetic retinopathy, a disorder frequently observed in people diagnosed with diabetes mellitus for an extended period. This condition can ultimately lead to vision loss as a result of pathological retinal microvascular leakage. Early diagnosis of any optic disc abnormalities is crucial for ophthalmologists to treat their patients effectively. Segmenting the optic disc to screen for glaucoma using the optic cup-to-disc ratio is also essential. Several retinal diseases can be significantly helped by automated screening of fundus images utilizing computational approaches. To enhance the efficiency of current annotated samples without necessitating thousands of examples of training for Optic Disc (OD) and Optic Cup (OC) Segmentation, we present the lightweight network known as the Transferable Attention U-Net (TAU-Net) model with Particle Swarm Optimization (PSO) for hyperparameter tuning. The model employs attention modules and two adversarial domain discriminators (feature and attention discriminators) to learn domain-invariant representations across DRISHTI-GS1, RIM-ONE, and REFUGE datasets, while structured dropout blocks reduce overfitting. Experimental results show that our method achieves superior segmentation performance compared to several state-of-the-art approaches. Specifically, for the optic cup, the TAU-PSO model attains 98.2% accuracy, 96.6% precision, 97.2% recall, and 96.4% Dice score, while for the optic disc, it achieves 97.1% accuracy, 96.0% precision, 96.5% recall, and 96.8% Dice score. Compared to baseline methods such as NFN+, SDU-Net, and DU-Net, our approach demonstrates consistent improvements of 2–4% across key metrics, highlighting the effectiveness of combining PSO-based hyperparameter optimization with domain-adversarial learning. The proposed TAU- PSO framework provides a robust and transferable solution for OD–OC segmentation under cross-dataset conditions, offering a reliable tool for early diagnosis of diabetic macular edema.