Leveraging Local and Global Features: A CNN-Transformer Hybrid for fMRI-Based Autism Diagnosis
Abstract
Autism Spectrum Disorder (ASD) diagnosis remains challenging because of its heterogeneity and reliance on subjective behavioral assessments. Resting-state functional MRI (fMRI) presents a compelling opportunity avenue for identifying objective biomarkers, but decoding its complex spatiotemporal patterns requires advanced computational models. While Deep Learning (DL) approaches have progressed, many struggle to concurrently capture local neural dynamics and global temporal dependencies. A novel end-to-end CNN-Transformer hybrid framework designed for fMRI-based autism diagnosis is proposed to address this. Our model leverages a convolutional module to extract localized spatiotemporal features, which are then processed by a Transformer encoder to model long-range, global dependencies through a Multi-Head Self-Attention (MHSA) mechanism. Evaluated on the large multi-site ABIDE-I dataset (N=1,035), the suggested model achieved state-of-the-art performance with an accuracy of 77.85%, a sensitivity of 76.52%, a specificity of 78.90%, and an F1-score of 77.71%. Ablation studies confirmed the critical contribution of each architectural component, and comparisons with pre-trained CNNs and other leading methods demonstrated superior and statistically significant performance (p<0.05). Despite an observed performance drop in site-specific evaluations, underscoring the challenge of scanner heterogeneity, our results affirm that the synergistic integration of local feature learning and global contextual modeling is a powerful paradigm for neuroimaging-based diagnostic applications.References
R. M. Hernández, J. C. Ponce-Meza, M. Á. Saavedra-López, W. A. C. Ugaz, R. M. Chanduvi, and W. C. Monteza, “Brain complexity and psychiatric disorders,” Iran J Psychiatry, vol. 18, no. 4, p. 493, 2023. https://doi.org/10.18502/ijps.v18i4.13637
M. R. Mohammadi et al., “Prevalence of autism and its comorbidities and the relationship with maternal psychopathology: a national population-based study,” Arch Iran Med, vol. 22, no. 10, pp. 546–553, 2019. http://eprints.mui.ac.ir/id/eprint/11250
J. Yang et al., “Towards an accurate autism spectrum disorder diagnosis: multiple connectome views from fMRI data,” Cerebral Cortex, vol. 34, no. 1, p. bhad477, 2024. https://doi.org/10.1093/cercor/bhad477
M. R. Mohammadi et al., “Prevalence and correlates of psychiatric disorders in a national survey of Iranian children and adolescents,” Iran J Psychiatry, vol. 14, no. 1, p. 1, 2019. https://pmc.ncbi.nlm.nih.gov/articles/PMC6505051/
M. KHARE, S. ACHARYA, S. SHUKLA, and A. SACHDEV, “Utilising Artificial Intelligence (AI) in the Diagnosis of Psychiatric Disorders: A Narrative Review.,” Journal of Clinical & Diagnostic Research, vol. 18, no. 4, 2024. DOI: 10.7860/JCDR/2023/61698.19249
N. Ghahari, F. Yousefian, S. Behzadi, and A. Jalilzadeh, “Rural-urban differences in age at autism diagnosis: a multiple model analysis,” Iran J Psychiatry, vol. 17, no. 3, p. 294, 2022. https://doi.org/10.18502/ijps.v17i3.9729
A. Khaleghi, H. Zarafshan, S. R. Vand, and M. R. Mohammadi, “Effects of non-invasive neurostimulation on autism spectrum disorder: a systematic review,” Clinical Psychopharmacology and Neuroscience, vol. 18, no. 4, p. 527, 2020. https://doi.org/10.9758/cpn.2020.18.4.527
S. Talepasand et al., “Psychiatric disorders in children and adolescents: prevalence and sociodemographic correlates in Semnan Province in Iran,” Asian J Psychiatr, vol. 40, pp. 9–14, 2019. https://doi.org/10.1016/j.ajp.2019.01.007
A. Marab and A. Bhadrashetty, “A novel approach using deep convolutional neural networks for automated dementia detection and classification,” Edraak, vol. 2023, pp. 16–20, 2023. https://doi.org/10.70470/EDRAAK/2023/004
N. Salari et al., “The global prevalence of autism spectrum disorder: a comprehensive systematic review and meta-analysis,” Ital J Pediatr, vol. 48, no. 1, p. 112, 2022. https://doi.org/10.1186/s13052-022-01310-w
E. Helmy et al., “Role of artificial intelligence for autism diagnosis using DTI and fMRI: A survey,” Biomedicines, vol. 11, no. 7, p. 1858, 2023. https://doi.org/10.3390/biomedicines11071858
A. Khaleghi, M. R. Mohammadi, G. P. Jahromi, and H. Zarafshan, “New ways to manage pandemics: using technologies in the era of COVID-19: a narrative review,” Iran J Psychiatry, vol. 15, no. 3, p. 236, 2020. https://doi.org/10.18502/ijps.v15i3.3816
H. Alkahtani, T. H. H. Aldhyani, and M. Y. Alzahrani, “Deep learning algorithms to identify autism spectrum disorder in children-based facial landmarks,” Applied Sciences, vol. 13, no. 8, p. 4855, 2023. https://doi.org/10.3390/app13084855
D. Pandian, S. S. Rajagopalan, and D. Jayagopi, “Detecting a child’s stimming behaviours for autism spectrum disorder diagnosis using rgbpose-slowfast network,” in 2022 IEEE International Conference on Image Processing (ICIP), IEEE, 2022, pp. 3356–3360. https://doi.org/10.1109/ICIP46576.2022.9897867
A. Zunino et al., “Video gesture analysis for autism spectrum disorder detection,” in 2018 24th international conference on pattern recognition (ICPR), IEEE, 2018, pp. 3421–3426.
A. Khaleghi, M. R. Mohammadi, K. Shahi, and A. M. Nasrabadi, “Computational neuroscience approach to psychiatry: a review on theory-driven approaches,” Clinical Psychopharmacology and Neuroscience, vol. 20, no. 1, p. 26, 2022. https://doi.org/10.1109/ICPR.2018.8545095
W. Wang and W. Beibei, “Improving Segmentation of MRI Images of Brain Glial Cells Using Fuzzy C-Means Algorithm and Combination of PET Images,” Journal of Artificial Intelligence and System Modelling, vol. 2, no. 04, pp. 47–59, 2024. https://doi.org/10.22034/jaism.2024.482743.1071
Z. Zhao et al., “Conventional machine learning and deep learning in Alzheimer’s disease diagnosis using neuroimaging: A review,” Front Comput Neurosci, vol. 17, p. 1038636, 2023. https://doi.org/10.3389/fncom.2023.1038636
B. Qiu, Q. Wang, X. Li, W. Li, W. Shao, and M. Wang, “Adaptive spatial-temporal neural network for ADHD identification using functional fMRI,” Front Neurosci, vol. 18, p. 1394234, 2024. https://doi.org/10.3389/fnins.2024.1394234
A. Chandra, S. Verma, A. S. Raghuvanshi, and N. K. Bodhey, “ASDC-Net: Optimized convolutional neural network-based automatic autism spectrum disorder classification using rs-fMRI Data,” IETE J Res, vol. 70, no. 4, pp. 4189–4202, 2024. https://doi.org/10.1080/03772063.2023.2196979
I. Chi, S. Tsai, C. Chen, and A. C. Yang, “Identifying Distinct Developmental Patterns of Brain Complexity in Autism: A Cross‐Sectional Cohort Analysis Using the Autism Brain Imaging Data Exchange,” Psychiatry Clin Neurosci, vol. 79, no. 3, pp. 98–107, 2025. https://doi.org/10.1111/pcn.13780
S. Gupta et al., “Enhancing autism spectrum disorder classification with lightweight quantized cnns and federated learning on abide-1 dataset,” Mathematics, vol. 12, no. 18, p. 2886, 2024. https://doi.org/10.3390/math12182886
S. Saponaro et al., “Deep learning based joint fusion approach to exploit anatomical and functional brain information in autism spectrum disorders,” Brain Inform, vol. 11, no. 1, p. 2, 2024. https://doi.org/10.1186/s40708-023-00217-4
X. Liu, M. R. Hasan, T. Gedeon, and M. Z. Hossain, “MADE-for-ASD: A multi-atlas deep ensemble network for diagnosing Autism Spectrum Disorder,” Comput Biol Med, vol. 182, p. 109083, 2024. https://doi.org/10.1016/j.compbiomed.2024.109083
R. R. Jha, A. Muralie, M. Daroch, A. Bhavsar, and A. Nigam, “Enhancing Autism Spectrum Disorder identification in multi-site MRI imaging: A multi-head cross-attention and multi-context approach for addressing variability in un-harmonized data,” Artif Intell Med, vol. 157, p. 102998, 2024. https://doi.org/10.1016/j.artmed.2024.102998
T. Iidaka, “Resting state functional magnetic resonance imaging and neural network classified autism and control,” Cortex, vol. 63, pp. 55–67, 2015. https://doi.org/10.1016/j.cortex.2014.08.011
M. Plitt, K. A. Barnes, and A. Martin, “Functional connectivity classification of autism identifies highly predictive brain features but falls short of biomarker standards,” Neuroimage Clin, vol. 7, pp. 359–366, 2015. https://doi.org/10.1016/j.nicl.2014.12.013
S. Parisot et al., “Disease prediction using graph convolutional networks: application to autism spectrum disorder and Alzheimer’s disease,” Med Image Anal, vol. 48, pp. 117–130, 2018. https://doi.org/10.1016/j.media.2018.06.001
B. Sen, N. C. Borle, R. Greiner, and M. R. G. Brown, “A general prediction model for the detection of ADHD and Autism using structural and functional MRI,” PLoS One, vol. 13, no. 4, p. e0194856, 2018. https://doi.org/10.1371/journal.pone.0194856
M. N. Parikh, H. Li, and L. He, “Enhancing diagnosis of autism with optimized machine learning models and personal characteristic data,” Front Comput Neurosci, vol. 13, p. 9, 2019. https://doi.org/10.3389/fncom.2019.00009
M. Z. Uddin, M. A. Shahriar, M. N. Mahamood, F. Alnajjar, M. I. Pramanik, and M. A. R. Ahad, “Deep learning with image-based autism spectrum disorder analysis: A systematic review,” Eng Appl Artif Intell, vol. 127, p. 107185, 2024. https://doi.org/10.1016/j.engappai.2023.107185
J. Shin, “Revolutionizing Medical Imaging with Artificial intelligence Real-Time Segmentation for Enhanced Diagnostics,” EDRAAK, vol. 2024, pp. 18–25, 2024. https://orcid.org/0000-0002-7476-2468
C. J. Brown, J. Kawahara, and G. Hamarneh, “Connectome priors in deep neural networks to predict autism,” in 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018), IEEE, 2018, pp. 110–113. https://doi.org/10.1109/ISBI.2018.8363534
A. S. Heinsfeld, A. R. Franco, R. C. Craddock, A. Buchweitz, and F. Meneguzzi, “Identification of autism spectrum disorder using deep learning and the ABIDE dataset,” Neuroimage Clin, vol. 17, pp. 16–23, 2018. https://doi.org/10.1016/j.nicl.2017.08.017
N. Qiang et al., “A deep learning method for autism spectrum disorder identification based on interactions of hierarchical brain networks,” Behavioural Brain Research, vol. 452, p. 114603, 2023. https://doi.org/10.1016/j.bbr.2023.114603
F. Z. Subah, K. Deb, P. K. Dhar, and T. Koshiba, “A deep learning approach to predict autism spectrum disorder using multisite resting-state fMRI,” Applied Sciences, vol. 11, no. 8, p. 3636, 2021. https://doi.org/10.3390/app11083636
F. Zhao et al., “Multi-head self-attention mechanism-based global feature learning model for ASD diagnosis,” Biomed Signal Process Control, vol. 91, p. 106090, 2024. https://doi.org/10.1016/j.bspc.2024.106090
S. Amemiya, H. Takao, and O. Abe, “Resting‐State fMRI: Emerging Concepts for Future Clinical Application,” Journal of Magnetic Resonance Imaging, vol. 59, no. 4, pp. 1135–1148, 2024. https://doi.org/10.1002/jmri.28894
W. Wei et al., “Analyzing 20 years of resting-state fMRI research: Trends and collaborative networks revealed,” Brain Res, vol. 1822, p. 148634, 2024. https://doi.org/10.1016/j.brainres.2023.148634
T. Eslami, V. Mirjalili, A. Fong, A. R. Laird, and F. Saeed, “ASD-DiagNet: a hybrid learning approach for detection of autism spectrum disorder using fMRI data,” Front Neuroinform, vol. 13, p. 70, 2019. https://doi.org/10.3389/fninf.2019.00070
Y. Ma et al., “Multi-scale dynamic graph learning for brain disorder detection with functional MRI,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 31, pp. 3501–3512, 2023.
F. Almuqhim and F. Saeed, “ASD-SAENet: a sparse autoencoder, and deep-neural network model for detecting autism spectrum disorder (ASD) using fMRI data,” Front Comput Neurosci, vol. 15, p. 654315, 2021. https://doi.org/10.3389/fncom.2021.654315
H. Zhao, H. Lou, L. Yao, and Y. Zhang, “Diffusion transformer-augmented fMRI functional connectivity for enhanced autism spectrum disorder diagnosis,” J Neural Eng, vol. 22, no. 1, p. 016044, 2025. DOI 10.1088/1741-2552/adb07a
J. Deng, M. R. Hasan, M. Mahmud, M. M. Hasan, K. A. Ahmed, and M. Z. Hossain, “Diagnosing autism spectrum disorder using ensemble 3D-CNN: A preliminary study,” in 2022 IEEE international conference on image processing (ICIP), IEEE, 2022, pp. 3480–3484.
A. Abraham et al., “Deriving reproducible biomarkers from multi-site resting-state data: An Autism-based example,” Neuroimage, vol. 147, pp. 736–745, 2017. https://doi.org/10.1016/j.neuroimage.2016.10.045
R. Anirudh and J. J. Thiagarajan, “Bootstrapping graph convolutional neural networks for autism spectrum disorder classification,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2019, pp. 3197–3201.
Z. Sherkatghanad et al., “Automated detection of autism spectrum disorder using a convolutional neural network,” Front Neurosci, vol. 13, p. 1325, 2020. https://doi.org/10.3389/fnins.2019.01325
Y. Liu, L. Xu, J. Li, J. Yu, and X. Yu, “Attentional connectivity-based prediction of autism using heterogeneous rs-fMRI data from CC200 atlas,” Exp Neurobiol, vol. 29, no. 1, p. 27, 2020. https://doi.org/10.5607/en.2020.29.1.27
Y. Wang, J. Wang, F.-X. Wu, R. Hayrat, and J. Liu, “AIMAFE: Autism spectrum disorder identification with multi-atlas deep feature representation and ensemble learning,” J Neurosci Methods, vol. 343, p. 108840, 2020. https://doi.org/10.1016/j.jneumeth.2020.108840
DOI:
https://doi.org/10.31449/inf.v50i5.12777Downloads
Published
How to Cite
Issue
Section
License
Authors retain copyright in their work. By submitting to and publishing with Informatica, authors grant the publisher (Slovene Society Informatika) the non-exclusive right to publish, reproduce, and distribute the article and to identify itself as the original publisher.
All articles are published under the Creative Commons Attribution license CC BY 3.0. Under this license, others may share and adapt the work for any purpose, provided appropriate credit is given and changes (if any) are indicated.
Authors may deposit and share the submitted version, accepted manuscript, and published version, provided the original publication in Informatica is properly cited.
