Fusion of Convolutional Architecture and Transformer Models for Enhanced Brain Tumor Classification

Abstract

Early detection of brain tumors based on MRI images has shown significant advancements with the advent of deep learning methods. However, achieving high accuracy and robustness in classification remains a challenge due to the complex and mixed nature of brain tumors and the clarity of samples. This study proposes a novel approach that integrates convolutional architectures with the transformer approach, which can lead to an optimal model. The convolutional neural networks (CNNs) excel in capturing local features and spatial hierarchies, while the transformer approach captures long-term dependencies and contextual information. By integrating these two robust architectures, our proposed model leverages the strengths of both to achieve superior performance. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) dataset is used to evaluate our model, which consists of 7023 samples across four classes. We compare the performance of the fusion model with that of the prescribed models. The results demonstrate that the fusion model significantly outperforms the standalone models, achieving a classification accuracy of 91.8%. The proposed approach also shows improved robustness in handling various tumor types and sizes, highlighting its potential for clinical application.