Hybrid vision transformer model for accurate prostate cancer classification in MRI images

Jashim, Farhan Bin and Refat, Fajle Rabbi and Karim, Mohammad Hasnatul and Mahmud, Farhad Uddin and Ashrafi, Fariha (2025) Hybrid vision transformer model for accurate prostate cancer classification in MRI images. International Journal of Science and Research Archive, 15 (2). pp. 1505-1517. ISSN 2582-8185

Prostate cancer remains one of the most prevalent malignancies among men globally, with early diagnosis complicated by its heterogeneous characteristics and the constraints of existing diagnostic approaches. This research introduces an advanced framework that integrates Convolutional Neural Networks (CNNs) with Vision Transformers (ViTs) to enhance the classification of prostate cancer using MRI scans. To mitigate class imbalance and improve generalization, we employed a combination of dual synthetic oversampling strategies along with data augmentation techniques. Our preprocessing workflow was designed to suppress image noise while maintaining edge integrity and enhancing local contrast without introducing artifacts. For robust feature representation, we extracted both Gray-Level Co-occurrence Matrix (GLCM) features and shape descriptors to capture the intricate patterns within the MRI data. Among the tested deep learning models, the ConvNeXt architecture delivered the highest performance. Specifically, using the SMOTE technique, it achieved an F1-score of 97.21% and a Matthews Correlation Coefficient (MCC) of 95.32%, while the application of ADASYN led to further gains, with an F1-score of 98.82% and an MCC of 97.86%. To support real-time clinical use, we also developed a web-based platform capable of analyzing prostate MRI scans interactively. These findings highlight the effectiveness and interpretability of our proposed method in facilitating accurate prostate cancer diagnosis.