Classifying the molecular subtype of breast cancer using vision transformer and convolutional neural network features

• Published in: Breast cancer research and treatment Vol. 210; no. 3; pp. 771 - 782
• Main Authors: Kai, Chiharu, Tamori, Hideaki, Ohtsuka, Tsunehiro, Nara, Miyako, Yoshida, Akifumi, Sato, Ikumi, Futamura, Hitoshi, Kodama, Naoki, Kasai, Satoshi
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.04.2025; Springer Nature B.V
• Subjects: Biomarkers, Tumor; Biopsy; Breast cancer; Breast Neoplasms - classification; Breast Neoplasms - diagnosis; Breast Neoplasms - diagnostic imaging; Breast Neoplasms - genetics; Breast Neoplasms - metabolism; Breast Neoplasms - pathology; Cancer screening; Classification; Convolutional Neural Networks; ErbB-2 protein; Female; Humans; Image processing; Invasiveness; Magnetic resonance imaging; Mammography; Mammography - methods; Medicine; Medicine & Public Health; Neural networks; Neural Networks, Computer; Oncology; Principal components analysis; Receptor, ErbB-2 - metabolism; Vision transformer; Mammogram; Breast cancer; Molecular subtype; Artificial intelligence; Convolutional neural network
• ISSN: 0167-6806; 1573-7217; 1573-7217
• DOI: 10.1007/s10549-025-07614-9

Purpose Identification of the molecular subtypes in breast cancer allows to optimize treatment strategies, but usually requires invasive needle biopsy. Recently, non-invasive imaging has emerged as promising means to classify them. Magnetic resonance imaging is often used for this purpose because it is three-dimensional and highly informative. Instead, only a few reports have documented the use of mammograms. Given that mammography is the first choice for breast cancer screening, using it to classify molecular subtypes would allow for early intervention on a much wider scale. Here, we aimed to evaluate the effectiveness of combining global and local mammographic features by using Vision Transformer (ViT) and Convolutional Neural Network (CNN) to classify molecular subtypes in breast cancer. Methods The feature values for binary classification were calculated using the ViT and EfficientnetV2 feature extractors, followed by dimensional compression via principal component analysis. LightGBM was used to perform binary classification of each molecular subtype: triple-negative, HER2-enriched, luminal A, and luminal B. Results The combination of ViT and CNN achieved higher accuracy than ViT or CNN alone. The sensitivity for triple-negative subtypes was very high (0.900, with F-value = 0.818); whereas F-value and sensitivity were 0.720 and 0.750 for HER2-enriched, 0.765 and 0.867 for luminal A, and 0.614 and 0.711 for luminal B subtypes, respectively. Conclusion Features obtained from mammograms by combining ViT and CNN allow the classification of molecular subtypes with high accuracy. This approach could streamline early treatment workflows and triage, especially for poor prognosis subtypes such as triple-negative breast cancer.