VTrans: A VAE-Based Pre-Trained Transformer Method for Microbiome Data Analysis

• Published in: Journal of computational biology Vol. 32; no. 9; pp. 85 - 864
• Main Authors: Shi, Xinyuan, Zhu, Fangfang, Min, Wenwen
• Format: Journal Article
• Language: English
• Published: United States Mary Ann Liebert, Inc., publishers 01.09.2025
• Subjects: Algorithms; Computational Biology - methods; Deep Learning; Humans; Microbiota - genetics; Neoplasms - genetics; Neoplasms - microbiology; Neoplasms - mortality; Original Articles; saliency map; Transformer; multihead-co-attention; variational autoencoder; microbiome data; pretraining
• ISSN: 1557-8666; 1557-8666
• DOI: 10.1089/cmb.2024.0884

Predicting the survival outcomes and assessing the risk of patients play a pivotal role in comprehending the microbial composition across various stages of cancer. With the ongoing advancements in deep learning, it has been substantiated that deep learning holds the potential to analyze patient survival risks based on microbial data. However, confronting a common challenge in individual cancer datasets involves the limited sample size and the high dimensionality of the feature space. This predicament often leads to overfitting issues in deep learning models, hindering their ability to effectively extract profound data representations and resulting in suboptimal model performance. To overcome these challenges, we advocate the utilization of pretraining and fine-tuning strategies, which have proven effective in addressing the constraint of having a smaller sample size in individual cancer datasets. In this study, we propose a deep learning model that amalgamates Transformer encoder and variational autoencoder (VAE), VTrans, employing both pre-training and fine-tuning strategies to predict the survival risk of cancer patients using microbial data. Furthermore, we highlight the potential of extending VTrans to integrate microbial multi-omics data. Our method is assessed on three distinct cancer datasets from The Cancer Genome Atlas Program, and the research findings demonstrated that (1) VTrans excels in terms of performance compared to conventional machine learning and other deep learning models. (2) The utilization of pretraning significantly enhances its performance. (3) In contrast to positional encoding, employing VAE encoding proves to be more effective in enriching data representation. (4) Using the idea of saliency map, it is possible to observe which microbes have a high contribution to the classification results. These results demonstrate the effectiveness of VTrans in prediting patient survival risk. Source code and all datasets used in this paper are available at https://github.com/wenwenmin/VTrans and https://doi.org/10.5281/zenodo.14166580 .