A review on the applications of Transformer-based language models for nucleotide sequence analysis

• Published in: Computational and structural biotechnology journal Vol. 27; pp. 1244 - 1254
• Main Authors: Ghosh, Nimisha, Santoni, Daniele, Saha, Indrajit, Felici, Giovanni
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 2025; Research Network of Computational and Structural Biotechnology
• Subjects: Bioinformatics; DNA/RNA sequences; Natural language processing; Nucleotide sequences; Review; Transformers; Nucleotide sequences; Transformers; DNA/RNA sequences; Natural language processing; Bioinformatics
• ISSN: 2001-0370; 2001-0370
• DOI: 10.1016/j.csbj.2025.03.024

Transformer-based language models are making an impact in the field of Natural Language Processing (NLP). As relevant parallels can be drawn between biological sequences and natural languages, the models used in NLP can be easily extended and adapted for applications in bioinformatics. This paper introduces the recent developments of Transformer-based models in the context of nucleotide sequences. We have reviewed and analysed a large number of application-based papers on this subject, giving evidence of the main characterizing features and to the different approaches that may be adopted to customize such powerful computational machines. Besides discussing what Transformers do and may do for the analysis of biological sequences, we also provide an overview of what Transformers are and why they work. We believe this review will help the scientific community in understanding the application of Transformer-based language models to nucleotide sequences, and that will motivate the readers to build on idea of Transformers as well as the discussed methodologies to tackle different problems in the field of bioinformatics. •Major developments of Transformer-based models for nucleotide sequences.•A general idea of Transformers that is easy to understand even for beginners.•Challenges and future directions provided in details to benefit research community.