A fast (CNN + MCWS-transformer) based architecture for protein function prediction

• Published in: Statistical applications in genetics and molecular biology Vol. 24; no. 1
• Main Authors: Mahala, Abhipsa, Ranjan, Ashish, Priyadarshini, Rojalina, Vikram, Raj, Dansena, Prabhat
• Format: Journal Article
• Language: English
• Published: Germany De Gruyter 01.01.2025; Walter de Gruyter GmbH
• Subjects: Algorithms; Artificial neural networks; Complexity; Computational Biology - methods; Databases, Protein; fast transformer architecture; MCWS-transformer; Neural Networks, Computer; protein function prediction; protein sequence; Proteins; Proteins - chemistry; Proteins - genetics; Proteins - physiology; Sequence Analysis, Protein - methods; MCWS-transformer; protein function prediction; fast transformer architecture; protein sequence
• ISSN: 2194-6302; 1544-6115; 1544-6115
• DOI: 10.1515/sagmb-2024-0027

The transformer model for sequence mining has brought a paradigmatic shift to many domains, including biological sequence mining. However, transformers suffer from quadratic complexity, i.e., O( ), where is the sequence length, which affects the training and prediction time. Therefore, the work herein introduces a simple, generalized, and fast transformer architecture for improved protein function prediction. The proposed architecture uses a combination of CNN and global-average pooling to effectively shorten the protein sequences. The shortening process helps reduce the quadratic complexity of the transformer, resulting in the complexity of O(( /2) ). This architecture is utilized to develop PFP solution at the sub-sequence level. Furthermore, focal loss is employed to ensure balanced training for the hard-classified examples. The multi sub-sequence-based proposed solution utilizing an average-pooling layer (with stride = 2) produced improvements of +2.50 % (BP) and +3.00 % (MF) when compared to Global-ProtEnc Plus. The corresponding improvements when compared to the Lite-SeqCNN are: +4.50 % (BP) and +2.30 % (MF).