A fingerprints based molecular property prediction method using the BERT model

• Published in: Journal of cheminformatics Vol. 14; no. 1; pp. 71 - 13
• Main Authors: Wen, Naifeng, Liu, Guanqun, Zhang, Jie, Zhang, Rubo, Fu, Yating, Han, Xu
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 21.10.2022; BioMed Central Ltd; Springer Nature B.V; BMC
• Subjects: Analysis; Artificial neural networks; Chemistry; Chemistry and Materials Science; Classification; Coders; Computational Biology/Bioinformatics; Computational linguistics; Computer Applications in Chemistry; Deep learning; Deep neural network; Documentation and Information in Chemistry; Drug discovery; Embedding; Feature extraction; Fingerprints; Language processing; Machine learning; Methods; Molecular modelling; Molecular properties; Molecular property prediction; Molecular representation; Natural language interfaces; Neural networks; Open access publishing; Pre-training language model; Prediction models; Quantitative structure-activity relationships; R&D; Representations; Research & development; Semantics; Theoretical and Computational Chemistry; Taiwan; Quantitative structure-activity relationships; Molecular representation; Pre-training language model; Deep neural network; Molecular property prediction
• ISSN: 1758-2946; 1758-2946
• DOI: 10.1186/s13321-022-00650-3

Molecular property prediction (MPP) is vital in drug discovery and drug reposition. Deep learning-based MPP models capture molecular property-related features from various molecule representations. In this paper, we propose a molecule sequence embedding and prediction model facing with MPP task. We pre-trained a bi-directional encoder representations from Transformers (BERT) encoder to obtain the semantic representation of compound fingerprints, called Fingerprints-BERT (FP-BERT), in a self-supervised learning manner. Then, the encoded molecular representation by the FP-BERT is input to the convolutional neural network (CNN) to extract higher-level abstract features, and the predicted properties of the molecule are finally obtained through fully connected layer for distinct classification or regression MPP tasks. Comparison with the baselines shows that the proposed model achieves high prediction performance on all of the classification tasks and regression tasks.