Evolutionary-scale prediction of atomic-level protein structure with a language model

• Published in: Science (American Association for the Advancement of Science) Vol. 379; no. 6637; pp. 1123 - 1130
• Main Authors: Lin, Zeming, Akin, Halil, Rao, Roshan, Hie, Brian, Zhu, Zhongkai, Lu, Wenting, Smetanin, Nikita, Verkuil, Robert, Kabeli, Ori, Shmueli, Yaniv, dos Santos Costa, Allan, Fazel-Zarandi, Maryam, Sercu, Tom, Candido, Salvatore, Rives, Alexander
• Format: Journal Article
• Language: English
• Published: United States The American Association for the Advancement of Science 17.03.2023
• Subjects: Amino Acid Sequence; Atomic structure; Evolution, Molecular; Language; Machine Learning; Metagenomics; Protein Conformation; Protein structure; Proteins; Proteins - chemistry; Sequence Analysis, Protein
• ISSN: 0036-8075; 1095-9203; 1095-9203
• DOI: 10.1126/science.ade2574

Recent advances in machine learning have leveraged evolutionary information in multiple sequence alignments to predict protein structure. We demonstrate direct inference of full atomic-level protein structure from primary sequence using a large language model. As language models of protein sequences are scaled up to 15 billion parameters, an atomic-resolution picture of protein structure emerges in the learned representations. This results in an order-of-magnitude acceleration of high-resolution structure prediction, which enables large-scale structural characterization of metagenomic proteins. We apply this capability to construct the ESM Metagenomic Atlas by predicting structures for >617 million metagenomic protein sequences, including >225 million that are predicted with high confidence, which gives a view into the vast breadth and diversity of natural proteins. Machine learning methods for protein structure prediction have taken advantage of the evolutionary information present in multiple sequence alignments to derive accurate structural information, but predicting structure accurately from a single sequence is much more difficult. Lin et al . trained transformer protein language models with up to 15 billion parameters on experimental and high-quality predicted structures and found that information about atomic-level structure emerged in the model as it was scaled up. They created ESMFold, a sequence-to-structure predictor that is nearly as accurate as alignment-based methods and considerably faster. The increased speed permitted the generation of a database, the ESM Metagenomic Atlas, containing more than 600 million metagenomic proteins. —MAF A protein language model enables structure prediction and analysis of more than 600 million metagenomic proteins.