NLM-Gene, a richly annotated gold standard dataset for gene entities that addresses ambiguity and multi-species gene recognition

• Published in: Journal of biomedical informatics Vol. 118; p. 103779
• Main Authors: Islamaj, Rezarta, Wei, Chih-Hsuan, Cissel, David, Miliaras, Nicholas, Printseva, Olga, Rodionov, Oleg, Sekiya, Keiko, Ward, Janice, Lu, Zhiyong
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.06.2021
• Subjects: Animals; Biomedical Text Mining; Data Mining; Deep Learning; Drosophila melanogaster; Gene entity recognition; Genes, vif; Manual annotation; National Library of Medicine (U.S.); Natural language processing; PubMed; Rats; United States; United States; Gene entity recognition; Natural language processing; Manual annotation; Biomedical Text Mining; Deep Learning
• ISSN: 1532-0464; 1532-0480
• DOI: 10.1016/j.jbi.2021.103779

[Display omitted] •Genes are one of the most searched bio-entities in biomedical literature.•NLM-Gene corpus is a novel resource of high-quality doubly-annotated articles.•NLM-Gene corpus articles are rich in bio-entities and gene mentions per article.•NLM-Gene corpus considers ambiguity of gene mentions and multiple model organisms.•NLM-Gene corpus can further improve accuracy of gene recognition algorithms. The automatic recognition of gene names and their corresponding database identifiers in biomedical text is an important first step for many downstream text-mining applications. While current methods for tagging gene entities have been developed for biomedical literature, their performance on species other than human is substantially lower due to the lack of annotation data. We therefore present the NLM-Gene corpus, a high-quality manually annotated corpus for genes developed at the US National Library of Medicine (NLM), covering ambiguous gene names, with an average of 29 gene mentions (10 unique identifiers) per document, and a broader representation of different species (including Homo sapiens, Mus musculus, Rattus norvegicus, Drosophila melanogaster, Arabidopsis thaliana, Danio rerio, etc.) when compared to previous gene annotation corpora. NLM-Gene consists of 550 PubMed abstracts from 156 biomedical journals, doubly annotated by six experienced NLM indexers, randomly paired for each document to control for bias. The annotators worked in three annotation rounds until they reached complete agreement. This gold-standard corpus can serve as a benchmark to develop & test new gene text mining algorithms. Using this new resource, we have developed a new gene finding algorithm based on deep learning which improved both on precision and recall from existing tools. The NLM-Gene annotated corpus is freely available at ftp://ftp.ncbi.nlm.nih.gov/pub/lu/NLMGene. We have also applied this tool to the entire PubMed/PMC with their results freely accessible through our web-based tool PubTator (www.ncbi.nlm.nih.gov/research/pubtator).