VAIV bio-discovery service using transformer model and retrieval augmented generation

• Published in: BMC bioinformatics Vol. 25; no. 1; pp. 273 - 25
• Main Authors: Kim, Seonho, Yoon, Juntae
• Format: Journal Article
• Language: English
• Published: London BioMed Central 21.08.2024; BioMed Central Ltd; BMC
• Subjects: Algorithms; Attention task; Big data management in biological domains; Bioinformatics; Biomedical and Life Sciences; Biomedical interaction extraction; Chatbots; Chemical compounds; Computational Biology/Bioinformatics; Computational linguistics; Computer Appl. in Life Sciences; Data mining; Data Mining - methods; Deep learning; Documents; Drug discovery; Drug interaction; Drugs; Engine blocks; Genes; Information Storage and Retrieval - methods; Keywords; Knowledge discovery; Knowledge Discovery - methods; Language; Language processing; Large language models; Life Sciences; LLM; Machine Learning; Medical research; Medical Subject Headings-MeSH; Medicine, Experimental; Methods; Microarrays; Natural language; Natural language interfaces; Natural Language Processing; Neural Networks, Computer; Proteins; PubMed; Queries; Query languages; RAG; Retrieval; Search Engine; Search engines; Semantics; Speech recognition; Technology application; Text mining; Transformer; Transformers; Trends; Unstructured data; South Korea; Text mining; T5; Neural search; Transformer; Biomedical interaction extraction; RAG; Embedding; Natural language processing; LLM
• ISSN: 1471-2105; 1471-2105
• DOI: 10.1186/s12859-024-05903-6

Background There has been a considerable advancement in AI technologies like LLM and machine learning to support biomedical knowledge discovery. Main body We propose a novel biomedical neural search service called ‘VAIV Bio-Discovery’, which supports enhanced knowledge discovery and document search on unstructured text such as PubMed. It mainly handles with information related to chemical compound/drugs, gene/proteins, diseases, and their interactions (chemical compounds/drugs-proteins/gene including drugs-targets, drug-drug, and drug-disease). To provide comprehensive knowledge, the system offers four search options: basic search, entity and interaction search, and natural language search. We employ T5slim_dec, which adapts the autoregressive generation task of the T5 (text-to-text transfer transformer) to the interaction extraction task by removing the self-attention layer in the decoder block. It also assists in interpreting research findings by summarizing the retrieved search results for a given natural language query with Retrieval Augmented Generation (RAG). The search engine is built with a hybrid method that combines neural search with the probabilistic search, BM25. Conclusion As a result, our system can better understand the context, semantics and relationships between terms within the document, enhancing search accuracy. This research contributes to the rapidly evolving biomedical field by introducing a new service to access and discover relevant knowledge.