Large language models help facilitate the automated synthesis of information on potential pest controllers

• Published in: bioRxiv
• Main Authors: Scheepens, Daan, Millard, Joseph, Farrell, Maxwell, Newbold, Tim
• Format: Paper
• Language: English
• Published: Cold Spring Harbor Cold Spring Harbor Laboratory Press 15.01.2024; Cold Spring Harbor Laboratory
• Edition: 1.1
• Subjects: Automation; Biodiversity; Data mining; Deep learning; Ecology; Information processing; Labeling; Language; Pest control; Predators; Taxonomy; Text Mining; GPT-4; Information Synthesis; Large Language Models
• ISSN: 2692-8205; 2692-8205
• DOI: 10.1101/2024.01.12.575330

The body of ecological literature, which informs much of our knowledge of the global loss of biodiversity, has been experiencing rapid growth in recent decades. The increasing difficulty to synthesise this literature manually has simultaneously resulted in a growing demand for automated text mining methods. Within the domain of deep learning, large language models (LLMs) have been the subject of considerable attention in recent years by virtue of great leaps in progress and a wide range of potential applications, however, quantitative investigation into their potential in ecology has so far been lacking. In this work, we analyse the ability of GPT-4 to extract information about invertebrate pests and pest controllers from abstracts of a body of literature on biological pest control, using a bespoke, zero-shot prompt. Our results show that the performance of GPT-4 is highly competitive with other state-of-the-art tools used for taxonomic named entity recognition and geographic location extraction tasks. On a held-out test set, we show that species and geographic locations are extracted with F1-scores of 99.8% and 95.3%, respectively, and highlight that the model is able to distinguish very effectively between the primary roles of interest (predators, parasitoids and pests). Moreover, we demonstrate the ability of the model to effectively extract and predict taxonomic information across various taxonomic ranks, and to automatically correct spelling mistakes. However, we do report a small number of cases of fabricated information (hallucinations). As a result of the current lack of specialised, pre-trained ecological language models, general-purpose LLMs may provide a promising way forward in ecology. Combined with tailored prompt engineering, such models can be employed for a wide range of text mining tasks in ecology, with the potential to greatly reduce time spent on manual screening and labelling of the literature.Competing Interest StatementThe authors have declared no competing interest.Footnotes* https://github.com/dscheepens/GPT4-pest-controllers