SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalog

We present SemEval-2025 Task 5: LLMs4Subjects, a shared task on automated subject tagging for scientific and technical records in English and German using the GND taxonomy. Participants developed LLM-based systems to recommend top-k subjects, evaluated through quantitative metrics (precision, recall...

Full description

Saved in:
Bibliographic Details
Main Authors D'Souza, Jennifer, Sadruddin, Sameer, Israel, Holger, Begoin, Mathias, Slawig, Diana
Format Journal Article
LanguageEnglish
Published 09.04.2025
Subjects
Computer Science - Artificial Intelligence
Computer Science - Computation and Language
Computer Science - Digital Libraries
Computer Science - Learning
Online AccessGet full text

Cover

More Information
Summary:We present SemEval-2025 Task 5: LLMs4Subjects, a shared task on automated subject tagging for scientific and technical records in English and German using the GND taxonomy. Participants developed LLM-based systems to recommend top-k subjects, evaluated through quantitative metrics (precision, recall, F1-score) and qualitative assessments by subject specialists. Results highlight the effectiveness of LLM ensembles, synthetic data generation, and multilingual processing, offering insights into applying LLMs for digital library classification.
DOI:10.48550/arxiv.2504.07199