MELO: An Evaluation Benchmark for Multilingual Entity Linking of Occupations

We present the Multilingual Entity Linking of Occupations (MELO) Benchmark, a new collection of 48 datasets for evaluating the linking of entity mentions in 21 languages to the ESCO Occupations multilingual taxonomy. MELO was built using high-quality, pre-existent human annotations. We conduct exper...

Full description

Saved in:
Bibliographic Details
Main Authors Retyk, Federico, Gasco, Luis, Carrino, Casimiro Pio, Deniz, Daniel, Zbib, Rabih
Format Journal Article
LanguageEnglish
Published 10.10.2024
Subjects
Computer Science - Computation and Language
Online AccessGet full text

Cover

More Information
Summary:We present the Multilingual Entity Linking of Occupations (MELO) Benchmark, a new collection of 48 datasets for evaluating the linking of entity mentions in 21 languages to the ESCO Occupations multilingual taxonomy. MELO was built using high-quality, pre-existent human annotations. We conduct experiments with simple lexical models and general-purpose sentence encoders, evaluated as bi-encoders in a zero-shot setup, to establish baselines for future research. The datasets and source code for standardized evaluation are publicly available at https://github.com/Avature/melo-benchmark
DOI:10.48550/arxiv.2410.08319