Overview of the CLEF–2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake News

We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, English, Spanish, and Turkish. Task 1 asks to predict which posts in a Twitter...

Full description

Saved in:
Bibliographic Details
Published inExperimental IR Meets Multilinguality, Multimodality, and Interaction Vol. 12880; pp. 264 - 291
Main Authors Nakov, Preslav, Da San Martino, Giovanni, Elsayed, Tamer, Barrón-Cedeño, Alberto, Míguez, Rubén, Shaar, Shaden, Alam, Firoj, Haouari, Fatima, Hasanain, Maram, Mansour, Watheq, Hamdan, Bayan, Ali, Zien Sheikh, Babulkov, Nikolay, Nikolov, Alex, Shahi, Gautam Kishore, Struß, Julia Maria, Mandl, Thomas, Kutlu, Mucahid, Kartal, Yavuz Selim
Format Book Chapter
LanguageEnglish
Published Switzerland Springer International Publishing AG 2021
Springer International Publishing
SeriesLecture Notes in Computer Science
Subjects
Check-worthiness estimation
COVID-19
Disinformation
Fact-checking
Fake news detection
Misinformation
Verified claim retrieval
Online AccessGet full text
ISBN3030852504
9783030852504
ISSN0302-9743
1611-3349
DOI10.1007/978-3-030-85251-1_19

Cover

More Information
Summary:We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, English, Spanish, and Turkish. Task 1 asks to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics (in all five languages). Task 2 asks to determine whether a claim in a tweet can be verified using a set of previously fact-checked claims (in Arabic and English). Task 3 asks to predict the veracity of a news article and its topical domain (in English). The evaluation is based on mean average precision or precision at rank k for the ranking tasks, and macro-F1 $$_1$$ for the classification tasks. This was the most popular CLEF-2021 lab in terms of team registrations: 132 teams. Nearly one-third of them participated: 15, 5, and 25 teams submitted official runs for tasks 1, 2, and 3, respectively.
Bibliography:Original Abstract: We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting tasks related to factuality, and covers Arabic, Bulgarian, English, Spanish, and Turkish. Task 1 asks to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics (in all five languages). Task 2 asks to determine whether a claim in a tweet can be verified using a set of previously fact-checked claims (in Arabic and English). Task 3 asks to predict the veracity of a news article and its topical domain (in English). The evaluation is based on mean average precision or precision at rank k for the ranking tasks, and macro-F1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$_1$$\end{document} for the classification tasks. This was the most popular CLEF-2021 lab in terms of team registrations: 132 teams. Nearly one-third of them participated: 15, 5, and 25 teams submitted official runs for tasks 1, 2, and 3, respectively.
B. Hamdan—Independent Researcher.
ISBN:3030852504
9783030852504
ISSN:0302-9743
1611-3349
DOI:10.1007/978-3-030-85251-1_19