StereoKG: Data-Driven Knowledge Graph Construction for Cultural Knowledge and Stereotypes

Analyzing ethnic or religious bias is important for improving fairness, accountability, and transparency of natural language processing models. However, many techniques rely on human-compiled lists of bias terms, which are expensive to create and are limited in coverage. In this study, we present a...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Deshpande, Awantee, Ruiter, Dana, Mosbach, Marius, Klakow, Dietrich
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 27.05.2022
Subjects
Bias
Knowledge representation
Natural language processing
Stereotypes
Online AccessGet full text

Cover

More Information
Summary:Analyzing ethnic or religious bias is important for improving fairness, accountability, and transparency of natural language processing models. However, many techniques rely on human-compiled lists of bias terms, which are expensive to create and are limited in coverage. In this study, we present a fully data-driven pipeline for generating a knowledge graph (KG) of cultural knowledge and stereotypes. Our resulting KG covers 5 religious groups and 5 nationalities and can easily be extended to include more entities. Our human evaluation shows that the majority (59.2%) of non-singleton entries are coherent and complete stereotypes. We further show that performing intermediate masked language model training on the verbalized KG leads to a higher level of cultural awareness in the model and has the potential to increase classification performance on knowledge-crucial samples on a related task, i.e., hate speech detection.
ISSN:2331-8422