The Defeat of the Winograd Schema Challenge
The Winograd Schema Challenge - a set of twin sentences involving pronoun reference disambiguation that seem to require the use of commonsense knowledge - was proposed by Hector Levesque in 2011. By 2019, a number of AI systems, based on large pre-trained transformer-based language models and fine-t...
Saved in:
Main Authors | , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
07.01.2022
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | The Winograd Schema Challenge - a set of twin sentences involving pronoun
reference disambiguation that seem to require the use of commonsense knowledge
- was proposed by Hector Levesque in 2011. By 2019, a number of AI systems,
based on large pre-trained transformer-based language models and fine-tuned on
these kinds of problems, achieved better than 90% accuracy. In this paper, we
review the history of the Winograd Schema Challenge and discuss the lasting
contributions of the flurry of research that has taken place on the WSC in the
last decade. We discuss the significance of various datasets developed for WSC,
and the research community's deeper understanding of the role of surrogate
tasks in assessing the intelligence of an AI system. |
---|---|
DOI: | 10.48550/arxiv.2201.02387 |