Assessing the Capacity of Transformer to Abstract Syntactic Representations: A Contrastive Analysis Based on Long-distance Agreement

• Published in: Transactions of the Association for Computational Linguistics Vol. 11; pp. 18 - 33
• Main Authors: Li, Bingzhi, Wisniewski, Guillaume, Crabbé, Benoît
• Format: Journal Article
• Language: English
• Published: One Broadway, 12th Floor, Cambridge, Massachusetts 02142, USA MIT Press 12.01.2023; MIT Press Journals, The; The MIT Press
• Subjects: Anaphora; Artificial Intelligence; Computational linguistics; Computer Science; Contrastive analysis; Document and Text Processing; French language; Grammatical agreement; Grammatical subject; Linguistics; Modelling; Neural networks; Representations; Syntactic analysis; Syntactic movement; Transformers; Verbs

Many studies have shown that transformers are able to predict subject-verb agreement, demonstrating their ability to uncover an abstract representation of the sentence in an unsupervised way. Recently, Li et al. ( ) found that transformers were also able to predict the object-past participle agreeme...