Time-Aware Language Models as Temporal Knowledge Bases

• Published in: Transactions of the Association for Computational Linguistics Vol. 10; pp. 257 - 273
• Main Authors: Dhingra, Bhuwan, Cole, Jeremy R., Eisenschlos, Julian Martin, Gillick, Daniel, Eisenstein, Jacob, Cohen, William W.
• Format: Journal Article
• Language: English
• Published: One Broadway, 12th Floor, Cambridge, Massachusetts 02142, USA MIT Press 18.03.2022; MIT Press Journals, The; The MIT Press
• Subjects: Calibration; Computational linguistics; Computerized corpora; Datasets; Knowledge; Knowledge bases (artificial intelligence); Language; Language modeling; Linguistics; Time periods

Many facts come with an expiration date, from the name of the President to the basketball team Lebron James plays for. However, most language models (LMs) are trained on snapshots of data collected at a specific moment in time. This can limit their utility, especially in the closed-book setting wher...