Learning Semantic Textual Similarity via Topic-informed Discrete Latent Variables

Recently, discrete latent variable models have received a surge of interest in both Natural Language Processing (NLP) and Computer Vision (CV), attributed to their comparable performance to the continuous counterparts in representation learning, while being more interpretable in their predictions. I...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Yu, Erxin, Du, Lan, Yuan, Jin, Zhepei Wei, Chang, Yi
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 07.11.2022
Subjects
Computer vision
English language
Learning
Natural language processing
Representations
Semantics
Similarity
Vector quantization
Online AccessGet full text

Cover

More Information
Summary:Recently, discrete latent variable models have received a surge of interest in both Natural Language Processing (NLP) and Computer Vision (CV), attributed to their comparable performance to the continuous counterparts in representation learning, while being more interpretable in their predictions. In this paper, we develop a topic-informed discrete latent variable model for semantic textual similarity, which learns a shared latent space for sentence-pair representation via vector quantization. Compared with previous models limited to local semantic contexts, our model can explore richer semantic information via topic modeling. We further boost the performance of semantic similarity by injecting the quantized representation into a transformer-based language model with a well-designed semantic-driven attention mechanism. We demonstrate, through extensive experiments across various English language datasets, that our model is able to surpass several strong neural baselines in semantic textual similarity tasks.
ISSN:2331-8422