A Multimodal Model for Predicting Conversational Feedbacks

• Published in: Text, Speech, and Dialogue Vol. 12848; pp. 537 - 549
• Main Authors: Boudin, Auriane, Bertrand, Roxane, Rauzy, Stéphane, Ochs, Magalie, Blache, Philippe
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2021; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Corpus study; Feedback; Linguistic interaction; Statistical model
• ISBN: 303083526X; 9783030835262
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-030-83527-9_46

We propose in this paper a statistical model in the perspective of predicting listener’s feedbacks in a conversation. The first contribution of the paper is a study of the prediction of all feedbacks, including those in overlap with the speaker with a good accuracy. Existing model are good at predicting feedbacks during a pause, but reach a very low success level for all feedbacks. We give in this paper a first step towards this complex problem. The second contribution is a model predicting precisely the type of the feedback (generic vs. specific) as well as other specific features (valence expectation) useful in particular for generating feedbacks in dialogue systems. This work relies on an original corpus.