Trilingual conversational intent decoding for response retrieval

• Published in: Knowledge and information systems Vol. 66; no. 1; pp. 535 - 556
• Main Authors: Godslove, Julius Femi, Nayak, Ajit Kumar
• Format: Journal Article
• Language: English
• Published: London Springer London 2024; Springer Nature B.V
• Subjects: Algorithms; Computer Science; Context; Conversation; Data Mining and Knowledge Discovery; Database Management; Datasets; Dialects; English language; Hindi language; Information retrieval; Information Storage and Retrieval; Information Systems and Communication Service; Information Systems Applications (incl.Internet); IT in Business; Modelling; Natural language processing; Performance measurement; Regular Paper; Yoruba language; Information retrieval; Trilingual conversational question answering; Natural language processing; Natural language understanding; Natural language inferencing
• ISSN: 0219-1377; 0219-3116
• DOI: 10.1007/s10115-023-01972-w

The rich diversity of human language allows speakers to seamlessly transition between multiple languages during conversations. While humans have the remarkable ability to become proficient in multiple languages in a short period, developing machines that can converse in multiple natural languages with an understanding of diverse dialects requires sophisticated Natural Language Processing (NLP) techniques such as dialect recognition and intent extraction. This facilitates mutual understanding between parties who use phrases, sentences, words, or expressions from multiple languages within a single context. The work in this paper, propose a trilingual approach to multi-dialect conversation modeling within the same conversational session and context for a mix of English, Hindi–English text, Hindi–Devanagari text and Yoruba text. The model identifies the language used and determines the intent behind a query to respond in the same dialect. Our model is capable of detecting the end of a conversation, and it also detects the predominant dialect and responds accordingly in scenarios where a user’s input query contains a mix of languages. This approach is particularly useful in situations where there is limited data available for multilingual or trilingual conversation tasks based on Intent Detection (ID). We evaluate our proposed pipeline and model on three benchmark ID datasets and a trilingual dialogue dataset for response retrieval by intent decoding. Our model outperforms existing approaches in terms of performance metrics and has faster training time. Moreover, our trilingual approach to multi-dialect conversation modeling provides a versatile tool for efficient and effective inter-dialect conversational automation, even when dealing with large datasets, with minimal parameters and low resource overhead. The lightweight architectural pipeline and efficient algorithms used in our model contribute to its high performance and versatility.