Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents

Document-level neural machine translation (DocNMT) achieves coherent translations by incorporating cross-sentence context. However, for most language pairs there's a shortage of parallel documents, although parallel sentences are readily available. In this paper, we study whether and how contex...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Zhang, Biao, Bapna, Ankur, Johnson, Melvin, Dabirmoghaddam, Ali, Arivazhagan, Naveen, Firat, Orhan
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 17.05.2022
Subjects
Languages
Machine translation
Modelling
Multilingualism
Schedules
Sentences
Teachers
Online AccessGet full text

Cover

More Information
Summary:Document-level neural machine translation (DocNMT) achieves coherent translations by incorporating cross-sentence context. However, for most language pairs there's a shortage of parallel documents, although parallel sentences are readily available. In this paper, we study whether and how contextual modeling in DocNMT is transferable via multilingual modeling. We focus on the scenario of zero-shot transfer from teacher languages with document level data to student languages with no documents but sentence level data, and for the first time treat document-level translation as a transfer learning problem. Using simple concatenation-based DocNMT, we explore the effect of 3 factors on the transfer: the number of teacher languages with document level data, the balance between document and sentence level data at training, and the data condition of parallel documents (genuine vs. backtranslated). Our experiments on Europarl-7 and IWSLT-10 show the feasibility of multilingual transfer for DocNMT, particularly on document-specific metrics. We observe that more teacher languages and adequate data balance both contribute to better transfer quality. Surprisingly, the transfer is less sensitive to the data condition, where multilingual DocNMT delivers decent performance with either backtranslated or genuine document pairs.
ISSN:2331-8422