To Diverge or Not to Diverge: A Morphosyntactic Perspective on Machine Translation vs Human Translation

• Published in: Transactions of the Association for Computational Linguistics Vol. 12; pp. 355 - 371
• Main Authors: Luo, Jiaming, Cherry, Colin, Foster, George
• Format: Journal Article
• Language: English
• Published: One Broadway, 12th Floor, Cambridge, Massachusetts 02142, USA MIT Press 08.04.2024; The MIT Press
• ISSN: 2307-387X; 2307-387X
• DOI: 10.1162/tacl_a_00645

We conduct a large-scale fine-grained comparative analysis of machine translations (MTs) against human translations (HTs) through the lens of morphosyntactic divergence. Across three language pairs and two types of divergence defined as the structural difference between the source and the target, MT is consistently more conservative than HT, with less morphosyntactic diversity, more convergent patterns, and more one-to-one alignments. Through analysis on different decoding algorithms, we attribute this discrepancy to the use of beam search that biases MT towards more convergent patterns. This bias is most amplified when the convergent pattern appears around 50% of the time in training data. Lastly, we show that for a majority of morphosyntactic divergences, their presence in HT is correlated with decreased MT performance, presenting a greater challenge for MT systems.