Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysis
Neural metrics for machine translation (MT) evaluation have become increasingly prominent due to their superior correlation with human judgments compared to traditional lexical metrics. Researchers have therefore utilized neural metrics through quality-informed decoding strategies, achieving better...
Saved in:
Main Authors | , , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
30.09.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Neural metrics for machine translation (MT) evaluation have become
increasingly prominent due to their superior correlation with human judgments
compared to traditional lexical metrics. Researchers have therefore utilized
neural metrics through quality-informed decoding strategies, achieving better
results than likelihood-based methods. With the rise of Large Language Models
(LLMs), preference-based alignment techniques have gained attention for their
potential to enhance translation quality by optimizing model weights directly
on preferences induced by quality estimators. This study focuses on Contrastive
Preference Optimization (CPO) and conducts extensive experiments to evaluate
the impact of preference-based alignment on translation quality. Our findings
indicate that while CPO consistently outperforms Supervised Fine-Tuning (SFT)
on high-quality data with regard to the alignment metric, it may lead to
instability across downstream evaluation metrics, particularly between neural
and lexical ones. Additionally, we demonstrate that relying solely on the base
model for generating candidate translations achieves performance comparable to
using multiple external systems, while ensuring better consistency across
downstream metrics. |
---|---|
AbstractList | Neural metrics for machine translation (MT) evaluation have become
increasingly prominent due to their superior correlation with human judgments
compared to traditional lexical metrics. Researchers have therefore utilized
neural metrics through quality-informed decoding strategies, achieving better
results than likelihood-based methods. With the rise of Large Language Models
(LLMs), preference-based alignment techniques have gained attention for their
potential to enhance translation quality by optimizing model weights directly
on preferences induced by quality estimators. This study focuses on Contrastive
Preference Optimization (CPO) and conducts extensive experiments to evaluate
the impact of preference-based alignment on translation quality. Our findings
indicate that while CPO consistently outperforms Supervised Fine-Tuning (SFT)
on high-quality data with regard to the alignment metric, it may lead to
instability across downstream evaluation metrics, particularly between neural
and lexical ones. Additionally, we demonstrate that relying solely on the base
model for generating candidate translations achieves performance comparable to
using multiple external systems, while ensuring better consistency across
downstream metrics. |
Author | Hudelot, Céline Guerreiro, Nuno M Gisserot-Boukhlef, Hippolyte Rei, Ricardo Colombo, Pierre Malherbe, Emmanuel |
Author_xml | – sequence: 1 givenname: Hippolyte surname: Gisserot-Boukhlef fullname: Gisserot-Boukhlef, Hippolyte – sequence: 2 givenname: Ricardo surname: Rei fullname: Rei, Ricardo – sequence: 3 givenname: Emmanuel surname: Malherbe fullname: Malherbe, Emmanuel – sequence: 4 givenname: Céline surname: Hudelot fullname: Hudelot, Céline – sequence: 5 givenname: Pierre surname: Colombo fullname: Colombo, Pierre – sequence: 6 givenname: Nuno M surname: Guerreiro fullname: Guerreiro, Nuno M |
BackLink | https://doi.org/10.48550/arXiv.2409.20059$$DView paper in arXiv |
BookMark | eNqFjs0KgkAURmdRi_4eoFX3BbTJFGoVGUZBUYv2cqlrDoxXmRkq3z6V9q2-88FZnKHocckkxHQh_XAVRXKO5qNefhDKtR9IGa0HIjtauBrKyBDfCbZaPbkgdg29sbbgcoKYrINL5VTJ4EpIOMfWPZ3OXoyWHnAzyFZjK2xgy5AUlTLqjro5qGur7Fj0M9SWJr8didk-ue0OXleUVkYVaOq0LUu7suV_4wv7pEVF |
ContentType | Journal Article |
Copyright | http://creativecommons.org/publicdomain/zero/1.0 |
Copyright_xml | – notice: http://creativecommons.org/publicdomain/zero/1.0 |
DBID | AKY GOX |
DOI | 10.48550/arxiv.2409.20059 |
DatabaseName | arXiv Computer Science arXiv.org |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: GOX name: arXiv.org url: http://arxiv.org/find sourceTypes: Open Access Repository |
DeliveryMethod | fulltext_linktorsrc |
ExternalDocumentID | 2409_20059 |
GroupedDBID | AKY GOX |
ID | FETCH-arxiv_primary_2409_200593 |
IEDL.DBID | GOX |
IngestDate | Thu Oct 03 12:25:34 EDT 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-arxiv_primary_2409_200593 |
OpenAccessLink | https://arxiv.org/abs/2409.20059 |
ParticipantIDs | arxiv_primary_2409_20059 |
PublicationCentury | 2000 |
PublicationDate | 2024-09-30 |
PublicationDateYYYYMMDD | 2024-09-30 |
PublicationDate_xml | – month: 09 year: 2024 text: 2024-09-30 day: 30 |
PublicationDecade | 2020 |
PublicationYear | 2024 |
Score | 3.8815377 |
SecondaryResourceType | preprint |
Snippet | Neural metrics for machine translation (MT) evaluation have become
increasingly prominent due to their superior correlation with human judgments
compared to... |
SourceID | arxiv |
SourceType | Open Access Repository |
SubjectTerms | Computer Science - Computation and Language |
Title | Is Preference Alignment Always the Best Option to Enhance LLM-Based Translation? An Empirical Analysis |
URI | https://arxiv.org/abs/2409.20059 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwdV09T8MwED21nVgQCFD5voHVgqROGk8oRSkFtZQBpGxRktoQCULVhK9_z-XcAks3y7ask0_2e2efnwHOVOoEFMpK4fWzQEj-yJ1YubgwAQVEpp-ns-ZoYHLnjx7lbezFLcDVW5h08VV8WH3grDonuOHHJJ5qQ9t1m5St62lsLydZimvZ_68fcUyu-gcSwy3YXLI7DK07tqGlyx0wNxXe_37ngeFL8cRX8FT6TL8rJAqGA9qcccrLF-s3jMrnxhs4Hk_EgHBmhowpNm_tEsMSo9d5weoeuJIV2YXTYfRwNRJsWTK3MhJJY3TCRvf2oEPBvu4C5tJRga9d5atM6p6f5srkxjFeYIwh7r8P3XWjHKxvOoQNl8DY5jkcQadevOtjAtM6O-EZ_QG9ynin |
link.rule.ids | 228,230,786,891 |
linkProvider | Cornell University |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Is+Preference+Alignment+Always+the+Best+Option+to+Enhance+LLM-Based+Translation%3F+An+Empirical+Analysis&rft.au=Gisserot-Boukhlef%2C+Hippolyte&rft.au=Rei%2C+Ricardo&rft.au=Malherbe%2C+Emmanuel&rft.au=Hudelot%2C+C%C3%A9line&rft.date=2024-09-30&rft_id=info:doi/10.48550%2Farxiv.2409.20059&rft.externalDocID=2409_20059 |