Pre-Training Multimodal Hallucination Detectors with Corrupted Grounding Data
Multimodal language models can exhibit hallucinations in their outputs, which limits their reliability. The ability to automatically detect these errors is important for mitigating them, but has been less explored and existing efforts do not localize hallucinations, instead framing this as a classif...
Saved in:
Main Authors | , , |
---|---|
Format | Journal Article |
Language | English |
Published |
30.08.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Multimodal language models can exhibit hallucinations in their outputs, which
limits their reliability. The ability to automatically detect these errors is
important for mitigating them, but has been less explored and existing efforts
do not localize hallucinations, instead framing this as a classification task.
In this work, we first pose multimodal hallucination detection as a sequence
labeling task where models must localize hallucinated text spans and present a
strong baseline model. Given the high cost of human annotations for this task,
we propose an approach to improve the sample efficiency of these models by
creating corrupted grounding data, which we use for pre-training. Leveraging
phrase grounding data, we generate hallucinations to replace grounded spans and
create hallucinated text. Experiments show that pre-training on this data
improves sample efficiency when fine-tuning, and that the learning signal from
the grounding data plays an important role in these improvements. |
---|---|
AbstractList | Multimodal language models can exhibit hallucinations in their outputs, which
limits their reliability. The ability to automatically detect these errors is
important for mitigating them, but has been less explored and existing efforts
do not localize hallucinations, instead framing this as a classification task.
In this work, we first pose multimodal hallucination detection as a sequence
labeling task where models must localize hallucinated text spans and present a
strong baseline model. Given the high cost of human annotations for this task,
we propose an approach to improve the sample efficiency of these models by
creating corrupted grounding data, which we use for pre-training. Leveraging
phrase grounding data, we generate hallucinations to replace grounded spans and
create hallucinated text. Experiments show that pre-training on this data
improves sample efficiency when fine-tuning, and that the learning signal from
the grounding data plays an important role in these improvements. |
Author | Whitehead, Spencer Hendryx, Sean Phillips, Jacob |
Author_xml | – sequence: 1 givenname: Spencer surname: Whitehead fullname: Whitehead, Spencer – sequence: 2 givenname: Jacob surname: Phillips fullname: Phillips, Jacob – sequence: 3 givenname: Sean surname: Hendryx fullname: Hendryx, Sean |
BackLink | https://doi.org/10.48550/arXiv.2409.00238$$DView paper in arXiv |
BookMark | eNqFjrsOgjAUhjvo4O0BnOwLgJVLgjOoLCQO7M0JVD1Jacmh9fL2BuLu9C3_l_9bspmxRjG2PYgwydJU7IHe-AyjRBxDIaI4W7DqSiqoCdCgufPKa4edbUHzErT2DRpwaA0vlFONszTwF7oHzy2R751q-YWsN-3oFuBgzeY30IPa_Lhiu_OpzstgOpY9YQf0kWOAnALi_4svwHQ9rg |
ContentType | Journal Article |
Copyright | http://creativecommons.org/licenses/by/4.0 |
Copyright_xml | – notice: http://creativecommons.org/licenses/by/4.0 |
DBID | AKY GOX |
DOI | 10.48550/arxiv.2409.00238 |
DatabaseName | arXiv Computer Science arXiv.org |
DatabaseTitleList | |
Database_xml | – sequence: 1 dbid: GOX name: arXiv.org url: http://arxiv.org/find sourceTypes: Open Access Repository |
DeliveryMethod | fulltext_linktorsrc |
ExternalDocumentID | 2409_00238 |
GroupedDBID | AKY GOX |
ID | FETCH-arxiv_primary_2409_002383 |
IEDL.DBID | GOX |
IngestDate | Thu Sep 05 12:20:22 EDT 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-arxiv_primary_2409_002383 |
OpenAccessLink | https://arxiv.org/abs/2409.00238 |
ParticipantIDs | arxiv_primary_2409_00238 |
PublicationCentury | 2000 |
PublicationDate | 2024-08-30 |
PublicationDateYYYYMMDD | 2024-08-30 |
PublicationDate_xml | – month: 08 year: 2024 text: 2024-08-30 day: 30 |
PublicationDecade | 2020 |
PublicationYear | 2024 |
Score | 3.8737671 |
SecondaryResourceType | preprint |
Snippet | Multimodal language models can exhibit hallucinations in their outputs, which
limits their reliability. The ability to automatically detect these errors is... |
SourceID | arxiv |
SourceType | Open Access Repository |
SubjectTerms | Computer Science - Computation and Language Computer Science - Computer Vision and Pattern Recognition |
Title | Pre-Training Multimodal Hallucination Detectors with Corrupted Grounding Data |
URI | https://arxiv.org/abs/2409.00238 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwdV1NSwMxEB3anryIRaV-VHPwurhNNm33WPrhUqh6qLC3JVkTKFRb0l3x5zuZrOilt5AMYZhA3uQl8wLwYNQwMVKkXnyQR4kxIhqnUmJLx6XUGjdAem3xPMzekmUu8xaw31oY5b43X0EfWB8eEW68nCTCShvanPsnW08vebicJCmuxv7PDnNM6voHEoszOG2yOzYJy9GFlvk8h9WrM9G6-YqBUcXrx-4dzTK13dblJvBxbGYqYtAPzHOjbLpzrt5jPsg8PUS1J2ymKnUB94v5eppF5ECxD2oRhfetIN_EJXTwTG96wLQYW2vFQAxUmdjUaoxMyUc6FlLwUaqvoHdsluvjQzdwwhFzifKMb6FTudr0ETMrfUeB-wHokXGE |
link.rule.ids | 228,230,786,891 |
linkProvider | Cornell University |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Pre-Training+Multimodal+Hallucination+Detectors+with+Corrupted+Grounding+Data&rft.au=Whitehead%2C+Spencer&rft.au=Phillips%2C+Jacob&rft.au=Hendryx%2C+Sean&rft.date=2024-08-30&rft_id=info:doi/10.48550%2Farxiv.2409.00238&rft.externalDocID=2409_00238 |