I3: Intent-Introspective Retrieval Conditioned on Instructions

Recent studies indicate that dense retrieval models struggle to perform well on a wide variety of retrieval tasks that lack dedicated training data, as different retrieval tasks often entail distinct search intents. To address this challenge, in this work we leverage instructions to flexibly describ...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Pan, Kaihang, Li, Juncheng, Wang, Wenjie, Hao Fei, Song, Hongye, Ji, Wei, Lin, Jun, Liu, Xiaozhong, Tat-Seng Chua, Tang, Siliang
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 25.04.2024
Subjects
Controllability
Image processing
Iterative methods
Large language models
Natural language
Retrieval
Training
Online AccessGet full text

Cover

More Information
Summary:Recent studies indicate that dense retrieval models struggle to perform well on a wide variety of retrieval tasks that lack dedicated training data, as different retrieval tasks often entail distinct search intents. To address this challenge, in this work we leverage instructions to flexibly describe retrieval intents and introduce I3, a unified retrieval system that performs Intent-Introspective retrieval across various tasks, conditioned on Instructions without any task-specific training. I3 innovatively incorporates a pluggable introspector in a parameter-isolated manner to comprehend specific retrieval intents by jointly reasoning over the input query and instruction, and seamlessly integrates the introspected intent into the original retrieval model for intent-aware retrieval. Furthermore, we propose progressively-pruned intent learning. It utilizes extensive LLM-generated data to train I3 phase-by-phase, embodying two key designs: progressive structure pruning and drawback extrapolation-based data refinement. Extensive experiments show that in the BEIR benchmark, I3 significantly outperforms baseline methods designed with task-specific retrievers, achieving state-of-the-art zero-shot performance without any task-specific tuning.
ISSN:2331-8422