MMMT-IF: A Challenging Multimodal Multi-Turn Instruction Following Benchmark

• Main Authors: Epstein, Elliot L, Yao, Kaisheng, Li, Jing, Bai, Xinyi, Palangi, Hamid
• Format: Journal Article
• Language: English
• Published: 26.09.2024
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computation and Language; Computer Science - Learning

Evaluating instruction following capabilities for multimodal, multi-turn dialogue is challenging. With potentially multiple instructions in the input model context, the task is time-consuming for human raters and we show LLM based judges are biased towards answers from the same model. We propose MMM...