VQA: Visual Question Answering www.visualqa.org

• Published in: International journal of computer vision Vol. 123; no. 1; pp. 4 - 31
• Main Authors: Agrawal, Aishwarya, Lu, Jiasen, Antol, Stanislaw, Mitchell, Margaret, Zitnick, C. Lawrence, Parikh, Devi, Batra, Dhruv
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.05.2017; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Computer Imaging; Computer Science; Computer vision; Datasets; Human performance; Image Processing and Computer Vision; Image processing systems; Information dissemination; Language; Natural language; Natural language (computers); Natural language processing; Pattern Recognition; Pattern Recognition and Graphics; Pizza; Questioning; Studies; Tasks; Texts; Vision; Vision systems; United States > US; Visual Question Answering
• ISSN: 0920-5691; 1573-1405
• DOI: 10.1007/s11263-016-0966-6

We propose the task of free-form and open-ended Visual Question Answering (VQA). Given an image and a natural language question about the image, the task is to provide an accurate natural language answer. Mirroring real-world scenarios, such as helping the visually impaired, both the questions and answers are open-ended. Visual questions selectively target different areas of an image, including background details and underlying context. As a result, a system that succeeds at VQA typically needs a more detailed understanding of the image and complex reasoning than a system producing generic image captions. Moreover, VQA is amenable to automatic evaluation, since many open-ended answers contain only a few words or a closed set of answers that can be provided in a multiple-choice format. We provide a dataset containing ∼ 0.25 M images, ∼ 0.76 M questions, and ∼ 10 M answers ( www.visualqa.org ), and discuss the information it provides. Numerous baselines and methods for VQA are provided and compared with human performance. Our VQA demo is available on CloudCV ( http://cloudcv.org/vqa ).