Can humans fly? Action understanding with multiple classes of actors

Can humans fly? Emphatically no. Can cars eat? Again, absolutely not. Yet, these absurd inferences result from the current disregard for particular types of actors in action understanding. There is no work we know of on simultaneously inferring actors and actions in the video, not to mention a datas...

Full description

Saved in:
Bibliographic Details
Published in2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp. 2264 - 2273
Main Authors Chenliang Xu, Shao-Hang Hsieh, Caiming Xiong, Corso, Jason J.
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2015
Subjects
Birds
Graphical models
Joints
Legged locomotion
Pediatrics
Semantics
Yttrium
Online AccessGet full text

Cover

More Information
Summary:Can humans fly? Emphatically no. Can cars eat? Again, absolutely not. Yet, these absurd inferences result from the current disregard for particular types of actors in action understanding. There is no work we know of on simultaneously inferring actors and actions in the video, not to mention a dataset to experiment with. Our paper hence marks the first effort in the computer vision community to jointly consider various types of actors undergoing various actions. To start with the problem, we collect a dataset of 3782 videos from YouTube and label both pixel-level actors and actions in each video. We formulate the general actor-action understanding problem and instantiate it at various granularities: both video-level single- and multiple-label actor-action recognition and pixel-level actor-action semantic segmentation. Our experiments demonstrate that inference jointly over actors and actions outperforms inference independently over them, and hence concludes our argument of the value of explicit consideration of various actors in comprehensive action understanding.
ISSN:1063-6919
DOI:10.1109/CVPR.2015.7298839