Layout-Induced Video Representation for Recognizing Agent-in-Place Actions

We address scene layout modeling for recognizing agent-in-place actions, which are actions associated with agents who perform them and the places where they occur, in the context of outdoor home surveillance. We introduce a novel representation to model the geometry and topology of scene layouts so...

Full description

Saved in:
Bibliographic Details
Published in2019 IEEE/CVF International Conference on Computer Vision (ICCV) pp. 1262 - 1272
Main Authors Yu, Ruichi, Wang, Hongcheng, Li, Ang, Zheng, Jingxiao, Morariu, Vlad, Davis, Larry
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.10.2019
Subjects
Aggregates
Feature extraction
Layout
Semantics
Surveillance
Three-dimensional displays
Transforms
Online AccessGet full text

Cover

More Information
Summary:We address scene layout modeling for recognizing agent-in-place actions, which are actions associated with agents who perform them and the places where they occur, in the context of outdoor home surveillance. We introduce a novel representation to model the geometry and topology of scene layouts so that a network can generalize from the layouts observed in the training scenes to unseen scenes in the test set. This Layout-Induced Video Representation (LIVR) abstracts away low-level appearance variance and encodes geometric and topological relationships of places to explicitly model scene layout. LIVR partitions the semantic features of a scene into different places to force the network to learn generic place-based feature descriptions which are independent of specific scene layouts; then, LIVR dynamically aggregates features based on connectivities of places in each specific scene to model its layout. We introduce a new Agent-in-Place Action (APA) dataset to show that our method allows neural network models to generalize significantly better to unseen scenes.
ISSN:2380-7504
DOI:10.1109/ICCV.2019.00135