Recurrent Flow-Guided Semantic Forecasting

Understanding the world around us and making decisions about the future is a critical component to human intelligence. As autonomous systems continue to develop, their ability to reason about the future will be the key to their success. Semantic anticipation is a relatively under-explored area for w...

Full description

Saved in:
Bibliographic Details
Published in2019 IEEE Winter Conference on Applications of Computer Vision (WACV) pp. 1703 - 1712
Main Authors Terwilliger, Adam, Brazil, Garrick, Liu, Xiaoming
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.01.2019
Subjects
Aggregates
Feature extraction
Forecasting
Motion segmentation
Predictive models
Semantics
Task analysis
Online AccessGet full text

Cover

More Information
Summary:Understanding the world around us and making decisions about the future is a critical component to human intelligence. As autonomous systems continue to develop, their ability to reason about the future will be the key to their success. Semantic anticipation is a relatively under-explored area for which autonomous vehicles could take advantage of (e.g., forecasting pedestrian trajectories). Motivated by the need for real-time prediction in autonomous systems, we propose to decompose the challenging semantic forecasting task into two subtasks: current frame segmentation and future optical flow prediction. Through this decomposition, we built an efficient, effective, low overhead model with three main components: flow prediction network, feature-flow aggregation LSTM, and end-to-end learnable warp layer. Our proposed method achieves state-of-the-art accuracy on short-term and moving objects semantic forecasting while simultaneously reducing model parameters by up to 95% and increasing efficiency by greater than 40x.
DOI:10.1109/WACV.2019.00186