SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warping

We present SMURF, a method for unsupervised learning of optical flow that improves state of the art on all benchmarks by 36% to 40% (over the prior best method UFlow) and even outperforms several supervised approaches such as PWC-Net and FlowNet2. Our method integrates architecture improvements from...

Full description

Saved in:
Bibliographic Details
Published in2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) pp. 3886 - 3895
Main Authors Stone, Austin, Maurer, Daniel, Ayvaci, Alper, Angelova, Anelia, Jonschkowski, Rico
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2021
Subjects
Benchmark testing
Computational modeling
Computer architecture
Computer vision
Data models
Estimation
Optical losses
Online AccessGet full text

Cover

More Information
Summary:We present SMURF, a method for unsupervised learning of optical flow that improves state of the art on all benchmarks by 36% to 40% (over the prior best method UFlow) and even outperforms several supervised approaches such as PWC-Net and FlowNet2. Our method integrates architecture improvements from supervised optical flow, i.e. the RAFT model, with new ideas for unsupervised learning that include a sequence-aware self-supervision loss, a technique for handling out-of-frame motion, and an approach for learning effectively from multi-frame video data while still only requiring two frames for inference.
ISSN:2575-7075
DOI:10.1109/CVPR46437.2021.00388