Scene Flow as a Partial Differential Equation

• Published in: arXiv.org
• Main Authors: Vedder, Kyle, Peri, Neehar, Khatri, Ishan, Li, Siyi, Eaton, Eric, Kocamaz, Mehmet, Wang, Yue, Yu, Zhiding, Ramanan, Deva, Pehserl, Joachim
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 02.10.2024
• Subjects: Moving object recognition; Partial differential equations; Three dimensional flow

We reframe scene flow as the problem of estimating a continuous space and time PDE that describes motion for an entire observation sequence, represented with a neural prior. Our resulting unsupervised method, EulerFlow, produces high quality scene flow on real-world data across multiple domains, including large-scale autonomous driving scenes and dynamic tabletop settings. Notably, EulerFlow produces high quality flow on small, fast moving objects like birds and tennis balls, and exhibits emergent 3D point tracking behavior by solving its estimated PDE over long time horizons. On the Argoverse 2 2024 Scene Flow Challenge, EulerFlow outperforms all prior art, beating the next best unsupervised method by over 2.5x and the next best supervised method by over 10%.