Real-Time Intermediate Flow Estimation for Video Frame Interpolation

• Published in: Computer Vision - ECCV 2022 Vol. 13674; pp. 624 - 642
• Main Authors: Huang, Zhewei, Zhang, Tianyuan, Heng, Wen, Shi, Boxin, Zhou, Shuchang
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer 01.01.2022; Springer Nature Switzerland
• Series: Lecture Notes in Computer Science
• DOI: 10.1007/978-3-031-19781-9_36
• ISBN: 9783031197802; 3031197801
• ISSN: 0302-9743

Real-time video frame interpolation (VFI) is very useful in video processing, media players, and display devices. We propose RIFE, a Real-time Intermediate Flow Estimation algorithm for VFI. To realize a high-quality flow-based VFI method, RIFE uses a neural network named IFNet that can estimate the intermediate flows end-to-end with much faster speed. A privileged distillation scheme is designed for stable IFNet training and improve the overall performance. RIFE does not rely on pre-trained optical flow models and can support arbitrary-timestep frame interpolation with the temporal encoding input. Experiments demonstrate that RIFE achieves state-of-the-art performance on several public benchmarks. Compared with the popular SuperSlomo and DAIN methods, RIFE is 4–27 times faster and produces better results. Furthermore, RIFE can be extended to wider applications thanks to temporal encoding. https://github.com/megvii-research/ECCV2022-RIFE