Improving Video Super-Resolution with Long-Term Self-Exemplars

Existing video super-resolution methods often utilize a few neighboring frames to generate a higher-resolution image for each frame. However, the abundant information in distant frames has not been fully exploited in these methods: corresponding patches of the same instance appear across distant fra...

Full description

Saved in:
Bibliographic Details
Published in2023 IEEE International Conference on Robotics and Automation (ICRA) pp. 5992 - 5998
Main Authors Meng, Guotao, Wu, Yue, Chen, Qifeng
Format Conference Proceeding
LanguageEnglish
Published IEEE 29.05.2023
Subjects
Automation
Cameras
Fuses
Robot vision systems
Superresolution
Surveillance
Training
Online AccessGet full text

Cover

More Information
Summary:Existing video super-resolution methods often utilize a few neighboring frames to generate a higher-resolution image for each frame. However, the abundant information in distant frames has not been fully exploited in these methods: corresponding patches of the same instance appear across distant frames at different scales. Based on this observation, we propose to improve the video super-resolution quality with long-term cross-scale aggregation that leverages similar patches (self-exemplars) across distant frames. Our method can be implemented as post-processing for any super-resolution methods to improve performance. Our model consists of a multi-reference alignment module to fuse the features derived from similar patches: we fuse the features of distant references to perform high-quality super-resolution. We also propose a novel and practical training strategy for reference-based super-resolution. To evaluate the performance of our proposed method, we conduct extensive experiments on our collected CarCam dataset, the Waymo Open dataset, and the REDS dataset, and the results demonstrate our method outperforms state-of-the-art methods.
DOI:10.1109/ICRA48891.2023.10160844